Transformer: TRANS1: AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)>
Hi,
I've encounted error when launching the Prepa the dataset
section (7th cell). Here is what it returned:
Process SpawnPoolWorker-1: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-2: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-3: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-4: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-5: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-6: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-7: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-8: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-9: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-10: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-11: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-12: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-13: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-14: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-15: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-16: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-17: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-18: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-19: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-20: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-21: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-22: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-23: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)> Process SpawnPoolWorker-24: Traceback (most recent call last): File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\pool.py", line 114, in worker task = get() File "C:\Users\alngu\anaconda3\envs\fidle\lib\multiprocessing\queues.py", line 358, in get return _ForkingPickler.loads(res) AttributeError: Can't get attribute 'tokenize_sample' on <module '__main__' (built-in)>
I run it on my laptop, I set n_cpus=6
and here is what the 1st (Imports and initialization) cell gives:
Tensorflow 2.8.0 #GPUs: 1 GPU 0: NVIDIA GeForce RTX 3080 Laptop GPU (UUID: GPU-18f8e617-13b2-0831-0bb2-8fcc1851d9c8) INFO:tensorflow:Mixed precision compatibility check (mixed_float16): OK Your GPU will likely run quickly with dtype policy mixed_float16 as it has compute capability of at least 7.0. Your GPU: NVIDIA GeForce RTX 3080 Laptop GPU, compute capability 8.6
Thanks for your help, Antoine