![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1807042%2F4a5debc3dd36e10a31aa8f9b046628c7%2Fbatchsize64.svg?generation=1593886097416388&alt=media\")
I recently joined the competition. The raw images take so long to load. My epochs time increased 3 times. So I decided to investigate further. This bugged me a lot.\nI compared 4 different methods of loading images, all using Pytorch Dataloaders:
\n\nI tried two different batch sizes: 64 and 128
\n\nQuite alarming that reading a batch of Raw images is 3 times slower on Kaggle then on My Laptop. Your mileage may vary.
\n\nBatch Size: 64\n
Batch Size: 128\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1807042%2F5642850c7c46d24dcb8a4285fa2563cf%2Fbatchsize128.svg?generation=1593886126333392&alt=media\")
The best option is to resize the Images beforehand.
\nHope this helps 👍
I recently joined the competition. The raw images take so long to load. My epochs time increased 3 times. So I decided to investigate further. This bugged me a lot.\nI compared 4 different methods of loading images, all using Pytorch Dataloaders:
\n\nI tried two different batch sizes: 64 and 128
\n\nQuite alarming that reading a batch of Raw images is 3 times slower on Kaggle then on My Laptop. Your mileage may vary.
\n\nBatch Size: 64\n
Batch Size: 128\n
The best option is to resize the Images beforehand.
\nHope this helps 👍
P.S: The only reason I can think of this delay is Server HardDrives are slow. Let me know if I am wrong.
", "rawMarkdown": "P.S: The only reason I can think of this delay is Server HardDrives are slow. Let me know if I am wrong.", "votes": null }, { "id": "916010", "postDate": "07/05/2020 09:23:42", "content": "Hey @spideysloth thanks for the benchmark!\nKaggle uses NFS (network file system) to mount the external data (i.e. the ../input folders)
\n\n\n\n\n!mount | grep input
\n
192.168.0.18:/data/kagglesdsdata/competitions/20270/1222630/c31vipkmu6ij on /kaggle/input/siim-isic-melanoma-classification type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.18,mountvers=3,mountport=2050,mountproto=tcp,local_lock=all,addr=192.168.0.18)
Hi :-) \nKaggle GPU and TPU kernels only have 2 CPU Cores and only pure CPU kernels have 4 cores.\nAdditionally, the input data is accessed via network.
\n\nYour notebook may have better specs. 4 Cores or more? Data local on SSD? Running natively on hardware, without any virtual environment.
\n\nAnd yes, it's important to use scaled down images, because a lot of the original images are pretty huge - 6000x4000 pixels.
", "rawMarkdown": "Hi :-) \nKaggle GPU and TPU kernels only have 2 CPU Cores and only pure CPU kernels have 4 cores.\nAdditionally, the input data is accessed via network.\n\nYour notebook may have better specs. 4 Cores or more? Data local on SSD? Running natively on hardware, without any virtual environment.\n\nAnd yes, it's important to use scaled down images, because a lot of the original images are pretty huge - 6000x4000 pixels.", "votes": null }, { "id": "916202", "postDate": "07/05/2020 13:19:02", "content": "I tried both kaggle GPU and CPU kernels. Doesn't make much difference. The network and storage is the bottleneck. Resizing seems to be the best option. \nP.S: My notebook has 6 cores and ssd. I know this is not a fair comparison but I just wanted to see how much time I can save by resizing beforehand.
", "rawMarkdown": "I tried both kaggle GPU and CPU kernels. Doesn't make much difference. The network and storage is the bottleneck. Resizing seems to be the best option. \nP.S: My notebook has 6 cores and ssd. I know this is not a fair comparison but I just wanted to see how much time I can save by resizing beforehand.", "votes": null } ], "comments": [ { "id": 915427, "author_name": "spideysloth", "author_url": "", "post_date": "07/04/2020 18:12:31", "content": "P.S: The only reason I can think of this delay is Server HardDrives are slow. Let me know if I am wrong.
", "votes": null, "replies": [] }, { "id": 916010, "author_name": "hmendonca", "author_url": "", "post_date": "07/05/2020 09:23:42", "content": "Hey @spideysloth thanks for the benchmark!\nKaggle uses NFS (network file system) to mount the external data (i.e. the ../input folders)
\n\n\n\n\n!mount | grep input
\n
192.168.0.18:/data/kagglesdsdata/competitions/20270/1222630/c31vipkmu6ij on /kaggle/input/siim-isic-melanoma-classification type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.18,mountvers=3,mountport=2050,mountproto=tcp,local_lock=all,addr=192.168.0.18)
Hi :-) \nKaggle GPU and TPU kernels only have 2 CPU Cores and only pure CPU kernels have 4 cores.\nAdditionally, the input data is accessed via network.
\n\nYour notebook may have better specs. 4 Cores or more? Data local on SSD? Running natively on hardware, without any virtual environment.
\n\nAnd yes, it's important to use scaled down images, because a lot of the original images are pretty huge - 6000x4000 pixels.
", "votes": null, "replies": [ { "id": 916202, "author_name": "spideysloth", "author_url": "", "post_date": "07/05/2020 13:19:02", "content": "I tried both kaggle GPU and CPU kernels. Doesn't make much difference. The network and storage is the bottleneck. Resizing seems to be the best option. \nP.S: My notebook has 6 cores and ssd. I know this is not a fair comparison but I just wanted to see how much time I can save by resizing beforehand.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "915424": "I recently joined the competition. The raw images take so long to load. My epochs time increased 3 times. So I decided to investigate further. This bugged me a lot.\nI compared 4 different methods of loading images, all using *Pytorch Dataloaders*:\n\n1. Reading Raw Images then resizing and normalizing them. \n2 Reading Resized Images and only normalizing them.\n3. Reading TFrecords as an iterator. \n4. Reading TFrecords and loading them in the memory (Not quite practical on Kaggle I can only load 256x256 tfreacords. RAM limitations)\n\nI tried two different batch sizes: **64** and **128**\n\nQuite alarming that reading a batch of Raw images is 3 times slower on Kaggle then on My Laptop. Your mileage may vary. \n\n**Batch Size: 64**\n\n\n\n**Batch Size: 128**\n\n\n\nThe best option is to resize the Images beforehand. \nHope this helps 👍", "915427": "P.S: The only reason I can think of this delay is Server HardDrives are slow. Let me know if I am wrong.", "916010": "Hey @spideysloth thanks for the benchmark!\nKaggle uses NFS (network file system) to mount the external data (i.e. the ../input folders) \n> !mount | grep input\n\n`192.168.0.18:/data/kagglesdsdata/competitions/20270/1222630/c31vipkmu6ij on /kaggle/input/siim-isic-melanoma-classification type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.18,mountvers=3,mountport=2050,mountproto=tcp,local_lock=all,addr=192.168.0.18)`", "916075": "Hi :-) \nKaggle GPU and TPU kernels only have 2 CPU Cores and only pure CPU kernels have 4 cores.\nAdditionally, the input data is accessed via network.\n\nYour notebook may have better specs. 4 Cores or more? Data local on SSD? Running natively on hardware, without any virtual environment.\n\nAnd yes, it's important to use scaled down images, because a lot of the original images are pretty huge - 6000x4000 pixels.", "916202": "I tried both kaggle GPU and CPU kernels. Doesn't make much difference. The network and storage is the bottleneck. Resizing seems to be the best option. \nP.S: My notebook has 6 cores and ssd. I know this is not a fair comparison but I just wanted to see how much time I can save by resizing beforehand." }, "source": "meta" }