Any idea how to best load the unlabelled images? When using keras ImageDataGenerator flow_from_directory(), it takes too long and I had a to cancel multiple times. A search on google shows others with same problem. One post indicated that it was slow because the images were being loaded from the internet and downloading on local machine made it faster. However, I think in case of Kaggle notebook it copied the data locally where the notebook is, so it doesn't sound like the internet should be the bottleneck. Any guidance would be helpful.
", "messages": [ { "id": "1269143", "postDate": "04/10/2021 08:00:11", "content": "Any idea how to best load the unlabelled images? When using keras ImageDataGenerator flow_from_directory(), it takes too long and I had a to cancel multiple times. A search on google shows others with same problem. One post indicated that it was slow because the images were being loaded from the internet and downloading on local machine made it faster. However, I think in case of Kaggle notebook it copied the data locally where the notebook is, so it doesn't sound like the internet should be the bottleneck. Any guidance would be helpful.
", "rawMarkdown": "Any idea how to best load the unlabelled images? When using keras ImageDataGenerator flow_from_directory(), it takes too long and I had a to cancel multiple times. A search on google shows others with same problem. One post indicated that it was slow because the images were being loaded from the internet and downloading on local machine made it faster. However, I think in case of Kaggle notebook it copied the data locally where the notebook is, so it doesn't sound like the internet should be the bottleneck. Any guidance would be helpful.", "votes": null }, { "id": "1270730", "postDate": "04/11/2021 23:44:18", "content": "Hi, I had the same problem but I still don't understand what the problem is, then I decided to stop using ImageDataGenerator to load large datasets. For an example of how I did it I leave my notebook: https://github.com/bezbahen0/MashineLearningExperiments/blob/master/VAE_generate/vae-generate-new-faces.ipynb
", "rawMarkdown": "Hi, I had the same problem but I still don't understand what the problem is, then I decided to stop using ImageDataGenerator to load large datasets. For an example of how I did it I leave my notebook: https://github.com/bezbahen0/MashineLearningExperiments/blob/master/VAE_generate/vae-generate-new-faces.ipynb", "votes": null } ], "comments": [ { "id": 1270730, "author_name": "dmitrysokolevskiy", "author_url": "", "post_date": "04/11/2021 23:44:18", "content": "Hi, I had the same problem but I still don't understand what the problem is, then I decided to stop using ImageDataGenerator to load large datasets. For an example of how I did it I leave my notebook: https://github.com/bezbahen0/MashineLearningExperiments/blob/master/VAE_generate/vae-generate-new-faces.ipynb
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1269143": "Any idea how to best load the unlabelled images? When using keras ImageDataGenerator flow_from_directory(), it takes too long and I had a to cancel multiple times. A search on google shows others with same problem. One post indicated that it was slow because the images were being loaded from the internet and downloading on local machine made it faster. However, I think in case of Kaggle notebook it copied the data locally where the notebook is, so it doesn't sound like the internet should be the bottleneck. Any guidance would be helpful.", "1270730": "Hi, I had the same problem but I still don't understand what the problem is, then I decided to stop using ImageDataGenerator to load large datasets. For an example of how I did it I leave my notebook: https://github.com/bezbahen0/MashineLearningExperiments/blob/master/VAE_generate/vae-generate-new-faces.ipynb" }, "source": "meta" }