A number of images in the training dataset have a filename such as XXXX.jpg, when they are in fact GIFs. This is a minor issue, since some libraries, including OpenCV, will not open GIF files. Some images (such as 121.jpg) also have transparency, which JPEG doesn't support.
\n\nIn the training set, the following are misnamed:
\n\nA number of images in the training dataset have a filename such as XXXX.jpg, when they are in fact GIFs. This is a minor issue, since some libraries, including OpenCV, will not open GIF files. Some images (such as 121.jpg) also have transparency, which JPEG doesn't support.
\n\nIn the training set, the following are misnamed:
\n\nAha! Thanks for tracking this down! I was already aware that there were a handful of files that opencv refused to open, but I hadn't figured out why. I think you are correct that these files are GIFs.
\n\nInstead of using opencv to open the files, another option is to use the Python image library (PIL) to open the files. You can convert the above files from GIF to JPG by opening them and resaving them with PIL.
", "rawMarkdown": "Aha! Thanks for tracking this down! I was already aware that there were a handful of files that opencv refused to open, but I hadn't figured out why. I think you are correct that these files are GIFs.\r\n\r\nInstead of using opencv to open the files, another option is to use the Python image library (PIL) to open the files. You can convert the above files from GIF to JPG by opening them and resaving them with PIL.", "votes": null }, { "id": "119942", "postDate": "05/13/2016 21:06:50", "content": "Here's a quick python script that implements small yellow duck's suggestion:
\n\nfrom PIL import Image\nimport os\n\ndirpath = './train/'\n\nfor i in [121, 32150, 32192, 32577, 38324, 43658, 50789, 65430, 69008, 77094, 80945, 84021, 84772, 95897, 97976]:\n print i\n\n input_filename = os.path.join(dirpath,str(i)+".jpg")\n input_filename_as_gif = os.path.join(dirpath,str(i)+".gif")\n\n os.rename(input_filename, input_filename_as_gif)\n\n img = Image.open( input_filename_as_gif ).convert('RGB')\n img.save( os.path.join(dirpath,str(i) + ".jpg"), "JPEG")\n\n os.remove(input_filename_as_gif)\nJust for completeness, please note these files from the test set are also GIFs:
\n\n1262.jpg: GIF image data, version 89a, 717 x 480\n38922.jpg: GIF image data, version 89a, 500 x 373\n83271.jpg: GIF image data, version 89a, 600 x 479\n85232.jpg: GIF image data, version 89a, 1200 x 360\nThere's also a number of PNG and BMP files throughout both sets that have .jpg file extensions. Almost all libraries should be smart enough to ignore the file extension and open them anyway though.
", "rawMarkdown": "Just for completeness, please note these files from the test set are also GIFs:\r\n\r\n 1262.jpg: GIF image data, version 89a, 717 x 480\r\n 38922.jpg: GIF image data, version 89a, 500 x 373\r\n 83271.jpg: GIF image data, version 89a, 600 x 479\r\n 85232.jpg: GIF image data, version 89a, 1200 x 360\r\n\r\nThere's also a number of PNG and BMP files throughout both sets that have .jpg file extensions. Almost all libraries should be smart enough to ignore the file extension and open them anyway though.", "votes": null }, { "id": "137571", "postDate": "10/02/2016 22:29:10", "content": "Thanks for tracking this down!
", "rawMarkdown": "Thanks for tracking this down!", "votes": null }, { "id": "137692", "postDate": "10/04/2016 05:40:46", "content": "My little research shows that there are probably a bit more non-jpegs:
\n\nAttaching lists (filenames + mime type) for each set.
", "rawMarkdown": "My little research shows that there are probably a bit more non-jpegs:\r\n\r\n - 183 in training sets\r\n - 46 in test set\r\n\r\nAttaching lists (filenames + mime type) for each set.", "votes": null } ], "comments": [ { "id": 119941, "author_name": "smallyellowduck", "author_url": "", "post_date": "05/13/2016 20:48:16", "content": "Aha! Thanks for tracking this down! I was already aware that there were a handful of files that opencv refused to open, but I hadn't figured out why. I think you are correct that these files are GIFs.
\n\nInstead of using opencv to open the files, another option is to use the Python image library (PIL) to open the files. You can convert the above files from GIF to JPG by opening them and resaving them with PIL.
", "votes": null, "replies": [] }, { "id": 119942, "author_name": "yhenon", "author_url": "", "post_date": "05/13/2016 21:06:50", "content": "Here's a quick python script that implements small yellow duck's suggestion:
\n\nfrom PIL import Image\nimport os\n\ndirpath = './train/'\n\nfor i in [121, 32150, 32192, 32577, 38324, 43658, 50789, 65430, 69008, 77094, 80945, 84021, 84772, 95897, 97976]:\n print i\n\n input_filename = os.path.join(dirpath,str(i)+".jpg")\n input_filename_as_gif = os.path.join(dirpath,str(i)+".gif")\n\n os.rename(input_filename, input_filename_as_gif)\n\n img = Image.open( input_filename_as_gif ).convert('RGB')\n img.save( os.path.join(dirpath,str(i) + ".jpg"), "JPEG")\n\n os.remove(input_filename_as_gif)\nJust for completeness, please note these files from the test set are also GIFs:
\n\n1262.jpg: GIF image data, version 89a, 717 x 480\n38922.jpg: GIF image data, version 89a, 500 x 373\n83271.jpg: GIF image data, version 89a, 600 x 479\n85232.jpg: GIF image data, version 89a, 1200 x 360\nThere's also a number of PNG and BMP files throughout both sets that have .jpg file extensions. Almost all libraries should be smart enough to ignore the file extension and open them anyway though.
", "votes": null, "replies": [] }, { "id": 137571, "author_name": "", "author_url": "", "post_date": "10/02/2016 22:29:10", "content": "Thanks for tracking this down!
", "votes": null, "replies": [] }, { "id": 137692, "author_name": "mikhailyurasov", "author_url": "", "post_date": "10/04/2016 05:40:46", "content": "My little research shows that there are probably a bit more non-jpegs:
\n\nAttaching lists (filenames + mime type) for each set.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "119937": "A number of images in the training dataset have a filename such as XXXX.jpg, when they are in fact GIFs. This is a minor issue, since some libraries, including OpenCV, will not open GIF files. Some images (such as 121.jpg) also have transparency, which JPEG doesn't support.\r\n\r\nIn the training set, the following are misnamed:\r\n\r\n- 121.jpg \r\n- 32150.jpg\r\n- 32192.jpg\r\n- 32577.jpg\r\n- 38324.jpg\r\n- 43658.jpg\r\n- 50789.jpg\r\n- 65430.jpg\r\n- 69008.jpg\r\n- 77094.jpg\r\n- 80945.jpg\r\n- 84021.jpg\r\n- 84772.jpg\r\n- 95897.jpg\r\n- 97976.jpg", "119941": "Aha! Thanks for tracking this down! I was already aware that there were a handful of files that opencv refused to open, but I hadn't figured out why. I think you are correct that these files are GIFs.\r\n\r\nInstead of using opencv to open the files, another option is to use the Python image library (PIL) to open the files. You can convert the above files from GIF to JPG by opening them and resaving them with PIL.", "119942": "Here's a quick python script that implements small yellow duck's suggestion:\r\n\r\n\r\n\tfrom PIL import Image\r\n\timport os\r\n\r\n\tdirpath = './train/'\r\n\r\n\tfor i in [121, 32150, 32192, 32577, 38324, 43658, 50789, 65430, 69008, 77094, 80945, 84021, 84772, 95897, 97976]:\r\n\t\tprint i\r\n\r\n\t\tinput_filename = os.path.join(dirpath,str(i)+\".jpg\")\r\n\t\tinput_filename_as_gif = os.path.join(dirpath,str(i)+\".gif\")\r\n\r\n\t\tos.rename(input_filename, input_filename_as_gif)\r\n\t\t\r\n\t\timg = Image.open( input_filename_as_gif ).convert('RGB')\r\n\t\timg.save( os.path.join(dirpath,str(i) + \".jpg\"), \"JPEG\")\r\n\r\n\t\tos.remove(input_filename_as_gif)", "120125": "Just for completeness, please note these files from the test set are also GIFs:\r\n\r\n 1262.jpg: GIF image data, version 89a, 717 x 480\r\n 38922.jpg: GIF image data, version 89a, 500 x 373\r\n 83271.jpg: GIF image data, version 89a, 600 x 479\r\n 85232.jpg: GIF image data, version 89a, 1200 x 360\r\n\r\nThere's also a number of PNG and BMP files throughout both sets that have .jpg file extensions. Almost all libraries should be smart enough to ignore the file extension and open them anyway though.", "137571": "Thanks for tracking this down!", "137692": "My little research shows that there are probably a bit more non-jpegs:\r\n\r\n - 183 in training sets\r\n - 46 in test set\r\n\r\nAttaching lists (filenames + mime type) for each set." }, "source": "meta" }