{
  "id": 23038,
  "title": "Keep reporting corrupt jpg data",
  "url": "/competitions/painter-by-numbers/discussion/23038",
  "author_name": "",
  "post_date": "2016-08-22T18:45:45.627Z",
  "votes": 1,
  "comment_count": 1,
  "views": 473,
  "content": "<p>Even after replacing the bad images, it keeps reporting corrupt jpg data when I load the train or test images, using opencv:</p>\n\n<p>time: 0.0159659385681  count: 0\nCorrupt JPEG data: 6 extraneous bytes before marker 0xda\ntime: 5.78732705116  count: 1000\ntime: 5.19732785225  count: 2000\nread none error: 83271.jpg\ntime: 6.01049399376  count: 3000\ntime: 5.07182312012  count: 4000\ntime: 4.42018699646  count: 5000\nCorrupt JPEG data: 3911 extraneous bytes before marker 0xd9\ntime: 3.7450940609  count: 6000\nCorrupt JPEG data: premature end of data segment\ntime: 4.5103559494  count: 7000\ntime: 4.67628693581  count: 8000\ntime: 4.3694050312  count: 9000\ntime: 4.00658798218  count: 10000\nread none error: 85232.jpg\nread none error: 1262.jpg\ntime: 4.30924606323  count: 11000\ntime: 4.92733097076  count: 12000\ntime: 5.36931705475  count: 13000\nCorrupt JPEG data: 10 extraneous bytes before marker 0xd9\ntime: 5.31384086609  count: 14000\ntime: 5.82300305367  count: 15000\ntime: 5.96512198448  count: 16000\ntime: 4.8245549202  count: 17000\nCorrupt JPEG data: premature end of data segment\nread none error: 38922.jpg\ntime: 4.69963908195  count: 18000\ntime: 5.35288500786  count: 19000\ntime: 5.82748007774  count: 20000\ntime: 6.28991293907  count: 21000\ntime: 17.0689280033  count: 22000\ntime: 9.87945103645  count: 23000</p>\n\n<p>Please help!</p>\n\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "132119",
      "postDate": "08/22/2016 18:45:45",
      "content": "<p>Even after replacing the bad images, it keeps reporting corrupt jpg data when I load the train or test images, using opencv:</p>\n\n<p>time: 0.0159659385681  count: 0\nCorrupt JPEG data: 6 extraneous bytes before marker 0xda\ntime: 5.78732705116  count: 1000\ntime: 5.19732785225  count: 2000\nread none error: 83271.jpg\ntime: 6.01049399376  count: 3000\ntime: 5.07182312012  count: 4000\ntime: 4.42018699646  count: 5000\nCorrupt JPEG data: 3911 extraneous bytes before marker 0xd9\ntime: 3.7450940609  count: 6000\nCorrupt JPEG data: premature end of data segment\ntime: 4.5103559494  count: 7000\ntime: 4.67628693581  count: 8000\ntime: 4.3694050312  count: 9000\ntime: 4.00658798218  count: 10000\nread none error: 85232.jpg\nread none error: 1262.jpg\ntime: 4.30924606323  count: 11000\ntime: 4.92733097076  count: 12000\ntime: 5.36931705475  count: 13000\nCorrupt JPEG data: 10 extraneous bytes before marker 0xd9\ntime: 5.31384086609  count: 14000\ntime: 5.82300305367  count: 15000\ntime: 5.96512198448  count: 16000\ntime: 4.8245549202  count: 17000\nCorrupt JPEG data: premature end of data segment\nread none error: 38922.jpg\ntime: 4.69963908195  count: 18000\ntime: 5.35288500786  count: 19000\ntime: 5.82748007774  count: 20000\ntime: 6.28991293907  count: 21000\ntime: 17.0689280033  count: 22000\ntime: 9.87945103645  count: 23000</p>\n\n<p>Please help!</p>\n\n<p>Thanks!</p>",
      "rawMarkdown": "Even after replacing the bad images, it keeps reporting corrupt jpg data when I load the train or test images, using opencv:\r\n\r\ntime: 0.0159659385681  count: 0\r\nCorrupt JPEG data: 6 extraneous bytes before marker 0xda\r\ntime: 5.78732705116  count: 1000\r\ntime: 5.19732785225  count: 2000\r\nread none error: 83271.jpg\r\ntime: 6.01049399376  count: 3000\r\ntime: 5.07182312012  count: 4000\r\ntime: 4.42018699646  count: 5000\r\nCorrupt JPEG data: 3911 extraneous bytes before marker 0xd9\r\ntime: 3.7450940609  count: 6000\r\nCorrupt JPEG data: premature end of data segment\r\ntime: 4.5103559494  count: 7000\r\ntime: 4.67628693581  count: 8000\r\ntime: 4.3694050312  count: 9000\r\ntime: 4.00658798218  count: 10000\r\nread none error: 85232.jpg\r\nread none error: 1262.jpg\r\ntime: 4.30924606323  count: 11000\r\ntime: 4.92733097076  count: 12000\r\ntime: 5.36931705475  count: 13000\r\nCorrupt JPEG data: 10 extraneous bytes before marker 0xd9\r\ntime: 5.31384086609  count: 14000\r\ntime: 5.82300305367  count: 15000\r\ntime: 5.96512198448  count: 16000\r\ntime: 4.8245549202  count: 17000\r\nCorrupt JPEG data: premature end of data segment\r\nread none error: 38922.jpg\r\ntime: 4.69963908195  count: 18000\r\ntime: 5.35288500786  count: 19000\r\ntime: 5.82748007774  count: 20000\r\ntime: 6.28991293907  count: 21000\r\ntime: 17.0689280033  count: 22000\r\ntime: 9.87945103645  count: 23000\r\n\r\n\r\nPlease help!\r\n\r\nThanks!",
      "votes": null
    },
    {
      "id": "132125",
      "postDate": "08/22/2016 19:12:31",
      "content": "<p>Trouble-shooting corrupt data is frustrating, but there are a couple things you can try.</p>\n\n<ol>\n<li>One thing to check is whether the images are truly corrupt. Can you open the problematic files with another program (Windows Paint, photoshop, Preview?) Do the images look OK? </li>\n<li><p>An alternative to opencv is the Python Image Library: PIL can often open images that openc chokes on. </p>\n\n<p>from PIL import Image</p>\n\n<p>im = Image.open('test/83271.jpg')</p>\n\n<p>plt.imshow(im, cmap=&quot;Greys_r&quot;)</p></li>\n</ol>\n\n<p>Image 83271 loads and can be viewed without issues with PIL. Once loaded, you can save it in a different format.</p>",
      "rawMarkdown": "Trouble-shooting corrupt data is frustrating, but there are a couple things you can try.\r\n\r\n 1. One thing to check is whether the images are truly corrupt. Can you open the problematic files with another program (Windows Paint, photoshop, Preview?) Do the images look OK? \r\n 2. An alternative to opencv is the Python Image Library: PIL can often open images that openc chokes on. \r\n\r\n    from PIL import Image\r\n\r\n    im = Image.open('test/83271.jpg')\r\n\r\n    plt.imshow(im, cmap=\"Greys_r\")\r\n\r\nImage 83271 loads and can be viewed without issues with PIL. Once loaded, you can save it in a different format.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 132125,
      "author_name": "smallyellowduck",
      "author_url": "",
      "post_date": "08/22/2016 19:12:31",
      "content": "<p>Trouble-shooting corrupt data is frustrating, but there are a couple things you can try.</p>\n\n<ol>\n<li>One thing to check is whether the images are truly corrupt. Can you open the problematic files with another program (Windows Paint, photoshop, Preview?) Do the images look OK? </li>\n<li><p>An alternative to opencv is the Python Image Library: PIL can often open images that openc chokes on. </p>\n\n<p>from PIL import Image</p>\n\n<p>im = Image.open('test/83271.jpg')</p>\n\n<p>plt.imshow(im, cmap=&quot;Greys_r&quot;)</p></li>\n</ol>\n\n<p>Image 83271 loads and can be viewed without issues with PIL. Once loaded, you can save it in a different format.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "132119": "Even after replacing the bad images, it keeps reporting corrupt jpg data when I load the train or test images, using opencv:\r\n\r\ntime: 0.0159659385681  count: 0\r\nCorrupt JPEG data: 6 extraneous bytes before marker 0xda\r\ntime: 5.78732705116  count: 1000\r\ntime: 5.19732785225  count: 2000\r\nread none error: 83271.jpg\r\ntime: 6.01049399376  count: 3000\r\ntime: 5.07182312012  count: 4000\r\ntime: 4.42018699646  count: 5000\r\nCorrupt JPEG data: 3911 extraneous bytes before marker 0xd9\r\ntime: 3.7450940609  count: 6000\r\nCorrupt JPEG data: premature end of data segment\r\ntime: 4.5103559494  count: 7000\r\ntime: 4.67628693581  count: 8000\r\ntime: 4.3694050312  count: 9000\r\ntime: 4.00658798218  count: 10000\r\nread none error: 85232.jpg\r\nread none error: 1262.jpg\r\ntime: 4.30924606323  count: 11000\r\ntime: 4.92733097076  count: 12000\r\ntime: 5.36931705475  count: 13000\r\nCorrupt JPEG data: 10 extraneous bytes before marker 0xd9\r\ntime: 5.31384086609  count: 14000\r\ntime: 5.82300305367  count: 15000\r\ntime: 5.96512198448  count: 16000\r\ntime: 4.8245549202  count: 17000\r\nCorrupt JPEG data: premature end of data segment\r\nread none error: 38922.jpg\r\ntime: 4.69963908195  count: 18000\r\ntime: 5.35288500786  count: 19000\r\ntime: 5.82748007774  count: 20000\r\ntime: 6.28991293907  count: 21000\r\ntime: 17.0689280033  count: 22000\r\ntime: 9.87945103645  count: 23000\r\n\r\n\r\nPlease help!\r\n\r\nThanks!",
    "132125": "Trouble-shooting corrupt data is frustrating, but there are a couple things you can try.\r\n\r\n 1. One thing to check is whether the images are truly corrupt. Can you open the problematic files with another program (Windows Paint, photoshop, Preview?) Do the images look OK? \r\n 2. An alternative to opencv is the Python Image Library: PIL can often open images that openc chokes on. \r\n\r\n    from PIL import Image\r\n\r\n    im = Image.open('test/83271.jpg')\r\n\r\n    plt.imshow(im, cmap=\"Greys_r\")\r\n\r\nImage 83271 loads and can be viewed without issues with PIL. Once loaded, you can save it in a different format."
  },
  "source": "meta"
}