{
  "id": 61757,
  "title": "Corrupted Images on AWS?",
  "url": "/competitions/google-ai-open-images-object-detection-track/discussion/61757",
  "author_name": "",
  "post_date": "2018-07-23T07:14:31.955905Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I'm seeing \"Decryption Failed or Bad Record\" errors on some files when syncing Open Images from AWS, based on the instructions at:\n<a href=\"https://github.com/cvdfoundation/open-images-dataset\">https://github.com/cvdfoundation/open-images-dataset</a></p>\n\n<blockquote>\n  <p>aws s3 --no-sign-request sync s3://open-images-dataset/train train</p>\n  \n  <p>download failed: s3://open-images-dataset/train/711bc12d088711f9.jpg\n  to train\\711bc12d088711f9.jpg [SSL:\n  DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record\n  mac (_ssl.c:2178)</p>\n</blockquote>\n\n<p>This happens sporadically when I run the \"aws s3 sync\" command on the \"train\" folder. I don't monitor the command line output, but sometimes when I view the output, I see these error lines.</p>\n\n<p>If I try to manually download that specific file again, it will fail with the same error too:</p>\n\n<blockquote>\n  <p>aws s3 --no-sign-request cp\n  s3://open-images-dataset/train/711bc12d088711f9.jpg</p>\n  \n  <p>download failed: s3://open-images-dataset/train/711bc12d088711f9.jpg\n  to .\\711bc12d088711f9.jpg [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC]\n  decryption failed or bad record mac (_ssl.c:2178)</p>\n</blockquote>\n\n<p>After many tries, it will sometimes finally succeed in downloading. Is this caused by corrupted images on some of AWS's replicated data servers?</p>\n\n<p>I found this (old) thread on Amazon with similar issue, which seems to indicate that it's AWS's problem:\n<a href=\"https://forums.aws.amazon.com/thread.jspa?threadID=126932\">https://forums.aws.amazon.com/thread.jspa?threadID=126932</a></p>",
  "messages": [
    {
      "id": "360759",
      "postDate": "07/23/2018 07:14:31",
      "content": "<p>I'm seeing \"Decryption Failed or Bad Record\" errors on some files when syncing Open Images from AWS, based on the instructions at:\n<a href=\"https://github.com/cvdfoundation/open-images-dataset\">https://github.com/cvdfoundation/open-images-dataset</a></p>\n\n<blockquote>\n  <p>aws s3 --no-sign-request sync s3://open-images-dataset/train train</p>\n  \n  <p>download failed: s3://open-images-dataset/train/711bc12d088711f9.jpg\n  to train\\711bc12d088711f9.jpg [SSL:\n  DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record\n  mac (_ssl.c:2178)</p>\n</blockquote>\n\n<p>This happens sporadically when I run the \"aws s3 sync\" command on the \"train\" folder. I don't monitor the command line output, but sometimes when I view the output, I see these error lines.</p>\n\n<p>If I try to manually download that specific file again, it will fail with the same error too:</p>\n\n<blockquote>\n  <p>aws s3 --no-sign-request cp\n  s3://open-images-dataset/train/711bc12d088711f9.jpg</p>\n  \n  <p>download failed: s3://open-images-dataset/train/711bc12d088711f9.jpg\n  to .\\711bc12d088711f9.jpg [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC]\n  decryption failed or bad record mac (_ssl.c:2178)</p>\n</blockquote>\n\n<p>After many tries, it will sometimes finally succeed in downloading. Is this caused by corrupted images on some of AWS's replicated data servers?</p>\n\n<p>I found this (old) thread on Amazon with similar issue, which seems to indicate that it's AWS's problem:\n<a href=\"https://forums.aws.amazon.com/thread.jspa?threadID=126932\">https://forums.aws.amazon.com/thread.jspa?threadID=126932</a></p>",
      "rawMarkdown": "I'm seeing \"Decryption Failed or Bad Record\" errors on some files when syncing Open Images from AWS, based on the instructions at:\nhttps://github.com/cvdfoundation/open-images-dataset\n\n&gt; aws s3 --no-sign-request sync s3://open-images-dataset/train train\n&gt; \n&gt; download failed: s3://open-images-dataset/train/711bc12d088711f9.jpg\n&gt; to train\\711bc12d088711f9.jpg [SSL:\n&gt; DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record\n&gt; mac (_ssl.c:2178)\n\nThis happens sporadically when I run the \"aws s3 sync\" command on the \"train\" folder. I don't monitor the command line output, but sometimes when I view the output, I see these error lines.\n\nIf I try to manually download that specific file again, it will fail with the same error too:\n\n&gt; aws s3 --no-sign-request cp\n&gt; s3://open-images-dataset/train/711bc12d088711f9.jpg\n&gt; \n&gt; download failed: s3://open-images-dataset/train/711bc12d088711f9.jpg\n&gt; to .\\711bc12d088711f9.jpg [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC]\n&gt; decryption failed or bad record mac (_ssl.c:2178)\n\nAfter many tries, it will sometimes finally succeed in downloading. Is this caused by corrupted images on some of AWS's replicated data servers?\n\nI found this (old) thread on Amazon with similar issue, which seems to indicate that it's AWS's problem:\nhttps://forums.aws.amazon.com/thread.jspa?threadID=126932",
      "votes": null
    },
    {
      "id": "360798",
      "postDate": "07/23/2018 09:01:12",
      "content": "<p>Hi Mongrel,</p>\n\n<p>I cannot reproduce this issue from my location, so it's possible that it's something related to AWS's replication... Sorry about that. I'll ping CVDF so they are aware of it.</p>\n\n<p>Best,</p>",
      "rawMarkdown": "Hi Mongrel,\n\nI cannot reproduce this issue from my location, so it's possible that it's something related to AWS's replication... Sorry about that. I'll ping CVDF so they are aware of it.\n\nBest,",
      "votes": null
    },
    {
      "id": "360809",
      "postDate": "07/23/2018 09:18:03",
      "content": "<p>Thank you for your prompt assistance. If I run the copy command for that individual file many times (say 10 or more), then eventually it gets through, so I suspect it's a problem at the server end.</p>\n\n<p>Another file I encountered the same error on (but works now after multiple attempts):</p>\n\n<blockquote>\n  <p>aws s3 --no-sign-request cp s3://open-images-dataset/train/6a494d82d30962df.jpg .</p>\n</blockquote>\n\n<p>The syncing also terminates randomly with connection errors (sometimes after an hour, sometimes a few hours), so I had to create a batch file that repeatedly runs the sync command, in order for it to resume syncing after the terminations. I think some other participants reported this problem too.</p>",
      "rawMarkdown": "Thank you for your prompt assistance. If I run the copy command for that individual file many times (say 10 or more), then eventually it gets through, so I suspect it's a problem at the server end.\n\nAnother file I encountered the same error on (but works now after multiple attempts):\n&gt; aws s3 --no-sign-request cp s3://open-images-dataset/train/6a494d82d30962df.jpg .\n\nThe syncing also terminates randomly with connection errors (sometimes after an hour, sometimes a few hours), so I had to create a batch file that repeatedly runs the sync command, in order for it to resume syncing after the terminations. I think some other participants reported this problem too.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 360798,
      "author_name": "jponttusset",
      "author_url": "",
      "post_date": "07/23/2018 09:01:12",
      "content": "<p>Hi Mongrel,</p>\n\n<p>I cannot reproduce this issue from my location, so it's possible that it's something related to AWS's replication... Sorry about that. I'll ping CVDF so they are aware of it.</p>\n\n<p>Best,</p>",
      "votes": null,
      "replies": [
        {
          "id": 360809,
          "author_name": "andykoh",
          "author_url": "",
          "post_date": "07/23/2018 09:18:03",
          "content": "<p>Thank you for your prompt assistance. If I run the copy command for that individual file many times (say 10 or more), then eventually it gets through, so I suspect it's a problem at the server end.</p>\n\n<p>Another file I encountered the same error on (but works now after multiple attempts):</p>\n\n<blockquote>\n  <p>aws s3 --no-sign-request cp s3://open-images-dataset/train/6a494d82d30962df.jpg .</p>\n</blockquote>\n\n<p>The syncing also terminates randomly with connection errors (sometimes after an hour, sometimes a few hours), so I had to create a batch file that repeatedly runs the sync command, in order for it to resume syncing after the terminations. I think some other participants reported this problem too.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "360759": "I'm seeing \"Decryption Failed or Bad Record\" errors on some files when syncing Open Images from AWS, based on the instructions at:\nhttps://github.com/cvdfoundation/open-images-dataset\n\n&gt; aws s3 --no-sign-request sync s3://open-images-dataset/train train\n&gt; \n&gt; download failed: s3://open-images-dataset/train/711bc12d088711f9.jpg\n&gt; to train\\711bc12d088711f9.jpg [SSL:\n&gt; DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record\n&gt; mac (_ssl.c:2178)\n\nThis happens sporadically when I run the \"aws s3 sync\" command on the \"train\" folder. I don't monitor the command line output, but sometimes when I view the output, I see these error lines.\n\nIf I try to manually download that specific file again, it will fail with the same error too:\n\n&gt; aws s3 --no-sign-request cp\n&gt; s3://open-images-dataset/train/711bc12d088711f9.jpg\n&gt; \n&gt; download failed: s3://open-images-dataset/train/711bc12d088711f9.jpg\n&gt; to .\\711bc12d088711f9.jpg [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC]\n&gt; decryption failed or bad record mac (_ssl.c:2178)\n\nAfter many tries, it will sometimes finally succeed in downloading. Is this caused by corrupted images on some of AWS's replicated data servers?\n\nI found this (old) thread on Amazon with similar issue, which seems to indicate that it's AWS's problem:\nhttps://forums.aws.amazon.com/thread.jspa?threadID=126932",
    "360798": "Hi Mongrel,\n\nI cannot reproduce this issue from my location, so it's possible that it's something related to AWS's replication... Sorry about that. I'll ping CVDF so they are aware of it.\n\nBest,",
    "360809": "Thank you for your prompt assistance. If I run the copy command for that individual file many times (say 10 or more), then eventually it gets through, so I suspect it's a problem at the server end.\n\nAnother file I encountered the same error on (but works now after multiple attempts):\n&gt; aws s3 --no-sign-request cp s3://open-images-dataset/train/6a494d82d30962df.jpg .\n\nThe syncing also terminates randomly with connection errors (sometimes after an hour, sometimes a few hours), so I had to create a batch file that repeatedly runs the sync command, in order for it to resume syncing after the terminations. I think some other participants reported this problem too."
  },
  "source": "meta"
}