{
  "id": 99133,
  "title": "Preprocessing Images",
  "url": "/competitions/aptos2019-blindness-detection/discussion/99133",
  "author_name": "",
  "post_date": "2019-07-09T04:18:20.677519200Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>How do you preprocess the unseen test examples? Currently I am trying to make a copy of every image with my preprocessing and saving it to a folder in the directory the notebook is running in, but I still get a submission error (regular running and committing looks fine).</p>\n\n<p>I am shrinking the test dataset to batches of 8 and size 600x450 images... So I don't think I have no memory problem. Is there an issue with read-only access to the folders on the hidden test set?</p>\n\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "571020",
      "postDate": "07/09/2019 04:18:20",
      "content": "<p>How do you preprocess the unseen test examples? Currently I am trying to make a copy of every image with my preprocessing and saving it to a folder in the directory the notebook is running in, but I still get a submission error (regular running and committing looks fine).</p>\n\n<p>I am shrinking the test dataset to batches of 8 and size 600x450 images... So I don't think I have no memory problem. Is there an issue with read-only access to the folders on the hidden test set?</p>\n\n<p>Thanks!</p>",
      "rawMarkdown": "How do you preprocess the unseen test examples? Currently I am trying to make a copy of every image with my preprocessing and saving it to a folder in the directory the notebook is running in, but I still get a submission error (regular running and committing looks fine).\n\nI am shrinking the test dataset to batches of 8 and size 600x450 images... So I don't think I have no memory problem. Is there an issue with read-only access to the folders on the hidden test set?\n\nThanks!",
      "votes": null
    },
    {
      "id": "571127",
      "postDate": "07/09/2019 07:41:25",
      "content": "<p>You will run out of space for sure at test time(unless you clean the output directory via rm-rf maybe) .. just use them on fly?</p>",
      "rawMarkdown": "You will run out of space for sure at test time(unless you clean the output directory via rm-rf maybe) .. just use them on fly?",
      "votes": null
    },
    {
      "id": "571377",
      "postDate": "07/09/2019 14:43:47",
      "content": "<p>In my opinion, simple <code>try:... except:...</code> at your preprocessing function will do the trick. However, it just helps to avoid problem (or invalid?) images.</p>",
      "rawMarkdown": "In my opinion, simple `try:... except:...` at your preprocessing function will do the trick. However, it just helps to avoid problem (or invalid?) images.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 571127,
      "author_name": "adityaecdrid",
      "author_url": "",
      "post_date": "07/09/2019 07:41:25",
      "content": "<p>You will run out of space for sure at test time(unless you clean the output directory via rm-rf maybe) .. just use them on fly?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 571377,
      "author_name": "bohdansafoniuk",
      "author_url": "",
      "post_date": "07/09/2019 14:43:47",
      "content": "<p>In my opinion, simple <code>try:... except:...</code> at your preprocessing function will do the trick. However, it just helps to avoid problem (or invalid?) images.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "571020": "How do you preprocess the unseen test examples? Currently I am trying to make a copy of every image with my preprocessing and saving it to a folder in the directory the notebook is running in, but I still get a submission error (regular running and committing looks fine).\n\nI am shrinking the test dataset to batches of 8 and size 600x450 images... So I don't think I have no memory problem. Is there an issue with read-only access to the folders on the hidden test set?\n\nThanks!",
    "571127": "You will run out of space for sure at test time(unless you clean the output directory via rm-rf maybe) .. just use them on fly?",
    "571377": "In my opinion, simple `try:... except:...` at your preprocessing function will do the trick. However, it just helps to avoid problem (or invalid?) images."
  },
  "source": "meta"
}