{
  "id": 104961,
  "title": "fastai preprocessing",
  "url": "/competitions/aptos2019-blindness-detection/discussion/104961",
  "author_name": "Ken Ho",
  "post_date": "2019-08-20T09:30:11.366000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>In general, there are 2 approaches for preprocessing:\n1. Preprocessing ALL images once before training --&gt; Memory problem\n2. Preprocessing in the image generator (which can be done in keras)</p>\n\n<p>Currently, I am trying to do preprocessing using ImageList.after_open. Is this a valid way for preprocessing? I have also seen ways of using custom transform for preprocessing but this may be difficult to be applied to test set.</p>\n\n<p>Can anyone give me some suggestions? Thanks in advance!</p>",
  "messages": [
    {
      "id": 603451,
      "postDate": "2019-08-20T09:30:11.367Z",
      "content": "<p>In general, there are 2 approaches for preprocessing:\n1. Preprocessing ALL images once before training --&gt; Memory problem\n2. Preprocessing in the image generator (which can be done in keras)</p>\n\n<p>Currently, I am trying to do preprocessing using ImageList.after_open. Is this a valid way for preprocessing? I have also seen ways of using custom transform for preprocessing but this may be difficult to be applied to test set.</p>\n\n<p>Can anyone give me some suggestions? Thanks in advance!</p>",
      "rawMarkdown": "In general, there are 2 approaches for preprocessing:\n1. Preprocessing ALL images once before training --&gt; Memory problem\n2. Preprocessing in the image generator (which can be done in keras)\n\nCurrently, I am trying to do preprocessing using ImageList.after_open. Is this a valid way for preprocessing? I have also seen ways of using custom transform for preprocessing but this may be difficult to be applied to test set.\n\nCan anyone give me some suggestions? Thanks in advance!",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "603451": "In general, there are 2 approaches for preprocessing:\n1. Preprocessing ALL images once before training --&gt; Memory problem\n2. Preprocessing in the image generator (which can be done in keras)\n\nCurrently, I am trying to do preprocessing using ImageList.after_open. Is this a valid way for preprocessing? I have also seen ways of using custom transform for preprocessing but this may be difficult to be applied to test set.\n\nCan anyone give me some suggestions? Thanks in advance!"
  }
}