{
  "id": 18555,
  "title": "Overfitting and validation curves ",
  "url": "/competitions/second-annual-data-science-bowl/discussion/18555",
  "author_name": "",
  "post_date": "2016-01-25T10:37:42.867Z",
  "votes": null,
  "comment_count": 2,
  "views": 735,
  "content": "<p>Some of  my models start to overfit fairly rapidly (see attached curve). From a theoretical perspective - is it possible that a &quot;good&quot; model shows this behaviour? (and that one just has to stop early / train with few epochs)? Or does a great model always show you a slowly declining validation curve that just flattens out or only curves up slightly after many epochs? I saw the validation curves on e.g.  <a href=\"http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/\">http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/</a> and I was wondering if and why this would be the ideal situation? </p>",
  "messages": [
    {
      "id": "105622",
      "postDate": "01/25/2016 10:37:42",
      "content": "<p>Some of  my models start to overfit fairly rapidly (see attached curve). From a theoretical perspective - is it possible that a &quot;good&quot; model shows this behaviour? (and that one just has to stop early / train with few epochs)? Or does a great model always show you a slowly declining validation curve that just flattens out or only curves up slightly after many epochs? I saw the validation curves on e.g.  <a href=\"http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/\">http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/</a> and I was wondering if and why this would be the ideal situation? </p>",
      "rawMarkdown": "Some of  my models start to overfit fairly rapidly (see attached curve). From a theoretical perspective - is it possible that a \"good\" model shows this behaviour? (and that one just has to stop early / train with few epochs)? Or does a great model always show you a slowly declining validation curve that just flattens out or only curves up slightly after many epochs? I saw the validation curves on e.g.  http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/ and I was wondering if and why this would be the ideal situation?",
      "votes": null
    },
    {
      "id": "105627",
      "postDate": "01/25/2016 12:39:16",
      "content": "<p>Are you using dropout? </p>",
      "rawMarkdown": "Are you using dropout?",
      "votes": null
    },
    {
      "id": "105674",
      "postDate": "01/25/2016 19:16:23",
      "content": "<p>I agree, dropout is a good idea, one can simply add <code>net = mx.sym.Dropout(net, p=0.25)</code> to some layers in the tutorial and control overfit. It worked for me.</p>",
      "rawMarkdown": "I agree, dropout is a good idea, one can simply add `net = mx.sym.Dropout(net, p=0.25)` to some layers in the tutorial and control overfit. It worked for me.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 105627,
      "author_name": "godaibo",
      "author_url": "",
      "post_date": "01/25/2016 12:39:16",
      "content": "<p>Are you using dropout? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 105674,
      "author_name": "phunter",
      "author_url": "",
      "post_date": "01/25/2016 19:16:23",
      "content": "<p>I agree, dropout is a good idea, one can simply add <code>net = mx.sym.Dropout(net, p=0.25)</code> to some layers in the tutorial and control overfit. It worked for me.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "105622": "Some of  my models start to overfit fairly rapidly (see attached curve). From a theoretical perspective - is it possible that a \"good\" model shows this behaviour? (and that one just has to stop early / train with few epochs)? Or does a great model always show you a slowly declining validation curve that just flattens out or only curves up slightly after many epochs? I saw the validation curves on e.g.  http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/ and I was wondering if and why this would be the ideal situation?",
    "105627": "Are you using dropout?",
    "105674": "I agree, dropout is a good idea, one can simply add `net = mx.sym.Dropout(net, p=0.25)` to some layers in the tutorial and control overfit. It worked for me."
  },
  "source": "meta"
}