{
  "id": 58406,
  "title": "How to retrain the NN model on all data without underfitting or overfitting?",
  "url": "/competitions/freesound-audio-tagging/discussion/58406",
  "author_name": "",
  "post_date": "2018-06-07T13:57:41.069080700Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Now , I train NN with training set and  validation set, so I can determine if the model is overfitting by validation set. At the end, I want to retrain the model on full data, how do I know how far to train ?</p>\n\n<p>Andrew Ng mentioned in his lecture that the model we eventually submitted to the user needed to be retrained with all the data, but didn't specify how to do it.</p>\n\n<p>I have tried to use average epochs in cross validation when the model converges, it did't improve my LB score.</p>\n\n<p>Is there any way to solve this problem instead of trying it out</p>",
  "messages": [
    {
      "id": "339719",
      "postDate": "06/07/2018 13:57:41",
      "content": "<p>Now , I train NN with training set and  validation set, so I can determine if the model is overfitting by validation set. At the end, I want to retrain the model on full data, how do I know how far to train ?</p>\n\n<p>Andrew Ng mentioned in his lecture that the model we eventually submitted to the user needed to be retrained with all the data, but didn't specify how to do it.</p>\n\n<p>I have tried to use average epochs in cross validation when the model converges, it did't improve my LB score.</p>\n\n<p>Is there any way to solve this problem instead of trying it out</p>",
      "rawMarkdown": "Now , I train NN with training set and  validation set, so I can determine if the model is overfitting by validation set. At the end, I want to retrain the model on full data, how do I know how far to train ?\n\nAndrew Ng mentioned in his lecture that the model we eventually submitted to the user needed to be retrained with all the data, but didn't specify how to do it.\n\nI have tried to use average epochs in cross validation when the model converges, it did't improve my LB score.\n\nIs there any way to solve this problem instead of trying it out",
      "votes": null
    },
    {
      "id": "339739",
      "postDate": "06/07/2018 14:34:57",
      "content": "<p>If you want to retrain your NN on all data, there is no way to guarantee the perfect moment for stopping.</p>\n\n<p>But you can use regularization techniques as data augmentation, dropout, etc. to make sure that the model does not overfit to the training data and then simply train until convergence (or for a number of epochs you think is reasonable). If your regularization works well the validation error will not go up again when training too long.</p>\n\n<p>Another possibility is to train with cross-validation and combine all the models afterwards. By doing that all the training data is being used.</p>",
      "rawMarkdown": "If you want to retrain your NN on all data, there is no way to guarantee the perfect moment for stopping.\n\nBut you can use regularization techniques as data augmentation, dropout, etc. to make sure that the model does not overfit to the training data and then simply train until convergence (or for a number of epochs you think is reasonable). If your regularization works well the validation error will not go up again when training too long.\n\nAnother possibility is to train with cross-validation and combine all the models afterwards. By doing that all the training data is being used.",
      "votes": null
    },
    {
      "id": "339948",
      "postDate": "06/08/2018 02:49:11",
      "content": "<p>Thanks, it's helpful !</p>",
      "rawMarkdown": "Thanks, it's helpful !",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 339739,
      "author_name": "kevinwilkinghoff",
      "author_url": "",
      "post_date": "06/07/2018 14:34:57",
      "content": "<p>If you want to retrain your NN on all data, there is no way to guarantee the perfect moment for stopping.</p>\n\n<p>But you can use regularization techniques as data augmentation, dropout, etc. to make sure that the model does not overfit to the training data and then simply train until convergence (or for a number of epochs you think is reasonable). If your regularization works well the validation error will not go up again when training too long.</p>\n\n<p>Another possibility is to train with cross-validation and combine all the models afterwards. By doing that all the training data is being used.</p>",
      "votes": null,
      "replies": [
        {
          "id": 339948,
          "author_name": "tauriel",
          "author_url": "",
          "post_date": "06/08/2018 02:49:11",
          "content": "<p>Thanks, it's helpful !</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "339719": "Now , I train NN with training set and  validation set, so I can determine if the model is overfitting by validation set. At the end, I want to retrain the model on full data, how do I know how far to train ?\n\nAndrew Ng mentioned in his lecture that the model we eventually submitted to the user needed to be retrained with all the data, but didn't specify how to do it.\n\nI have tried to use average epochs in cross validation when the model converges, it did't improve my LB score.\n\nIs there any way to solve this problem instead of trying it out",
    "339739": "If you want to retrain your NN on all data, there is no way to guarantee the perfect moment for stopping.\n\nBut you can use regularization techniques as data augmentation, dropout, etc. to make sure that the model does not overfit to the training data and then simply train until convergence (or for a number of epochs you think is reasonable). If your regularization works well the validation error will not go up again when training too long.\n\nAnother possibility is to train with cross-validation and combine all the models afterwards. By doing that all the training data is being used.",
    "339948": "Thanks, it's helpful !"
  },
  "source": "meta"
}