{
  "id": 131877,
  "title": "Loss is not decreasing ",
  "url": "/competitions/deepfake-detection-challenge/discussion/131877",
  "author_name": "",
  "post_date": "2020-02-22T10:08:42.676974500Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>We have tried various models in the past few weeks but our loss gets stuck. We have tried tuning the learning rate, batch size and so on but to no avail. Any ideas?</p>",
  "messages": [
    {
      "id": "753523",
      "postDate": "02/22/2020 10:08:42",
      "content": "<p>We have tried various models in the past few weeks but our loss gets stuck. We have tried tuning the learning rate, batch size and so on but to no avail. Any ideas?</p>",
      "rawMarkdown": "We have tried various models in the past few weeks but our loss gets stuck. We have tried tuning the learning rate, batch size and so on but to no avail. Any ideas?",
      "votes": null
    },
    {
      "id": "753548",
      "postDate": "02/22/2020 11:00:20",
      "content": "<p>What helped me the most was 1) adding more data for training (when I only had 10% of all the chunks I overfitted severely) 2) working out a way to balance originals vs fakes during training. Afterwards you can try several models to see what fits best.</p>",
      "rawMarkdown": "What helped me the most was 1) adding more data for training (when I only had 10% of all the chunks I overfitted severely) 2) working out a way to balance originals vs fakes during training. Afterwards you can try several models to see what fits best.",
      "votes": null
    },
    {
      "id": "753879",
      "postDate": "02/22/2020 19:10:45",
      "content": "<p>The difference of reals and fakes are often subtle and not every system can pick up the signal within the noise.</p>\n\n<p>For me, this happened in two cases: the preprocessing was not good enough and the model being actually too complex ...and the problem you are going into next will be fighting against overfitting ;)</p>",
      "rawMarkdown": "The difference of reals and fakes are often subtle and not every system can pick up the signal within the noise.\n\nFor me, this happened in two cases: the preprocessing was not good enough and the model being actually too complex ...and the problem you are going into next will be fighting against overfitting ;)",
      "votes": null
    },
    {
      "id": "753974",
      "postDate": "02/22/2020 22:00:02",
      "content": "<p>There must be something wrong with your training or submission code, cause your score is similar to what you'd get with a random guesses. Any of the models I have tried so far get a better score only after 2 or 3 epochs of training. What are your training and validation losses?</p>",
      "rawMarkdown": "There must be something wrong with your training or submission code, cause your score is similar to what you'd get with a random guesses. Any of the models I have tried so far get a better score only after 2 or 3 epochs of training. What are your training and validation losses?",
      "votes": null
    },
    {
      "id": "754621",
      "postDate": "02/23/2020 20:55:08",
      "content": "<p>Try to focus on data processing (that's what I am working on now). Some things to try as suggested by others already:</p>\n\n<ul>\n<li>Try to have a balanced dataset during training </li>\n<li>Start with simpler models since this dataset is prone to overfitting</li>\n<li>Investigate the wrong predictions you are making and try to understand the reason</li>\n<li>Check your code for any obvious bugs</li>\n</ul>\n\n<p>Best of luck! =)</p>",
      "rawMarkdown": "Try to focus on data processing (that's what I am working on now). Some things to try as suggested by others already:\n\n- Try to have a balanced dataset during training \n- Start with simpler models since this dataset is prone to overfitting\n- Investigate the wrong predictions you are making and try to understand the reason\n- Check your code for any obvious bugs\n\nBest of luck! =)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 753548,
      "author_name": "rafiko1",
      "author_url": "",
      "post_date": "02/22/2020 11:00:20",
      "content": "<p>What helped me the most was 1) adding more data for training (when I only had 10% of all the chunks I overfitted severely) 2) working out a way to balance originals vs fakes during training. Afterwards you can try several models to see what fits best.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 753879,
      "author_name": "dagnelies",
      "author_url": "",
      "post_date": "02/22/2020 19:10:45",
      "content": "<p>The difference of reals and fakes are often subtle and not every system can pick up the signal within the noise.</p>\n\n<p>For me, this happened in two cases: the preprocessing was not good enough and the model being actually too complex ...and the problem you are going into next will be fighting against overfitting ;)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 753974,
      "author_name": "ngcferreira",
      "author_url": "",
      "post_date": "02/22/2020 22:00:02",
      "content": "<p>There must be something wrong with your training or submission code, cause your score is similar to what you'd get with a random guesses. Any of the models I have tried so far get a better score only after 2 or 3 epochs of training. What are your training and validation losses?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 754621,
      "author_name": "yassinealouini",
      "author_url": "",
      "post_date": "02/23/2020 20:55:08",
      "content": "<p>Try to focus on data processing (that's what I am working on now). Some things to try as suggested by others already:</p>\n\n<ul>\n<li>Try to have a balanced dataset during training </li>\n<li>Start with simpler models since this dataset is prone to overfitting</li>\n<li>Investigate the wrong predictions you are making and try to understand the reason</li>\n<li>Check your code for any obvious bugs</li>\n</ul>\n\n<p>Best of luck! =)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "753523": "We have tried various models in the past few weeks but our loss gets stuck. We have tried tuning the learning rate, batch size and so on but to no avail. Any ideas?",
    "753548": "What helped me the most was 1) adding more data for training (when I only had 10% of all the chunks I overfitted severely) 2) working out a way to balance originals vs fakes during training. Afterwards you can try several models to see what fits best.",
    "753879": "The difference of reals and fakes are often subtle and not every system can pick up the signal within the noise.\n\nFor me, this happened in two cases: the preprocessing was not good enough and the model being actually too complex ...and the problem you are going into next will be fighting against overfitting ;)",
    "753974": "There must be something wrong with your training or submission code, cause your score is similar to what you'd get with a random guesses. Any of the models I have tried so far get a better score only after 2 or 3 epochs of training. What are your training and validation losses?",
    "754621": "Try to focus on data processing (that's what I am working on now). Some things to try as suggested by others already:\n\n- Try to have a balanced dataset during training \n- Start with simpler models since this dataset is prone to overfitting\n- Investigate the wrong predictions you are making and try to understand the reason\n- Check your code for any obvious bugs\n\nBest of luck! =)"
  },
  "source": "meta"
}