{
  "id": 137109,
  "title": "[Solved] is training loss goes to ~ 0 (1e-12) quickly ?",
  "url": "/competitions/deepfake-detection-challenge/discussion/137109",
  "author_name": "SeshuRaju 🧘‍♂️",
  "post_date": "2020-03-19T06:58:12.714000",
  "votes": 5,
  "comment_count": 12,
  "views": 0,
  "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761268%2Fada3062628ae3efd26a92f6be0039526%2FScreenshot%202020-03-19%20at%2012.27.46%20PM.png?generation=1584601086298610&amp;alt=media\" alt=\"\"></p>\n\n<p>Training with <a href=\"https://www.kaggle.com/unkownhihi/dfdc-lrcn-training\">https://www.kaggle.com/unkownhihi/dfdc-lrcn-training</a>\n<a href=\"/unkownhihi\">@unkownhihi</a> is it normal or is my train/test split issue ? can i trust training loss ?</p>",
  "messages": [
    {
      "id": 779278,
      "postDate": "2020-03-19T06:58:12.713Z",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761268%2Fada3062628ae3efd26a92f6be0039526%2FScreenshot%202020-03-19%20at%2012.27.46%20PM.png?generation=1584601086298610&amp;alt=media\" alt=\"\"></p>\n\n<p>Training with <a href=\"https://www.kaggle.com/unkownhihi/dfdc-lrcn-training\">https://www.kaggle.com/unkownhihi/dfdc-lrcn-training</a>\n<a href=\"/unkownhihi\">@unkownhihi</a> is it normal or is my train/test split issue ? can i trust training loss ?</p>",
      "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761268%2Fada3062628ae3efd26a92f6be0039526%2FScreenshot%202020-03-19%20at%2012.27.46%20PM.png?generation=1584601086298610&amp;alt=media)\n\n\nTraining with https://www.kaggle.com/unkownhihi/dfdc-lrcn-training\n@unkownhihi is it normal or is my train/test split issue ? can i trust training loss ?\n",
      "votes": 5
    },
    {
      "id": 779750,
      "postDate": "2020-03-19T16:42:01.357Z",
      "content": "<p><a href=\"/unkownhihi\">@unkownhihi</a> You motivated me to do best in this competition thanks. 👍 </p>",
      "rawMarkdown": "@unkownhihi You motivated me to do best in this competition thanks. 👍 ",
      "votes": 1
    },
    {
      "id": 779647,
      "postDate": "2020-03-19T15:05:40.553Z",
      "content": "<p>I never meet this before. When it started training, it will bounce around 0.6-0.7, but never went to this low before.</p>",
      "rawMarkdown": "I never meet this before. When it started training, it will bounce around 0.6-0.7, but never went to this low before.",
      "votes": 1,
      "replies": [
        {
          "id": 779664,
          "postDate": "2020-03-19T15:23:16.703Z",
          "content": "<p>Yes after i fix my input data, it reach between 0.4 to 0.5 Thanks <a href=\"/unkownhihi\">@unkownhihi</a> But i got val_loss 0.0063, i need to improve this.</p>\n\n<p>when i try split based on folders, training going closer to zero again :(. is it means training samples are more similar ?</p>",
          "rawMarkdown": "Yes after i fix my input data, it reach between 0.4 to 0.5 Thanks @unkownhihi But i got val_loss 0.0063, i need to improve this.\n\nwhen i try split based on folders, training going closer to zero again :(. is it means training samples are more similar ?"
        },
        {
          "id": 779711,
          "postDate": "2020-03-19T16:07:11.247Z",
          "content": "<p><a href=\"/seshurajup\">@seshurajup</a> did you use my exact same model, or you used your custom model?</p>",
          "rawMarkdown": "@seshurajup did you use my exact same model, or you used your custom model?"
        },
        {
          "id": 779716,
          "postDate": "2020-03-19T16:10:52.447Z",
          "content": "<p><a href=\"/unkownhihi\">@unkownhihi</a>  same model with 0 to 39 as training and 40 to 49 as validation</p>",
          "rawMarkdown": "@unkownhihi  same model with 0 to 39 as training and 40 to 49 as validation"
        },
        {
          "id": 779728,
          "postDate": "2020-03-19T16:17:48.747Z",
          "content": "<p>well, first of all, I do not recommend copying the exact same model. Because 0.38 is its limit.\nSecond, it shouldn't happen. Check your code, data, etc. There's a slight possibility that your model is actually performing soooooo good, that it can get to ~0 loss.</p>",
          "rawMarkdown": "well, first of all, I do not recommend copying the exact same model. Because 0.38 is its limit.\nSecond, it shouldn't happen. Check your code, data, etc. There's a slight possibility that your model is actually performing soooooo good, that it can get to ~0 loss."
        },
        {
          "id": 779730,
          "postDate": "2020-03-19T16:19:22.467Z",
          "content": "<p>Thanks, i will cross check the issue.</p>",
          "rawMarkdown": "Thanks, i will cross check the issue.",
          "votes": 1
        }
      ]
    },
    {
      "id": 779308,
      "postDate": "2020-03-19T07:39:14.310Z",
      "content": "<p>Loss and accuracy are too random at starting when we train the model. don't be panic, it will be stable after 65% of train samples.</p>\n\n<p>I stopped multiple times to test is my train/split have issue. Thanks, maybe it will help others not to confuse with loss and accuracy</p>",
      "rawMarkdown": "Loss and accuracy are too random at starting when we train the model. don't be panic, it will be stable after 65% of train samples.\n\nI stopped multiple times to test is my train/split have issue. Thanks, maybe it will help others not to confuse with loss and accuracy",
      "votes": 2
    },
    {
      "id": 789260,
      "postDate": "2020-03-28T14:06:05.947Z",
      "content": "<p>Had very low training loss as well. Found a bug and fixed it! \nNow, I still have low validation loss, will check my code more and maybe add data augmentation. \nThanks for this reassuring thread. Hopefully, I find and correct the bug before the end of the competition. ;)</p>",
      "rawMarkdown": "Had very low training loss as well. Found a bug and fixed it! \nNow, I still have low validation loss, will check my code more and maybe add data augmentation. \nThanks for this reassuring thread. Hopefully, I find and correct the bug before the end of the competition. ;)",
      "replies": [
        {
          "id": 790557,
          "postDate": "2020-03-29T17:04:00.593Z",
          "content": "<p>Same issue with u. I'm trying to change my data creation now.</p>",
          "rawMarkdown": "Same issue with u. I'm trying to change my data creation now.",
          "votes": 1
        },
        {
          "id": 790563,
          "postDate": "2020-03-29T17:11:22.263Z",
          "content": "<p>Good luck to you! I am trying to resample the data to avoid overfitting. I have few ideas to try but time is running low. :p </p>",
          "rawMarkdown": "Good luck to you! I am trying to resample the data to avoid overfitting. I have few ideas to try but time is running low. :p "
        }
      ]
    },
    {
      "id": 789707,
      "postDate": "2020-03-28T23:08:16.800Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 779750,
      "author_name": "SeshuRaju 🧘‍♂️",
      "author_url": "",
      "post_date": "2020-03-19T16:42:01.357000",
      "content": "<p><a href=\"/unkownhihi\">@unkownhihi</a> You motivated me to do best in this competition thanks. 👍 </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 779647,
      "author_name": "Shangqiu Li",
      "author_url": "",
      "post_date": "2020-03-19T15:05:40.553000",
      "content": "<p>I never meet this before. When it started training, it will bounce around 0.6-0.7, but never went to this low before.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 779664,
          "author_name": "SeshuRaju 🧘‍♂️",
          "author_url": "",
          "post_date": "2020-03-19T15:23:16.703000",
          "content": "<p>Yes after i fix my input data, it reach between 0.4 to 0.5 Thanks <a href=\"/unkownhihi\">@unkownhihi</a> But i got val_loss 0.0063, i need to improve this.</p>\n\n<p>when i try split based on folders, training going closer to zero again :(. is it means training samples are more similar ?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 779711,
          "author_name": "Shangqiu Li",
          "author_url": "",
          "post_date": "2020-03-19T16:07:11.247000",
          "content": "<p><a href=\"/seshurajup\">@seshurajup</a> did you use my exact same model, or you used your custom model?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 779716,
          "author_name": "SeshuRaju 🧘‍♂️",
          "author_url": "",
          "post_date": "2020-03-19T16:10:52.447000",
          "content": "<p><a href=\"/unkownhihi\">@unkownhihi</a>  same model with 0 to 39 as training and 40 to 49 as validation</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 779728,
          "author_name": "Shangqiu Li",
          "author_url": "",
          "post_date": "2020-03-19T16:17:48.747000",
          "content": "<p>well, first of all, I do not recommend copying the exact same model. Because 0.38 is its limit.\nSecond, it shouldn't happen. Check your code, data, etc. There's a slight possibility that your model is actually performing soooooo good, that it can get to ~0 loss.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 779730,
          "author_name": "SeshuRaju 🧘‍♂️",
          "author_url": "",
          "post_date": "2020-03-19T16:19:22.467000",
          "content": "<p>Thanks, i will cross check the issue.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 779308,
      "author_name": "SeshuRaju 🧘‍♂️",
      "author_url": "",
      "post_date": "2020-03-19T07:39:14.310000",
      "content": "<p>Loss and accuracy are too random at starting when we train the model. don't be panic, it will be stable after 65% of train samples.</p>\n\n<p>I stopped multiple times to test is my train/split have issue. Thanks, maybe it will help others not to confuse with loss and accuracy</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 789260,
      "author_name": "Yassine Alouini",
      "author_url": "",
      "post_date": "2020-03-28T14:06:05.947000",
      "content": "<p>Had very low training loss as well. Found a bug and fixed it! \nNow, I still have low validation loss, will check my code more and maybe add data augmentation. \nThanks for this reassuring thread. Hopefully, I find and correct the bug before the end of the competition. ;)</p>",
      "votes": 0,
      "replies": [
        {
          "id": 790557,
          "author_name": "Liang Su",
          "author_url": "",
          "post_date": "2020-03-29T17:04:00.593000",
          "content": "<p>Same issue with u. I'm trying to change my data creation now.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 790563,
          "author_name": "Yassine Alouini",
          "author_url": "",
          "post_date": "2020-03-29T17:11:22.263000",
          "content": "<p>Good luck to you! I am trying to resample the data to avoid overfitting. I have few ideas to try but time is running low. :p </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 789707,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-03-28T23:08:16.800000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "779278": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F761268%2Fada3062628ae3efd26a92f6be0039526%2FScreenshot%202020-03-19%20at%2012.27.46%20PM.png?generation=1584601086298610&amp;alt=media)\n\n\nTraining with https://www.kaggle.com/unkownhihi/dfdc-lrcn-training\n@unkownhihi is it normal or is my train/test split issue ? can i trust training loss ?\n",
    "779750": "@unkownhihi You motivated me to do best in this competition thanks. 👍 ",
    "779647": "I never meet this before. When it started training, it will bounce around 0.6-0.7, but never went to this low before.",
    "779308": "Loss and accuracy are too random at starting when we train the model. don't be panic, it will be stable after 65% of train samples.\n\nI stopped multiple times to test is my train/split have issue. Thanks, maybe it will help others not to confuse with loss and accuracy",
    "789260": "Had very low training loss as well. Found a bug and fixed it! \nNow, I still have low validation loss, will check my code more and maybe add data augmentation. \nThanks for this reassuring thread. Hopefully, I find and correct the bug before the end of the competition. ;)",
    "789707": ""
  }
}