{
  "id": 132748,
  "title": "Jumping val loss",
  "url": "/competitions/deepfake-detection-challenge/discussion/132748",
  "author_name": "gody7334",
  "post_date": "2020-02-27T15:44:51.948000",
  "votes": 0,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi kagglers,\nJust find this paper's experiment data, \n<a href=\"https://www.dessa.com/post/deepfake-detection-that-actually-works\">Towards Deepfake Detection That Actually Works</a>\nseems like they also have jumping val loss issue.\nIt also match to my experiment result.</p>\n\n<p>I am curious why val loss will jump around but accuracy/roc_auc can stay high and stable?\nUsually is another way around, as we directly optimize train loss. Val loss should be closer reflect to train loss, not accuracy.</p>\n\n<p>Thanks,</p>",
  "messages": [
    {
      "id": 758345,
      "postDate": "2020-02-27T16:51:31.983Z",
      "content": "<p>I see, thanks <a href=\"/jpandas\">@jpandas</a> \nThis competition uses cross entropy as evaluation.</p>\n\n<p>Does it mean the val will getting worse when we train long anyway?\nSo we must do some trick like early stop, or modify prediction probability to avoid it explode?</p>\n\n<p>Thanks</p>",
      "rawMarkdown": "I see, thanks @jpandas \nThis competition uses cross entropy as evaluation.\n\nDoes it mean the val will getting worse when we train long anyway?\nSo we must do some trick like early stop, or modify prediction probability to avoid it explode?\n\nThanks",
      "replies": [
        {
          "id": 758396,
          "postDate": "2020-02-27T17:46:55.550Z",
          "rawMarkdown": "",
          "votes": 1,
          "isDeleted": true
        }
      ]
    },
    {
      "id": 758268,
      "postDate": "2020-02-27T15:44:51.947Z",
      "content": "<p>Hi kagglers,\nJust find this paper's experiment data, \n<a href=\"https://www.dessa.com/post/deepfake-detection-that-actually-works\">Towards Deepfake Detection That Actually Works</a>\nseems like they also have jumping val loss issue.\nIt also match to my experiment result.</p>\n\n<p>I am curious why val loss will jump around but accuracy/roc_auc can stay high and stable?\nUsually is another way around, as we directly optimize train loss. Val loss should be closer reflect to train loss, not accuracy.</p>\n\n<p>Thanks,</p>",
      "rawMarkdown": "Hi kagglers,\nJust find this paper's experiment data, \n[Towards Deepfake Detection That Actually Works](https://www.dessa.com/post/deepfake-detection-that-actually-works)\nseems like they also have jumping val loss issue.\nIt also match to my experiment result.\n\nI am curious why val loss will jump around but accuracy/roc_auc can stay high and stable?\nUsually is another way around, as we directly optimize train loss. Val loss should be closer reflect to train loss, not accuracy.\n\nThanks,\n\n"
    },
    {
      "id": 758325,
      "postDate": "2020-02-27T16:33:23.127Z",
      "rawMarkdown": "",
      "votes": 1,
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 758345,
      "author_name": "gody7334",
      "author_url": "",
      "post_date": "2020-02-27T16:51:31.983000",
      "content": "<p>I see, thanks <a href=\"/jpandas\">@jpandas</a> \nThis competition uses cross entropy as evaluation.</p>\n\n<p>Does it mean the val will getting worse when we train long anyway?\nSo we must do some trick like early stop, or modify prediction probability to avoid it explode?</p>\n\n<p>Thanks</p>",
      "votes": 0,
      "replies": [
        {
          "id": 758396,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-02-27T17:46:55.550000",
          "content": "",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 758325,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-02-27T16:33:23.127000",
      "content": "",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "758345": "I see, thanks @jpandas \nThis competition uses cross entropy as evaluation.\n\nDoes it mean the val will getting worse when we train long anyway?\nSo we must do some trick like early stop, or modify prediction probability to avoid it explode?\n\nThanks",
    "758268": "Hi kagglers,\nJust find this paper's experiment data, \n[Towards Deepfake Detection That Actually Works](https://www.dessa.com/post/deepfake-detection-that-actually-works)\nseems like they also have jumping val loss issue.\nIt also match to my experiment result.\n\nI am curious why val loss will jump around but accuracy/roc_auc can stay high and stable?\nUsually is another way around, as we directly optimize train loss. Val loss should be closer reflect to train loss, not accuracy.\n\nThanks,\n\n",
    "758325": ""
  }
}