{
  "id": 138318,
  "title": "What makes overfitting?",
  "url": "/competitions/deepfake-detection-challenge/discussion/138318",
  "author_name": "",
  "post_date": "2020-03-24T14:12:21.900135400Z",
  "votes": 8,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Just catching up with this competition. When doing baseline, I found (unsurprisingly😧 ) that after a certain point validation loss increases when training loss continues to decrease. Apparently this is overfitting, as <strong>the model learns features which are not generalizable</strong></p>\n\n<p>Has anyone investigated what are these un-generalizable features? Three hypotheses are:\n- The model is memorizing the background (I use image with padding)\n- The model is memorizing the actors\n- The model is memorizing the attack method.</p>\n\n<p>(3) is definitely in play honestly with 7 days left I cannot think of an effective way to deal with it yet.\nAre (1) and (2) contributing greatly to overfitting, to what degree are we certain? What are some other variables which the models confound with real / fake features?</p>",
  "messages": [
    {
      "id": "784806",
      "postDate": "03/24/2020 14:12:21",
      "content": "<p>Just catching up with this competition. When doing baseline, I found (unsurprisingly😧 ) that after a certain point validation loss increases when training loss continues to decrease. Apparently this is overfitting, as <strong>the model learns features which are not generalizable</strong></p>\n\n<p>Has anyone investigated what are these un-generalizable features? Three hypotheses are:\n- The model is memorizing the background (I use image with padding)\n- The model is memorizing the actors\n- The model is memorizing the attack method.</p>\n\n<p>(3) is definitely in play honestly with 7 days left I cannot think of an effective way to deal with it yet.\nAre (1) and (2) contributing greatly to overfitting, to what degree are we certain? What are some other variables which the models confound with real / fake features?</p>",
      "rawMarkdown": "Just catching up with this competition. When doing baseline, I found (unsurprisingly😧 ) that after a certain point validation loss increases when training loss continues to decrease. Apparently this is overfitting, as **the model learns features which are not generalizable**\n\nHas anyone investigated what are these un-generalizable features? Three hypotheses are:\n- The model is memorizing the background (I use image with padding)\n- The model is memorizing the actors\n- The model is memorizing the attack method.\n\n(3) is definitely in play honestly with 7 days left I cannot think of an effective way to deal with it yet.\nAre (1) and (2) contributing greatly to overfitting, to what degree are we certain? What are some other variables which the models confound with real / fake features?",
      "votes": null
    },
    {
      "id": "785557",
      "postDate": "03/25/2020 06:47:54",
      "content": "<p>I'm experiencing the same (using the lrcnn model shared in the public kernel), but #1 seems unlikely since I have no padding.  I'm wondering if using model pretrained on face would make #2 more likely </p>",
      "rawMarkdown": "I'm experiencing the same (using the lrcnn model shared in the public kernel), but #1 seems unlikely since I have no padding.  I'm wondering if using model pretrained on face would make #2 more likely",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 785557,
      "author_name": "nyleve",
      "author_url": "",
      "post_date": "03/25/2020 06:47:54",
      "content": "<p>I'm experiencing the same (using the lrcnn model shared in the public kernel), but #1 seems unlikely since I have no padding.  I'm wondering if using model pretrained on face would make #2 more likely </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "784806": "Just catching up with this competition. When doing baseline, I found (unsurprisingly😧 ) that after a certain point validation loss increases when training loss continues to decrease. Apparently this is overfitting, as **the model learns features which are not generalizable**\n\nHas anyone investigated what are these un-generalizable features? Three hypotheses are:\n- The model is memorizing the background (I use image with padding)\n- The model is memorizing the actors\n- The model is memorizing the attack method.\n\n(3) is definitely in play honestly with 7 days left I cannot think of an effective way to deal with it yet.\nAre (1) and (2) contributing greatly to overfitting, to what degree are we certain? What are some other variables which the models confound with real / fake features?",
    "785557": "I'm experiencing the same (using the lrcnn model shared in the public kernel), but #1 seems unlikely since I have no padding.  I'm wondering if using model pretrained on face would make #2 more likely"
  },
  "source": "meta"
}