{
  "id": 79203,
  "title": "Why so many kernel us batchnorm with momentum=0.5?",
  "url": "/competitions/quora-insincere-questions-classification/discussion/79203",
  "author_name": "",
  "post_date": "2019-02-01T06:58:55.670788Z",
  "votes": 3,
  "comment_count": 1,
  "views": 0,
  "content": "<p>The default momentum is 0.999, but most high-score kernel set it much smaller, which seems to make the slip-average of BN effectless? Why?</p>",
  "messages": [
    {
      "id": "464621",
      "postDate": "02/01/2019 06:58:55",
      "content": "<p>The default momentum is 0.999, but most high-score kernel set it much smaller, which seems to make the slip-average of BN effectless? Why?</p>",
      "rawMarkdown": "The default momentum is 0.999, but most high-score kernel set it much smaller, which seems to make the slip-average of BN effectless? Why?",
      "votes": null
    },
    {
      "id": "465534",
      "postDate": "02/03/2019 11:53:58",
      "content": "<p>I haven't put much thought into it, but maybe because in this competition we only train a few epochs. Having a high momentum might make sense when you plan to train for a hundred epochs or so, but since we only have 3-6 here, it may be preferable to let the estimates change rapidly.</p>",
      "rawMarkdown": "I haven't put much thought into it, but maybe because in this competition we only train a few epochs. Having a high momentum might make sense when you plan to train for a hundred epochs or so, but since we only have 3-6 here, it may be preferable to let the estimates change rapidly.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 465534,
      "author_name": "mschumacher",
      "author_url": "",
      "post_date": "02/03/2019 11:53:58",
      "content": "<p>I haven't put much thought into it, but maybe because in this competition we only train a few epochs. Having a high momentum might make sense when you plan to train for a hundred epochs or so, but since we only have 3-6 here, it may be preferable to let the estimates change rapidly.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "464621": "The default momentum is 0.999, but most high-score kernel set it much smaller, which seems to make the slip-average of BN effectless? Why?",
    "465534": "I haven't put much thought into it, but maybe because in this competition we only train a few epochs. Having a high momentum might make sense when you plan to train for a hundred epochs or so, but since we only have 3-6 here, it may be preferable to let the estimates change rapidly."
  },
  "source": "meta"
}