{
  "id": 317628,
  "title": "Deciding Learning Rate Scheduler",
  "url": "/competitions/happy-whale-and-dolphin/discussion/317628",
  "author_name": "",
  "post_date": "2022-04-08T05:09:02.163962900Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I have trained multiple models using efficient nets and everytime I got stuck at a loss of around 13. I used onecyclelr as my scheduler and adam as optimizer. Everything else is set to commonly used parameter values. Can anyone help me to get low loss in training?</p>",
  "messages": [
    {
      "id": "1748915",
      "postDate": "04/08/2022 05:09:02",
      "content": "<p>I have trained multiple models using efficient nets and everytime I got stuck at a loss of around 13. I used onecyclelr as my scheduler and adam as optimizer. Everything else is set to commonly used parameter values. Can anyone help me to get low loss in training?</p>",
      "rawMarkdown": "I have trained multiple models using efficient nets and everytime I got stuck at a loss of around 13. I used onecyclelr as my scheduler and adam as optimizer. Everything else is set to commonly used parameter values. Can anyone help me to get low loss in training?",
      "votes": null
    },
    {
      "id": "1749012",
      "postDate": "04/08/2022 07:10:28",
      "content": "<p>Schedulers used till now</p>\n<ul>\n<li>OneCycleLR</li>\n<li>CosineAnnealingLR</li>\n<li>ReduceLROnPlateau</li>\n</ul>\n<p>I was able to achieve training loss of 0.002 in one model with OneCycleLR (got score of 71.9 on public submission)but using the same thing using with cleaned data doesn't give me the same result and every models get stuck at a loss of 15-13.</p>",
      "rawMarkdown": "Schedulers used till now\n- OneCycleLR\n- CosineAnnealingLR\n- ReduceLROnPlateau\n\nI was able to achieve training loss of 0.002 in one model with OneCycleLR (got score of 71.9 on public submission)but using the same thing using with cleaned data doesn't give me the same result and every models get stuck at a loss of 15-13.",
      "votes": null
    },
    {
      "id": "1749039",
      "postDate": "04/08/2022 07:32:51",
      "content": "<p>I suggest you to take a look at the paper of arcface loss. And they said adding constants to fearture level is helpful. I'm still working on it.</p>",
      "rawMarkdown": "I suggest you to take a look at the paper of arcface loss. And they said adding constants to fearture level is helpful. I'm still working on it.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1749012,
      "author_name": "jainishsavalia",
      "author_url": "",
      "post_date": "04/08/2022 07:10:28",
      "content": "<p>Schedulers used till now</p>\n<ul>\n<li>OneCycleLR</li>\n<li>CosineAnnealingLR</li>\n<li>ReduceLROnPlateau</li>\n</ul>\n<p>I was able to achieve training loss of 0.002 in one model with OneCycleLR (got score of 71.9 on public submission)but using the same thing using with cleaned data doesn't give me the same result and every models get stuck at a loss of 15-13.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1749039,
      "author_name": "jimmysmith1009",
      "author_url": "",
      "post_date": "04/08/2022 07:32:51",
      "content": "<p>I suggest you to take a look at the paper of arcface loss. And they said adding constants to fearture level is helpful. I'm still working on it.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1748915": "I have trained multiple models using efficient nets and everytime I got stuck at a loss of around 13. I used onecyclelr as my scheduler and adam as optimizer. Everything else is set to commonly used parameter values. Can anyone help me to get low loss in training?",
    "1749012": "Schedulers used till now\n- OneCycleLR\n- CosineAnnealingLR\n- ReduceLROnPlateau\n\nI was able to achieve training loss of 0.002 in one model with OneCycleLR (got score of 71.9 on public submission)but using the same thing using with cleaned data doesn't give me the same result and every models get stuck at a loss of 15-13.",
    "1749039": "I suggest you to take a look at the paper of arcface loss. And they said adding constants to fearture level is helpful. I'm still working on it."
  },
  "source": "meta"
}