{
  "id": 118342,
  "title": "Optimizer choise",
  "url": "/competitions/understanding_cloud_organization/discussion/118342",
  "author_name": "",
  "post_date": "2019-11-20T21:14:52.660658500Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hello everybody,</p>\n\n<p>What optimizer did you use for this competition and why ?\nDo you always use a specific one in segmentation applications, do you chose it by a certain criteria of what kind of data do you have or other application  custom nature (what criteria ?), or try multiple and see what it works best ?</p>",
  "messages": [
    {
      "id": "677989",
      "postDate": "11/20/2019 21:14:52",
      "content": "<p>Hello everybody,</p>\n\n<p>What optimizer did you use for this competition and why ?\nDo you always use a specific one in segmentation applications, do you chose it by a certain criteria of what kind of data do you have or other application  custom nature (what criteria ?), or try multiple and see what it works best ?</p>",
      "rawMarkdown": "Hello everybody,\n\nWhat optimizer did you use for this competition and why ?\nDo you always use a specific one in segmentation applications, do you chose it by a certain criteria of what kind of data do you have or other application  custom nature (what criteria ?), or try multiple and see what it works best ?",
      "votes": null
    },
    {
      "id": "678021",
      "postDate": "11/20/2019 22:36:20",
      "content": "<p>At first, we tried a bunch of different optimizers: SGD, Adam, RAdam, RALAMB and Over9000\nOver9000 scored 1-2% better in all experiments, so we decided to stick to it.</p>",
      "rawMarkdown": "At first, we tried a bunch of different optimizers: SGD, Adam, RAdam, RALAMB and Over9000\nOver9000 scored 1-2% better in all experiments, so we decided to stick to it.",
      "votes": null
    },
    {
      "id": "678276",
      "postDate": "11/21/2019 07:20:08",
      "content": "<p>1-2% is a important number. Congratulations of finding this advantage of Over9000 \nThere are little references in literature about over9000 optimizer.\nFrom what I see here (<a href=\"https://github.com/mgrankin/over9000/blob/master/over9000.py\">https://github.com/mgrankin/over9000/blob/master/over9000.py</a>) it is a combination of RAdam + LARS + LookAHead. So a Ralamb with a LookAHead </p>",
      "rawMarkdown": "1-2% is a important number. Congratulations of finding this advantage of Over9000 \nThere are little references in literature about over9000 optimizer.\nFrom what I see here (https://github.com/mgrankin/over9000/blob/master/over9000.py) it is a combination of RAdam + LARS + LookAHead. So a Ralamb with a LookAHead",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 678021,
      "author_name": "spsancti",
      "author_url": "",
      "post_date": "11/20/2019 22:36:20",
      "content": "<p>At first, we tried a bunch of different optimizers: SGD, Adam, RAdam, RALAMB and Over9000\nOver9000 scored 1-2% better in all experiments, so we decided to stick to it.</p>",
      "votes": null,
      "replies": [
        {
          "id": 678276,
          "author_name": "vladvdv",
          "author_url": "",
          "post_date": "11/21/2019 07:20:08",
          "content": "<p>1-2% is a important number. Congratulations of finding this advantage of Over9000 \nThere are little references in literature about over9000 optimizer.\nFrom what I see here (<a href=\"https://github.com/mgrankin/over9000/blob/master/over9000.py\">https://github.com/mgrankin/over9000/blob/master/over9000.py</a>) it is a combination of RAdam + LARS + LookAHead. So a Ralamb with a LookAHead </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "677989": "Hello everybody,\n\nWhat optimizer did you use for this competition and why ?\nDo you always use a specific one in segmentation applications, do you chose it by a certain criteria of what kind of data do you have or other application  custom nature (what criteria ?), or try multiple and see what it works best ?",
    "678021": "At first, we tried a bunch of different optimizers: SGD, Adam, RAdam, RALAMB and Over9000\nOver9000 scored 1-2% better in all experiments, so we decided to stick to it.",
    "678276": "1-2% is a important number. Congratulations of finding this advantage of Over9000 \nThere are little references in literature about over9000 optimizer.\nFrom what I see here (https://github.com/mgrankin/over9000/blob/master/over9000.py) it is a combination of RAdam + LARS + LookAHead. So a Ralamb with a LookAHead"
  },
  "source": "meta"
}