{
  "id": 71325,
  "title": "Learning Rate for UNet with Resnet34 Encoder",
  "url": "/competitions/airbus-ship-detection/discussion/71325",
  "author_name": "",
  "post_date": "2018-11-12T16:16:51.681128100Z",
  "votes": null,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Can someone suggest LR for Unet with resnet34 encoder and decoder.\nI have tried Lr between  0.1 to .001, with these rates my validation score is very high around 500,000. and train_score is less than 1.0 \nAny suggestions on why it is like this.\nI have tried the same on subest of data (with ships), validation score now becomes NaN.\nI didn't find any bugs as of now.</p>",
  "messages": [
    {
      "id": "419843",
      "postDate": "11/12/2018 16:16:51",
      "content": "<p>Can someone suggest LR for Unet with resnet34 encoder and decoder.\nI have tried Lr between  0.1 to .001, with these rates my validation score is very high around 500,000. and train_score is less than 1.0 \nAny suggestions on why it is like this.\nI have tried the same on subest of data (with ships), validation score now becomes NaN.\nI didn't find any bugs as of now.</p>",
      "rawMarkdown": "Can someone suggest LR for Unet with resnet34 encoder and decoder.\nI have tried Lr between  0.1 to .001, with these rates my validation score is very high around 500,000. and train_score is less than 1.0 \nAny suggestions on why it is like this.\nI have tried the same on subest of data (with ships), validation score now becomes NaN.\nI didn't find any bugs as of now.",
      "votes": null
    },
    {
      "id": "419879",
      "postDate": "11/12/2018 17:32:16",
      "content": "<p>Unless you are freezing layers, i wouldn't go much more (if any) above 1e-4 for LR. I guess it depends on your optimizer, but I'm using Adam.</p>",
      "rawMarkdown": "Unless you are freezing layers, i wouldn't go much more (if any) above 1e-4 for LR. I guess it depends on your optimizer, but I'm using Adam.",
      "votes": null
    },
    {
      "id": "420027",
      "postDate": "11/13/2018 00:07:33",
      "content": "<p>sgd &lt;1e-2 or adam &lt; 1e-4</p>",
      "rawMarkdown": "sgd &lt;1e-2 or adam &lt; 1e-4",
      "votes": null
    },
    {
      "id": "420202",
      "postDate": "11/13/2018 08:59:56",
      "content": "<p>Please check your loss function and metrics function because there could be a bug\nLoss function needs to be <strong>continuous</strong> - check standard implementations of Focal Loss or Dice loss.</p>\n\n<p>... I did the same mistake in previous competitions.</p>",
      "rawMarkdown": "Please check your loss function and metrics function because there could be a bug\nLoss function needs to be **continuous** - check standard implementations of Focal Loss or Dice loss.\n\n ... I did the same mistake in previous competitions.",
      "votes": null
    },
    {
      "id": "420273",
      "postDate": "11/13/2018 11:30:36",
      "content": "<p>1e-4 (min, not max) adam unfrozen</p>",
      "rawMarkdown": "1e-4 (min, not max) adam unfrozen",
      "votes": null
    },
    {
      "id": "420326",
      "postDate": "11/13/2018 13:07:06",
      "content": "<p>thanks for the response everyone,\nI used SGD with 1e-2, worked fine, had no idea why I was getting huge validation loss previously. \nwill update if i find the cause.</p>",
      "rawMarkdown": "thanks for the response everyone,\nI used SGD with 1e-2, worked fine, had no idea why I was getting huge validation loss previously. \nwill update if i find the cause.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 419879,
      "author_name": "oewyn000",
      "author_url": "",
      "post_date": "11/12/2018 17:32:16",
      "content": "<p>Unless you are freezing layers, i wouldn't go much more (if any) above 1e-4 for LR. I guess it depends on your optimizer, but I'm using Adam.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 420027,
      "author_name": "yaroshevskiy",
      "author_url": "",
      "post_date": "11/13/2018 00:07:33",
      "content": "<p>sgd &lt;1e-2 or adam &lt; 1e-4</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 420202,
      "author_name": "remidi",
      "author_url": "",
      "post_date": "11/13/2018 08:59:56",
      "content": "<p>Please check your loss function and metrics function because there could be a bug\nLoss function needs to be <strong>continuous</strong> - check standard implementations of Focal Loss or Dice loss.</p>\n\n<p>... I did the same mistake in previous competitions.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 420273,
      "author_name": "robga",
      "author_url": "",
      "post_date": "11/13/2018 11:30:36",
      "content": "<p>1e-4 (min, not max) adam unfrozen</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 420326,
      "author_name": "ravichbits",
      "author_url": "",
      "post_date": "11/13/2018 13:07:06",
      "content": "<p>thanks for the response everyone,\nI used SGD with 1e-2, worked fine, had no idea why I was getting huge validation loss previously. \nwill update if i find the cause.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "419843": "Can someone suggest LR for Unet with resnet34 encoder and decoder.\nI have tried Lr between  0.1 to .001, with these rates my validation score is very high around 500,000. and train_score is less than 1.0 \nAny suggestions on why it is like this.\nI have tried the same on subest of data (with ships), validation score now becomes NaN.\nI didn't find any bugs as of now.",
    "419879": "Unless you are freezing layers, i wouldn't go much more (if any) above 1e-4 for LR. I guess it depends on your optimizer, but I'm using Adam.",
    "420027": "sgd &lt;1e-2 or adam &lt; 1e-4",
    "420202": "Please check your loss function and metrics function because there could be a bug\nLoss function needs to be **continuous** - check standard implementations of Focal Loss or Dice loss.\n\n ... I did the same mistake in previous competitions.",
    "420273": "1e-4 (min, not max) adam unfrozen",
    "420326": "thanks for the response everyone,\nI used SGD with 1e-2, worked fine, had no idea why I was getting huge validation loss previously. \nwill update if i find the cause."
  },
  "source": "meta"
}