{
  "id": 20459,
  "title": "(Hyper) Parameters tuning",
  "url": "/competitions/state-farm-distracted-driver-detection/discussion/20459",
  "author_name": "",
  "post_date": "2016-04-26T19:23:23.027Z",
  "votes": null,
  "comment_count": 4,
  "views": 861,
  "content": "<p>Hi, I have been breaking my head over these things so I thought I might as well as the people with experience. I am trying to train a CNN, starting off from the script shared by ZFTurbo:</p>\n\n<p>1)What should be the size of the images i.e how much should I downsample by? 24 x 32 or 48 x 64?</p>\n\n<p>2) What optimizer is preferred? I have been using SGD with lr=0.1</p>\n\n<p>3)How many layers in the CNN would be 'optimal'?  </p>\n\n<p>At present, I am getting a LB score of 1.5</p>",
  "messages": [
    {
      "id": "116975",
      "postDate": "04/26/2016 19:23:23",
      "content": "<p>Hi, I have been breaking my head over these things so I thought I might as well as the people with experience. I am trying to train a CNN, starting off from the script shared by ZFTurbo:</p>\n\n<p>1)What should be the size of the images i.e how much should I downsample by? 24 x 32 or 48 x 64?</p>\n\n<p>2) What optimizer is preferred? I have been using SGD with lr=0.1</p>\n\n<p>3)How many layers in the CNN would be 'optimal'?  </p>\n\n<p>At present, I am getting a LB score of 1.5</p>",
      "rawMarkdown": "Hi, I have been breaking my head over these things so I thought I might as well as the people with experience. I am trying to train a CNN, starting off from the script shared by ZFTurbo:\r\n\r\n1)What should be the size of the images i.e how much should I downsample by? 24 x 32 or 48 x 64?\r\n\r\n2) What optimizer is preferred? I have been using SGD with lr=0.1\r\n\r\n3)How many layers in the CNN would be 'optimal'?  \r\n\r\nAt present, I am getting a LB score of 1.5",
      "votes": null
    },
    {
      "id": "117279",
      "postDate": "04/28/2016 10:38:46",
      "content": "<p>From my experience, I used 96x96 image with Adam and lr 1e-4, plus resent-34 trained from scratch, it achieve 1.08 LB</p>",
      "rawMarkdown": "From my experience, I used 96x96 image with Adam and lr 1e-4, plus resent-34 trained from scratch, it achieve 1.08 LB",
      "votes": null
    },
    {
      "id": "117468",
      "postDate": "04/29/2016 08:14:57",
      "content": "<p>Can you tell me in why did you choose Adam, and not some other optimizer like SGD?</p>",
      "rawMarkdown": "Can you tell me in why did you choose Adam, and not some other optimizer like SGD?",
      "votes": null
    },
    {
      "id": "117478",
      "postDate": "04/29/2016 09:18:59",
      "content": "<p>sgd usually converge slower , Adam generally works better in multiple task that I recently came across, you can check the math behind Adam. Just try both of them and compare, in the deep learning world, you need to practice </p>",
      "rawMarkdown": "sgd usually converge slower , Adam generally works better in multiple task that I recently came across, you can check the math behind Adam. Just try both of them and compare, in the deep learning world, you need to practice",
      "votes": null
    },
    {
      "id": "117941",
      "postDate": "05/01/2016 21:47:56",
      "content": "<p>[quote=SecondPlan;117279]</p>\n\n<p>From my experience, I used 96x96 image with Adam and lr 1e-4, plus resent-34 trained from scratch, it achieve 1.08 LB</p>\n\n<p>[/quote]\n96 x 96 image size is giving mean memory error. Also, why are you reshaping is to a square? Aren't you losing information by changing the aspect ratio?</p>",
      "rawMarkdown": "[quote=SecondPlan;117279]\r\n\r\nFrom my experience, I used 96x96 image with Adam and lr 1e-4, plus resent-34 trained from scratch, it achieve 1.08 LB\r\n\r\n[/quote]\r\n96 x 96 image size is giving mean memory error. Also, why are you reshaping is to a square? Aren't you losing information by changing the aspect ratio?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 117279,
      "author_name": "usixuz",
      "author_url": "",
      "post_date": "04/28/2016 10:38:46",
      "content": "<p>From my experience, I used 96x96 image with Adam and lr 1e-4, plus resent-34 trained from scratch, it achieve 1.08 LB</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117468,
      "author_name": "abhijayvuyyuru",
      "author_url": "",
      "post_date": "04/29/2016 08:14:57",
      "content": "<p>Can you tell me in why did you choose Adam, and not some other optimizer like SGD?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117478,
      "author_name": "usixuz",
      "author_url": "",
      "post_date": "04/29/2016 09:18:59",
      "content": "<p>sgd usually converge slower , Adam generally works better in multiple task that I recently came across, you can check the math behind Adam. Just try both of them and compare, in the deep learning world, you need to practice </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 117941,
      "author_name": "abhijayvuyyuru",
      "author_url": "",
      "post_date": "05/01/2016 21:47:56",
      "content": "<p>[quote=SecondPlan;117279]</p>\n\n<p>From my experience, I used 96x96 image with Adam and lr 1e-4, plus resent-34 trained from scratch, it achieve 1.08 LB</p>\n\n<p>[/quote]\n96 x 96 image size is giving mean memory error. Also, why are you reshaping is to a square? Aren't you losing information by changing the aspect ratio?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "116975": "Hi, I have been breaking my head over these things so I thought I might as well as the people with experience. I am trying to train a CNN, starting off from the script shared by ZFTurbo:\r\n\r\n1)What should be the size of the images i.e how much should I downsample by? 24 x 32 or 48 x 64?\r\n\r\n2) What optimizer is preferred? I have been using SGD with lr=0.1\r\n\r\n3)How many layers in the CNN would be 'optimal'?  \r\n\r\nAt present, I am getting a LB score of 1.5",
    "117279": "From my experience, I used 96x96 image with Adam and lr 1e-4, plus resent-34 trained from scratch, it achieve 1.08 LB",
    "117468": "Can you tell me in why did you choose Adam, and not some other optimizer like SGD?",
    "117478": "sgd usually converge slower , Adam generally works better in multiple task that I recently came across, you can check the math behind Adam. Just try both of them and compare, in the deep learning world, you need to practice",
    "117941": "[quote=SecondPlan;117279]\r\n\r\nFrom my experience, I used 96x96 image with Adam and lr 1e-4, plus resent-34 trained from scratch, it achieve 1.08 LB\r\n\r\n[/quote]\r\n96 x 96 image size is giving mean memory error. Also, why are you reshaping is to a square? Aren't you losing information by changing the aspect ratio?"
  },
  "source": "meta"
}