{
  "id": 255945,
  "title": "Smart Hyperparameter finding",
  "url": "/competitions/siim-covid19-detection/discussion/255945",
  "author_name": "",
  "post_date": "2021-07-29T23:27:15.104405400Z",
  "votes": 6,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hello Everyone</p>\n<p>For me this is the first competition that Im trying hard to get a highest place :)<br>\nAs probably all of us we are doing several other activities, based on this time optimization for testing iteration is critical.</p>\n<p>For me would be good to test several different hyperparameters, just as an example, for coarse drop out, I was not able to improve my results, I think I might need to find the correct hyperparameters, additionally other augmentations like rotation, brightness etc.</p>\n<p>In my case I made some iterations, but not sure that they were the best hyperparameter settings for all hyperparameters (learning rate, batch size, etc,etc), for example in my case I trained one model with some hyperparameters with all data (5 stratified folds and checking CV) , create inference, then running PB score, after that I repeated the cycle several times. but for one model this process could take several hours.</p>\n<p>I would like to know if you use some better optimization techniques for finding the best hyperparameters, I just can imagine in testing with less samples, and analyze this results only with the CV result, but:</p>\n<p>**Do you know some methodic approach which can really provide an smart and secure way to find the best hyperparameters in the best possible time? **</p>\n<p>Would be great if you can share some of your techniques</p>\n<p>Thank you!</p>",
  "messages": [
    {
      "id": "1404495",
      "postDate": "07/29/2021 23:27:15",
      "content": "<p>Hello Everyone</p>\n<p>For me this is the first competition that Im trying hard to get a highest place :)<br>\nAs probably all of us we are doing several other activities, based on this time optimization for testing iteration is critical.</p>\n<p>For me would be good to test several different hyperparameters, just as an example, for coarse drop out, I was not able to improve my results, I think I might need to find the correct hyperparameters, additionally other augmentations like rotation, brightness etc.</p>\n<p>In my case I made some iterations, but not sure that they were the best hyperparameter settings for all hyperparameters (learning rate, batch size, etc,etc), for example in my case I trained one model with some hyperparameters with all data (5 stratified folds and checking CV) , create inference, then running PB score, after that I repeated the cycle several times. but for one model this process could take several hours.</p>\n<p>I would like to know if you use some better optimization techniques for finding the best hyperparameters, I just can imagine in testing with less samples, and analyze this results only with the CV result, but:</p>\n<p>**Do you know some methodic approach which can really provide an smart and secure way to find the best hyperparameters in the best possible time? **</p>\n<p>Would be great if you can share some of your techniques</p>\n<p>Thank you!</p>",
      "rawMarkdown": "Hello Everyone\n\nFor me this is the first competition that Im trying hard to get a highest place :)\nAs probably all of us we are doing several other activities, based on this time optimization for testing iteration is critical.\n\nFor me would be good to test several different hyperparameters, just as an example, for coarse drop out, I was not able to improve my results, I think I might need to find the correct hyperparameters, additionally other augmentations like rotation, brightness etc.\n\nIn my case I made some iterations, but not sure that they were the best hyperparameter settings for all hyperparameters (learning rate, batch size, etc,etc), for example in my case I trained one model with some hyperparameters with all data (5 stratified folds and checking CV) , create inference, then running PB score, after that I repeated the cycle several times. but for one model this process could take several hours.\n\nI would like to know if you use some better optimization techniques for finding the best hyperparameters, I just can imagine in testing with less samples, and analyze this results only with the CV result, but:\n\n**Do you know some methodic approach which can really provide an smart and secure way to find the best hyperparameters in the best possible time? **\n\nWould be great if you can share some of your techniques\n\nThank you!",
      "votes": null
    },
    {
      "id": "1404519",
      "postDate": "07/30/2021 00:30:05",
      "content": "<p>I would highly recommend checking out Optuna. </p>\n<ul>\n<li><a href=\"https://optuna.org/\" target=\"_blank\">Optuna Website</a>. </li>\n<li><a href=\"https://github.com/optuna/optuna\" target=\"_blank\">Optuna Github Page</a></li>\n</ul>\n<p>I have used their framework to optimize hyperparameters for both gradient boosted decision trees and neural networks, and it has been beneficial for both. They have some great tutorials to get you started on their website and a github repository with some more reading. </p>\n<p>If these examples are not enough, feel free to check out this notebook I made a while back using the framework --&gt; <a href=\"https://www.kaggle.com/brendanartley/riiid-lgb-model-attempt\" target=\"_blank\">Riiid Model Attempt</a></p>\n<p>Hope this helps!</p>",
      "rawMarkdown": "I would highly recommend checking out Optuna. \n- [Optuna Website](https://optuna.org/). \n- [Optuna Github Page](https://github.com/optuna/optuna)\n\nI have used their framework to optimize hyperparameters for both gradient boosted decision trees and neural networks, and it has been beneficial for both. They have some great tutorials to get you started on their website and a github repository with some more reading. \n\nIf these examples are not enough, feel free to check out this notebook I made a while back using the framework --> [Riiid Model Attempt](https://www.kaggle.com/brendanartley/riiid-lgb-model-attempt)\n\nHope this helps!",
      "votes": null
    },
    {
      "id": "1405629",
      "postDate": "07/31/2021 05:45:32",
      "content": "<p>Thank you! Brendan I will have a look here :)</p>",
      "rawMarkdown": "Thank you! Brendan I will have a look here :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1404519,
      "author_name": "brendanartley",
      "author_url": "",
      "post_date": "07/30/2021 00:30:05",
      "content": "<p>I would highly recommend checking out Optuna. </p>\n<ul>\n<li><a href=\"https://optuna.org/\" target=\"_blank\">Optuna Website</a>. </li>\n<li><a href=\"https://github.com/optuna/optuna\" target=\"_blank\">Optuna Github Page</a></li>\n</ul>\n<p>I have used their framework to optimize hyperparameters for both gradient boosted decision trees and neural networks, and it has been beneficial for both. They have some great tutorials to get you started on their website and a github repository with some more reading. </p>\n<p>If these examples are not enough, feel free to check out this notebook I made a while back using the framework --&gt; <a href=\"https://www.kaggle.com/brendanartley/riiid-lgb-model-attempt\" target=\"_blank\">Riiid Model Attempt</a></p>\n<p>Hope this helps!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1405629,
          "author_name": "omararias86",
          "author_url": "",
          "post_date": "07/31/2021 05:45:32",
          "content": "<p>Thank you! Brendan I will have a look here :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1404495": "Hello Everyone\n\nFor me this is the first competition that Im trying hard to get a highest place :)\nAs probably all of us we are doing several other activities, based on this time optimization for testing iteration is critical.\n\nFor me would be good to test several different hyperparameters, just as an example, for coarse drop out, I was not able to improve my results, I think I might need to find the correct hyperparameters, additionally other augmentations like rotation, brightness etc.\n\nIn my case I made some iterations, but not sure that they were the best hyperparameter settings for all hyperparameters (learning rate, batch size, etc,etc), for example in my case I trained one model with some hyperparameters with all data (5 stratified folds and checking CV) , create inference, then running PB score, after that I repeated the cycle several times. but for one model this process could take several hours.\n\nI would like to know if you use some better optimization techniques for finding the best hyperparameters, I just can imagine in testing with less samples, and analyze this results only with the CV result, but:\n\n**Do you know some methodic approach which can really provide an smart and secure way to find the best hyperparameters in the best possible time? **\n\nWould be great if you can share some of your techniques\n\nThank you!",
    "1404519": "I would highly recommend checking out Optuna. \n- [Optuna Website](https://optuna.org/). \n- [Optuna Github Page](https://github.com/optuna/optuna)\n\nI have used their framework to optimize hyperparameters for both gradient boosted decision trees and neural networks, and it has been beneficial for both. They have some great tutorials to get you started on their website and a github repository with some more reading. \n\nIf these examples are not enough, feel free to check out this notebook I made a while back using the framework --> [Riiid Model Attempt](https://www.kaggle.com/brendanartley/riiid-lgb-model-attempt)\n\nHope this helps!",
    "1405629": "Thank you! Brendan I will have a look here :)"
  },
  "source": "meta"
}