{
  "id": 160782,
  "title": "Choice of Models",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/160782",
  "author_name": "",
  "post_date": "2020-06-22T16:26:41.971543Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hey guys,</p>\n\n<p>I am new to Kaggle Competitions and I am trying to find my way around here. I wanted to post a quick question to find out how you guys make your decisions on what models to use. </p>\n\n<p>Typically most models are those with pre-built architectures and pre-trained weights, to begin with. In this competition, I took the time to look at the notebooks and I noticed people using various versions of EfficientNets, SeResNet, SeResNeXt, ResNeXt, ResUnet, and some more complicated names!</p>\n\n<p>Is there a specific method that you work with which helps you make these choices? Is it just your experience that tells you that a certain model will be good to use here? What should a beginner's approach be? I started with a simple ResNet34 and then moved to ResNet50 to see it there is an improvement and then I plan to keep making the model more and more complex..</p>\n\n<p>Also, I would be very interested to know how you keep up-to-date with the latest developments in the field? Any particular forums you follow?</p>\n\n<p>Thanks for your time!</p>",
  "messages": [
    {
      "id": "897125",
      "postDate": "06/22/2020 16:26:41",
      "content": "<p>Hey guys,</p>\n\n<p>I am new to Kaggle Competitions and I am trying to find my way around here. I wanted to post a quick question to find out how you guys make your decisions on what models to use. </p>\n\n<p>Typically most models are those with pre-built architectures and pre-trained weights, to begin with. In this competition, I took the time to look at the notebooks and I noticed people using various versions of EfficientNets, SeResNet, SeResNeXt, ResNeXt, ResUnet, and some more complicated names!</p>\n\n<p>Is there a specific method that you work with which helps you make these choices? Is it just your experience that tells you that a certain model will be good to use here? What should a beginner's approach be? I started with a simple ResNet34 and then moved to ResNet50 to see it there is an improvement and then I plan to keep making the model more and more complex..</p>\n\n<p>Also, I would be very interested to know how you keep up-to-date with the latest developments in the field? Any particular forums you follow?</p>\n\n<p>Thanks for your time!</p>",
      "rawMarkdown": "Hey guys,\n\nI am new to Kaggle Competitions and I am trying to find my way around here. I wanted to post a quick question to find out how you guys make your decisions on what models to use. \n\nTypically most models are those with pre-built architectures and pre-trained weights, to begin with. In this competition, I took the time to look at the notebooks and I noticed people using various versions of EfficientNets, SeResNet, SeResNeXt, ResNeXt, ResUnet, and some more complicated names!\n\nIs there a specific method that you work with which helps you make these choices? Is it just your experience that tells you that a certain model will be good to use here? What should a beginner's approach be? I started with a simple ResNet34 and then moved to ResNet50 to see it there is an improvement and then I plan to keep making the model more and more complex..\n\nAlso, I would be very interested to know how you keep up-to-date with the latest developments in the field? Any particular forums you follow?\n\nThanks for your time!",
      "votes": null
    },
    {
      "id": "897155",
      "postDate": "06/22/2020 16:51:33",
      "content": "<p>As being a beginner too , i too perform the same step like you do. Trying one model first and then another to check if there is any improvement or not. But a really cool thing happen , after many such tries , you get some experience and an understanding with which to try first on such a dataset , even though you give many tries further too. But soon you get to realize there are some models that performs good almost any kind of data, and soon there will be some 2-3 model architectures which you start using in such competition . Sometime , i just have at some public notebooks and see what they have used and think what if i had used this one and tuned it to get better results.</p>\n\n<p>So as a conclusion , i would suggest you to just give these kind of tries , try as many competitions you can and after you can't think of any new thing , have a look at other's approach and understand what they have done.\nIt will help you a lot to learn amazing things.</p>\n\n<p>i hope this helps. !</p>",
      "rawMarkdown": "As being a beginner too , i too perform the same step like you do. Trying one model first and then another to check if there is any improvement or not. But a really cool thing happen , after many such tries , you get some experience and an understanding with which to try first on such a dataset , even though you give many tries further too. But soon you get to realize there are some models that performs good almost any kind of data, and soon there will be some 2-3 model architectures which you start using in such competition . Sometime , i just have at some public notebooks and see what they have used and think what if i had used this one and tuned it to get better results.\n\nSo as a conclusion , i would suggest you to just give these kind of tries , try as many competitions you can and after you can't think of any new thing , have a look at other's approach and understand what they have done.\nIt will help you a lot to learn amazing things.\n\ni hope this helps. !",
      "votes": null
    },
    {
      "id": "897329",
      "postDate": "06/22/2020 19:26:46",
      "content": "<p>Just to give you more knowledge on the architectures you listed (i assume you dont know haha), EfficientNets, SeResNet, SeResNeXt all contain SE blocks (squeeze and excite) which usually boosts the performance by helping the model learn more powerful representations of the image. Here is the paper if you want to learn more about it: <a href=\"https://arxiv.org/pdf/1709.01507.pdf\">https://arxiv.org/pdf/1709.01507.pdf</a></p>\n\n<p>I suggest you start with small models and try different augmentations and optimize your pipeline, then use bigger models. Efficientnets are great for that, start with B0 and then increase the size if you want slight performance boosts!</p>",
      "rawMarkdown": "Just to give you more knowledge on the architectures you listed (i assume you dont know haha), EfficientNets, SeResNet, SeResNeXt all contain SE blocks (squeeze and excite) which usually boosts the performance by helping the model learn more powerful representations of the image. Here is the paper if you want to learn more about it: https://arxiv.org/pdf/1709.01507.pdf\n\nI suggest you start with small models and try different augmentations and optimize your pipeline, then use bigger models. Efficientnets are great for that, start with B0 and then increase the size if you want slight performance boosts!",
      "votes": null
    },
    {
      "id": "897559",
      "postDate": "06/23/2020 00:42:10",
      "content": "<p>Aah no.. I studied the model architectures and the intuition from all the various whitepapers. I just wanted to get an idea of what everyone's thought process is for choosing the models they choose.</p>\n\n<p>I understand your suggestion and will proceed in that direction.</p>\n\n<p>Thanks </p>",
      "rawMarkdown": "Aah no.. I studied the model architectures and the intuition from all the various whitepapers. I just wanted to get an idea of what everyone's thought process is for choosing the models they choose.\n\nI understand your suggestion and will proceed in that direction.\n\nThanks",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 897155,
      "author_name": "prashantarorat",
      "author_url": "",
      "post_date": "06/22/2020 16:51:33",
      "content": "<p>As being a beginner too , i too perform the same step like you do. Trying one model first and then another to check if there is any improvement or not. But a really cool thing happen , after many such tries , you get some experience and an understanding with which to try first on such a dataset , even though you give many tries further too. But soon you get to realize there are some models that performs good almost any kind of data, and soon there will be some 2-3 model architectures which you start using in such competition . Sometime , i just have at some public notebooks and see what they have used and think what if i had used this one and tuned it to get better results.</p>\n\n<p>So as a conclusion , i would suggest you to just give these kind of tries , try as many competitions you can and after you can't think of any new thing , have a look at other's approach and understand what they have done.\nIt will help you a lot to learn amazing things.</p>\n\n<p>i hope this helps. !</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 897329,
      "author_name": "yannmajewski",
      "author_url": "",
      "post_date": "06/22/2020 19:26:46",
      "content": "<p>Just to give you more knowledge on the architectures you listed (i assume you dont know haha), EfficientNets, SeResNet, SeResNeXt all contain SE blocks (squeeze and excite) which usually boosts the performance by helping the model learn more powerful representations of the image. Here is the paper if you want to learn more about it: <a href=\"https://arxiv.org/pdf/1709.01507.pdf\">https://arxiv.org/pdf/1709.01507.pdf</a></p>\n\n<p>I suggest you start with small models and try different augmentations and optimize your pipeline, then use bigger models. Efficientnets are great for that, start with B0 and then increase the size if you want slight performance boosts!</p>",
      "votes": null,
      "replies": [
        {
          "id": 897559,
          "author_name": "eadhunath",
          "author_url": "",
          "post_date": "06/23/2020 00:42:10",
          "content": "<p>Aah no.. I studied the model architectures and the intuition from all the various whitepapers. I just wanted to get an idea of what everyone's thought process is for choosing the models they choose.</p>\n\n<p>I understand your suggestion and will proceed in that direction.</p>\n\n<p>Thanks </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "897125": "Hey guys,\n\nI am new to Kaggle Competitions and I am trying to find my way around here. I wanted to post a quick question to find out how you guys make your decisions on what models to use. \n\nTypically most models are those with pre-built architectures and pre-trained weights, to begin with. In this competition, I took the time to look at the notebooks and I noticed people using various versions of EfficientNets, SeResNet, SeResNeXt, ResNeXt, ResUnet, and some more complicated names!\n\nIs there a specific method that you work with which helps you make these choices? Is it just your experience that tells you that a certain model will be good to use here? What should a beginner's approach be? I started with a simple ResNet34 and then moved to ResNet50 to see it there is an improvement and then I plan to keep making the model more and more complex..\n\nAlso, I would be very interested to know how you keep up-to-date with the latest developments in the field? Any particular forums you follow?\n\nThanks for your time!",
    "897155": "As being a beginner too , i too perform the same step like you do. Trying one model first and then another to check if there is any improvement or not. But a really cool thing happen , after many such tries , you get some experience and an understanding with which to try first on such a dataset , even though you give many tries further too. But soon you get to realize there are some models that performs good almost any kind of data, and soon there will be some 2-3 model architectures which you start using in such competition . Sometime , i just have at some public notebooks and see what they have used and think what if i had used this one and tuned it to get better results.\n\nSo as a conclusion , i would suggest you to just give these kind of tries , try as many competitions you can and after you can't think of any new thing , have a look at other's approach and understand what they have done.\nIt will help you a lot to learn amazing things.\n\ni hope this helps. !",
    "897329": "Just to give you more knowledge on the architectures you listed (i assume you dont know haha), EfficientNets, SeResNet, SeResNeXt all contain SE blocks (squeeze and excite) which usually boosts the performance by helping the model learn more powerful representations of the image. Here is the paper if you want to learn more about it: https://arxiv.org/pdf/1709.01507.pdf\n\nI suggest you start with small models and try different augmentations and optimize your pipeline, then use bigger models. Efficientnets are great for that, start with B0 and then increase the size if you want slight performance boosts!",
    "897559": "Aah no.. I studied the model architectures and the intuition from all the various whitepapers. I just wanted to get an idea of what everyone's thought process is for choosing the models they choose.\n\nI understand your suggestion and will proceed in that direction.\n\nThanks"
  },
  "source": "meta"
}