{
  "id": 111781,
  "title": "Deeper, Stronger, Better?",
  "url": "/competitions/understanding_cloud_organization/discussion/111781",
  "author_name": "",
  "post_date": "2019-10-09T00:20:35.410046500Z",
  "votes": 3,
  "comment_count": 6,
  "views": 0,
  "content": "<p>I have been experimenting with deeper encoders like <code>resnext50_32x4d</code> and <code>efficientnet-b5</code> but their performance is worse than lighter models like <code>resnet18</code>. Has anyone achieved better score with deeper models?\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1528571%2F96ec6549cc2006e59ea05830b18fa53a%2Fdeeper.png?generation=1570580423647693&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": "644533",
      "postDate": "10/09/2019 00:20:35",
      "content": "<p>I have been experimenting with deeper encoders like <code>resnext50_32x4d</code> and <code>efficientnet-b5</code> but their performance is worse than lighter models like <code>resnet18</code>. Has anyone achieved better score with deeper models?\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1528571%2F96ec6549cc2006e59ea05830b18fa53a%2Fdeeper.png?generation=1570580423647693&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "I have been experimenting with deeper encoders like `resnext50_32x4d` and `efficientnet-b5` but their performance is worse than lighter models like `resnet18`. Has anyone achieved better score with deeper models?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1528571%2F96ec6549cc2006e59ea05830b18fa53a%2Fdeeper.png?generation=1570580423647693&amp;alt=media)",
      "votes": null
    },
    {
      "id": "644844",
      "postDate": "10/09/2019 12:19:50",
      "content": "<p>I am thinking also about this and it seems according to the results that deeper is not better in this competition (at least not much better).</p>\n\n<p>I wonder should we concentrate on the things like post-processing and leave simple architecture which works such as resnet18? </p>",
      "rawMarkdown": "I am thinking also about this and it seems according to the results that deeper is not better in this competition (at least not much better).\n\nI wonder should we concentrate on the things like post-processing and leave simple architecture which works such as resnet18?",
      "votes": null
    },
    {
      "id": "644861",
      "postDate": "10/09/2019 12:43:58",
      "content": "<p>I would say deep models have bigger potential. </p>",
      "rawMarkdown": "I would say deep models have bigger potential.",
      "votes": null
    },
    {
      "id": "645132",
      "postDate": "10/09/2019 20:43:05",
      "content": "<p>Why? There have been a lot of recent competitions where light architectures like resnet34 performed best.</p>",
      "rawMarkdown": "Why? There have been a lot of recent competitions where light architectures like resnet34 performed best.",
      "votes": null
    },
    {
      "id": "645280",
      "postDate": "10/10/2019 00:46:45",
      "content": "<p>Maybe deep model can learn better with more data.</p>",
      "rawMarkdown": "Maybe deep model can learn better with more data.",
      "votes": null
    },
    {
      "id": "645335",
      "postDate": "10/10/2019 02:32:48",
      "content": "<p>Deeper architectures also didn't help me.</p>",
      "rawMarkdown": "Deeper architectures also didn't help me.",
      "votes": null
    },
    {
      "id": "646300",
      "postDate": "10/11/2019 05:13:49",
      "content": "<p>This is a very noise filled data set.  Try doing some of the work yourself on the web site - after about 25 images I got a bit more skill - after about 300 I got a lot more lazy.   If I had to redo the 300 images I tried, pretty sure they would be a whole lot different than the first time around.  You would have to hold a gun or a ton of dollars to my head to get me to do the full set of images.</p>\n\n<p>I also saw that deeper did not help.  Not a complete surprise, but still makes you wonder.</p>\n\n<p>What did surprise was that more augmentation I used the less help and the more hurt.   Leader board score dropped each time I added a new augmentation to the mix.  Never did a vision model in my two years here on Kaggle where augmentation hurt.</p>\n\n<p>I think final leader board will be a huge shakeup.   Despite a lower public leader board score I am going with deeper and lots of augmentation.   Will know in a month if that is a great judgement call or a real stupid move.</p>",
      "rawMarkdown": "This is a very noise filled data set.  Try doing some of the work yourself on the web site - after about 25 images I got a bit more skill - after about 300 I got a lot more lazy.   If I had to redo the 300 images I tried, pretty sure they would be a whole lot different than the first time around.  You would have to hold a gun or a ton of dollars to my head to get me to do the full set of images.\n\nI also saw that deeper did not help.  Not a complete surprise, but still makes you wonder.\n\nWhat did surprise was that more augmentation I used the less help and the more hurt.   Leader board score dropped each time I added a new augmentation to the mix.  Never did a vision model in my two years here on Kaggle where augmentation hurt.\n\nI think final leader board will be a huge shakeup.   Despite a lower public leader board score I am going with deeper and lots of augmentation.   Will know in a month if that is a great judgement call or a real stupid move.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 644844,
      "author_name": "demonplus",
      "author_url": "",
      "post_date": "10/09/2019 12:19:50",
      "content": "<p>I am thinking also about this and it seems according to the results that deeper is not better in this competition (at least not much better).</p>\n\n<p>I wonder should we concentrate on the things like post-processing and leave simple architecture which works such as resnet18? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 644861,
      "author_name": "gogo827jz",
      "author_url": "",
      "post_date": "10/09/2019 12:43:58",
      "content": "<p>I would say deep models have bigger potential. </p>",
      "votes": null,
      "replies": [
        {
          "id": 645132,
          "author_name": "christofhenkel",
          "author_url": "",
          "post_date": "10/09/2019 20:43:05",
          "content": "<p>Why? There have been a lot of recent competitions where light architectures like resnet34 performed best.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 645280,
          "author_name": "gogo827jz",
          "author_url": "",
          "post_date": "10/10/2019 00:46:45",
          "content": "<p>Maybe deep model can learn better with more data.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 645335,
      "author_name": "artgor",
      "author_url": "",
      "post_date": "10/10/2019 02:32:48",
      "content": "<p>Deeper architectures also didn't help me.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 646300,
      "author_name": "pcjimmmy",
      "author_url": "",
      "post_date": "10/11/2019 05:13:49",
      "content": "<p>This is a very noise filled data set.  Try doing some of the work yourself on the web site - after about 25 images I got a bit more skill - after about 300 I got a lot more lazy.   If I had to redo the 300 images I tried, pretty sure they would be a whole lot different than the first time around.  You would have to hold a gun or a ton of dollars to my head to get me to do the full set of images.</p>\n\n<p>I also saw that deeper did not help.  Not a complete surprise, but still makes you wonder.</p>\n\n<p>What did surprise was that more augmentation I used the less help and the more hurt.   Leader board score dropped each time I added a new augmentation to the mix.  Never did a vision model in my two years here on Kaggle where augmentation hurt.</p>\n\n<p>I think final leader board will be a huge shakeup.   Despite a lower public leader board score I am going with deeper and lots of augmentation.   Will know in a month if that is a great judgement call or a real stupid move.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "644533": "I have been experimenting with deeper encoders like `resnext50_32x4d` and `efficientnet-b5` but their performance is worse than lighter models like `resnet18`. Has anyone achieved better score with deeper models?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1528571%2F96ec6549cc2006e59ea05830b18fa53a%2Fdeeper.png?generation=1570580423647693&amp;alt=media)",
    "644844": "I am thinking also about this and it seems according to the results that deeper is not better in this competition (at least not much better).\n\nI wonder should we concentrate on the things like post-processing and leave simple architecture which works such as resnet18?",
    "644861": "I would say deep models have bigger potential.",
    "645132": "Why? There have been a lot of recent competitions where light architectures like resnet34 performed best.",
    "645280": "Maybe deep model can learn better with more data.",
    "645335": "Deeper architectures also didn't help me.",
    "646300": "This is a very noise filled data set.  Try doing some of the work yourself on the web site - after about 25 images I got a bit more skill - after about 300 I got a lot more lazy.   If I had to redo the 300 images I tried, pretty sure they would be a whole lot different than the first time around.  You would have to hold a gun or a ton of dollars to my head to get me to do the full set of images.\n\nI also saw that deeper did not help.  Not a complete surprise, but still makes you wonder.\n\nWhat did surprise was that more augmentation I used the less help and the more hurt.   Leader board score dropped each time I added a new augmentation to the mix.  Never did a vision model in my two years here on Kaggle where augmentation hurt.\n\nI think final leader board will be a huge shakeup.   Despite a lower public leader board score I am going with deeper and lots of augmentation.   Will know in a month if that is a great judgement call or a real stupid move."
  },
  "source": "meta"
}