{
  "id": 476093,
  "title": "Did anyone achieve better results with EfficientNet (B3, B4, B5) compared to B0?",
  "url": "/competitions/hms-harmful-brain-activity-classification/discussion/476093",
  "author_name": "",
  "post_date": "2024-02-11T06:25:32.550950Z",
  "votes": 8,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I've been trying to use heavier architectures on kaggle spectrograms, but they're overfitting a lot even after adding augmentations. Freezing some initial layers decreases the CV significantly. I would like to know about others' experiences. <br>\nAlso are their any other methods besides regularization, freezing and augmentation that helps in preventing overfitting?</p>",
  "messages": [
    {
      "id": "2646691",
      "postDate": "02/11/2024 06:25:32",
      "content": "<p>I've been trying to use heavier architectures on kaggle spectrograms, but they're overfitting a lot even after adding augmentations. Freezing some initial layers decreases the CV significantly. I would like to know about others' experiences. <br>\nAlso are their any other methods besides regularization, freezing and augmentation that helps in preventing overfitting?</p>",
      "rawMarkdown": "I've been trying to use heavier architectures on kaggle spectrograms, but they're overfitting a lot even after adding augmentations. Freezing some initial layers decreases the CV significantly. I would like to know about others' experiences. \nAlso are their any other methods besides regularization, freezing and augmentation that helps in preventing overfitting?",
      "votes": null
    },
    {
      "id": "2646896",
      "postDate": "02/11/2024 09:14:33",
      "content": "<p>I had better result than b0 using heavier architecture atleast till b2 … augmentation was the only way for me …</p>",
      "rawMarkdown": "I had better result than b0 using heavier architecture atleast till b2 ... augmentation was the only way for me ...",
      "votes": null
    },
    {
      "id": "2646930",
      "postDate": "02/11/2024 09:43:21",
      "content": "<p>Not here yet but it happened to me in previous competitions. I think the reason is it's harder to converge larger models.</p>",
      "rawMarkdown": "Not here yet but it happened to me in previous competitions. I think the reason is it's harder to converge larger models.",
      "votes": null
    },
    {
      "id": "2648616",
      "postDate": "02/12/2024 10:25:20",
      "content": "<p>My best cv at the moment is b3, 0.56, but the lb is unchanged.</p>",
      "rawMarkdown": "My best cv at the moment is b3, 0.56, but the lb is unchanged.",
      "votes": null
    },
    {
      "id": "2649155",
      "postDate": "02/12/2024 16:39:23",
      "content": "<p>Also have seen this in previous competitions.    There seems to be a very clear, but well hidden, relationship between the amount of data and the results from varying model sizes.    This data set is pretty small so it would be very surprising to me if b4 or larger showed real value.   </p>",
      "rawMarkdown": "Also have seen this in previous competitions.    There seems to be a very clear, but well hidden, relationship between the amount of data and the results from varying model sizes.    This data set is pretty small so it would be very surprising to me if b4 or larger showed real value.",
      "votes": null
    },
    {
      "id": "2649679",
      "postDate": "02/13/2024 03:24:25",
      "content": "<p>experiencing the same CV is improving with various methods but LB is like stuck at one point best so far got <code>0.575052675</code> . Not sure need to focus maybe more on the data side 😏</p>\n<p>Also seeing some torch models having worse cv giving better LB scores </p>",
      "rawMarkdown": "experiencing the same CV is improving with various methods but LB is like stuck at one point best so far got `0.575052675` . Not sure need to focus maybe more on the data side 😏\n\nAlso seeing some torch models having worse cv giving better LB scores",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2646896,
      "author_name": "phoenix9032",
      "author_url": "",
      "post_date": "02/11/2024 09:14:33",
      "content": "<p>I had better result than b0 using heavier architecture atleast till b2 … augmentation was the only way for me …</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2646930,
      "author_name": "gunesevitan",
      "author_url": "",
      "post_date": "02/11/2024 09:43:21",
      "content": "<p>Not here yet but it happened to me in previous competitions. I think the reason is it's harder to converge larger models.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2649155,
          "author_name": "pcjimmmy",
          "author_url": "",
          "post_date": "02/12/2024 16:39:23",
          "content": "<p>Also have seen this in previous competitions.    There seems to be a very clear, but well hidden, relationship between the amount of data and the results from varying model sizes.    This data set is pretty small so it would be very surprising to me if b4 or larger showed real value.   </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2648616,
      "author_name": "horikitasaku",
      "author_url": "",
      "post_date": "02/12/2024 10:25:20",
      "content": "<p>My best cv at the moment is b3, 0.56, but the lb is unchanged.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2649679,
      "author_name": "gauravbrills",
      "author_url": "",
      "post_date": "02/13/2024 03:24:25",
      "content": "<p>experiencing the same CV is improving with various methods but LB is like stuck at one point best so far got <code>0.575052675</code> . Not sure need to focus maybe more on the data side 😏</p>\n<p>Also seeing some torch models having worse cv giving better LB scores </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2646691": "I've been trying to use heavier architectures on kaggle spectrograms, but they're overfitting a lot even after adding augmentations. Freezing some initial layers decreases the CV significantly. I would like to know about others' experiences. \nAlso are their any other methods besides regularization, freezing and augmentation that helps in preventing overfitting?",
    "2646896": "I had better result than b0 using heavier architecture atleast till b2 ... augmentation was the only way for me ...",
    "2646930": "Not here yet but it happened to me in previous competitions. I think the reason is it's harder to converge larger models.",
    "2648616": "My best cv at the moment is b3, 0.56, but the lb is unchanged.",
    "2649155": "Also have seen this in previous competitions.    There seems to be a very clear, but well hidden, relationship between the amount of data and the results from varying model sizes.    This data set is pretty small so it would be very surprising to me if b4 or larger showed real value.",
    "2649679": "experiencing the same CV is improving with various methods but LB is like stuck at one point best so far got `0.575052675` . Not sure need to focus maybe more on the data side 😏\n\nAlso seeing some torch models having worse cv giving better LB scores"
  },
  "source": "meta"
}