{
  "id": 506908,
  "title": "model not learning",
  "url": "/competitions/rsna-2024-lumbar-spine-degenerative-classification/discussion/506908",
  "author_name": "snehal",
  "post_date": "2024-05-23T16:19:37.394000",
  "votes": 2,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I spent a day trying a bunch of things to increase validation score and realized no matter what I tried my train and validation loss stayed exactly the same. Then I tried inputing random noise to the model and still got the same train and validation loss and realized model was just learning optimal train means. I've tried increasing model complexity, image resolution, lr, epochs, scheduler and I can't find any issue in my train loop if there is one. My notebook is below and I was hoping for a second set of eyes to see if someone else can find what I'm missing.</p>\n<p>notebook: <a href=\"https://www.kaggle.com/code/snehalverma10/rsna2024-public\" target=\"_blank\">https://www.kaggle.com/code/snehalverma10/rsna2024-public</a></p>",
  "messages": [
    {
      "id": 2833006,
      "postDate": "2024-05-24T03:07:16.083Z",
      "content": "<p>when you use bs=1,you can change BN to GroupNorm  or freeze BN.<br>\nyou can fix num-slice in order to make your code simple. </p>",
      "rawMarkdown": "when you use bs=1,you can change BN to GroupNorm  or freeze BN.\nyou can fix num-slice in order to make your code simple. ",
      "votes": 1,
      "replies": [
        {
          "id": 2834654,
          "postDate": "2024-05-24T22:00:23.620Z",
          "content": "<p>I totally missed that, thanks so much for taking the time to look over my code. I tried freezing bn, instance norm (couldnt get group norm to work with efficientnet), and fixing num slice to 64 by padding short sequences up and resampling longer ones down and then training with batch size 4 (didnt have enough gpu memory for higher bs). However none of these had an effect on training and i’m still getting the same results as before. Do you have any other ideas on what might be causing the issue? </p>",
          "rawMarkdown": "I totally missed that, thanks so much for taking the time to look over my code. I tried freezing bn, instance norm (couldnt get group norm to work with efficientnet), and fixing num slice to 64 by padding short sequences up and resampling longer ones down and then training with batch size 4 (didnt have enough gpu memory for higher bs). However none of these had an effect on training and i’m still getting the same results as before. Do you have any other ideas on what might be causing the issue? "
        },
        {
          "id": 2834690,
          "postDate": "2024-05-24T23:36:54.487Z",
          "content": "<p>I've also tried weight normalized networks (timm eca_nfnet_l0.ra2_in1k) and still no luck so there is definitely some issue other than normalization.</p>",
          "rawMarkdown": "I've also tried weight normalized networks (timm eca_nfnet_l0.ra2_in1k) and still no luck so there is definitely some issue other than normalization."
        },
        {
          "id": 2834818,
          "postDate": "2024-05-25T03:42:38.980Z",
          "content": "<p>Due to the difference in the distribution of the z-axis after axial resampling, I also conducted experiments with batchsize=1 and replacing BN with GN, but the results do not seem ideal.🥲</p>",
          "rawMarkdown": "Due to the difference in the distribution of the z-axis after axial resampling, I also conducted experiments with batchsize=1 and replacing BN with GN, but the results do not seem ideal.🥲",
          "replies": [
            {
              "id": 2834876,
              "postDate": "2024-05-25T04:56:06.817Z",
              "rawMarkdown": "",
              "isDeleted": true
            }
          ]
        }
      ]
    },
    {
      "id": 2831343,
      "postDate": "2024-05-23T16:19:37.393Z",
      "content": "<p>I spent a day trying a bunch of things to increase validation score and realized no matter what I tried my train and validation loss stayed exactly the same. Then I tried inputing random noise to the model and still got the same train and validation loss and realized model was just learning optimal train means. I've tried increasing model complexity, image resolution, lr, epochs, scheduler and I can't find any issue in my train loop if there is one. My notebook is below and I was hoping for a second set of eyes to see if someone else can find what I'm missing.</p>\n<p>notebook: <a href=\"https://www.kaggle.com/code/snehalverma10/rsna2024-public\" target=\"_blank\">https://www.kaggle.com/code/snehalverma10/rsna2024-public</a></p>",
      "rawMarkdown": "I spent a day trying a bunch of things to increase validation score and realized no matter what I tried my train and validation loss stayed exactly the same. Then I tried inputing random noise to the model and still got the same train and validation loss and realized model was just learning optimal train means. I've tried increasing model complexity, image resolution, lr, epochs, scheduler and I can't find any issue in my train loop if there is one. My notebook is below and I was hoping for a second set of eyes to see if someone else can find what I'm missing.\n\nnotebook: https://www.kaggle.com/code/snehalverma10/rsna2024-public",
      "votes": 2
    }
  ],
  "comments": [
    {
      "id": 2833006,
      "author_name": "patriot",
      "author_url": "",
      "post_date": "2024-05-24T03:07:16.083000",
      "content": "<p>when you use bs=1,you can change BN to GroupNorm  or freeze BN.<br>\nyou can fix num-slice in order to make your code simple. </p>",
      "votes": 1,
      "replies": [
        {
          "id": 2834654,
          "author_name": "snehal",
          "author_url": "",
          "post_date": "2024-05-24T22:00:23.620000",
          "content": "<p>I totally missed that, thanks so much for taking the time to look over my code. I tried freezing bn, instance norm (couldnt get group norm to work with efficientnet), and fixing num slice to 64 by padding short sequences up and resampling longer ones down and then training with batch size 4 (didnt have enough gpu memory for higher bs). However none of these had an effect on training and i’m still getting the same results as before. Do you have any other ideas on what might be causing the issue? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2834690,
          "author_name": "snehal",
          "author_url": "",
          "post_date": "2024-05-24T23:36:54.487000",
          "content": "<p>I've also tried weight normalized networks (timm eca_nfnet_l0.ra2_in1k) and still no luck so there is definitely some issue other than normalization.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2834818,
          "author_name": "",
          "author_url": "",
          "post_date": "2024-05-25T03:42:38.980000",
          "content": "<p>Due to the difference in the distribution of the z-axis after axial resampling, I also conducted experiments with batchsize=1 and replacing BN with GN, but the results do not seem ideal.🥲</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2834876,
              "author_name": "",
              "author_url": "",
              "post_date": "2024-05-25T04:56:06.817000",
              "content": "",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2833006": "when you use bs=1,you can change BN to GroupNorm  or freeze BN.\nyou can fix num-slice in order to make your code simple. ",
    "2831343": "I spent a day trying a bunch of things to increase validation score and realized no matter what I tried my train and validation loss stayed exactly the same. Then I tried inputing random noise to the model and still got the same train and validation loss and realized model was just learning optimal train means. I've tried increasing model complexity, image resolution, lr, epochs, scheduler and I can't find any issue in my train loop if there is one. My notebook is below and I was hoping for a second set of eyes to see if someone else can find what I'm missing.\n\nnotebook: https://www.kaggle.com/code/snehalverma10/rsna2024-public"
  }
}