{
  "id": 403325,
  "title": "improving score from 0.71",
  "url": "/competitions/birdclef-2023/discussion/403325",
  "author_name": "",
  "post_date": "2023-04-22T12:57:00.620618400Z",
  "votes": 3,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Hii everyone,<br>\nI am trying many audio augmentation techniques and the validation loss and accuracy also improved in training.<br>\nBut the leaderboard score is always remains same(0.71).<br>\ncan anyone suggests on how to improve the score?<br>\nHere are my training and submission notebooks:<br>\n<a href=\"https://www.kaggle.com/code/rounakkumbhakar/birdclef23-training\" target=\"_blank\">training notebook</a><br>\n<a href=\"https://www.kaggle.com/code/rounakkumbhakar/birdclef23\" target=\"_blank\">submission notebook</a></p>",
  "messages": [
    {
      "id": "2230518",
      "postDate": "04/22/2023 12:57:00",
      "content": "<p>Hii everyone,<br>\nI am trying many audio augmentation techniques and the validation loss and accuracy also improved in training.<br>\nBut the leaderboard score is always remains same(0.71).<br>\ncan anyone suggests on how to improve the score?<br>\nHere are my training and submission notebooks:<br>\n<a href=\"https://www.kaggle.com/code/rounakkumbhakar/birdclef23-training\" target=\"_blank\">training notebook</a><br>\n<a href=\"https://www.kaggle.com/code/rounakkumbhakar/birdclef23\" target=\"_blank\">submission notebook</a></p>",
      "rawMarkdown": "Hii everyone,\nI am trying many audio augmentation techniques and the validation loss and accuracy also improved in training.\nBut the leaderboard score is always remains same(0.71).\ncan anyone suggests on how to improve the score?\nHere are my training and submission notebooks:\n[training notebook](https://www.kaggle.com/code/rounakkumbhakar/birdclef23-training)\n[submission notebook](https://www.kaggle.com/code/rounakkumbhakar/birdclef23)",
      "votes": null
    },
    {
      "id": "2230743",
      "postDate": "04/22/2023 17:49:22",
      "content": "<p>Hey Rounak,</p>\n<p>I suggest pretraining your model on the data from the 21 and 22 BirdCLEF competition as this is probably the easiest way to achieve better results. You could also try a slightly bigger model like efficientnet_b1 which performs better than b0 in my case.</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Rounak,\n\nI suggest pretraining your model on the data from the 21 and 22 BirdCLEF competition as this is probably the easiest way to achieve better results. You could also try a slightly bigger model like efficientnet_b1 which performs better than b0 in my case.\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2230810",
      "postDate": "04/22/2023 18:38:15",
      "content": "<p>0.71 is the same result you’d get from submitting all zeros or completely random probabilities 🙂 so it’s likely that you’ve got a bug with your submission notebook or your model is overfitting way too much to the training data, or just not training at all.</p>\n<p>Glancing at your code, here are some possibilities for bugs to watch out for:</p>\n<ol>\n<li>Make sure you’re mapping target indices to bird names the same way in both notebooks. (Btw glob may not always return stuff in the same order, so consider sorting it: <a href=\"https://stackoverflow.com/questions/6773584/how-are-glob-globs-return-values-ordered\" target=\"_blank\">https://stackoverflow.com/questions/6773584/how-are-glob-globs-return-values-ordered</a>)</li>\n<li>Also you might try making sure the columns in the submission are in the same order as the example submission notebook. Not sure if that should matter so long as the column name is right, but it’s worth trying.</li>\n<li>Make sure that the data prep like spectrogram loading/normalization + activation is exactly the same in both notebooks. Run your model in both using the same data prep on some samples just to make sure the input looks the same and outputs the same probabilities.</li>\n</ol>",
      "rawMarkdown": "0.71 is the same result you’d get from submitting all zeros or completely random probabilities 🙂 so it’s likely that you’ve got a bug with your submission notebook or your model is overfitting way too much to the training data, or just not training at all.\n\nGlancing at your code, here are some possibilities for bugs to watch out for:\n1. Make sure you’re mapping target indices to bird names the same way in both notebooks. (Btw glob may not always return stuff in the same order, so consider sorting it: https://stackoverflow.com/questions/6773584/how-are-glob-globs-return-values-ordered)\n2. Also you might try making sure the columns in the submission are in the same order as the example submission notebook. Not sure if that should matter so long as the column name is right, but it’s worth trying.\n3. Make sure that the data prep like spectrogram loading/normalization + activation is exactly the same in both notebooks. Run your model in both using the same data prep on some samples just to make sure the input looks the same and outputs the same probabilities.",
      "votes": null
    },
    {
      "id": "2231072",
      "postDate": "04/23/2023 04:34:40",
      "content": "<p>sorted the targets like as you suggested and the score is now 0.74 😃<br>\nThank you!</p>",
      "rawMarkdown": "sorted the targets like as you suggested and the score is now 0.74 😃\nThank you!",
      "votes": null
    },
    {
      "id": "2231951",
      "postDate": "04/23/2023 21:19:05",
      "content": "<p>Awesome, great to hear! I would still double check the spectrogram prep and activation logic just to make sure there's isn't an issue there. I had a case where my activation wasn't the same which brought my score down significantly.</p>",
      "rawMarkdown": "Awesome, great to hear! I would still double check the spectrogram prep and activation logic just to make sure there's isn't an issue there. I had a case where my activation wasn't the same which brought my score down significantly.",
      "votes": null
    },
    {
      "id": "2246067",
      "postDate": "05/04/2023 20:22:27",
      "content": "<p>but it becames 0.80 in medal zone atm….</p>",
      "rawMarkdown": "but it becames 0.80 in medal zone atm....",
      "votes": null
    },
    {
      "id": "2247513",
      "postDate": "05/06/2023 05:03:20",
      "content": "<p>Hi , i am getting the similar issue. I am not using glob instead my code is like this:</p>\n<p><code>bird_cols = list(pd.get_dummies(df_train['primary_label']).columns)</code></p>\n<p>which is already sorted.</p>",
      "rawMarkdown": "Hi , i am getting the similar issue. I am not using glob instead my code is like this:\n\n`bird_cols = list(pd.get_dummies(df_train['primary_label']).columns)`\n\nwhich is already sorted.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2230743,
      "author_name": "janbrederecke",
      "author_url": "",
      "post_date": "04/22/2023 17:49:22",
      "content": "<p>Hey Rounak,</p>\n<p>I suggest pretraining your model on the data from the 21 and 22 BirdCLEF competition as this is probably the easiest way to achieve better results. You could also try a slightly bigger model like efficientnet_b1 which performs better than b0 in my case.</p>\n<p>Best,<br>\nJan</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2230810,
      "author_name": "robbynevels",
      "author_url": "",
      "post_date": "04/22/2023 18:38:15",
      "content": "<p>0.71 is the same result you’d get from submitting all zeros or completely random probabilities 🙂 so it’s likely that you’ve got a bug with your submission notebook or your model is overfitting way too much to the training data, or just not training at all.</p>\n<p>Glancing at your code, here are some possibilities for bugs to watch out for:</p>\n<ol>\n<li>Make sure you’re mapping target indices to bird names the same way in both notebooks. (Btw glob may not always return stuff in the same order, so consider sorting it: <a href=\"https://stackoverflow.com/questions/6773584/how-are-glob-globs-return-values-ordered\" target=\"_blank\">https://stackoverflow.com/questions/6773584/how-are-glob-globs-return-values-ordered</a>)</li>\n<li>Also you might try making sure the columns in the submission are in the same order as the example submission notebook. Not sure if that should matter so long as the column name is right, but it’s worth trying.</li>\n<li>Make sure that the data prep like spectrogram loading/normalization + activation is exactly the same in both notebooks. Run your model in both using the same data prep on some samples just to make sure the input looks the same and outputs the same probabilities.</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 2231072,
          "author_name": "rounakkumbhakar",
          "author_url": "",
          "post_date": "04/23/2023 04:34:40",
          "content": "<p>sorted the targets like as you suggested and the score is now 0.74 😃<br>\nThank you!</p>",
          "votes": null,
          "replies": [
            {
              "id": 2231951,
              "author_name": "robbynevels",
              "author_url": "",
              "post_date": "04/23/2023 21:19:05",
              "content": "<p>Awesome, great to hear! I would still double check the spectrogram prep and activation logic just to make sure there's isn't an issue there. I had a case where my activation wasn't the same which brought my score down significantly.</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 2247513,
          "author_name": "himanshunayal",
          "author_url": "",
          "post_date": "05/06/2023 05:03:20",
          "content": "<p>Hi , i am getting the similar issue. I am not using glob instead my code is like this:</p>\n<p><code>bird_cols = list(pd.get_dummies(df_train['primary_label']).columns)</code></p>\n<p>which is already sorted.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2246067,
      "author_name": "tixmhl",
      "author_url": "",
      "post_date": "05/04/2023 20:22:27",
      "content": "<p>but it becames 0.80 in medal zone atm….</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2230518": "Hii everyone,\nI am trying many audio augmentation techniques and the validation loss and accuracy also improved in training.\nBut the leaderboard score is always remains same(0.71).\ncan anyone suggests on how to improve the score?\nHere are my training and submission notebooks:\n[training notebook](https://www.kaggle.com/code/rounakkumbhakar/birdclef23-training)\n[submission notebook](https://www.kaggle.com/code/rounakkumbhakar/birdclef23)",
    "2230743": "Hey Rounak,\n\nI suggest pretraining your model on the data from the 21 and 22 BirdCLEF competition as this is probably the easiest way to achieve better results. You could also try a slightly bigger model like efficientnet_b1 which performs better than b0 in my case.\n\nBest,\nJan",
    "2230810": "0.71 is the same result you’d get from submitting all zeros or completely random probabilities 🙂 so it’s likely that you’ve got a bug with your submission notebook or your model is overfitting way too much to the training data, or just not training at all.\n\nGlancing at your code, here are some possibilities for bugs to watch out for:\n1. Make sure you’re mapping target indices to bird names the same way in both notebooks. (Btw glob may not always return stuff in the same order, so consider sorting it: https://stackoverflow.com/questions/6773584/how-are-glob-globs-return-values-ordered)\n2. Also you might try making sure the columns in the submission are in the same order as the example submission notebook. Not sure if that should matter so long as the column name is right, but it’s worth trying.\n3. Make sure that the data prep like spectrogram loading/normalization + activation is exactly the same in both notebooks. Run your model in both using the same data prep on some samples just to make sure the input looks the same and outputs the same probabilities.",
    "2231072": "sorted the targets like as you suggested and the score is now 0.74 😃\nThank you!",
    "2231951": "Awesome, great to hear! I would still double check the spectrogram prep and activation logic just to make sure there's isn't an issue there. I had a case where my activation wasn't the same which brought my score down significantly.",
    "2246067": "but it becames 0.80 in medal zone atm....",
    "2247513": "Hi , i am getting the similar issue. I am not using glob instead my code is like this:\n\n`bird_cols = list(pd.get_dummies(df_train['primary_label']).columns)`\n\nwhich is already sorted."
  },
  "source": "meta"
}