{ "id": 318081, "title": "My Experiments ... (public 0.71)", "url": "/competitions/birdclef-2022/discussion/318081", "author_name": "", "post_date": "2022-04-10T13:03:52.565081700Z", "votes": 156, "comment_count": 31, "views": 0, "content": "

I spent a lot of time on this competition to get the solo gold.

\n

However, my private has become so busy that I can no longer devote enough time to this competition.

\n

I will share the results of my experiments so far and hope you will make use of them (If you give me
\nupvotes, I'll be happy).

\n

Best Score Inference : https://www.kaggle.com/code/kaerunantoka/birdclef2022-ex005-f0-infer/notebook
\nTraining code : https://www.kaggle.com/code/kaerunantoka/birdclef2022-use-2nd-label-f0/data
\nDataset 1/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-1-4 (re-created)
\nDataset 2/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-2-4
\nDataset 3/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-3-4
\nDataset 4/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-4-4

\n

Good Luck.

\n

Kaerururu

", "messages": [ { "id": "1751172", "postDate": "04/10/2022 13:03:52", "content": "

I spent a lot of time on this competition to get the solo gold.

\n

However, my private has become so busy that I can no longer devote enough time to this competition.

\n

I will share the results of my experiments so far and hope you will make use of them (If you give me
\nupvotes, I'll be happy).

\n

Best Score Inference : https://www.kaggle.com/code/kaerunantoka/birdclef2022-ex005-f0-infer/notebook
\nTraining code : https://www.kaggle.com/code/kaerunantoka/birdclef2022-use-2nd-label-f0/data
\nDataset 1/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-1-4 (re-created)
\nDataset 2/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-2-4
\nDataset 3/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-3-4
\nDataset 4/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-4-4

\n

Good Luck.

\n

Kaerururu

", "rawMarkdown": "I spent a lot of time on this competition to get the solo gold.\n\nHowever, my private has become so busy that I can no longer devote enough time to this competition.\n\nI will share the results of my experiments so far and hope you will make use of them (If you give me \nupvotes, I'll be happy).\n\nBest Score Inference : https://www.kaggle.com/code/kaerunantoka/birdclef2022-ex005-f0-infer/notebook\nTraining code : https://www.kaggle.com/code/kaerunantoka/birdclef2022-use-2nd-label-f0/data\nDataset 1/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-1-4 (re-created)\nDataset 2/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-2-4\nDataset 3/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-3-4\nDataset 4/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-4-4\n\nGood Luck.\n\nKaerururu", "votes": null }, { "id": "1751182", "postDate": "04/10/2022 13:16:43", "content": "

!!!! You are a good person .Thank you ^W^

", "rawMarkdown": "!!!! You are a good person .Thank you ^W^", "votes": null }, { "id": "1752157", "postDate": "04/11/2022 13:32:07", "content": "

Hi @kaerunantoka,

\n

I checked your inference notebook and it looks suspicious. Final submission.csv seems to different in comparison with submission.csv from https://www.kaggle.com/code/stefankahl/how-to-submit-to-birdclef-2022/notebook

", "rawMarkdown": "Hi @kaerunantoka,\n\nI checked your inference notebook and it looks suspicious. Final submission.csv seems to different in comparison with submission.csv from https://www.kaggle.com/code/stefankahl/how-to-submit-to-birdclef-2022/notebook", "votes": null }, { "id": "1752174", "postDate": "04/11/2022 13:45:57", "content": "

Thank you for your comment.

\n

I'll check it. However I don't have enough time as written above. update will be later.

", "rawMarkdown": "Thank you for your comment.\n\nI'll check it. However I don't have enough time as written above. update will be later.", "votes": null }, { "id": "1752230", "postDate": "04/11/2022 14:44:11", "content": "

wow I respect your devotion
\nthanks for sharing

", "rawMarkdown": "wow I respect your devotion\nthanks for sharing", "votes": null }, { "id": "1752323", "postDate": "04/11/2022 15:59:39", "content": "

does anyone else find it frustrating that people's months of work on submissions that didn't get to 0.71 aren't competitive in this comp now with the release of this notebook. there are seemingly 40+ people with \"0.71\".

", "rawMarkdown": "does anyone else find it frustrating that people's months of work on submissions that didn't get to 0.71 aren't competitive in this comp now with the release of this notebook. there are seemingly 40+ people with \"0.71\".", "votes": null }, { "id": "1752389", "postDate": "04/11/2022 16:58:12", "content": "

This is not a problem. To be competitive in the end one probably needs LB>0.80.

", "rawMarkdown": "This is not a problem. To be competitive in the end one probably needs LB>0.80.", "votes": null }, { "id": "1752497", "postDate": "04/11/2022 19:51:58", "content": "

Hmm… I tried to modify the inference notebook in accordance with \"how to submit to birdclef 2022\" and got LB=0.71 back. I do not understand what really happens. 😐

", "rawMarkdown": "Hmm... I tried to modify the inference notebook in accordance with \"how to submit to birdclef 2022\" and got LB=0.71 back. I do not understand what really happens. 😐", "votes": null }, { "id": "1752561", "postDate": "04/11/2022 23:38:26", "content": "

I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".

\n

Consider it done.

\n

I think many kagglers read and don't get the whole thing. Probably, because they click on mobiles without analysing anything. Exemplifying, this topic had 42 votes, however the Notebooks got only one or two votes. Those with more votes had bronze medals. With 42 votes (if they had understood) you should get at least silver medals on your 6 (Kaggle Notebooks) training code. Am I right?

", "rawMarkdown": "I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".\n\nConsider it done.\n\nI think many kagglers read and don't get the whole thing. Probably, because they click on mobiles without analysing anything. Exemplifying, this topic had 42 votes, however the Notebooks got only one or two votes. Those with more votes had bronze medals. With 42 votes (if they had understood) you should get at least silver medals on your 6 (Kaggle Notebooks) training code. Am I right?", "votes": null }, { "id": "1752641", "postDate": "04/12/2022 02:34:21", "content": "

I have released this considering that we have more 1+ month.
\nI wish other people take advantage of my experiments and achieve LB>0.8 .

", "rawMarkdown": "I have released this considering that we have more 1+ month.\nI wish other people take advantage of my experiments and achieve LB>0.8 .", "votes": null }, { "id": "1752643", "postDate": "04/12/2022 02:37:41", "content": "

Ya, you are right.

\n

If my notebooks achieve silver medal, it's nice.

", "rawMarkdown": "Ya, you are right.\n\nIf my notebooks achieve silver medal, it's nice.", "votes": null }, { "id": "1752647", "postDate": "04/12/2022 02:45:11", "content": "

Thank you more experiment.

\n

Maybe, both my notebook and \"how to submit to birdclef 2022\" notebook retrun the same format csv (col name, row number, row name, …) when they are rerun.
\nMy inference code generate sample submission when not rerun.

", "rawMarkdown": "Thank you more experiment.\n\nMaybe, both my notebook and \"how to submit to birdclef 2022\" notebook retrun the same format csv (col name, row number, row name, ...) when they are rerun.\nMy inference code generate sample submission when not rerun.", "votes": null }, { "id": "1753104", "postDate": "04/12/2022 13:47:41", "content": "

Thank you. Its really helpful!!!!!

", "rawMarkdown": "Thank you. Its really helpful!!!!!", "votes": null }, { "id": "1753305", "postDate": "04/12/2022 16:41:32", "content": "

I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".

", "rawMarkdown": "I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".", "votes": null }, { "id": "1753531", "postDate": "04/12/2022 23:40:14", "content": "

Thank you. It helps a lot!

", "rawMarkdown": "Thank you. It helps a lot!", "votes": null }, { "id": "1754029", "postDate": "04/13/2022 10:19:46", "content": "

Thanks for sharing.

", "rawMarkdown": "Thanks for sharing.", "votes": null }, { "id": "1754732", "postDate": "04/13/2022 23:48:00", "content": "

Thanks for sharing @kaerunantoka

", "rawMarkdown": "Thanks for sharing @kaerunantoka", "votes": null }, { "id": "1754764", "postDate": "04/14/2022 01:53:28", "content": "

Thank you for sharing your experiment. There is a big difference in CV between Fold 0 and Fold 1, do you have any idea what could be the reason ? I tried with other base model and this difference is still there.

", "rawMarkdown": "Thank you for sharing your experiment. There is a big difference in CV between Fold 0 and Fold 1, do you have any idea what could be the reason ? I tried with other base model and this difference is still there.", "votes": null }, { "id": "1755406", "postDate": "04/14/2022 15:20:23", "content": "

I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".

", "rawMarkdown": "I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".", "votes": null }, { "id": "1755476", "postDate": "04/14/2022 16:37:01", "content": "

Oh …., I'm sorry for my mistake.
\nI have not know because I experimented in my local env, always run 5fold training.
\nIf you run my training code per fold, as written below, var fold may always be 0.

\n

So, It's leakage.
\nIn fold1, trained with fold 0,2,3,4 and calc cv with fold0 (not fold1 !!).

\n
def calc_cv(model_paths):\n    df = pd.read_csv('train_folds.csv')\n    y_true = []\n    y_pred = []\n    for fold, model_path in enumerate(model_paths):\n        #################################\n        # var fold is always 0 if len(model_paths) == 1\n        #################################\n        model = TimmSED(\n            base_model_name=CFG.base_model_name,\n            pretrained=CFG.pretrained,\n            num_classes=CFG.num_classes,\n            in_channels=CFG.in_channels)\n\n        model.to(device)\n        model.load_state_dict(torch.load(model_path))\n        model.eval()\n\n        val_df = df[df.kfold == fold].reset_index(drop=True)\n        dataset = WaveformDataset(df=val_df, mode='valid')\n        dataloader = torch.utils.data.DataLoader(\n            dataset, batch_size=CFG.valid_bs, num_workers=0, pin_memory=True, shuffle=False\n        )\n\n        final_output, final_target = inference_fn(model, dataloader, device)\n        y_pred.extend(final_output)\n        y_true.extend(final_target)\n        torch.cuda.empty_cache()\n\n        f1_03 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.3, average=\"micro\")\n        print(f'micro f1_0.3 {f1_03}')\n\n    f1_03 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.3, average=\"micro\")\n    f1_05 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.5, average=\"micro\")\n\n    print(f'overall micro f1_0.3 {f1_03}')\n    print(f'overall micro f1_0.5 {f1_05}')\n    return\n
", "rawMarkdown": "Oh ...., I'm sorry for my mistake.\nI have not know because I experimented in my local env, always run 5fold training.\nIf you run my training code per fold, as written below, var fold may always be 0.\n\nSo, It's leakage. \nIn fold1, trained with fold 0,2,3,4 and calc cv with fold0 (not fold1 !!).\n\n```\ndef calc_cv(model_paths):\n df = pd.read_csv('train_folds.csv')\n y_true = []\n y_pred = []\n for fold, model_path in enumerate(model_paths):\n #################################\n # var fold is always 0 if len(model_paths) == 1\n #################################\n model = TimmSED(\n base_model_name=CFG.base_model_name,\n pretrained=CFG.pretrained,\n num_classes=CFG.num_classes,\n in_channels=CFG.in_channels)\n\n model.to(device)\n model.load_state_dict(torch.load(model_path))\n model.eval()\n\n val_df = df[df.kfold == fold].reset_index(drop=True)\n dataset = WaveformDataset(df=val_df, mode='valid')\n dataloader = torch.utils.data.DataLoader(\n dataset, batch_size=CFG.valid_bs, num_workers=0, pin_memory=True, shuffle=False\n )\n\n final_output, final_target = inference_fn(model, dataloader, device)\n y_pred.extend(final_output)\n y_true.extend(final_target)\n torch.cuda.empty_cache()\n\n f1_03 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.3, average=\"micro\")\n print(f'micro f1_0.3 {f1_03}')\n\n f1_03 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.3, average=\"micro\")\n f1_05 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.5, average=\"micro\")\n\n print(f'overall micro f1_0.3 {f1_03}')\n print(f'overall micro f1_0.5 {f1_05}')\n return\n```", "votes": null }, { "id": "1756407", "postDate": "04/15/2022 13:50:08", "content": "

Thank you for your reply. I should be the one saying sorry for not reading the code carefully (I'm too eager to run them first)

", "rawMarkdown": "Thank you for your reply. I should be the one saying sorry for not reading the code carefully (I'm too eager to run them first)", "votes": null }, { "id": "1757208", "postDate": "04/16/2022 11:41:22", "content": "

Thank you. Its helpful!!!!!

", "rawMarkdown": "Thank you. Its helpful!!!!!", "votes": null }, { "id": "1757805", "postDate": "04/17/2022 03:27:55", "content": "

Thanks for the good work, wasn't familiar with using the framewise_logit as an auxilary for the loss.

\n

I do have a question regarding the mixup, why did you decide to use loss mixup over label mixup??

", "rawMarkdown": "Thanks for the good work, wasn't familiar with using the framewise_logit as an auxilary for the loss.\n\nI do have a question regarding the mixup, why did you decide to use loss mixup over label mixup??", "votes": null }, { "id": "1758050", "postDate": "04/17/2022 09:15:29", "content": "

Thanks for sharing. You deserve a gold medal :)

", "rawMarkdown": "Thanks for sharing. You deserve a gold medal :)", "votes": null }, { "id": "1758226", "postDate": "04/17/2022 13:23:05", "content": "

Thanks for your sharing. It is a good source to reference.

", "rawMarkdown": "Thanks for your sharing. It is a good source to reference.", "votes": null }, { "id": "1758614", "postDate": "04/17/2022 21:26:36", "content": "

The code you have shared is really helpful. I hope you will continue to share more code andexperiments in the future.\"

", "rawMarkdown": "The code you have shared is really helpful. I hope you will continue to share more code andexperiments in the future.\"", "votes": null }, { "id": "1762176", "postDate": "04/20/2022 13:35:55", "content": "

Thanks! That helps a lot. :-)

", "rawMarkdown": "Thanks! That helps a lot. :-)", "votes": null }, { "id": "1781793", "postDate": "05/09/2022 00:40:21", "content": "

@kaerunantoka Thanks for sharing. You work helps me a lot!
\nBut I have a problem. Is your CV score consistent with the public leaderboard score?

\n

I opened a discuss. https://www.kaggle.com/competitions/birdclef-2022/discussion/323809

", "rawMarkdown": "kaerunantoka Thanks for sharing. You work helps me a lot!\nBut I have a problem. Is your CV score consistent with the public leaderboard score?\n\nI opened a discuss. [https://www.kaggle.com/competitions/birdclef-2022/discussion/323809](url)", "votes": null }, { "id": "1792019", "postDate": "05/16/2022 15:03:01", "content": "

Has anyone tried to run this code on colab? It runs slowly - around 20 min on kaggle VS 1 hour on colab with the same and running on GPU. Has anyone encountered similar problem?

", "rawMarkdown": "Has anyone tried to run this code on colab? It runs slowly - around 20 min on kaggle VS 1 hour on colab with the same and running on GPU. Has anyone encountered similar problem?", "votes": null }, { "id": "1792037", "postDate": "05/16/2022 15:26:06", "content": "

I also faced the same problem. Even V100 is slower than kaggle notebook. I don't know much about the reason, but I suspect it is due to different versions of PyTorch and transformers.

", "rawMarkdown": "I also faced the same problem. Even V100 is slower than kaggle notebook. I don't know much about the reason, but I suspect it is due to different versions of PyTorch and transformers.", "votes": null }, { "id": "1792376", "postDate": "05/16/2022 23:19:47", "content": "

oh ok, I see, thank you for your response

", "rawMarkdown": "oh ok, I see, thank you for your response", "votes": null }, { "id": "1796881", "postDate": "05/21/2022 09:14:44", "content": "

Hi, if you have data on drive, then you need to transfer it to /content. Drive has a horribly slow I/O.

\n

This takes ~25 min for me every time.

", "rawMarkdown": "Hi, if you have data on drive, then you need to transfer it to /content. Drive has a horribly slow I/O.\n\nThis takes ~25 min for me every time.", "votes": null } ], "comments": [ { "id": 1751182, "author_name": "yoyobar", "author_url": "", "post_date": "04/10/2022 13:16:43", "content": "

!!!! You are a good person .Thank you ^W^

", "votes": null, "replies": [] }, { "id": 1752157, "author_name": "egortrushin", "author_url": "", "post_date": "04/11/2022 13:32:07", "content": "

Hi @kaerunantoka,

\n

I checked your inference notebook and it looks suspicious. Final submission.csv seems to different in comparison with submission.csv from https://www.kaggle.com/code/stefankahl/how-to-submit-to-birdclef-2022/notebook

", "votes": null, "replies": [ { "id": 1752174, "author_name": "kaerunantoka", "author_url": "", "post_date": "04/11/2022 13:45:57", "content": "

Thank you for your comment.

\n

I'll check it. However I don't have enough time as written above. update will be later.

", "votes": null, "replies": [] }, { "id": 1752497, "author_name": "egortrushin", "author_url": "", "post_date": "04/11/2022 19:51:58", "content": "

Hmm… I tried to modify the inference notebook in accordance with \"how to submit to birdclef 2022\" and got LB=0.71 back. I do not understand what really happens. 😐

", "votes": null, "replies": [] }, { "id": 1752647, "author_name": "kaerunantoka", "author_url": "", "post_date": "04/12/2022 02:45:11", "content": "

Thank you more experiment.

\n

Maybe, both my notebook and \"how to submit to birdclef 2022\" notebook retrun the same format csv (col name, row number, row name, …) when they are rerun.
\nMy inference code generate sample submission when not rerun.

", "votes": null, "replies": [] } ] }, { "id": 1752230, "author_name": "bell2psy", "author_url": "", "post_date": "04/11/2022 14:44:11", "content": "

wow I respect your devotion
\nthanks for sharing

", "votes": null, "replies": [] }, { "id": 1752323, "author_name": "dryanfurman", "author_url": "", "post_date": "04/11/2022 15:59:39", "content": "

does anyone else find it frustrating that people's months of work on submissions that didn't get to 0.71 aren't competitive in this comp now with the release of this notebook. there are seemingly 40+ people with \"0.71\".

", "votes": null, "replies": [ { "id": 1752389, "author_name": "egortrushin", "author_url": "", "post_date": "04/11/2022 16:58:12", "content": "

This is not a problem. To be competitive in the end one probably needs LB>0.80.

", "votes": null, "replies": [] }, { "id": 1752641, "author_name": "kaerunantoka", "author_url": "", "post_date": "04/12/2022 02:34:21", "content": "

I have released this considering that we have more 1+ month.
\nI wish other people take advantage of my experiments and achieve LB>0.8 .

", "votes": null, "replies": [] } ] }, { "id": 1752561, "author_name": "mpwolke", "author_url": "", "post_date": "04/11/2022 23:38:26", "content": "

I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".

\n

Consider it done.

\n

I think many kagglers read and don't get the whole thing. Probably, because they click on mobiles without analysing anything. Exemplifying, this topic had 42 votes, however the Notebooks got only one or two votes. Those with more votes had bronze medals. With 42 votes (if they had understood) you should get at least silver medals on your 6 (Kaggle Notebooks) training code. Am I right?

", "votes": null, "replies": [ { "id": 1752643, "author_name": "kaerunantoka", "author_url": "", "post_date": "04/12/2022 02:37:41", "content": "

Ya, you are right.

\n

If my notebooks achieve silver medal, it's nice.

", "votes": null, "replies": [] } ] }, { "id": 1753104, "author_name": "piyushbaramkar", "author_url": "", "post_date": "04/12/2022 13:47:41", "content": "

Thank you. Its really helpful!!!!!

", "votes": null, "replies": [] }, { "id": 1753305, "author_name": "mvpankit", "author_url": "", "post_date": "04/12/2022 16:41:32", "content": "

I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".

", "votes": null, "replies": [] }, { "id": 1753531, "author_name": "arti1117", "author_url": "", "post_date": "04/12/2022 23:40:14", "content": "

Thank you. It helps a lot!

", "votes": null, "replies": [] }, { "id": 1754029, "author_name": "magnsuhyun", "author_url": "", "post_date": "04/13/2022 10:19:46", "content": "

Thanks for sharing.

", "votes": null, "replies": [] }, { "id": 1754732, "author_name": "ishanmehta115", "author_url": "", "post_date": "04/13/2022 23:48:00", "content": "

Thanks for sharing @kaerunantoka

", "votes": null, "replies": [] }, { "id": 1754764, "author_name": "nyleve", "author_url": "", "post_date": "04/14/2022 01:53:28", "content": "

Thank you for sharing your experiment. There is a big difference in CV between Fold 0 and Fold 1, do you have any idea what could be the reason ? I tried with other base model and this difference is still there.

", "votes": null, "replies": [ { "id": 1755476, "author_name": "kaerunantoka", "author_url": "", "post_date": "04/14/2022 16:37:01", "content": "

Oh …., I'm sorry for my mistake.
\nI have not know because I experimented in my local env, always run 5fold training.
\nIf you run my training code per fold, as written below, var fold may always be 0.

\n

So, It's leakage.
\nIn fold1, trained with fold 0,2,3,4 and calc cv with fold0 (not fold1 !!).

\n
def calc_cv(model_paths):\n    df = pd.read_csv('train_folds.csv')\n    y_true = []\n    y_pred = []\n    for fold, model_path in enumerate(model_paths):\n        #################################\n        # var fold is always 0 if len(model_paths) == 1\n        #################################\n        model = TimmSED(\n            base_model_name=CFG.base_model_name,\n            pretrained=CFG.pretrained,\n            num_classes=CFG.num_classes,\n            in_channels=CFG.in_channels)\n\n        model.to(device)\n        model.load_state_dict(torch.load(model_path))\n        model.eval()\n\n        val_df = df[df.kfold == fold].reset_index(drop=True)\n        dataset = WaveformDataset(df=val_df, mode='valid')\n        dataloader = torch.utils.data.DataLoader(\n            dataset, batch_size=CFG.valid_bs, num_workers=0, pin_memory=True, shuffle=False\n        )\n\n        final_output, final_target = inference_fn(model, dataloader, device)\n        y_pred.extend(final_output)\n        y_true.extend(final_target)\n        torch.cuda.empty_cache()\n\n        f1_03 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.3, average=\"micro\")\n        print(f'micro f1_0.3 {f1_03}')\n\n    f1_03 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.3, average=\"micro\")\n    f1_05 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.5, average=\"micro\")\n\n    print(f'overall micro f1_0.3 {f1_03}')\n    print(f'overall micro f1_0.5 {f1_05}')\n    return\n
", "votes": null, "replies": [] }, { "id": 1756407, "author_name": "nyleve", "author_url": "", "post_date": "04/15/2022 13:50:08", "content": "

Thank you for your reply. I should be the one saying sorry for not reading the code carefully (I'm too eager to run them first)

", "votes": null, "replies": [] } ] }, { "id": 1755406, "author_name": "razexpro", "author_url": "", "post_date": "04/14/2022 15:20:23", "content": "

I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".

", "votes": null, "replies": [] }, { "id": 1757208, "author_name": "jacopobonato", "author_url": "", "post_date": "04/16/2022 11:41:22", "content": "

Thank you. Its helpful!!!!!

", "votes": null, "replies": [] }, { "id": 1757805, "author_name": "ksmcg90", "author_url": "", "post_date": "04/17/2022 03:27:55", "content": "

Thanks for the good work, wasn't familiar with using the framewise_logit as an auxilary for the loss.

\n

I do have a question regarding the mixup, why did you decide to use loss mixup over label mixup??

", "votes": null, "replies": [] }, { "id": 1758050, "author_name": "yangranran", "author_url": "", "post_date": "04/17/2022 09:15:29", "content": "

Thanks for sharing. You deserve a gold medal :)

", "votes": null, "replies": [] }, { "id": 1758226, "author_name": "trminhnam", "author_url": "", "post_date": "04/17/2022 13:23:05", "content": "

Thanks for your sharing. It is a good source to reference.

", "votes": null, "replies": [] }, { "id": 1758614, "author_name": "", "author_url": "", "post_date": "04/17/2022 21:26:36", "content": "

The code you have shared is really helpful. I hope you will continue to share more code andexperiments in the future.\"

", "votes": null, "replies": [] }, { "id": 1762176, "author_name": "dysonlin", "author_url": "", "post_date": "04/20/2022 13:35:55", "content": "

Thanks! That helps a lot. :-)

", "votes": null, "replies": [] }, { "id": 1781793, "author_name": "zhoumichael", "author_url": "", "post_date": "05/09/2022 00:40:21", "content": "

@kaerunantoka Thanks for sharing. You work helps me a lot!
\nBut I have a problem. Is your CV score consistent with the public leaderboard score?

\n

I opened a discuss. https://www.kaggle.com/competitions/birdclef-2022/discussion/323809

", "votes": null, "replies": [] }, { "id": 1792019, "author_name": "azamat25", "author_url": "", "post_date": "05/16/2022 15:03:01", "content": "

Has anyone tried to run this code on colab? It runs slowly - around 20 min on kaggle VS 1 hour on colab with the same and running on GPU. Has anyone encountered similar problem?

", "votes": null, "replies": [ { "id": 1792037, "author_name": "tc0000", "author_url": "", "post_date": "05/16/2022 15:26:06", "content": "

I also faced the same problem. Even V100 is slower than kaggle notebook. I don't know much about the reason, but I suspect it is due to different versions of PyTorch and transformers.

", "votes": null, "replies": [] }, { "id": 1792376, "author_name": "azamat25", "author_url": "", "post_date": "05/16/2022 23:19:47", "content": "

oh ok, I see, thank you for your response

", "votes": null, "replies": [] }, { "id": 1796881, "author_name": "jayeonyi", "author_url": "", "post_date": "05/21/2022 09:14:44", "content": "

Hi, if you have data on drive, then you need to transfer it to /content. Drive has a horribly slow I/O.

\n

This takes ~25 min for me every time.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1751172": "I spent a lot of time on this competition to get the solo gold.\n\nHowever, my private has become so busy that I can no longer devote enough time to this competition.\n\nI will share the results of my experiments so far and hope you will make use of them (If you give me \nupvotes, I'll be happy).\n\nBest Score Inference : https://www.kaggle.com/code/kaerunantoka/birdclef2022-ex005-f0-infer/notebook\nTraining code : https://www.kaggle.com/code/kaerunantoka/birdclef2022-use-2nd-label-f0/data\nDataset 1/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-1-4 (re-created)\nDataset 2/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-2-4\nDataset 3/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-3-4\nDataset 4/4 : https://www.kaggle.com/kaerunantoka/birdclef2022-audio-to-numpy-4-4\n\nGood Luck.\n\nKaerururu", "1751182": "!!!! You are a good person .Thank you ^W^", "1752157": "Hi @kaerunantoka,\n\nI checked your inference notebook and it looks suspicious. Final submission.csv seems to different in comparison with submission.csv from https://www.kaggle.com/code/stefankahl/how-to-submit-to-birdclef-2022/notebook", "1752174": "Thank you for your comment.\n\nI'll check it. However I don't have enough time as written above. update will be later.", "1752230": "wow I respect your devotion\nthanks for sharing", "1752323": "does anyone else find it frustrating that people's months of work on submissions that didn't get to 0.71 aren't competitive in this comp now with the release of this notebook. there are seemingly 40+ people with \"0.71\".", "1752389": "This is not a problem. To be competitive in the end one probably needs LB>0.80.", "1752497": "Hmm... I tried to modify the inference notebook in accordance with \"how to submit to birdclef 2022\" and got LB=0.71 back. I do not understand what really happens. 😐", "1752561": "I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".\n\nConsider it done.\n\nI think many kagglers read and don't get the whole thing. Probably, because they click on mobiles without analysing anything. Exemplifying, this topic had 42 votes, however the Notebooks got only one or two votes. Those with more votes had bronze medals. With 42 votes (if they had understood) you should get at least silver medals on your 6 (Kaggle Notebooks) training code. Am I right?", "1752641": "I have released this considering that we have more 1+ month.\nI wish other people take advantage of my experiments and achieve LB>0.8 .", "1752643": "Ya, you are right.\n\nIf my notebooks achieve silver medal, it's nice.", "1752647": "Thank you more experiment.\n\nMaybe, both my notebook and \"how to submit to birdclef 2022\" notebook retrun the same format csv (col name, row number, row name, ...) when they are rerun.\nMy inference code generate sample submission when not rerun.", "1753104": "Thank you. Its really helpful!!!!!", "1753305": "I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".", "1753531": "Thank you. It helps a lot!", "1754029": "Thanks for sharing.", "1754732": "Thanks for sharing @kaerunantoka", "1754764": "Thank you for sharing your experiment. There is a big difference in CV between Fold 0 and Fold 1, do you have any idea what could be the reason ? I tried with other base model and this difference is still there.", "1755406": "I endorsed your honesty: \"If you give me upvotes, I'll be happy\". (your words). \"I spent a lot of time on this competition to get the solo gold\".", "1755476": "Oh ...., I'm sorry for my mistake.\nI have not know because I experimented in my local env, always run 5fold training.\nIf you run my training code per fold, as written below, var fold may always be 0.\n\nSo, It's leakage. \nIn fold1, trained with fold 0,2,3,4 and calc cv with fold0 (not fold1 !!).\n\n```\ndef calc_cv(model_paths):\n df = pd.read_csv('train_folds.csv')\n y_true = []\n y_pred = []\n for fold, model_path in enumerate(model_paths):\n #################################\n # var fold is always 0 if len(model_paths) == 1\n #################################\n model = TimmSED(\n base_model_name=CFG.base_model_name,\n pretrained=CFG.pretrained,\n num_classes=CFG.num_classes,\n in_channels=CFG.in_channels)\n\n model.to(device)\n model.load_state_dict(torch.load(model_path))\n model.eval()\n\n val_df = df[df.kfold == fold].reset_index(drop=True)\n dataset = WaveformDataset(df=val_df, mode='valid')\n dataloader = torch.utils.data.DataLoader(\n dataset, batch_size=CFG.valid_bs, num_workers=0, pin_memory=True, shuffle=False\n )\n\n final_output, final_target = inference_fn(model, dataloader, device)\n y_pred.extend(final_output)\n y_true.extend(final_target)\n torch.cuda.empty_cache()\n\n f1_03 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.3, average=\"micro\")\n print(f'micro f1_0.3 {f1_03}')\n\n f1_03 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.3, average=\"micro\")\n f1_05 = metrics.f1_score(np.array(y_true), np.array(y_pred) > 0.5, average=\"micro\")\n\n print(f'overall micro f1_0.3 {f1_03}')\n print(f'overall micro f1_0.5 {f1_05}')\n return\n```", "1756407": "Thank you for your reply. I should be the one saying sorry for not reading the code carefully (I'm too eager to run them first)", "1757208": "Thank you. Its helpful!!!!!", "1757805": "Thanks for the good work, wasn't familiar with using the framewise_logit as an auxilary for the loss.\n\nI do have a question regarding the mixup, why did you decide to use loss mixup over label mixup??", "1758050": "Thanks for sharing. You deserve a gold medal :)", "1758226": "Thanks for your sharing. It is a good source to reference.", "1758614": "The code you have shared is really helpful. I hope you will continue to share more code andexperiments in the future.\"", "1762176": "Thanks! That helps a lot. :-)", "1781793": "kaerunantoka Thanks for sharing. You work helps me a lot!\nBut I have a problem. Is your CV score consistent with the public leaderboard score?\n\nI opened a discuss. [https://www.kaggle.com/competitions/birdclef-2022/discussion/323809](url)", "1792019": "Has anyone tried to run this code on colab? It runs slowly - around 20 min on kaggle VS 1 hour on colab with the same and running on GPU. Has anyone encountered similar problem?", "1792037": "I also faced the same problem. Even V100 is slower than kaggle notebook. I don't know much about the reason, but I suspect it is due to different versions of PyTorch and transformers.", "1792376": "oh ok, I see, thank you for your response", "1796881": "Hi, if you have data on drive, then you need to transfer it to /content. Drive has a horribly slow I/O.\n\nThis takes ~25 min for me every time." }, "source": "meta" }