11 Days ago I submitted a blend using his public weights because I wanted to try incorporating it into my ensemble. Unfortunately, I could not figure out how to reconcile his GPU implentation of Melspec with my CPU implementation of Melspec (my submission relied on CPU for the longer runtime; I tried using the same parameters but it was not successful).
\nSimply blend 3 of his weights and you could have gotten a very high silver. Well done to his training script, sorry I could not share this earlier.
\nKernel: https://www.kaggle.com/returnofsputnik/bird-submission?scriptVersionId=42007314
", "messages": [ { "id": "1012222", "postDate": "09/16/2020 01:22:13", "content": "11 Days ago I submitted a blend using his public weights because I wanted to try incorporating it into my ensemble. Unfortunately, I could not figure out how to reconcile his GPU implentation of Melspec with my CPU implementation of Melspec (my submission relied on CPU for the longer runtime; I tried using the same parameters but it was not successful).
\nSimply blend 3 of his weights and you could have gotten a very high silver. Well done to his training script, sorry I could not share this earlier.
\nKernel: https://www.kaggle.com/returnofsputnik/bird-submission?scriptVersionId=42007314
", "rawMarkdown": "11 Days ago I submitted a blend using his public weights because I wanted to try incorporating it into my ensemble. Unfortunately, I could not figure out how to reconcile his GPU implentation of Melspec with my CPU implementation of Melspec (my submission relied on CPU for the longer runtime; I tried using the same parameters but it was not successful).\n\nSimply blend 3 of his weights and you could have gotten a very high silver. Well done to his training script, sorry I could not share this earlier.\n\nKernel: https://www.kaggle.com/returnofsputnik/bird-submission?scriptVersionId=42007314", "votes": null }, { "id": "1012235", "postDate": "09/16/2020 01:31:49", "content": "Was the training script ever shared ? It would be interesting to know how the training was done
", "rawMarkdown": "Was the training script ever shared ? It would be interesting to know how the training was done", "votes": null }, { "id": "1012237", "postDate": "09/16/2020 01:34:24", "content": "It was not shared. Well done yukiya, we both silver medal!
", "rawMarkdown": "It was not shared. Well done yukiya, we both silver medal!", "votes": null }, { "id": "1012261", "postDate": "09/16/2020 02:07:30", "content": "Thanks, and congrats to you too !
\nI found something interesting in your kernel. Why take sqrt of probability ?
\nEDIT : I just submitted and the private LB is a lot less without sqrt
Squaring the predictions, averaging the squares, then taking square root brought significant improvement for me. It was a tip that Mario Lasseck used to win previous BirdCLEF competitions.
", "rawMarkdown": "Squaring the predictions, averaging the squares, then taking square root brought significant improvement for me. It was a tip that Mario Lasseck used to win previous BirdCLEF competitions.", "votes": null }, { "id": "1012331", "postDate": "09/16/2020 03:07:36", "content": "I discovered the same things. Actually, a simple ensemble of his weights can get you 17th place. Check this notebook https://www.kaggle.com/nvnnghia/bird-submission-fun-test?scriptVersionId=41638486
", "rawMarkdown": "I discovered the same things. Actually, a simple ensemble of his weights can get you 17th place. Check this notebook https://www.kaggle.com/nvnnghia/bird-submission-fun-test?scriptVersionId=41638486", "votes": null }, { "id": "1012347", "postDate": "09/16/2020 03:26:46", "content": "Wow! I feel bad that he did not come in 17th place. But this shows he is a strong competitor nonetheless.
", "rawMarkdown": "Wow! I feel bad that he did not come in 17th place. But this shows he is a strong competitor nonetheless.", "votes": null }, { "id": "1012383", "postDate": "09/16/2020 04:01:07", "content": "Yes:) Indeed he is very strong competitor, quick to come up with great ideas. We won gold in Bengali because last few hours he come up with unique approach :) Fun fact: He is just finishing high school :)
", "rawMarkdown": "Yes:) Indeed he is very strong competitor, quick to come up with great ideas. We won gold in Bengali because last few hours he come up with unique approach :) Fun fact: He is just finishing high school :)", "votes": null }, { "id": "1012874", "postDate": "09/16/2020 11:11:00", "content": "Hahaha I'm glad the public weights are of help to u guys. It's a bummer our team didn't make the best submission possible--CV-LB's too unstable for us to do so, and hindsight's always 20-20. Congrats to the high rank!
", "rawMarkdown": "Hahaha I'm glad the public weights are of help to u guys. It's a bummer our team didn't make the best submission possible--CV-LB's too unstable for us to do so, and hindsight's always 20-20. Congrats to the high rank!", "votes": null }, { "id": "1013173", "postDate": "09/16/2020 14:46:26", "content": "@roguekk007 I think you are the only one who uses 3 different frequency resolution as input. Any insight to that, or perhaps possibility to show us how the training is done ?
", "rawMarkdown": "roguekk007 I think you are the only one who uses 3 different frequency resolution as input. Any insight to that, or perhaps possibility to show us how the training is done ?", "votes": null }, { "id": "1013498", "postDate": "09/16/2020 18:11:46", "content": "I wanted to use that technique since it gives more information than just copying 1 melspectrogram into three channels, only costing extra preprocessing time but doesn't cost more model parameters. But not enough time!
", "rawMarkdown": "I wanted to use that technique since it gives more information than just copying 1 melspectrogram into three channels, only costing extra preprocessing time but doesn't cost more model parameters. But not enough time!", "votes": null } ], "comments": [ { "id": 1012235, "author_name": "nyleve", "author_url": "", "post_date": "09/16/2020 01:31:49", "content": "Was the training script ever shared ? It would be interesting to know how the training was done
", "votes": null, "replies": [ { "id": 1012237, "author_name": "returnofsputnik", "author_url": "", "post_date": "09/16/2020 01:34:24", "content": "It was not shared. Well done yukiya, we both silver medal!
", "votes": null, "replies": [] }, { "id": 1012261, "author_name": "nyleve", "author_url": "", "post_date": "09/16/2020 02:07:30", "content": "Thanks, and congrats to you too !
\nI found something interesting in your kernel. Why take sqrt of probability ?
\nEDIT : I just submitted and the private LB is a lot less without sqrt
Squaring the predictions, averaging the squares, then taking square root brought significant improvement for me. It was a tip that Mario Lasseck used to win previous BirdCLEF competitions.
", "votes": null, "replies": [] } ] }, { "id": 1012331, "author_name": "nvnnghia", "author_url": "", "post_date": "09/16/2020 03:07:36", "content": "I discovered the same things. Actually, a simple ensemble of his weights can get you 17th place. Check this notebook https://www.kaggle.com/nvnnghia/bird-submission-fun-test?scriptVersionId=41638486
", "votes": null, "replies": [ { "id": 1012347, "author_name": "returnofsputnik", "author_url": "", "post_date": "09/16/2020 03:26:46", "content": "Wow! I feel bad that he did not come in 17th place. But this shows he is a strong competitor nonetheless.
", "votes": null, "replies": [] }, { "id": 1012383, "author_name": "drhabib", "author_url": "", "post_date": "09/16/2020 04:01:07", "content": "Yes:) Indeed he is very strong competitor, quick to come up with great ideas. We won gold in Bengali because last few hours he come up with unique approach :) Fun fact: He is just finishing high school :)
", "votes": null, "replies": [] } ] }, { "id": 1012874, "author_name": "roguekk007", "author_url": "", "post_date": "09/16/2020 11:11:00", "content": "Hahaha I'm glad the public weights are of help to u guys. It's a bummer our team didn't make the best submission possible--CV-LB's too unstable for us to do so, and hindsight's always 20-20. Congrats to the high rank!
", "votes": null, "replies": [ { "id": 1013173, "author_name": "nyleve", "author_url": "", "post_date": "09/16/2020 14:46:26", "content": "@roguekk007 I think you are the only one who uses 3 different frequency resolution as input. Any insight to that, or perhaps possibility to show us how the training is done ?
", "votes": null, "replies": [] }, { "id": 1013498, "author_name": "returnofsputnik", "author_url": "", "post_date": "09/16/2020 18:11:46", "content": "I wanted to use that technique since it gives more information than just copying 1 melspectrogram into three channels, only costing extra preprocessing time but doesn't cost more model parameters. But not enough time!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1012222": "11 Days ago I submitted a blend using his public weights because I wanted to try incorporating it into my ensemble. Unfortunately, I could not figure out how to reconcile his GPU implentation of Melspec with my CPU implementation of Melspec (my submission relied on CPU for the longer runtime; I tried using the same parameters but it was not successful).\n\nSimply blend 3 of his weights and you could have gotten a very high silver. Well done to his training script, sorry I could not share this earlier.\n\nKernel: https://www.kaggle.com/returnofsputnik/bird-submission?scriptVersionId=42007314", "1012235": "Was the training script ever shared ? It would be interesting to know how the training was done", "1012237": "It was not shared. Well done yukiya, we both silver medal!", "1012261": "Thanks, and congrats to you too !\nI found something interesting in your kernel. Why take sqrt of probability ? \nEDIT : I just submitted and the private LB is a lot less without sqrt", "1012296": "Squaring the predictions, averaging the squares, then taking square root brought significant improvement for me. It was a tip that Mario Lasseck used to win previous BirdCLEF competitions.", "1012331": "I discovered the same things. Actually, a simple ensemble of his weights can get you 17th place. Check this notebook https://www.kaggle.com/nvnnghia/bird-submission-fun-test?scriptVersionId=41638486", "1012347": "Wow! I feel bad that he did not come in 17th place. But this shows he is a strong competitor nonetheless.", "1012383": "Yes:) Indeed he is very strong competitor, quick to come up with great ideas. We won gold in Bengali because last few hours he come up with unique approach :) Fun fact: He is just finishing high school :)", "1012874": "Hahaha I'm glad the public weights are of help to u guys. It's a bummer our team didn't make the best submission possible--CV-LB's too unstable for us to do so, and hindsight's always 20-20. Congrats to the high rank!", "1013173": "roguekk007 I think you are the only one who uses 3 different frequency resolution as input. Any insight to that, or perhaps possibility to show us how the training is done ?", "1013498": "I wanted to use that technique since it gives more information than just copying 1 melspectrogram into three channels, only costing extra preprocessing time but doesn't cost more model parameters. But not enough time!" }, "source": "meta" }