Hi all.
\nFirst, thanks for a great competition. Thats my favorite type of competitions where there are incomplete or noisy labels and you have to think how to deal with them.
\nSecond, thanks to those who shared code. In particular to the authors of the following kernels:
My code is in the following notebook - https://www.kaggle.com/vzaguskin/training-rfcx-tensorflow-tpu-effnet-b2-with-fp. The final submission is a merge of several versions of submissions from that code(and similar code on colab) plus s3 trick.
\nNow, how I got from initial 80+ in the starter notebook to 90+ and silver zone.
\nThe best version of that code got .89+ on private LB.
\nThan ensembling goes - I just collect all the well scoring submissions(.87+ public) and average them. Ranking average seem to work slightly better than simple average(by 0.001 approximately).
\nThe best private score I could get with that approach is .912
My version of s3 trick is that I multiply s3 by 2.5 and s7 by 2. That gave me .93 on private LB(.918 public). The version I selected for final had same .918 public and .929 private which is pretty much the same.
\nAgain, thanks a lot for the competition. Learned many things and had a lot of fun.
\nUpd: I've added postprocessing from Chris and now this kernel scores 0.95467 private (gold zone) - which means complete training and inference on Kaggle TPU only within less than 3 hours and gold level score.
", "messages": [ { "id": 1208105, "postDate": "2021-02-18T06:54:49.807Z", "content": "Hi all.
\nFirst, thanks for a great competition. Thats my favorite type of competitions where there are incomplete or noisy labels and you have to think how to deal with them.
\nSecond, thanks to those who shared code. In particular to the authors of the following kernels:
My code is in the following notebook - https://www.kaggle.com/vzaguskin/training-rfcx-tensorflow-tpu-effnet-b2-with-fp. The final submission is a merge of several versions of submissions from that code(and similar code on colab) plus s3 trick.
\nNow, how I got from initial 80+ in the starter notebook to 90+ and silver zone.
\nThe best version of that code got .89+ on private LB.
\nThan ensembling goes - I just collect all the well scoring submissions(.87+ public) and average them. Ranking average seem to work slightly better than simple average(by 0.001 approximately).
\nThe best private score I could get with that approach is .912
My version of s3 trick is that I multiply s3 by 2.5 and s7 by 2. That gave me .93 on private LB(.918 public). The version I selected for final had same .918 public and .929 private which is pretty much the same.
\nAgain, thanks a lot for the competition. Learned many things and had a lot of fun.
\nUpd: I've added postprocessing from Chris and now this kernel scores 0.95467 private (gold zone) - which means complete training and inference on Kaggle TPU only within less than 3 hours and gold level score.
", "rawMarkdown": "Hi all.\nFirst, thanks for a great competition. Thats my favorite type of competitions where there are incomplete or noisy labels and you have to think how to deal with them.\nSecond, thanks to those who shared code. In particular to the authors of the following kernels:\n1. https://www.kaggle.com/ashusma/training-rfcx-tensorflow-tpu-effnet-b2 - that was a great starter and I was just doing edits of that kernel to move on\n2. https://www.kaggle.com/aikhmelnytskyy/resnet-tpu-on-colab-and-kaggle - that showed how you can train on colab as well\n\nMy code is in the following notebook - https://www.kaggle.com/vzaguskin/training-rfcx-tensorflow-tpu-effnet-b2-with-fp. The final submission is a merge of several versions of submissions from that code(and similar code on colab) plus s3 trick. \n\nNow, how I got from initial 80+ in the starter notebook to 90+ and silver zone.\n1. 5-second cut worked better then initial 10 second\n2. Added FP data with masked BCE/BCEFocal loss. I simply calculate BCE loss only on the label I know is missing(and just usual BCE with label smoothing 0.2/Usual BCEFocal on TP data)\n3. Use heavier model(B4)\n4. Added mixup/cutmix\n\nThe best version of that code got .89+ on private LB. \nThan ensembling goes - I just collect all the well scoring submissions(.87+ public) and average them. Ranking average seem to work slightly better than simple average(by 0.001 approximately).\nThe best private score I could get with that approach is .912\n\nMy version of s3 trick is that I multiply s3 by 2.5 and s7 by 2. That gave me .93 on private LB(.918 public). The version I selected for final had same .918 public and .929 private which is pretty much the same.\n\nAgain, thanks a lot for the competition. Learned many things and had a lot of fun.\n\n**Upd:** I've added postprocessing from Chris and now [this kernel](https://www.kaggle.com/vzaguskin/training-rfcx-tensorflow-tpu-effnet-b2-with-fp) scores 0.95467 private (gold zone) - which means complete training and inference on Kaggle TPU only within less than 3 hours and gold level score.", "votes": 18 }, { "id": 1209271, "postDate": "2021-02-18T19:39:50.550Z", "content": "Congratz !
\nIt's getting harder and harder these days to grab a medal only using Kaggle & colab, so extra credits for that :)
Can you please explain to me what masked BCE/BCEFocal loss is? Why does it help you improve your performance in this task? Thank you.
", "rawMarkdown": "Can you please explain to me what masked BCE/BCEFocal loss is? Why does it help you improve your performance in this task? Thank you." }, { "id": 1208160, "postDate": "2021-02-18T07:30:58.110Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 1208189, "postDate": "2021-02-18T07:48:49.923Z", "content": "First I noticed that my CV score for several classes is much lower than average. S3 was the most obvious. I've tried submitting all 0.5 for s3 and it gave me a score boost close to 0.01.
\nI started investigating and found that I actually underpredicting s3(and probaly s7 as well)
\nSo I tried upscaling the prediction for those two classes that would give the best boost in local CV on one of the folds and got those values.
\nI guess one could do much better than that(like https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220389)
Congratz !
\nIt's getting harder and harder these days to grab a medal only using Kaggle & colab, so extra credits for that :)
Can you please explain to me what masked BCE/BCEFocal loss is? Why does it help you improve your performance in this task? Thank you.
", "votes": 0, "replies": [] }, { "id": 1208160, "author_name": "", "author_url": "", "post_date": "2021-02-18T07:30:58.110000", "content": "", "votes": 1, "replies": [ { "id": 1208189, "author_name": "Victor Zaguskin", "author_url": "", "post_date": "2021-02-18T07:48:49.923000", "content": "First I noticed that my CV score for several classes is much lower than average. S3 was the most obvious. I've tried submitting all 0.5 for s3 and it gave me a score boost close to 0.01.
\nI started investigating and found that I actually underpredicting s3(and probaly s7 as well)
\nSo I tried upscaling the prediction for those two classes that would give the best boost in local CV on one of the folds and got those values.
\nI guess one could do much better than that(like https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220389)