Thank you for opening this competition
\nAlso the notebooks and discussions have helped me a lot, thank you all!
My solution was similar to Beluga & Peter in 7th Place.
\n@https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220443.
[hand labels]
\n While observing the data from t_min and t_max in the given train__tp.csv, I found that there are many kinds of bird calls mixed together.
\n So I decided to treat it as a multi-class multi-label problem.
\n It was also mentioned in the discussion that the test labels were eventually carefully labeled by humans.
\n The TPs in the given t_min, t_max range are all easy to understand, but there are many TPs in the 60s clip that are difficult to understand and not labeled.
\n I thought it would be better to label them carefully by myself to make the condition as close to test as possible in case such incomprehensible calls are also labeled in test.
\nAnd I was thinking of doing Pseudo labeling after the accuracy of the model improves.
I trimmed the 5s~ around t_min and t_max in train__tp.csv.
\n Hand labels took about a week.
\n As a result, a total of 2428 clips and 5s chunks were used as train_data.
\n The distribution of the train_data classes looks like this
\n(I couldn't upload the image, so I'll post it later)
class nb
\ns3 1257
\ns12 520
\ns18 512
\n:
\ns16 100
\ns17 100
\ns6 97
I can see that there is a label imbalance, especially for s3, s12, and s18, because their labels co-occur among the other classes of clips.
\nIn particular, s3 is dominant, so it tends to output high probability, while a few classes output low probability, so i thought this is a bad problem for this evaluation index.
\nTherefore, in order to achieve a more balanced distribution, I oversampled the minority classes and undersampled the majority classes.
\n However, the LB became worse.
\n Looking back, I didn't think of approaching the test distribution, as Chris pointed out.
\nhttps://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220389
Eventually, i ensembled 15 models with various losses(FocalLoss,FbetaLoss+FocalLoss,ResampleLoss).
\n[trainning]
\nexample single model
\nPANNsDense161 (public_LB 0.95548, private_LB 0.96300)
I also tried EfficientNet_b0, Dense121, etc., but Dense161 worked well.
\ntrain_data(sr=48000,5s)
\n window_size=2048,hop_size=512,mel_bins=256
\n MultilabelStratifiedKFold 5fold
\n BCEFocalLoss(α=0.25,β=2)
\n GradualWarmupScheduler,CosineAnnealingLR(lr = 0.001,multiplier=10,epo35)
Augmentation
\n GaussianNoise(p=0.5)
\n GaussianSNR(p=0.5)
\n FrequencyMask(min_frequency_band=0.0, max_frequency_band=0.2, p=0.3)
\n TimeMask(min_band_part=0.0, max_band_part=0.2, p=0.8)
\n PitchShift(min_semitones=-0.5, max_semitones=0.5, p=0.1)
\n Shift(p=0.1)
\n Gain(p=0.2)
[inference]
\n stride=1
\n framewise_output max
\n No TTA (I used it in the final ensemble model)
Finally, I've uploaded the train_data_wav (sr=48000) and csv that I used.
\nhttps://www.kaggle.com/shinoda18/rainforest-data
Thank you for opening this competition
\nAlso the notebooks and discussions have helped me a lot, thank you all!
My solution was similar to Beluga & Peter in 7th Place.
\n@https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220443.
[hand labels]
\n While observing the data from t_min and t_max in the given train__tp.csv, I found that there are many kinds of bird calls mixed together.
\n So I decided to treat it as a multi-class multi-label problem.
\n It was also mentioned in the discussion that the test labels were eventually carefully labeled by humans.
\n The TPs in the given t_min, t_max range are all easy to understand, but there are many TPs in the 60s clip that are difficult to understand and not labeled.
\n I thought it would be better to label them carefully by myself to make the condition as close to test as possible in case such incomprehensible calls are also labeled in test.
\nAnd I was thinking of doing Pseudo labeling after the accuracy of the model improves.
I trimmed the 5s~ around t_min and t_max in train__tp.csv.
\n Hand labels took about a week.
\n As a result, a total of 2428 clips and 5s chunks were used as train_data.
\n The distribution of the train_data classes looks like this
\n(I couldn't upload the image, so I'll post it later)
class nb
\ns3 1257
\ns12 520
\ns18 512
\n:
\ns16 100
\ns17 100
\ns6 97
I can see that there is a label imbalance, especially for s3, s12, and s18, because their labels co-occur among the other classes of clips.
\nIn particular, s3 is dominant, so it tends to output high probability, while a few classes output low probability, so i thought this is a bad problem for this evaluation index.
\nTherefore, in order to achieve a more balanced distribution, I oversampled the minority classes and undersampled the majority classes.
\n However, the LB became worse.
\n Looking back, I didn't think of approaching the test distribution, as Chris pointed out.
\nhttps://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220389
Eventually, i ensembled 15 models with various losses(FocalLoss,FbetaLoss+FocalLoss,ResampleLoss).
\n[trainning]
\nexample single model
\nPANNsDense161 (public_LB 0.95548, private_LB 0.96300)
I also tried EfficientNet_b0, Dense121, etc., but Dense161 worked well.
\ntrain_data(sr=48000,5s)
\n window_size=2048,hop_size=512,mel_bins=256
\n MultilabelStratifiedKFold 5fold
\n BCEFocalLoss(α=0.25,β=2)
\n GradualWarmupScheduler,CosineAnnealingLR(lr = 0.001,multiplier=10,epo35)
Augmentation
\n GaussianNoise(p=0.5)
\n GaussianSNR(p=0.5)
\n FrequencyMask(min_frequency_band=0.0, max_frequency_band=0.2, p=0.3)
\n TimeMask(min_band_part=0.0, max_band_part=0.2, p=0.8)
\n PitchShift(min_semitones=-0.5, max_semitones=0.5, p=0.1)
\n Shift(p=0.1)
\n Gain(p=0.2)
[inference]
\n stride=1
\n framewise_output max
\n No TTA (I used it in the final ensemble model)
Finally, I've uploaded the train_data_wav (sr=48000) and csv that I used.
\nhttps://www.kaggle.com/shinoda18/rainforest-data
a strong effort! a solo Gold… I think u and Chris were the only solos on the Top 10 list. Brilliant!
", "rawMarkdown": "a strong effort! a solo Gold... I think u and Chris were the only solos on the Top 10 list. Brilliant!", "votes": null }, { "id": "1212172", "postDate": "02/21/2021 00:44:53", "content": "Thank you!
\nI saw your discussion.
\nGreat insights and hard work!
\nI'll see you at another convention somewhere!
\"In my case, Pseudo labeling did not work well, so I did not use it.\"
\nsince you have both hand label and pseudo label, I suggest you can do this:
\nif you can find out the reasons, why it doesn't work and then make pseudo label works, I think it is useful for future competition.
\nthe issue here is I think pseudo label is not as good as your hand label. But we want to ask the question:
\nI am always amazed at your discussions and solutions!
\nYou are one of the kaggler I personally respect!
Sorry, there is a mistake in my text!
\nI tried the hand label=> Pseudo label
\nI did not try Pseudo label from the beginning.
\n\nhow much difference is there between the 2 set?
\n
So I don't have a set of pseudo labels with me.
\nI was not able to compare them.
\nI can't say that the pseudo labels did not work. I will fix it.
\n\ncan I modify so that the 2 distribution are more similar?
\n
I've looked at all the train_data.
\ns3,s7,s11,s12,s18 seem to be particularly frequent.
\nIn Chris's notebook.
\nhttps://www.kaggle.com/cdeotte/rainforest-post-process-lb-0-970
\n[113,204,44,923,53,41,3,213,44,23,26,149,255,14,123,222,46,6,474,4,17,18,23,72].
\nThe distribution is similar to
If we do a Pseudo label using all the train_data, we may get a distribution exactly like test_data.
\nThe hand label=> Pseudo label I tried mainly upsampled the tail class to eliminate the class imbalance, so it did not work well because it was far from the test_data distribution.
", "rawMarkdown": "I am always amazed at your discussions and solutions!\nYou are one of the kaggler I personally respect!\n\nSorry, there is a mistake in my text!\nI tried the hand label=> Pseudo label\nI did not try Pseudo label from the beginning.\n\n> how much difference is there between the 2 set?\n\nSo I don't have a set of pseudo labels with me.\nI was not able to compare them.\nI can't say that the pseudo labels did not work. I will fix it.\n\n> can I modify so that the 2 distribution are more similar?\n\nI've looked at all the train_data.\ns3,s7,s11,s12,s18 seem to be particularly frequent.\nIn Chris's notebook.\nhttps://www.kaggle.com/cdeotte/rainforest-post-process-lb-0-970\n[113,204,44,923,53,41,3,213,44,23,26,149,255,14,123,222,46,6,474,4,17,18,23,72].\nThe distribution is similar to\n\nIf we do a Pseudo label using all the train_data, we may get a distribution exactly like test_data.\n\n\nThe hand label=> Pseudo label I tried mainly upsampled the tail class to eliminate the class imbalance, so it did not work well because it was far from the test_data distribution.", "votes": null } ], "comments": [ { "id": 1211942, "author_name": "allohvk", "author_url": "", "post_date": "02/20/2021 17:23:22", "content": "a strong effort! a solo Gold… I think u and Chris were the only solos on the Top 10 list. Brilliant!
", "votes": null, "replies": [ { "id": 1212172, "author_name": "shinoda18", "author_url": "", "post_date": "02/21/2021 00:44:53", "content": "Thank you!
\nI saw your discussion.
\nGreat insights and hard work!
\nI'll see you at another convention somewhere!
\"In my case, Pseudo labeling did not work well, so I did not use it.\"
\nsince you have both hand label and pseudo label, I suggest you can do this:
\nif you can find out the reasons, why it doesn't work and then make pseudo label works, I think it is useful for future competition.
\nthe issue here is I think pseudo label is not as good as your hand label. But we want to ask the question:
\nI am always amazed at your discussions and solutions!
\nYou are one of the kaggler I personally respect!
Sorry, there is a mistake in my text!
\nI tried the hand label=> Pseudo label
\nI did not try Pseudo label from the beginning.
\n\nhow much difference is there between the 2 set?
\n
So I don't have a set of pseudo labels with me.
\nI was not able to compare them.
\nI can't say that the pseudo labels did not work. I will fix it.
\n\ncan I modify so that the 2 distribution are more similar?
\n
I've looked at all the train_data.
\ns3,s7,s11,s12,s18 seem to be particularly frequent.
\nIn Chris's notebook.
\nhttps://www.kaggle.com/cdeotte/rainforest-post-process-lb-0-970
\n[113,204,44,923,53,41,3,213,44,23,26,149,255,14,123,222,46,6,474,4,17,18,23,72].
\nThe distribution is similar to
If we do a Pseudo label using all the train_data, we may get a distribution exactly like test_data.
\nThe hand label=> Pseudo label I tried mainly upsampled the tail class to eliminate the class imbalance, so it did not work well because it was far from the test_data distribution.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1210666": "Thank you for opening this competition\nAlso the notebooks and discussions have helped me a lot, thank you all!\n\nMy solution was similar to Beluga & Peter in 7th Place.\n@https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220443.\n- Multi-class multi-label problem\n- Data cleaning train_data with hand labels\n- SED model\n\n~~In my case, Pseudo labeling did not work well, so I did not use it.~~\n\n**[hand labels]**\n While observing the data from t_min and t_max in the given train__tp.csv, I found that there are many kinds of bird calls mixed together.\n So I decided to treat it as a multi-class multi-label problem.\n It was also mentioned in the discussion that the test labels were eventually carefully labeled by humans.\n The TPs in the given t_min, t_max range are all easy to understand, but there are many TPs in the 60s clip that are difficult to understand and not labeled.\n I thought it would be better to label them carefully by myself to make the condition as close to test as possible in case such incomprehensible calls are also labeled in test.\nAnd I was thinking of doing Pseudo labeling after the accuracy of the model improves.\n\n I trimmed the 5s~ around t_min and t_max in train__tp.csv.\n Hand labels took about a week.\n As a result, a total of 2428 clips and 5s chunks were used as train_data.\n The distribution of the train_data classes looks like this\n(I couldn't upload the image, so I'll post it later)\n\nclass nb\ns3 1257\ns12 520\ns18 512\n:\ns16 100\ns17 100\ns6 97\n\n I can see that there is a label imbalance, especially for s3, s12, and s18, because their labels co-occur among the other classes of clips.\n\n In particular, s3 is dominant, so it tends to output high probability, while a few classes output low probability, so i thought this is a bad problem for this evaluation index.\nTherefore, in order to achieve a more balanced distribution, I oversampled the minority classes and undersampled the majority classes.\n However, the LB became worse.\n Looking back, I didn't think of approaching the test distribution, as Chris pointed out.\nhttps://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220389\n\nEventually, i ensembled 15 models with various losses(FocalLoss,FbetaLoss+FocalLoss,ResampleLoss).\n\n**[trainning]**\nexample single model \nPANNsDense161 (public_LB 0.95548, private_LB 0.96300)\n\nI also tried EfficientNet_b0, Dense121, etc., but Dense161 worked well.\n\n train_data(sr=48000,5s)\n window_size=2048,hop_size=512,mel_bins=256\n MultilabelStratifiedKFold 5fold\n BCEFocalLoss(α=0.25,β=2)\n GradualWarmupScheduler,CosineAnnealingLR(lr = 0.001,multiplier=10,epo35)\n\n**Augmentation**\n GaussianNoise(p=0.5)\n GaussianSNR(p=0.5)\n FrequencyMask(min_frequency_band=0.0, max_frequency_band=0.2, p=0.3)\n TimeMask(min_band_part=0.0, max_band_part=0.2, p=0.8)\n PitchShift(min_semitones=-0.5, max_semitones=0.5, p=0.1)\n Shift(p=0.1)\n Gain(p=0.2)\n\n**[inference]**\n stride=1\n framewise_output max\n No TTA (I used it in the final ensemble model)\n\nFinally, I've uploaded the train_data_wav (sr=48000) and csv that I used. \nhttps://www.kaggle.com/shinoda18/rainforest-data", "1211942": "a strong effort! a solo Gold... I think u and Chris were the only solos on the Top 10 list. Brilliant!", "1212172": "Thank you!\nI saw your discussion.\nGreat insights and hard work!\nI'll see you at another convention somewhere!", "1212175": "\"In my case, Pseudo labeling did not work well, so I did not use it.\"\n\nsince you have both hand label and pseudo label, I suggest you can do this:\n1. measure label purity of pseudo label, based on your hand label\n2. train with a subset of the purer pseudo label and see if results improve.\n\nif you can find out the reasons, why it doesn't work and then make pseudo label works, I think it is useful for future competition.\n\nthe issue here is I think pseudo label is not as good as your hand label. But we want to ask the question:\n- how much difference is there between the 2 set?\n- can I modify so that the 2 distribution are more similar?", "1212201": "I am always amazed at your discussions and solutions!\nYou are one of the kaggler I personally respect!\n\nSorry, there is a mistake in my text!\nI tried the hand label=> Pseudo label\nI did not try Pseudo label from the beginning.\n\n> how much difference is there between the 2 set?\n\nSo I don't have a set of pseudo labels with me.\nI was not able to compare them.\nI can't say that the pseudo labels did not work. I will fix it.\n\n> can I modify so that the 2 distribution are more similar?\n\nI've looked at all the train_data.\ns3,s7,s11,s12,s18 seem to be particularly frequent.\nIn Chris's notebook.\nhttps://www.kaggle.com/cdeotte/rainforest-post-process-lb-0-970\n[113,204,44,923,53,41,3,213,44,23,26,149,255,14,123,222,46,6,474,4,17,18,23,72].\nThe distribution is similar to\n\nIf we do a Pseudo label using all the train_data, we may get a distribution exactly like test_data.\n\n\nThe hand label=> Pseudo label I tried mainly upsampled the tail class to eliminate the class imbalance, so it did not work well because it was far from the test_data distribution." }, "source": "meta" }