Training with submission takes ~ 1hr 5 minutes. I train a pretrained resnet34 on precalculated and uploaded spectrograms (I haven't tried but I assume this offers a significant speedup vs converting audio to spectrograms in training as Kaggle kernels come with only 2 CPU cores). In order to meet the single dataset limit of 20 GB, I limit the spectrograms to only the first 25 seconds or so of each audio file.
\n\nThere are ways you could probably improve on this result, for instance via training longer or training a bigger model.
\n\nTruth be told, I was hoping this notebook would perform better. Unfortunately, I am a little bit lost in this competition. Using xenocanto recordings as validation set doesn't work that well - local validation scores even on as much as 20% of recordings stratified by class do not track the results on public LB too well (wondering if others are also experiencing this?)
\n\nUsing soundscape recordings shared in the External Data/Pre-Trained Models Disclosure Thread also doesn't work very well as validation.
\n\nThere is a set of methods that could be implemented if we had access to the test set (such as the really cool arch here, you can find my implementation here - didn't manage to get it to work, potentially due the soundscape recordings I used not being representative of the test data or for some other reason).
\n\nDespite all that, I am assuming our only hope is to make the representation of data between train and test as similar as possible. Not sure exactly how to address this but will keep trying 🙂
", "messages": [ { "id": 927421, "postDate": "2020-07-13T11:20:41.180Z", "content": "\n\nTraining with submission takes ~ 1hr 5 minutes. I train a pretrained resnet34 on precalculated and uploaded spectrograms (I haven't tried but I assume this offers a significant speedup vs converting audio to spectrograms in training as Kaggle kernels come with only 2 CPU cores). In order to meet the single dataset limit of 20 GB, I limit the spectrograms to only the first 25 seconds or so of each audio file.
\n\nThere are ways you could probably improve on this result, for instance via training longer or training a bigger model.
\n\nTruth be told, I was hoping this notebook would perform better. Unfortunately, I am a little bit lost in this competition. Using xenocanto recordings as validation set doesn't work that well - local validation scores even on as much as 20% of recordings stratified by class do not track the results on public LB too well (wondering if others are also experiencing this?)
\n\nUsing soundscape recordings shared in the External Data/Pre-Trained Models Disclosure Thread also doesn't work very well as validation.
\n\nThere is a set of methods that could be implemented if we had access to the test set (such as the really cool arch here, you can find my implementation here - didn't manage to get it to work, potentially due the soundscape recordings I used not being representative of the test data or for some other reason).
\n\nDespite all that, I am assuming our only hope is to make the representation of data between train and test as similar as possible. Not sure exactly how to address this but will keep trying 🙂
", "rawMarkdown": "[Link to Starter Pack](https://www.kaggle.com/radek1/esp-starter-pack-from-training-to-submission)\n\nTraining with submission takes ~ 1hr 5 minutes. I train a pretrained resnet34 on [precalculated and uploaded spectrograms](https://www.kaggle.com/radek1/esp-starter-pack-from-training-to-submission) (I haven't tried but I assume this offers a significant speedup vs converting audio to spectrograms in training as Kaggle kernels come with only 2 CPU cores). In order to meet the single dataset limit of 20 GB, I limit the spectrograms to only the first 25 seconds or so of each audio file.\n\nThere are ways you could probably improve on this result, for instance via training longer or training a bigger model.\n\nTruth be told, I was hoping this notebook would perform better. Unfortunately, I am a little bit lost in this competition. Using xenocanto recordings as validation set doesn't work that well - local validation scores even on as much as 20% of recordings stratified by class do not track the results on public LB too well (wondering if others are also experiencing this?)\n\nUsing soundscape recordings shared in the [External Data/Pre-Trained Models Disclosure Thread](https://www.kaggle.com/c/birdsong-recognition/discussion/158877) also doesn't work very well as validation.\n\nThere is a set of methods that could be implemented if we had access to the test set (such as the really cool arch [here](https://arxiv.org/abs/1409.7495), you can find my implementation [here](https://github.com/earthspecies/birdcall/blob/master/02k_train_on_melspectrograms_pytorch_lme_pool_all_classes_simple_minmax_domain_adaptation.ipynb) - didn't manage to get it to work, potentially due the soundscape recordings I used not being representative of the test data or for some other reason). \n\nDespite all that, I am assuming our only hope is to make the representation of data between train and test as similar as possible. Not sure exactly how to address this but will keep trying 🙂", "votes": 14 } ], "comments": [], "raw_markdown_by_id": { "927421": "[Link to Starter Pack](https://www.kaggle.com/radek1/esp-starter-pack-from-training-to-submission)\n\nTraining with submission takes ~ 1hr 5 minutes. I train a pretrained resnet34 on [precalculated and uploaded spectrograms](https://www.kaggle.com/radek1/esp-starter-pack-from-training-to-submission) (I haven't tried but I assume this offers a significant speedup vs converting audio to spectrograms in training as Kaggle kernels come with only 2 CPU cores). In order to meet the single dataset limit of 20 GB, I limit the spectrograms to only the first 25 seconds or so of each audio file.\n\nThere are ways you could probably improve on this result, for instance via training longer or training a bigger model.\n\nTruth be told, I was hoping this notebook would perform better. Unfortunately, I am a little bit lost in this competition. Using xenocanto recordings as validation set doesn't work that well - local validation scores even on as much as 20% of recordings stratified by class do not track the results on public LB too well (wondering if others are also experiencing this?)\n\nUsing soundscape recordings shared in the [External Data/Pre-Trained Models Disclosure Thread](https://www.kaggle.com/c/birdsong-recognition/discussion/158877) also doesn't work very well as validation.\n\nThere is a set of methods that could be implemented if we had access to the test set (such as the really cool arch [here](https://arxiv.org/abs/1409.7495), you can find my implementation [here](https://github.com/earthspecies/birdcall/blob/master/02k_train_on_melspectrograms_pytorch_lme_pool_all_classes_simple_minmax_domain_adaptation.ipynb) - didn't manage to get it to work, potentially due the soundscape recordings I used not being representative of the test data or for some other reason). \n\nDespite all that, I am assuming our only hope is to make the representation of data between train and test as similar as possible. Not sure exactly how to address this but will keep trying 🙂" } }