It was a great pleasure for me to compete in the latest Kaggle audio detection competition which is not too different from this one: definitely, audio detection is now drawing more attention on Kaggle than ever.
\nBased on my last experience, I will be sharing some common techniques that could be used to deal with audio data classification/labeling.
\nThe first idea could be to treat the audio records as simple time series. Hence, any recurrent NN could be used (LSTM, GRU, TCN, …). That could be a great way even if many SOTA audio detection/classification models don't go that way
\nThe most used approach when dealing with audio data is this one. The waveforms are transformed into images using, for example MFCCs. Those images are fed to a computer vision model (EfficientNet, ResNet, ResNext …). Hence, audio detection can fully leverage all the advancements in computer vision.
\nHopefully, I will be sharing some kernels here whenever I got some time off.
\nAs promised, I've released my first model. It's clearly a more interesting baseline for next RFCX audio detection models (I'm sure there will be plenty of models since the dataset seems very clean and not too big)
\n\n", "messages": [ { "id": "1082432", "postDate": "11/17/2020 22:11:21", "content": "To be continued
\n
It was a great pleasure for me to compete in the latest Kaggle audio detection competition which is not too different from this one: definitely, audio detection is now drawing more attention on Kaggle than ever.
\nBased on my last experience, I will be sharing some common techniques that could be used to deal with audio data classification/labeling.
\nThe first idea could be to treat the audio records as simple time series. Hence, any recurrent NN could be used (LSTM, GRU, TCN, …). That could be a great way even if many SOTA audio detection/classification models don't go that way
\nThe most used approach when dealing with audio data is this one. The waveforms are transformed into images using, for example MFCCs. Those images are fed to a computer vision model (EfficientNet, ResNet, ResNext …). Hence, audio detection can fully leverage all the advancements in computer vision.
\nHopefully, I will be sharing some kernels here whenever I got some time off.
\nAs promised, I've released my first model. It's clearly a more interesting baseline for next RFCX audio detection models (I'm sure there will be plenty of models since the dataset seems very clean and not too big)
\n\n", "rawMarkdown": "It was a great pleasure for me to compete in [the latest Kaggle audio detection competition](https://www.kaggle.com/c/birdsong-recognition) which is not too different from this one: definitely, audio detection is now drawing more attention on Kaggle than ever.\n\n Based on my last experience, I will be sharing some common techniques that could be used to deal with audio data classification/labeling.\n\n## Recurrent NN models\nThe first idea could be to treat the audio records as simple time series. Hence, any recurrent NN could be used (LSTM, GRU, TCN, ...). That could be a great way even if many SOTA audio detection/classification models don't go that way\n\n## From audio to image : audio detection as computer vision problem\nThe most used approach when dealing with audio data is this one. The waveforms are transformed into images using, for example MFCCs. Those images are fed to a computer vision model (EfficientNet, ResNet, ResNext ...). Hence, audio detection can fully leverage all the advancements in computer vision.\n\nHopefully, I will be sharing some kernels here whenever I got some time off.\n\n\n## # Update 1\nAs promised, I've [released my first model](https://www.kaggle.com/kneroma/inference-resnest-rfcx-audio-detection). It's clearly a more interesting baseline for next RFCX audio detection models (I'm sure there will be plenty of models since the dataset seems very clean and not too big)\n\n\n> To be continued", "votes": null }, { "id": "1091288", "postDate": "11/25/2020 23:10:09", "content": "To be continued
\n
thanks for your sharing! I have a basic question about how the problem is framed and its target. I saw in your kernel that seems like you were training on a random crop of an audio clip, but in train_tp.csv it shows the target is only present in a period of time i.e. t_min and t_max, so wouldn't a random cropping likely miss this period of clip and thus the target should be null or something? Couldn't we just feed the entire audio clip to cnn as input instead of small random crops?
We could feed the entire audio for sure. It's up to you :) .
\nBut, if you're cropping, its means that the target will depend on the cropped chunk. For example, if you crop a chunk which overlap with zero (t_min, t_max), all 24 targets must be set to zeros.
I see… Just want to confirm, in your shared notebook I'm a bit confused that you set the specie_id to empty list: data[\"species_id\"] = [[] for _ in range(len(data))] but in your dataset you get the target by setting the index of the class (the specie id) to 1, is this because the kernel is for inference, and for training we should match the cropped chunk to the target period accordingly based on train_tp?
Yes, It's because It's inference. I need not the species IDs (we're supposed to predict them for the test set), I just put them there to make my code work.
", "rawMarkdown": "Yes, It's because It's inference. I need not the species IDs (we're supposed to predict them for the test set), I just put them there to make my code work.", "votes": null }, { "id": "1105535", "postDate": "12/08/2020 01:27:46", "content": "@kneroma thanks for sharing! I got a question similar to @samshipengs. Let's say we take advantage of t_min and t_max to better detect species rather than using random crops, for example, by using small time windows of 10 seconds which start at t_min and contain t_max.
Then when it comes to the test set which has audios of 60 seconds with no t_min nor _t_max information, should we randomly take 10 second clips and test on those? Should we take 6 audio samples of 10 seconds each and average predictions for each sample? How do you tackle this issue?
I posted a similar question here
\nThanks in advance!
", "rawMarkdown": "kneroma thanks for sharing! I got a question similar to @samshipengs. Let's say we take advantage of `t_min` and `t_max` to better detect species rather than using random crops, for example, by using small time windows of 10 seconds which start at `t_min` and contain `t_max`. \n\nThen when it comes to the test set which has audios of 60 seconds with no `t_min` nor `_t_max` information, should we randomly take 10 second clips and test on those? Should we take 6 audio samples of 10 seconds each and average predictions for each sample? How do you tackle this issue?\n\nI posted a similar question [here](https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/201827)\n\nThanks in advance!", "votes": null } ], "comments": [ { "id": 1091288, "author_name": "samshipengs", "author_url": "", "post_date": "11/25/2020 23:10:09", "content": "thanks for your sharing! I have a basic question about how the problem is framed and its target. I saw in your kernel that seems like you were training on a random crop of an audio clip, but in train_tp.csv it shows the target is only present in a period of time i.e. t_min and t_max, so wouldn't a random cropping likely miss this period of clip and thus the target should be null or something? Couldn't we just feed the entire audio clip to cnn as input instead of small random crops?
We could feed the entire audio for sure. It's up to you :) .
\nBut, if you're cropping, its means that the target will depend on the cropped chunk. For example, if you crop a chunk which overlap with zero (t_min, t_max), all 24 targets must be set to zeros.
I see… Just want to confirm, in your shared notebook I'm a bit confused that you set the specie_id to empty list: data[\"species_id\"] = [[] for _ in range(len(data))] but in your dataset you get the target by setting the index of the class (the specie id) to 1, is this because the kernel is for inference, and for training we should match the cropped chunk to the target period accordingly based on train_tp?
Yes, It's because It's inference. I need not the species IDs (we're supposed to predict them for the test set), I just put them there to make my code work.
", "votes": null, "replies": [] } ] }, { "id": 1105535, "author_name": "alejopaullier", "author_url": "", "post_date": "12/08/2020 01:27:46", "content": "@kneroma thanks for sharing! I got a question similar to @samshipengs. Let's say we take advantage of t_min and t_max to better detect species rather than using random crops, for example, by using small time windows of 10 seconds which start at t_min and contain t_max.
Then when it comes to the test set which has audios of 60 seconds with no t_min nor _t_max information, should we randomly take 10 second clips and test on those? Should we take 6 audio samples of 10 seconds each and average predictions for each sample? How do you tackle this issue?
I posted a similar question here
\nThanks in advance!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1082432": "It was a great pleasure for me to compete in [the latest Kaggle audio detection competition](https://www.kaggle.com/c/birdsong-recognition) which is not too different from this one: definitely, audio detection is now drawing more attention on Kaggle than ever.\n\n Based on my last experience, I will be sharing some common techniques that could be used to deal with audio data classification/labeling.\n\n## Recurrent NN models\nThe first idea could be to treat the audio records as simple time series. Hence, any recurrent NN could be used (LSTM, GRU, TCN, ...). That could be a great way even if many SOTA audio detection/classification models don't go that way\n\n## From audio to image : audio detection as computer vision problem\nThe most used approach when dealing with audio data is this one. The waveforms are transformed into images using, for example MFCCs. Those images are fed to a computer vision model (EfficientNet, ResNet, ResNext ...). Hence, audio detection can fully leverage all the advancements in computer vision.\n\nHopefully, I will be sharing some kernels here whenever I got some time off.\n\n\n## # Update 1\nAs promised, I've [released my first model](https://www.kaggle.com/kneroma/inference-resnest-rfcx-audio-detection). It's clearly a more interesting baseline for next RFCX audio detection models (I'm sure there will be plenty of models since the dataset seems very clean and not too big)\n\n\n> To be continued", "1091288": "thanks for your sharing! I have a basic question about how the problem is framed and its target. I saw in your kernel that seems like you were training on a random crop of an audio clip, but in `train_tp.csv` it shows the target is only present in a period of time i.e. `t_min` and `t_max`, so wouldn't a random cropping likely miss this period of clip and thus the target should be null or something? Couldn't we just feed the entire audio clip to cnn as input instead of small random crops?", "1091313": "We could feed the entire audio for sure. It's up to you :) .\nBut, if you're cropping, its means that the target will depend on the cropped chunk. For example, if you crop a chunk which overlap with zero (t_min, t_max), all 24 targets must be set to zeros.", "1091373": "I see... Just want to confirm, in your shared notebook I'm a bit confused that you set the specie_id to empty list: `data[\"species_id\"] = [[] for _ in range(len(data))]` but in your dataset you get the target by setting the index of the class (the specie id) to 1, is this because the kernel is for inference, and for training we should match the cropped chunk to the target period accordingly based on `train_tp`?", "1091410": "Yes, It's because It's inference. I need not the species IDs (we're supposed to predict them for the test set), I just put them there to make my code work.", "1105535": "kneroma thanks for sharing! I got a question similar to @samshipengs. Let's say we take advantage of `t_min` and `t_max` to better detect species rather than using random crops, for example, by using small time windows of 10 seconds which start at `t_min` and contain `t_max`. \n\nThen when it comes to the test set which has audios of 60 seconds with no `t_min` nor `_t_max` information, should we randomly take 10 second clips and test on those? Should we take 6 audio samples of 10 seconds each and average predictions for each sample? How do you tackle this issue?\n\nI posted a similar question [here](https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/201827)\n\nThanks in advance!" }, "source": "meta" }