If you want to save time during training and inference, it is a good thing to precompute the spectrograms and save them in your disk. You can easily get a huge (x1000) speed-up on your loading times by doing so.
\nI've made public a script that precomputes spectrograms. Generation is done with multi-processing and takes about 1h30 for the train data and 45min for the test data with n_mels=128:
\n \n\n
You can change the parameters and re-generate the data if you feel like it.
\nHere are the associated datasets :
\n\n\nTest data : https://www.kaggle.com/theoviel/rcfx-test-spectrograms
\n
\n Train data : https://www.kaggle.com/theoviel/rcfx-train-spectrograms
\n\nTest + Train data : https://www.kaggle.com/theoviel/rcfx-spectrograms-32-khz
\n
\n\nTest + Train data : https://www.kaggle.com/theoviel/rcfx-spectrograms-32-khz-64-mels
\n
Please let me know if there are any issues with the code.
", "messages": [ { "id": 1083903, "postDate": "2020-11-19T14:14:39.283Z", "content": "If you want to save time during training and inference, it is a good thing to precompute the spectrograms and save them in your disk. You can easily get a huge (x1000) speed-up on your loading times by doing so.
\nI've made public a script that precomputes spectrograms. Generation is done with multi-processing and takes about 1h30 for the train data and 45min for the test data with n_mels=128:
\n \n\n
You can change the parameters and re-generate the data if you feel like it.
\nHere are the associated datasets :
\n\n\nTest data : https://www.kaggle.com/theoviel/rcfx-test-spectrograms
\n
\n Train data : https://www.kaggle.com/theoviel/rcfx-train-spectrograms
\n\nTest + Train data : https://www.kaggle.com/theoviel/rcfx-spectrograms-32-khz
\n
\n\nTest + Train data : https://www.kaggle.com/theoviel/rcfx-spectrograms-32-khz-64-mels
\n
Please let me know if there are any issues with the code.
", "rawMarkdown": "If you want to save time during training and inference, it is a good thing to precompute the spectrograms and save them in your disk. You can easily get a huge (x1000) speed-up on your loading times by doing so.\n\n\nI've made public a script that precomputes spectrograms. Generation is done with multi-processing and takes about 1h30 for the train data and 45min for the test data with `n_mels=128`: \n> https://www.kaggle.com/theoviel/spectrogram-generation\n\nYou can change the parameters and re-generate the data if you feel like it.\n\nHere are the associated datasets :\n\n- 16 kHz, 128 mels (as mentioned by @cpmpml 16 kHz is not a great choice, using 32 kHz should be better)\n> Test data : https://www.kaggle.com/theoviel/rcfx-test-spectrograms\n> Train data : https://www.kaggle.com/theoviel/rcfx-train-spectrograms\n\n- 32 kHz, 128 mels\n> Test + Train data : https://www.kaggle.com/theoviel/rcfx-spectrograms-32-khz\n\n- 32 kHz, 64 mels\n> Test + Train data : https://www.kaggle.com/theoviel/rcfx-spectrograms-32-khz-64-mels\n\n\nPlease let me know if there are any issues with the code.", "votes": 57 }, { "id": 1167563, "postDate": "2021-01-24T11:04:44.660Z", "content": "You know that sampling at 16kHz removes frequencies above 8kHz right? This means that some species are much harder to detect.
", "rawMarkdown": "You know that sampling at 16kHz removes frequencies above 8kHz right? This means that some species are much harder to detect.", "votes": 4, "replies": [ { "id": 1167884, "postDate": "2021-01-24T14:46:31.880Z", "content": "Yup, it doesn't really make sense to use a sampling rate of 16kHz but for some reason that's the one I used originally.
", "rawMarkdown": "Yup, it doesn't really make sense to use a sampling rate of 16kHz but for some reason that's the one I used originally.", "votes": 2 }, { "id": 1167954, "postDate": "2021-01-24T15:52:27.557Z", "content": "Good for you, but not for all those who read your post ;) I preferred to warn them.
", "rawMarkdown": "Good for you, but not for all those who read your post ;) I preferred to warn them.", "votes": 4 }, { "id": 1167976, "postDate": "2021-01-24T16:10:06.083Z", "content": "True, I'll update the post
", "rawMarkdown": "True, I'll update the post", "votes": 3 } ] }, { "id": 1165768, "postDate": "2021-01-23T08:21:04.027Z", "content": "I was looking for such a script and datasets. Thanks for sharing!
", "rawMarkdown": "I was looking for such a script and datasets. Thanks for sharing!" }, { "id": 1093334, "postDate": "2020-11-27T16:11:15.903Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1084216, "postDate": "2020-11-19T20:48:43.420Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1102034, "postDate": "2020-12-04T13:56:01.620Z", "content": "Thanks for this tip :)
", "rawMarkdown": "Thanks for this tip :)", "votes": 1 } ], "comments": [ { "id": 1167563, "author_name": "CPMP", "author_url": "", "post_date": "2021-01-24T11:04:44.660000", "content": "You know that sampling at 16kHz removes frequencies above 8kHz right? This means that some species are much harder to detect.
", "votes": 4, "replies": [ { "id": 1167884, "author_name": "Theo Viel", "author_url": "", "post_date": "2021-01-24T14:46:31.880000", "content": "Yup, it doesn't really make sense to use a sampling rate of 16kHz but for some reason that's the one I used originally.
", "votes": 2, "replies": [] }, { "id": 1167954, "author_name": "CPMP", "author_url": "", "post_date": "2021-01-24T15:52:27.557000", "content": "Good for you, but not for all those who read your post ;) I preferred to warn them.
", "votes": 4, "replies": [] }, { "id": 1167976, "author_name": "Theo Viel", "author_url": "", "post_date": "2021-01-24T16:10:06.083000", "content": "True, I'll update the post
", "votes": 3, "replies": [] } ] }, { "id": 1165768, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2021-01-23T08:21:04.027000", "content": "I was looking for such a script and datasets. Thanks for sharing!
", "votes": 0, "replies": [] }, { "id": 1093334, "author_name": "", "author_url": "", "post_date": "2020-11-27T16:11:15.903000", "content": "", "votes": 0, "replies": [] }, { "id": 1084216, "author_name": "", "author_url": "", "post_date": "2020-11-19T20:48:43.420000", "content": "", "votes": 0, "replies": [] }, { "id": 1102034, "author_name": "Buffalo Spdwy", "author_url": "", "post_date": "2020-12-04T13:56:01.620000", "content": "Thanks for this tip :)
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "1083903": "If you want to save time during training and inference, it is a good thing to precompute the spectrograms and save them in your disk. You can easily get a huge (x1000) speed-up on your loading times by doing so.\n\n\nI've made public a script that precomputes spectrograms. Generation is done with multi-processing and takes about 1h30 for the train data and 45min for the test data with `n_mels=128`: \n> https://www.kaggle.com/theoviel/spectrogram-generation\n\nYou can change the parameters and re-generate the data if you feel like it.\n\nHere are the associated datasets :\n\n- 16 kHz, 128 mels (as mentioned by @cpmpml 16 kHz is not a great choice, using 32 kHz should be better)\n> Test data : https://www.kaggle.com/theoviel/rcfx-test-spectrograms\n> Train data : https://www.kaggle.com/theoviel/rcfx-train-spectrograms\n\n- 32 kHz, 128 mels\n> Test + Train data : https://www.kaggle.com/theoviel/rcfx-spectrograms-32-khz\n\n- 32 kHz, 64 mels\n> Test + Train data : https://www.kaggle.com/theoviel/rcfx-spectrograms-32-khz-64-mels\n\n\nPlease let me know if there are any issues with the code.", "1167563": "You know that sampling at 16kHz removes frequencies above 8kHz right? This means that some species are much harder to detect.", "1165768": "I was looking for such a script and datasets. Thanks for sharing!", "1093334": "", "1084216": "", "1102034": "Thanks for this tip :)" } }