{ "id": 450606, "title": "🥈24th in two weeks and `No space left on device`!", "url": "/competitions/bengaliai-speech/writeups/bayartsogt-yadamsuren-24th-in-two-weeks-and-no-spa", "author_name": "", "post_date": "2023-10-25T01:56:29.600493800Z", "votes": 10, "comment_count": 2, "views": 0, "content": "

First of all, THANK YOU ALL!

\n

As a late joiner, It was so helpful to read insightful discussions of @imtiazprio @reasat @tugstugi @hengck23 @mbmmurad and list goes on!

\n

Approach

\n\n

Important lesson for future me!

\n\n

Guilt of Overfitting to LB!

\n

Because test data (Out of Distribution) data was so different from train datasets, it was really about overfitting to public leaderboard.

\n
!kaggle competitions submissions -v bengaliai-speech >> ./bengali-speech-submissions.csv\ndf = pd.read_csv()\nnp.corrcoef(df.publicScore, df.privateScore)[, ]\n\n...\nsns.lineplot(df, y=, x=, hue=)\n
\n

\"\"

\n

In the End

\n

It is all about learning!
\nEven though it is always so frustrating to feel you were so close or so much could have done or should have done, I appreciate this learning path and that's why I joined to Kaggle in the first place! 🫡

", "messages": [ { "id": "2497907", "postDate": "10/25/2023 01:56:29", "content": "

First of all, THANK YOU ALL!

\n

As a late joiner, It was so helpful to read insightful discussions of @imtiazprio @reasat @tugstugi @hengck23 @mbmmurad and list goes on!

\n

Approach

\n\n

Important lesson for future me!

\n\n

Guilt of Overfitting to LB!

\n

Because test data (Out of Distribution) data was so different from train datasets, it was really about overfitting to public leaderboard.

\n
!kaggle competitions submissions -v bengaliai-speech >> ./bengali-speech-submissions.csv\ndf = pd.read_csv()\nnp.corrcoef(df.publicScore, df.privateScore)[, ]\n\n...\nsns.lineplot(df, y=, x=, hue=)\n
\n

\"\"

\n

In the End

\n

It is all about learning!
\nEven though it is always so frustrating to feel you were so close or so much could have done or should have done, I appreciate this learning path and that's why I joined to Kaggle in the first place! 🫡

", "rawMarkdown": "## First of all, THANK YOU ALL!\nAs a late joiner, It was so helpful to read insightful discussions of @imtiazprio @reasat @tugstugi @hengck23 @mbmmurad and list goes on!\n\n## Approach\n* Code: https://github.com/bayartsogt-ya/bengali-speech-2023\n* Inference: https://www.kaggle.com/code/bayartsogtya/submit-to-restore-punctuation/notebook\n* Backbone Model: [`facebook/wav2vec2-xls-r-300m`](https://huggingface.co/facebook/wav2vec2-xls-r-300m)\n* LM: KenLM 5-gram (16G) trained on [IndicCorpV2 corpus](https://github.com/AI4Bharat/IndicBERT#indiccorp-v2) and [Bengali Hate Speech Dataset](https://github.com/rezacsedu/Bengali-Hate-Speech-Dataset/tree/main)\n* More Data: Competition data + MadASR2023 + OpenSLR53\n* Data Augmentation: `audiomentations.AddBackgroundNoise` using subset of \"Bollywood Music\", \"Applause\" and \"Theme Music\" from [AudioSet dataset](https://research.google.com/audioset/dataset/index.html)\n* Restore Punctuation https://github.com/xashru/punctuation-restoration\n\n## Important lesson for future me!\n* **`[No space left on device]`** Just write your own custom dataset class!!!\n * Look at https://github.com/bayartsogt-ya/bengali-speech-2023/blob/main/train2.py.\n * Just increase `dataloader_num_workers` if you have enough cores. Preparing input and use `datasets.Dataset.set_transform` is complicated and **not** efficient.\n * Be simple! read it from a file system in `__getitem__` and apply whatever you want on the fly!\n* **`[Quality vs Quantity]`** 0.475 on only validation VS 0.421 on train (filtered) validation madasr openslr53 😂\n * It is obvious that filtering on big datasets helps!\n* **`[Manually Check Output]`** See where your model is making mistake on your validation data.\n * This helped me to see that punctuations (dari, comma, question mark, etc...) were counted as substitutions and deleted.\n* **`[Stop procrastinating on small things]`** You could have checked different chunk_length_s way before deadline. But you did not here! -> This is not calling out you did not try to train whisper!\n\n## Guilt of Overfitting to LB!\n\nBecause test data (Out of Distribution) data was so different from train datasets, it was really about overfitting to public leaderboard.\n```python\n!kaggle competitions submissions -v bengaliai-speech >> ./bengali-speech-submissions.csv\n>>> df = pd.read_csv(\"./data/bengali-speech-submissions.csv\")\n>>> np.corrcoef(df.publicScore, df.privateScore)[0, 1]\n0.990815988867224\n...\n>>> sns.lineplot(df, y=\"score\", x=\"date\", hue=\"split\")\n```\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5055010%2Ff73eba4301b1d7ec468eb29fba3ddb02%2Fpublic_vs_private.png?generation=1698196214469291&alt=media)\n\n## In the End\n\nIt is all about learning!\nEven though it is always so frustrating to feel you were so close or so much could have done or should have done, I appreciate this learning path and that's why I joined to Kaggle in the first place! 🫡", "votes": null }, { "id": "2498065", "postDate": "10/25/2023 05:44:03", "content": "

Considering the short time investment, I think the result is fantastic!!
\nBest regards and all the best for future competitions. The write up is clean and well structured @bayartsogtya
\nCongratulations!!

", "rawMarkdown": "Considering the short time investment, I think the result is fantastic!! \nBest regards and all the best for future competitions. The write up is clean and well structured @bayartsogtya \nCongratulations!!", "votes": null }, { "id": "2498766", "postDate": "10/25/2023 14:13:22", "content": "

Thanks @ravi20076

", "rawMarkdown": "Thanks @ravi20076", "votes": null } ], "comments": [ { "id": 2498065, "author_name": "ravi20076", "author_url": "", "post_date": "10/25/2023 05:44:03", "content": "

Considering the short time investment, I think the result is fantastic!!
\nBest regards and all the best for future competitions. The write up is clean and well structured @bayartsogtya
\nCongratulations!!

", "votes": null, "replies": [ { "id": 2498766, "author_name": "bayartsogtya", "author_url": "", "post_date": "10/25/2023 14:13:22", "content": "

Thanks @ravi20076

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2497907": "## First of all, THANK YOU ALL!\nAs a late joiner, It was so helpful to read insightful discussions of @imtiazprio @reasat @tugstugi @hengck23 @mbmmurad and list goes on!\n\n## Approach\n* Code: https://github.com/bayartsogt-ya/bengali-speech-2023\n* Inference: https://www.kaggle.com/code/bayartsogtya/submit-to-restore-punctuation/notebook\n* Backbone Model: [`facebook/wav2vec2-xls-r-300m`](https://huggingface.co/facebook/wav2vec2-xls-r-300m)\n* LM: KenLM 5-gram (16G) trained on [IndicCorpV2 corpus](https://github.com/AI4Bharat/IndicBERT#indiccorp-v2) and [Bengali Hate Speech Dataset](https://github.com/rezacsedu/Bengali-Hate-Speech-Dataset/tree/main)\n* More Data: Competition data + MadASR2023 + OpenSLR53\n* Data Augmentation: `audiomentations.AddBackgroundNoise` using subset of \"Bollywood Music\", \"Applause\" and \"Theme Music\" from [AudioSet dataset](https://research.google.com/audioset/dataset/index.html)\n* Restore Punctuation https://github.com/xashru/punctuation-restoration\n\n## Important lesson for future me!\n* **`[No space left on device]`** Just write your own custom dataset class!!!\n * Look at https://github.com/bayartsogt-ya/bengali-speech-2023/blob/main/train2.py.\n * Just increase `dataloader_num_workers` if you have enough cores. Preparing input and use `datasets.Dataset.set_transform` is complicated and **not** efficient.\n * Be simple! read it from a file system in `__getitem__` and apply whatever you want on the fly!\n* **`[Quality vs Quantity]`** 0.475 on only validation VS 0.421 on train (filtered) validation madasr openslr53 😂\n * It is obvious that filtering on big datasets helps!\n* **`[Manually Check Output]`** See where your model is making mistake on your validation data.\n * This helped me to see that punctuations (dari, comma, question mark, etc...) were counted as substitutions and deleted.\n* **`[Stop procrastinating on small things]`** You could have checked different chunk_length_s way before deadline. But you did not here! -> This is not calling out you did not try to train whisper!\n\n## Guilt of Overfitting to LB!\n\nBecause test data (Out of Distribution) data was so different from train datasets, it was really about overfitting to public leaderboard.\n```python\n!kaggle competitions submissions -v bengaliai-speech >> ./bengali-speech-submissions.csv\n>>> df = pd.read_csv(\"./data/bengali-speech-submissions.csv\")\n>>> np.corrcoef(df.publicScore, df.privateScore)[0, 1]\n0.990815988867224\n...\n>>> sns.lineplot(df, y=\"score\", x=\"date\", hue=\"split\")\n```\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5055010%2Ff73eba4301b1d7ec468eb29fba3ddb02%2Fpublic_vs_private.png?generation=1698196214469291&alt=media)\n\n## In the End\n\nIt is all about learning!\nEven though it is always so frustrating to feel you were so close or so much could have done or should have done, I appreciate this learning path and that's why I joined to Kaggle in the first place! 🫡", "2498065": "Considering the short time investment, I think the result is fantastic!! \nBest regards and all the best for future competitions. The write up is clean and well structured @bayartsogtya \nCongratulations!!", "2498766": "Thanks @ravi20076" }, "source": "meta" }