{
  "id": 432791,
  "title": "[LB: 0.471] Public Wav2Vec2.0 w/ Language Model Baseline",
  "url": "/competitions/bengaliai-speech/discussion/432791",
  "author_name": "Tawara",
  "post_date": "2023-08-19T00:19:19.127000",
  "votes": 27,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hi, I'm a new comer and this is my first speech2text competition.  <br>\nI start from just inference test data using public trained models.</p>\n<p>After reading  <a href=\"https://www.kaggle.com/competitions/bengaliai-speech/discussion/425496\" target=\"_blank\">this topic</a>, I've been interested in trying a combination of public Wav2Vec2.0 and Language Model.</p>\n<p>I used two public models from hugging faces:</p>\n<ul>\n<li><code>https://huggingface.co/ai4bharat/indicwav2vec_v1_bengali</code> for Wav2vec2CTC Model only</li>\n<li><code>https://huggingface.co/arijitx/wav2vec2-xls-r-300m-bengali</code> for Language Model</li>\n</ul>\n<p>Surprisingly, the combination got 0.471 on Public LB(16th now).</p>\n<ul>\n<li><p>model downloading notebook:  <br>\n<a href=\"https://www.kaggle.com/code/ttahara/bengali-sr-download-public-trained-models\" target=\"_blank\">https://www.kaggle.com/code/ttahara/bengali-sr-download-public-trained-models</a></p></li>\n<li><p>inference notebook:  <br>\n<a href=\"https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-baseline\" target=\"_blank\">https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-baseline</a></p></li>\n</ul>\n<p>I didn't train these models with this competition data. So  we may get higher and higher score with fine-tuning.</p>\n<p>Happy Kaggling ;)</p>\n<h3>Edit</h3>\n<p>I check WER for valid by the following notebook:  <br>\n<a href=\"https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-validation\" target=\"_blank\">https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-validation</a></p>\n<p>valid mean WER:  0.298</p>",
  "messages": [
    {
      "id": 2397353,
      "postDate": "2023-08-19T00:19:19.127Z",
      "content": "<p>Hi, I'm a new comer and this is my first speech2text competition.  <br>\nI start from just inference test data using public trained models.</p>\n<p>After reading  <a href=\"https://www.kaggle.com/competitions/bengaliai-speech/discussion/425496\" target=\"_blank\">this topic</a>, I've been interested in trying a combination of public Wav2Vec2.0 and Language Model.</p>\n<p>I used two public models from hugging faces:</p>\n<ul>\n<li><code>https://huggingface.co/ai4bharat/indicwav2vec_v1_bengali</code> for Wav2vec2CTC Model only</li>\n<li><code>https://huggingface.co/arijitx/wav2vec2-xls-r-300m-bengali</code> for Language Model</li>\n</ul>\n<p>Surprisingly, the combination got 0.471 on Public LB(16th now).</p>\n<ul>\n<li><p>model downloading notebook:  <br>\n<a href=\"https://www.kaggle.com/code/ttahara/bengali-sr-download-public-trained-models\" target=\"_blank\">https://www.kaggle.com/code/ttahara/bengali-sr-download-public-trained-models</a></p></li>\n<li><p>inference notebook:  <br>\n<a href=\"https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-baseline\" target=\"_blank\">https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-baseline</a></p></li>\n</ul>\n<p>I didn't train these models with this competition data. So  we may get higher and higher score with fine-tuning.</p>\n<p>Happy Kaggling ;)</p>\n<h3>Edit</h3>\n<p>I check WER for valid by the following notebook:  <br>\n<a href=\"https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-validation\" target=\"_blank\">https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-validation</a></p>\n<p>valid mean WER:  0.298</p>",
      "rawMarkdown": "Hi, I'm a new comer and this is my first speech2text competition.  \nI start from just inference test data using public trained models.\n\nAfter reading  [this topic](https://www.kaggle.com/competitions/bengaliai-speech/discussion/425496), I've been interested in trying a combination of public Wav2Vec2.0 and Language Model.\n\nI used two public models from hugging faces:\n\n* `https://huggingface.co/ai4bharat/indicwav2vec_v1_bengali` for Wav2vec2CTC Model only\n* `https://huggingface.co/arijitx/wav2vec2-xls-r-300m-bengali` for Language Model\n\n\nSurprisingly, the combination got 0.471 on Public LB(16th now).\n\n* model downloading notebook:  \nhttps://www.kaggle.com/code/ttahara/bengali-sr-download-public-trained-models\n\n* inference notebook:  \n https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-baseline\n\nI didn't train these models with this competition data. So  we may get higher and higher score with fine-tuning.\n\nHappy Kaggling ;)\n\n\n### Edit\n\nI check WER for valid by the following notebook:  \nhttps://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-validation\n\nvalid mean WER:  0.298\n",
      "votes": 27
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2397353": "Hi, I'm a new comer and this is my first speech2text competition.  \nI start from just inference test data using public trained models.\n\nAfter reading  [this topic](https://www.kaggle.com/competitions/bengaliai-speech/discussion/425496), I've been interested in trying a combination of public Wav2Vec2.0 and Language Model.\n\nI used two public models from hugging faces:\n\n* `https://huggingface.co/ai4bharat/indicwav2vec_v1_bengali` for Wav2vec2CTC Model only\n* `https://huggingface.co/arijitx/wav2vec2-xls-r-300m-bengali` for Language Model\n\n\nSurprisingly, the combination got 0.471 on Public LB(16th now).\n\n* model downloading notebook:  \nhttps://www.kaggle.com/code/ttahara/bengali-sr-download-public-trained-models\n\n* inference notebook:  \n https://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-baseline\n\nI didn't train these models with this competition data. So  we may get higher and higher score with fine-tuning.\n\nHappy Kaggling ;)\n\n\n### Edit\n\nI check WER for valid by the following notebook:  \nhttps://www.kaggle.com/code/ttahara/bengali-sr-public-wav2vec2-0-w-lm-validation\n\nvalid mean WER:  0.298\n"
  }
}