{ "id": 425241, "title": "Approaching Audio to Text Tasks", "url": "/competitions/bengaliai-speech/discussion/425241", "author_name": "", "post_date": "2023-07-17T21:50:52.311092600Z", "votes": 23, "comment_count": 7, "views": 0, "content": "

This is essentially an audio to text challenge. The part that makes this challenge especially complex is that this is an audio to text task for the Bengali language.

\n

Here is an idea on how to approach this type of task:

\n
    \n
  1. Convert audio files to spectrograms
    2. \n
  2. Feed the spectrograms to a convolutional neural network
    3. \n
  3. CNN generates feature embeddings
    4. \n
  4. Feed the feature embeddings to a text generation neural network
    5. \n
  5. Evaluate the results of the generated text.
    6. \n
\n

Audio to Features

\n\n

Features to Text

\n\n

References

\n

https://pylessons.com/speech-recognition
\nhttps://towardsdatascience.com/audio-deep-learning-made-simple-automatic-speech-recognition-asr-how-it-works-716cfce4c706
\nhttps://huggingface.co/docs/transformers/model_doc/speech_to_text
\nhttps://wingedsheep.com/building-a-language-model/

", "messages": [ { "id": "2348705", "postDate": "07/17/2023 21:50:52", "content": "

This is essentially an audio to text challenge. The part that makes this challenge especially complex is that this is an audio to text task for the Bengali language.

\n

Here is an idea on how to approach this type of task:

\n
    \n
  1. Convert audio files to spectrograms
    2. \n
  2. Feed the spectrograms to a convolutional neural network
    3. \n
  3. CNN generates feature embeddings
    4. \n
  4. Feed the feature embeddings to a text generation neural network
    5. \n
  5. Evaluate the results of the generated text.
    6. \n
\n

Audio to Features

\n\n

Features to Text

\n\n

References

\n

https://pylessons.com/speech-recognition
\nhttps://towardsdatascience.com/audio-deep-learning-made-simple-automatic-speech-recognition-asr-how-it-works-716cfce4c706
\nhttps://huggingface.co/docs/transformers/model_doc/speech_to_text
\nhttps://wingedsheep.com/building-a-language-model/

", "rawMarkdown": "This is essentially an audio to text challenge. The part that makes this challenge especially complex is that this is an audio to text task for the Bengali language.\n\nHere is an idea on how to approach this type of task:\n1. Convert audio files to spectrograms\n2. Feed the spectrograms to a convolutional neural network\n3. CNN generates feature embeddings\n4. Feed the feature embeddings to a text generation neural network\n5. Evaluate the results of the generated text.\n\n# Audio to Features\n- you can create your spectrogram using a library such as librosa\n- there are many place to find CNN model architectures such as HuggingFace\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6537187%2F93fb4090ce9ddc8b6bb837e98a04de30%2FAudio%20to%20Features.png?generation=1689629891639847&alt=media)\n\n# Features to Text\n- you want to feed the embedding map to a text generation model such as a bi-directional lstm or a text generation transformer\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6537187%2Ffa84a28468b2d0f7da95eab1e1f722c0%2FFeatures%20to%20Text.png?generation=1689630172589929&alt=media)\n\n# References\nhttps://pylessons.com/speech-recognition\nhttps://towardsdatascience.com/audio-deep-learning-made-simple-automatic-speech-recognition-asr-how-it-works-716cfce4c706\nhttps://huggingface.co/docs/transformers/model_doc/speech_to_text\nhttps://wingedsheep.com/building-a-language-model/", "votes": null }, { "id": "2357687", "postDate": "07/25/2023 04:08:29", "content": "

Thanks for sharing! @ravishah1 This would be helpful for beginners like me:)

", "rawMarkdown": "Thanks for sharing! @ravishah1 This would be helpful for beginners like me:)", "votes": null }, { "id": "2358685", "postDate": "07/25/2023 16:49:09", "content": "

@jaygun84 I'm glad you found it helpful!

", "rawMarkdown": "jaygun84 I'm glad you found it helpful!", "votes": null }, { "id": "2368213", "postDate": "08/01/2023 04:20:19", "content": "

Here are a few additional resources about audio to text tasks that I found on YouTube:

\n", "rawMarkdown": "Here are a few additional resources about audio to text tasks that I found on YouTube:\n- https://www.youtube.com/watch?v=YereI6Gn3bM\n- https://www.youtube.com/watch?v=Qf4YJcHXtcY\n- https://www.youtube.com/watch?v=qKz_lmgad3o", "votes": null }, { "id": "2368263", "postDate": "08/01/2023 04:56:31", "content": "

Helpful post @ravishah1. I just started out on Kaggle, how exactly does the language involved being Bengali make the task more difficult than a more commonly spoken language? Or do you think that does not matter?

", "rawMarkdown": "Helpful post @ravishah1. I just started out on Kaggle, how exactly does the language involved being Bengali make the task more difficult than a more commonly spoken language? Or do you think that does not matter?", "votes": null }, { "id": "2369048", "postDate": "08/01/2023 14:05:04", "content": "

@arnavmodi It can be trickier because there are less online resources and fewer pre-trained models for Bengali.

", "rawMarkdown": "arnavmodi It can be trickier because there are less online resources and fewer pre-trained models for Bengali.", "votes": null }, { "id": "2381027", "postDate": "08/09/2023 00:16:01", "content": "

Thanks for sharing @ravishah1! Great tip, I'll start from here🙌

", "rawMarkdown": "Thanks for sharing @ravishah1! Great tip, I'll start from here🙌", "votes": null }, { "id": "2429399", "postDate": "09/08/2023 15:23:31", "content": "

Very informative guidance to starts, thanks @ravishah1

", "rawMarkdown": "Very informative guidance to starts, thanks @ravishah1", "votes": null } ], "comments": [ { "id": 2357687, "author_name": "jaygun84", "author_url": "", "post_date": "07/25/2023 04:08:29", "content": "

Thanks for sharing! @ravishah1 This would be helpful for beginners like me:)

", "votes": null, "replies": [ { "id": 2358685, "author_name": "ravishah1", "author_url": "", "post_date": "07/25/2023 16:49:09", "content": "

@jaygun84 I'm glad you found it helpful!

", "votes": null, "replies": [] } ] }, { "id": 2368213, "author_name": "ravishah1", "author_url": "", "post_date": "08/01/2023 04:20:19", "content": "

Here are a few additional resources about audio to text tasks that I found on YouTube:

\n", "votes": null, "replies": [] }, { "id": 2368263, "author_name": "arnavmodi", "author_url": "", "post_date": "08/01/2023 04:56:31", "content": "

Helpful post @ravishah1. I just started out on Kaggle, how exactly does the language involved being Bengali make the task more difficult than a more commonly spoken language? Or do you think that does not matter?

", "votes": null, "replies": [ { "id": 2369048, "author_name": "ravishah1", "author_url": "", "post_date": "08/01/2023 14:05:04", "content": "

@arnavmodi It can be trickier because there are less online resources and fewer pre-trained models for Bengali.

", "votes": null, "replies": [] } ] }, { "id": 2381027, "author_name": "haridasabr", "author_url": "", "post_date": "08/09/2023 00:16:01", "content": "

Thanks for sharing @ravishah1! Great tip, I'll start from here🙌

", "votes": null, "replies": [] }, { "id": 2429399, "author_name": "liushundawang", "author_url": "", "post_date": "09/08/2023 15:23:31", "content": "

Very informative guidance to starts, thanks @ravishah1

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2348705": "This is essentially an audio to text challenge. The part that makes this challenge especially complex is that this is an audio to text task for the Bengali language.\n\nHere is an idea on how to approach this type of task:\n1. Convert audio files to spectrograms\n2. Feed the spectrograms to a convolutional neural network\n3. CNN generates feature embeddings\n4. Feed the feature embeddings to a text generation neural network\n5. Evaluate the results of the generated text.\n\n# Audio to Features\n- you can create your spectrogram using a library such as librosa\n- there are many place to find CNN model architectures such as HuggingFace\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6537187%2F93fb4090ce9ddc8b6bb837e98a04de30%2FAudio%20to%20Features.png?generation=1689629891639847&alt=media)\n\n# Features to Text\n- you want to feed the embedding map to a text generation model such as a bi-directional lstm or a text generation transformer\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6537187%2Ffa84a28468b2d0f7da95eab1e1f722c0%2FFeatures%20to%20Text.png?generation=1689630172589929&alt=media)\n\n# References\nhttps://pylessons.com/speech-recognition\nhttps://towardsdatascience.com/audio-deep-learning-made-simple-automatic-speech-recognition-asr-how-it-works-716cfce4c706\nhttps://huggingface.co/docs/transformers/model_doc/speech_to_text\nhttps://wingedsheep.com/building-a-language-model/", "2357687": "Thanks for sharing! @ravishah1 This would be helpful for beginners like me:)", "2358685": "jaygun84 I'm glad you found it helpful!", "2368213": "Here are a few additional resources about audio to text tasks that I found on YouTube:\n- https://www.youtube.com/watch?v=YereI6Gn3bM\n- https://www.youtube.com/watch?v=Qf4YJcHXtcY\n- https://www.youtube.com/watch?v=qKz_lmgad3o", "2368263": "Helpful post @ravishah1. I just started out on Kaggle, how exactly does the language involved being Bengali make the task more difficult than a more commonly spoken language? Or do you think that does not matter?", "2369048": "arnavmodi It can be trickier because there are less online resources and fewer pre-trained models for Bengali.", "2381027": "Thanks for sharing @ravishah1! Great tip, I'll start from here🙌", "2429399": "Very informative guidance to starts, thanks @ravishah1" }, "source": "meta" }