In this competition I have tried to use the Wisper model of openai. For anyone interested, here are the notebooks I used for training and inference using LoRA for fine tuning the models.
\n\nAlso, here is a cleaned and preprocessed dataset I used for training:
\n", "messages": [ { "id": "2467238", "postDate": "10/04/2023 11:25:01", "content": "In this competition I have tried to use the Wisper model of openai. For anyone interested, here are the notebooks I used for training and inference using LoRA for fine tuning the models.
\n\nAlso, here is a cleaned and preprocessed dataset I used for training:
\n", "rawMarkdown": "In this competition I have tried to use the Wisper model of openai. For anyone interested, here are the notebooks I used for training and inference using LoRA for fine tuning the models.\n- [fine-tuning-whisper-with-lora](https://www.kaggle.com/code/davidramos18/fork-of-fine-tuning-whisper-with-lora)\n- [inference-whisper-lora](https://www.kaggle.com/code/davidramos18/inference-whisper-lora)\n- [inference-whisper](https://www.kaggle.com/code/davidramos18/inference-whisper)\n\nAlso, here is a cleaned and preprocessed dataset I used for training:\n- [Preprocessed Common Voice 13 | Bengali](https://www.kaggle.com/datasets/davidramos18/preprocessed-common-voice-13-bengali)", "votes": null }, { "id": "2467382", "postDate": "10/04/2023 14:02:49", "content": "Why the score is so low, even when trained on a good dataset. We must have missed something
", "rawMarkdown": "Why the score is so low, even when trained on a good dataset. We must have missed something", "votes": null }, { "id": "2467818", "postDate": "10/04/2023 21:41:12", "content": "It is something I don't understand either. It could be several things, to begin with, I don't know if it will influence anything but the torch.autocast part can be a bit of a problem (without that line, the trainer raises an error). Also, maybe the amount of data with which the training has been done is not enough and the training time is not enough either. Perhaps I am missing something and some part of the code is wrong, but I have checked it several times and I have not found any answer.
", "rawMarkdown": "It is something I don't understand either. It could be several things, to begin with, I don't know if it will influence anything but the torch.autocast part can be a bit of a problem (without that line, the trainer raises an error). Also, maybe the amount of data with which the training has been done is not enough and the training time is not enough either. Perhaps I am missing something and some part of the code is wrong, but I have checked it several times and I have not found any answer.", "votes": null } ], "comments": [ { "id": 2467382, "author_name": "nyleve", "author_url": "", "post_date": "10/04/2023 14:02:49", "content": "Why the score is so low, even when trained on a good dataset. We must have missed something
", "votes": null, "replies": [ { "id": 2467818, "author_name": "davidramos18", "author_url": "", "post_date": "10/04/2023 21:41:12", "content": "It is something I don't understand either. It could be several things, to begin with, I don't know if it will influence anything but the torch.autocast part can be a bit of a problem (without that line, the trainer raises an error). Also, maybe the amount of data with which the training has been done is not enough and the training time is not enough either. Perhaps I am missing something and some part of the code is wrong, but I have checked it several times and I have not found any answer.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2467238": "In this competition I have tried to use the Wisper model of openai. For anyone interested, here are the notebooks I used for training and inference using LoRA for fine tuning the models.\n- [fine-tuning-whisper-with-lora](https://www.kaggle.com/code/davidramos18/fork-of-fine-tuning-whisper-with-lora)\n- [inference-whisper-lora](https://www.kaggle.com/code/davidramos18/inference-whisper-lora)\n- [inference-whisper](https://www.kaggle.com/code/davidramos18/inference-whisper)\n\nAlso, here is a cleaned and preprocessed dataset I used for training:\n- [Preprocessed Common Voice 13 | Bengali](https://www.kaggle.com/datasets/davidramos18/preprocessed-common-voice-13-bengali)", "2467382": "Why the score is so low, even when trained on a good dataset. We must have missed something", "2467818": "It is something I don't understand either. It could be several things, to begin with, I don't know if it will influence anything but the torch.autocast part can be a bit of a problem (without that line, the trainer raises an error). Also, maybe the amount of data with which the training has been done is not enough and the training time is not enough either. Perhaps I am missing something and some part of the code is wrong, but I have checked it several times and I have not found any answer." }, "source": "meta" }