{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":73047,"databundleVersionId":8149390,"sourceType":"competition"}],"dockerImageVersionId":30698,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<div style=\"background-color: #ff4d4d; color: #ffffff; padding: 20px; border-radius: 10px; text-align: center; font-family: 'Indie Flower', cursive;\">\n    <h1 style=\"font-size: 36px; font-weight: bold; margin-bottom: 10px;\">ভাষা-বিচিত্রা: ASR for Regional Dialects</h1>\n    <h2 style=\"font-size: 24px; font-weight: bold;\">Team: DU Metro Rail</h2>\n    <marquee behavior=\"scroll\" direction=\"left\" style=\"margin-top: 20px; font-size: 20px;\">\n        INFERENCE NOTEBOOK\n    </marquee>\n</div>\n","metadata":{}},{"cell_type":"code","source":"import os\n\nimport pandas as pd\n\nimport librosa\nimport librosa.display\n\nimport numpy as np\n\nimport IPython.display as ipd\n\nimport matplotlib.pyplot as plt\n\nimport random\n\nfrom collections import Counter\n\nfrom sklearn.model_selection import train_test_split\n\nimport torch\nimport torchaudio\n\nfrom dataclasses import dataclass\nfrom typing import Any, Dict, List, Union\nfrom datasets import DatasetDict\nfrom datasets import Dataset as DS\n\nfrom transformers import (\n    WhisperFeatureExtractor,\n    WhisperTokenizer,\n    WhisperProcessor,\n    WhisperForConditionalGeneration,\n    Seq2SeqTrainingArguments,\n    Seq2SeqTrainer,\n    TrainerCallback,\n    TrainingArguments,\n    TrainerState,\n    TrainerControl,\n    EarlyStoppingCallback,\n    pipeline\n)\n\nfrom torchmetrics.text import WordErrorRate, CharErrorRate","metadata":{"execution":{"iopub.status.busy":"2024-04-24T19:52:52.446273Z","iopub.execute_input":"2024-04-24T19:52:52.446691Z","iopub.status.idle":"2024-04-24T19:53:21.191731Z","shell.execute_reply.started":"2024-04-24T19:52:52.446659Z","shell.execute_reply":"2024-04-24T19:53:21.190390Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"BASE_DIR = '/kaggle/input/ben10/ben10'\ntrain_data_dir = f\"{BASE_DIR}/16_kHz_train_audio/\"\ntest_data_dir = f\"{BASE_DIR}/16_kHz_valid_audio/\"\ndata_path = f\"{BASE_DIR}/train.csv\"","metadata":{"execution":{"iopub.status.busy":"2024-04-24T19:53:22.585331Z","iopub.execute_input":"2024-04-24T19:53:22.586299Z","iopub.status.idle":"2024-04-24T19:53:22.593982Z","shell.execute_reply.started":"2024-04-24T19:53:22.586255Z","shell.execute_reply":"2024-04-24T19:53:22.592050Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pipe = pipeline(\n    \"automatic-speech-recognition\",\n    model='abdullaharean/reg-bn-asr',#Our Model Hosted on Hugging Face\n    chunk_length_s=30,\n    device=0,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def pretty_sort(filename):\n    name, number_str = filename.split(\" (\")\n    number = int(number_str.split(\")\")[0])\n    return name, number","metadata":{"execution":{"iopub.status.busy":"2024-04-24T19:54:29.548438Z","iopub.status.idle":"2024-04-24T19:54:29.548877Z","shell.execute_reply.started":"2024-04-24T19:54:29.548670Z","shell.execute_reply":"2024-04-24T19:54:29.548687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ids = []\npreds=[]","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for root, dirs, files in os.walk(test_data_dir):\n    files = sorted(files, key=pretty_sort)\n    \n    shift = files[1070 : 1202]\n    \n    files = shift + files[:1070] + files[1202:]\n    ids = files.copy()\n    \n    for file in files:\n        composed_path = f\"{test_data_dir}{file}\"\n        audio, sr = librosa.load(composed_path, sr=16_000)\n        text = pipe(audio)[\"text\"]\n        preds.append(text)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_df = pd.DataFrame()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_df[\"id\"] = ids\nsub_df[\"sentence\"] = preds","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_df.head(20)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_df.to_csv(\"submission.csv\", index=False)","metadata":{},"execution_count":null,"outputs":[]}]}