{ "id": 425458, "title": "Bengali.AI Speech Wav Dataset", "url": "/competitions/bengaliai-speech/discussion/425458", "author_name": "SeshuRaju 🧘‍♂️", "post_date": "2023-07-18T20:23:56.450000", "votes": 4, "comment_count": 0, "views": 0, "content": "\n

Credits to @mbmmurad for his convert notebook Convert mp3s to wavs

\n
 os\n cv2\n skimage.io\n tqdm.notebook  tqdm\n zipfile\n pandas  pd\n numpy  np\n shutil\n\n pydub  AudioSegment\n joblib  Parallel, delayed\n\ndf = pd.read_csv()\ndf.shape\n\nROOT_PATH = \nOUTPUT_DIR = \nos.makedirs(OUTPUT_DIR, exist_ok=)\n\ndf[] = df[].apply( x: x[])\ndf[] = df[].apply( x: x[])\n\nfolders = (((df[].tolist())))\nfolders1 = (((df[].tolist())))\n(, folders, folders1)\n(, (folders), (folders1))\n folder  folders:\n     folder  [,,,,,]:\n        \n     folder1  folders:\n        (folder, folder1, )\n        audio_files = df[((df[] == folder) & (df[] == folder1))][].apply( x:x+).tolist()\n         ():\n            folder = filename[:]\n            path = \n            save_path = \n              os.path.exists(save_path):\n                os.makedirs(save_path, exist_ok=)\n\n             os.path.exists(path):\n                :\n                    sound = AudioSegment.from_mp3(path)\n                    sound.export(, =)\n                :\n                    (path)\n\n         multiprocessing\n        num_cores = multiprocessing.cpu_count()\n\n         time\n        start = time.time()\n\n        Parallel(n_jobs=, backend=)(\n            delayed(save_fn)(filename)  filename  tqdm(audio_files)\n        )\n\n        end = time.time()\n        (folder, folder1, .(x=end-start))\n
", "messages": [ { "id": 2349966, "postDate": "2023-07-18T20:23:56.450Z", "content": "\n

Credits to @mbmmurad for his convert notebook Convert mp3s to wavs

\n
 os\n cv2\n skimage.io\n tqdm.notebook  tqdm\n zipfile\n pandas  pd\n numpy  np\n shutil\n\n pydub  AudioSegment\n joblib  Parallel, delayed\n\ndf = pd.read_csv()\ndf.shape\n\nROOT_PATH = \nOUTPUT_DIR = \nos.makedirs(OUTPUT_DIR, exist_ok=)\n\ndf[] = df[].apply( x: x[])\ndf[] = df[].apply( x: x[])\n\nfolders = (((df[].tolist())))\nfolders1 = (((df[].tolist())))\n(, folders, folders1)\n(, (folders), (folders1))\n folder  folders:\n     folder  [,,,,,]:\n        \n     folder1  folders:\n        (folder, folder1, )\n        audio_files = df[((df[] == folder) & (df[] == folder1))][].apply( x:x+).tolist()\n         ():\n            folder = filename[:]\n            path = \n            save_path = \n              os.path.exists(save_path):\n                os.makedirs(save_path, exist_ok=)\n\n             os.path.exists(path):\n                :\n                    sound = AudioSegment.from_mp3(path)\n                    sound.export(, =)\n                :\n                    (path)\n\n         multiprocessing\n        num_cores = multiprocessing.cpu_count()\n\n         time\n        start = time.time()\n\n        Parallel(n_jobs=, backend=)(\n            delayed(save_fn)(filename)  filename  tqdm(audio_files)\n        )\n\n        end = time.time()\n        (folder, folder1, .(x=end-start))\n
", "rawMarkdown": "- [Dataset 0](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-0)\n- [Dataset 1](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-1)\n- [Dataset 2](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-2)\n- [Dataset 3](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-3)\n- [Dataset 4](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-4)\n- [Dataset 5](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-5)\n- [Dataset 6](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-6)\n- [Dataset 7](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-7)\n- [Dataset 8](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-8)\n- [Dataset 9](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-9)\n- [Dataset a](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-a)\n- [Dataset b](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-b)\n- [Dataset c](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-c)\n- [Dataset d](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-d)\n- [Dataset e](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-e)\n- [Dataset f](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-f)\n\n\nCredits to @mbmmurad for his convert notebook [Convert mp3s to wavs](https://www.kaggle.com/code/mbmmurad/convert-mp3s-to-wavs)\n\n```python\nimport os\nimport cv2\nimport skimage.io\nfrom tqdm.notebook import tqdm\nimport zipfile\nimport pandas as pd\nimport numpy as np\nimport shutil\n\nfrom pydub import AudioSegment\nfrom joblib import Parallel, delayed\n\ndf = pd.read_csv(\"../input/bengaliai-speech/train.csv\")\ndf.shape\n\nROOT_PATH = \"../input/bengaliai-speech/train_mp3s\"\nOUTPUT_DIR = \"../input/train_files_wav\"\nos.makedirs(OUTPUT_DIR, exist_ok=True)\n\ndf['folder'] = df['id'].apply(lambda x: x[0])\ndf['folder1'] = df['id'].apply(lambda x: x[1])\n\nfolders = sorted(list(set(df['folder'].tolist())))\nfolders1 = sorted(list(set(df['folder1'].tolist())))\nprint(\"Folders\", folders, folders1)\nprint(\"Total Folders\", len(folders), len(folders1))\nfor folder in folders:\n if folder in ['0','1','2','3','4','5']:\n continue\n for folder1 in folders:\n print(folder, folder1, \"Started Folder\")\n audio_files = df[((df['folder'] == folder) & (df['folder1'] == folder1))]['id'].apply(lambda x:x+\".mp3\").tolist()\n def save_fn(filename):\n folder = filename[0:1]\n path = f\"{ROOT_PATH}/{filename}\"\n save_path = f\"{OUTPUT_DIR}/{folder}/wav/{folder1}\"\n if not os.path.exists(save_path):\n os.makedirs(save_path, exist_ok=True)\n\n if os.path.exists(path):\n try:\n sound = AudioSegment.from_mp3(path)\n sound.export(f\"{save_path}/{filename[:-4]}.wav\", format=\"wav\")\n except:\n print(path)\n\n import multiprocessing\n num_cores = multiprocessing.cpu_count()\n \n import time\n start = time.time()\n\n Parallel(n_jobs=72, backend=\"multiprocessing\")(\n delayed(save_fn)(filename) for filename in tqdm(audio_files)\n )\n\n end = time.time()\n print(folder, folder1, \"total time to process: {x} seconds\".format(x=end-start))\n\n\n```\n", "votes": 4 } ], "comments": [], "raw_markdown_by_id": { "2349966": "- [Dataset 0](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-0)\n- [Dataset 1](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-1)\n- [Dataset 2](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-2)\n- [Dataset 3](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-3)\n- [Dataset 4](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-4)\n- [Dataset 5](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-5)\n- [Dataset 6](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-6)\n- [Dataset 7](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-7)\n- [Dataset 8](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-8)\n- [Dataset 9](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-9)\n- [Dataset a](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-a)\n- [Dataset b](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-b)\n- [Dataset c](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-c)\n- [Dataset d](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-d)\n- [Dataset e](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-e)\n- [Dataset f](https://www.kaggle.com/datasets/seshurajup/bengaliai-speech-wav-dataset-f)\n\n\nCredits to @mbmmurad for his convert notebook [Convert mp3s to wavs](https://www.kaggle.com/code/mbmmurad/convert-mp3s-to-wavs)\n\n```python\nimport os\nimport cv2\nimport skimage.io\nfrom tqdm.notebook import tqdm\nimport zipfile\nimport pandas as pd\nimport numpy as np\nimport shutil\n\nfrom pydub import AudioSegment\nfrom joblib import Parallel, delayed\n\ndf = pd.read_csv(\"../input/bengaliai-speech/train.csv\")\ndf.shape\n\nROOT_PATH = \"../input/bengaliai-speech/train_mp3s\"\nOUTPUT_DIR = \"../input/train_files_wav\"\nos.makedirs(OUTPUT_DIR, exist_ok=True)\n\ndf['folder'] = df['id'].apply(lambda x: x[0])\ndf['folder1'] = df['id'].apply(lambda x: x[1])\n\nfolders = sorted(list(set(df['folder'].tolist())))\nfolders1 = sorted(list(set(df['folder1'].tolist())))\nprint(\"Folders\", folders, folders1)\nprint(\"Total Folders\", len(folders), len(folders1))\nfor folder in folders:\n if folder in ['0','1','2','3','4','5']:\n continue\n for folder1 in folders:\n print(folder, folder1, \"Started Folder\")\n audio_files = df[((df['folder'] == folder) & (df['folder1'] == folder1))]['id'].apply(lambda x:x+\".mp3\").tolist()\n def save_fn(filename):\n folder = filename[0:1]\n path = f\"{ROOT_PATH}/{filename}\"\n save_path = f\"{OUTPUT_DIR}/{folder}/wav/{folder1}\"\n if not os.path.exists(save_path):\n os.makedirs(save_path, exist_ok=True)\n\n if os.path.exists(path):\n try:\n sound = AudioSegment.from_mp3(path)\n sound.export(f\"{save_path}/{filename[:-4]}.wav\", format=\"wav\")\n except:\n print(path)\n\n import multiprocessing\n num_cores = multiprocessing.cpu_count()\n \n import time\n start = time.time()\n\n Parallel(n_jobs=72, backend=\"multiprocessing\")(\n delayed(save_fn)(filename) for filename in tqdm(audio_files)\n )\n\n end = time.time()\n print(folder, folder1, \"total time to process: {x} seconds\".format(x=end-start))\n\n\n```\n" } }