{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

PogChamp2- Music Genre Classification

\n

PogChamp2 EDA + Audio ANIMATION + Baseline

\n\n\n

\n \n

\n\n\n# 🍁 Introduction:\n> Following one month of the pogchamp1, PogChamp2 has been launched. It is a much more interesting competition now that audio data is being used. It was almost destined to have an image competition this time, but there is always next time.. Nonetheless, you can treat audio competition as an image competition by converting the data into spectograms, or if you are into NLP you can train a model based on a sequential model. But I think creating malspectograms and using them as image data is a state-of-the-art approach [what most people do]. \n\n> You should give this competition a go if you're just starting out with kaggle, machine learning, or audio data. If you are starting with audio data, then this video https://youtu.be/ZqpSb5p1xQo by [@robikscube](https://www.kaggle.com/robikscube) is the best. In addition, there are lots of prizes at stack this time, as the winner of the competition will receive the NVIDIA 3080 Ti GPU. The runners-up and the third-place finalists will also receive access to Deep Learning courses by NVIDIA. But it is also true that, regardless of the prize we receive, we will still participate in the future pogchamp series.\n\n\n>

\n\n

\n\n> But the catch is, there is only one GPU for the whole winning team, so if you team up with people, make sure to discuss who is gonna take the GPU if you folks end up winning, if you guys cant decide, I can always help [if you know what I mean].\n\n>

\n\n

\n\n\n# 📌 About the NB:\n> I have created a detailed EDA and baseline model but creating the audio animation took a lot of time. Making these animations felt like a great way to visualize the audio data, its looking decent now, I hope you enjoy.\n\n> - [AUDIO ANIMATION](#🧩AUDIO-ANIMATION:)\n> - [File Metadata Analysis](#🚒Metadata-Analysis:)\n> - [RF Model Using Metadata](#✂Model-on-Metadata:)","metadata":{}},{"cell_type":"code","source":"!pip install -q audio-metadata","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-03-14T16:56:53.25248Z","iopub.execute_input":"2022-03-14T16:56:53.25337Z","iopub.status.idle":"2022-03-14T16:57:11.942925Z","shell.execute_reply.started":"2022-03-14T16:56:53.253258Z","shell.execute_reply":"2022-03-14T16:57:11.941769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📙 Importing Libraries:","metadata":{}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport random\nimport cv2\nfrom tqdm import tqdm_notebook as tqdm\nfrom plotly.offline import iplot\nimport plotly as py\nimport plotly.tools as tls\nimport cufflinks as cf\nfrom IPython.core.display import display, HTML\nimport IPython.display as ipd\nimport librosa\nfrom glob import glob\nimport matplotlib.pylab as plt\nimport IPython\nprint(\"librosa version: \", librosa.__version__)\nfrom matplotlib import lines\nfrom matplotlib import animation, rc\nfrom numpy.fft import rfft\n\n\nplt.style.use('ggplot')\npy.offline.init_notebook_mode(connected = True)\ncf.go_offline()\n%matplotlib inline","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T16:57:11.94505Z","iopub.execute_input":"2022-03-14T16:57:11.94542Z","iopub.status.idle":"2022-03-14T16:57:17.080686Z","shell.execute_reply.started":"2022-03-14T16:57:11.945372Z","shell.execute_reply":"2022-03-14T16:57:17.079693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🔎 Lets checkout the training data:","metadata":{}},{"cell_type":"code","source":"DIR = \"../input/kaggle-pog-series-s01e02/\"\ntrain_df = pd.read_csv(\"../input/kaggle-pog-series-s01e02/train.csv\")\ntrain_df.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T16:57:17.083915Z","iopub.execute_input":"2022-03-14T16:57:17.084242Z","iopub.status.idle":"2022-03-14T16:57:17.170775Z","shell.execute_reply.started":"2022-03-14T16:57:17.084205Z","shell.execute_reply":"2022-03-14T16:57:17.170063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"genre\"].value_counts().iplot(kind='bar',color='orange',xTitle=\"genre\",yTitle=\"count\",title=\"Music Genre Count\") # thank you for making the problem interesting","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T16:57:17.172285Z","iopub.execute_input":"2022-03-14T16:57:17.172817Z","iopub.status.idle":"2022-03-14T16:57:18.003089Z","shell.execute_reply.started":"2022-03-14T16:57:17.17277Z","shell.execute_reply":"2022-03-14T16:57:18.002218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> Seems like we have to deal with class imbalance here too. Im not sure we need to do any noise removal or not, like birdclef'22.","metadata":{}},{"cell_type":"markdown","source":"# 🧩AUDIO ANIMATION:\n> It is very boring to compare a audio file with a static waveform[amplitude] plot of the same audio file, you cant identify where the amplitude is changing. This is where this animation plot helps. This animation is consists of two plots one is a amplitude plot over time and the other one is a fft of a sliding window of size 10k, this sliding window moves linearly from left ro right, with will help to see the modulation changes. Im not a audio data expert, please let me know if there is any mistake.\n\n>
\nNote: Please turn on the animation plot and the audio at the same time, to hear the simultaneous changes of the audio.\n
","metadata":{}},{"cell_type":"code","source":"# to calculate audio duration \ndef output_duration(length):\n hours = length // 3600 # calculate in hours\n length %= 3600\n mins = length // 60 # calculate in minutes\n length %= 60\n seconds = length # calculate in seconds\n \n return hours, mins, seconds\n\n# sample_audio_file = \"../input/kaggle-pog-series-s01e02/train/000025.ogg\" \n# data, sr = librosa.load(sample_audio_file)\n# len_data = len(data) # holds length of the numpy array\n \n# t = len_data / sr # returns duration but in floats\n \n# hours, mins, seconds = output_duration(int(t))\n# print('Total Duration: {}:{}:{}'.format(hours, mins, seconds))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T16:57:18.005454Z","iopub.execute_input":"2022-03-14T16:57:18.005778Z","iopub.status.idle":"2022-03-14T16:57:18.011361Z","shell.execute_reply.started":"2022-03-14T16:57:18.005742Z","shell.execute_reply":"2022-03-14T16:57:18.010693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class animation_plot(object):\n \n # setting up sub-plots, first one is for Fourier, second one for Amplitude\n fig, ax = plt.subplots(1, 2,figsize=(18,6))\n min_index = 0\n \n def __init__(self, file_name):\n self.file_name = file_name\n self.data, self.sr = librosa.load(self.file_name)\n \n \n \n # both the plots has two functions, one to generate a blank function [init_blue_line and init_rfft] and \n # the other function is for feeding the each frame for the animation [animate_blue_line,animate_rfft]\n # we pass the combined init and main function in the matplotlib FuncAnimation for animation\n\n # Blue sliding line, to show which part is playing\n def init_blue_line(self, axi=ax):\n global blue_line\n axi.plot(self.data, color=\"skyblue\")\n blue_line, = axi.plot([self.min_index, self.min_index], [0, 0], color=\"blue\")\n\n return axi,\n\n def animate_blue_line(self,num,axi=ax):\n blue_line.set_data([num, num], [+0.9, -0.8])\n return axi,\n\n # Fourier Transformation to show what's \"happening\" at that time\n # this both init and the main function is kind of like a sliding window, which takes a slice of 10k then performs\n # rfft [reduced to 10k/2 +1 ] and take another slice of 1k and display that as a animation.\n def init_rfft(self,axi=ax):\n global rfft_line\n fourier = rfft(self.data[self.min_index:self.min_index+10000])[1:1001]\n fourier = abs(fourier)\n axi.set_xlim(0, 1000)\n axi.set_ylim(fourier.min(), fourier.max())\n rfft_line, = axi.plot([],[], color=\"blue\")\n\n def animate_rfft(self,num, axi=ax):\n fourier = rfft(self.data[num:num+10000])[1:1001]\n fourier = abs(fourier)\n rfft_line.set_data(np.arange(0,len(fourier)), fourier)\n\n\n def init_all(self):\n self.init_rfft(animation_plot.ax[0])\n self.init_blue_line(animation_plot.ax[1])\n\n # this is just how matplotlib animation works - we need one function to update the whole frame\n def animate_all(self,num):\n self.animate_rfft(num, animation_plot.ax[0])\n self.animate_blue_line(num, animation_plot.ax[1])","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-03-14T16:57:18.012783Z","iopub.execute_input":"2022-03-14T16:57:18.013299Z","iopub.status.idle":"2022-03-14T16:57:18.531482Z","shell.execute_reply.started":"2022-03-14T16:57:18.013264Z","shell.execute_reply":"2022-03-14T16:57:18.530645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def draw_plot(fname):\n# plt.figure().clear()\n# # plt.close()\n# plt.cla()\n# plt.clf()\n y = 0\n anim = animation_plot(fname)\n y, sr = librosa.load(fname)\n \n len_data = len(y) # holds length of the numpy array\n t = len_data / sr # returns duration but in floats\n _, _, seconds = output_duration(int(t))\n \n rc('animation', html='jshtml')\n AUDIO_RATE = len(y)\n # setting video rate to 70, for 'smooth' animation\n VIDEO_RATE = 70\n ani_interval = 1000 * seconds\n \n # defining frames\n frames = np.arange(0, len(y), AUDIO_RATE/VIDEO_RATE,dtype=int)#[:24]\n \n # creating animation\n anim = animation.FuncAnimation(animation_plot.fig, anim.animate_all, init_func=anim.init_all, frames=frames, interval=ani_interval/VIDEO_RATE, blit=False)\n plt.figure().clear()\n plt.cla()\n plt.clf()\n return anim","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T16:57:18.532707Z","iopub.execute_input":"2022-03-14T16:57:18.532935Z","iopub.status.idle":"2022-03-14T16:57:18.540797Z","shell.execute_reply.started":"2022-03-14T16:57:18.532906Z","shell.execute_reply":"2022-03-14T16:57:18.540123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from random import sample\ngenre_list = list(np.unique(train_df[\"genre\"]))\ngenre_file_list = []\nfor i in range(len(genre_list)):\n genre_file_list.append(sample(train_df[train_df[\"genre\"] == genre_list[i]][\"filepath\"].to_list(),1)[0])\n \nreplace_dict = {17: \"train/007675.ogg\", 18: \"train/003552.ogg\", 0: \"train/001302.ogg\", 1: \"train/011873.ogg\", 6: \"train/016862.ogg\" , 8: \"train/007103.ogg\"}\nfor i in replace_dict.keys():\n genre_file_list[i] =replace_dict[i] ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T16:57:18.541732Z","iopub.execute_input":"2022-03-14T16:57:18.542248Z","iopub.status.idle":"2022-03-14T16:57:18.649879Z","shell.execute_reply.started":"2022-03-14T16:57:18.542213Z","shell.execute_reply":"2022-03-14T16:57:18.648872Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fname_1 = DIR + genre_file_list[0]\nana_k = draw_plot(fname_1)\ndisplay(HTML(f'

Genre: {genre_list[0]} [{genre_file_list[0]}]

'))\nana_k","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:15:32.516021Z","iopub.execute_input":"2022-03-14T17:15:32.516366Z","iopub.status.idle":"2022-03-14T17:16:14.549831Z","shell.execute_reply.started":"2022-03-14T17:15:32.516331Z","shell.execute_reply":"2022-03-14T17:16:14.548837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"file name: \",genre_file_list[0])\nIPython.display.Audio(fname_1)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:22:31.474361Z","iopub.execute_input":"2022-03-14T17:22:31.474686Z","iopub.status.idle":"2022-03-14T17:22:31.498769Z","shell.execute_reply.started":"2022-03-14T17:22:31.474655Z","shell.execute_reply":"2022-03-14T17:22:31.497794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🔮 Waveform vs Spectogram:","metadata":{}},{"cell_type":"code","source":"for genre_file_idx in range(len(genre_file_list)):\n file_path = DIR + genre_file_list[genre_file_idx]\n y_data, sr = librosa.load(file_path)\n plt.figure(1, figsize=(18,6))\n\n plot_a = plt.subplot(211)\n plot_a.plot(y_data, color='olive')\n plt.title(f\"waveform and frequency spectrum of {genre_list[genre_file_idx]} [{genre_file_list[genre_file_idx]}]\")\n plot_a.set_xlabel('sample rate * time')\n plot_a.set_ylabel('energy')\n\n plot_b = plt.subplot(212)\n plot_b.specgram(y_data, NFFT=1024, Fs=sr, noverlap=900)\n plot_b.set_xlabel('Time')\n plot_b.set_ylabel('Frequency')\n\n plt.show()\n# IPython.display.Audio(file_path) ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:11:58.189928Z","iopub.execute_input":"2022-03-14T17:11:58.190307Z","iopub.status.idle":"2022-03-14T17:12:43.660088Z","shell.execute_reply.started":"2022-03-14T17:11:58.190271Z","shell.execute_reply":"2022-03-14T17:12:43.659192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🚒Metadata Analysis:\n## 🏀 Metadata Extraction:","metadata":{}},{"cell_type":"code","source":"import audio_metadata\ndata = {\n \"filesize\":[],\n \"duration\": [],\n \"channels\": [],\n \"bitrate\": [],\n \"max_bitrate\": [],\n \"min_bitrate\": [],\n \"nominal_bitrate\": [],\n \"sample_rate\": [],\n \"genre\":[],\n}\n\nmissing_file_list = []\n\nfor item in tqdm(range(len(train_df)), total=len(train_df)):\n# print(item)\n# break\n file_path = train_df[\"filepath\"].iloc[item]\n \n try:\n meta = audio_metadata.load(f\"../input/kaggle-pog-series-s01e02/{file_path}\")\n meta_list = [meta[\"filesize\"],meta[\"streaminfo\"][\"duration\"],meta[\"streaminfo\"][\"channels\"],\n meta[\"streaminfo\"][\"bitrate\"],meta[\"streaminfo\"][\"max_bitrate\"],meta[\"streaminfo\"][\"min_bitrate\"],\n meta[\"streaminfo\"][\"nominal_bitrate\"],meta[\"streaminfo\"][\"sample_rate\"],train_df[\"genre_id\"].iloc[item]]\n for num in range(len(data.keys())):\n data[list(data.keys())[num]].append(meta_list[num])\n except:\n missing_file_list.append(file_path)\n print(f\"could not found: {file_path}\")\naudio_meta = pd.DataFrame(data)\naudio_meta.head() ","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-03-14T16:59:42.572778Z","iopub.execute_input":"2022-03-14T16:59:42.573132Z","iopub.status.idle":"2022-03-14T17:03:01.353096Z","shell.execute_reply.started":"2022-03-14T16:59:42.573086Z","shell.execute_reply":"2022-03-14T17:03:01.352235Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### Summary of the metadata","metadata":{}},{"cell_type":"code","source":"audio_meta.describe()","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:01.354548Z","iopub.execute_input":"2022-03-14T17:03:01.354797Z","iopub.status.idle":"2022-03-14T17:03:01.398782Z","shell.execute_reply.started":"2022-03-14T17:03:01.354769Z","shell.execute_reply":"2022-03-14T17:03:01.397956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### Metadata Dataframe","metadata":{}},{"cell_type":"code","source":"# dropping max_bitrate and min_bitrate, they are useless\naudio_meta.drop([\"max_bitrate\",\"min_bitrate\"],axis=1,inplace=True)\naudio_meta.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:03:01.399818Z","iopub.execute_input":"2022-03-14T17:03:01.400015Z","iopub.status.idle":"2022-03-14T17:03:01.414264Z","shell.execute_reply.started":"2022-03-14T17:03:01.399987Z","shell.execute_reply":"2022-03-14T17:03:01.413136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# for doing bar plot using matplotlib\ndef bar_plot(data, cat1):\n data_df = data.value_counts().rename_axis(cat1).reset_index(name=\"count\")\n data_count = data_df[\"count\"].to_list()\n data_name = [str(i) for i in data_df[cat1]]\n plt.figure(figsize=(18,6))\n plt.bar(data_name, data_count, width=.4)\n plt.title(f'Bar plot of {cat1}') \n plt.xlabel(f'{cat1}') \n plt.ylabel('count')\n\n for i, v in enumerate(data_count):\n plt.text(i, v, str(v),fontsize=20)\n\n plt.show();","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:03:01.415784Z","iopub.execute_input":"2022-03-14T17:03:01.416265Z","iopub.status.idle":"2022-03-14T17:03:01.425358Z","shell.execute_reply.started":"2022-03-14T17:03:01.416182Z","shell.execute_reply":"2022-03-14T17:03:01.424599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### Histogram of bitrate\n> We can find that bitrate is in a normal distribution ","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(18,6))\nplt.hist(audio_meta[\"bitrate\"], bins=50, edgecolor='#169acf', linewidth=0.7);\nplt.title('Histogram of bitrate') \nplt.xlabel('Bins') \nplt.ylabel('Values')\nplt.show();","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:03:01.426499Z","iopub.execute_input":"2022-03-14T17:03:01.426738Z","iopub.status.idle":"2022-03-14T17:03:01.697957Z","shell.execute_reply.started":"2022-03-14T17:03:01.42671Z","shell.execute_reply":"2022-03-14T17:03:01.697007Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### Histogram of filesize:\nLooking like the filesize is also normally distributed","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(18,6))\nplt.hist(audio_meta[\"filesize\"], bins=90, edgecolor='#169acf', linewidth=0.5);\nplt.title('Histogram of filesize') \nplt.xlabel('Bins') \nplt.ylabel('Values')\nplt.show();","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:03:01.699367Z","iopub.execute_input":"2022-03-14T17:03:01.699675Z","iopub.status.idle":"2022-03-14T17:03:02.282741Z","shell.execute_reply.started":"2022-03-14T17:03:01.699634Z","shell.execute_reply":"2022-03-14T17:03:02.28187Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### Sample Rate count plot ","metadata":{}},{"cell_type":"code","source":"bar_plot(audio_meta[\"sample_rate\"],\"sample_rate\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:03:02.285465Z","iopub.execute_input":"2022-03-14T17:03:02.285743Z","iopub.status.idle":"2022-03-14T17:03:02.450016Z","shell.execute_reply.started":"2022-03-14T17:03:02.28571Z","shell.execute_reply":"2022-03-14T17:03:02.44932Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### Duration Count Plot:\n> Maybe we need to delete those short audio files ","metadata":{}},{"cell_type":"code","source":"bar_plot(audio_meta[\"duration\"].astype(np.int8),\"duration\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:03:02.45127Z","iopub.execute_input":"2022-03-14T17:03:02.452117Z","iopub.status.idle":"2022-03-14T17:03:02.628033Z","shell.execute_reply.started":"2022-03-14T17:03:02.45207Z","shell.execute_reply":"2022-03-14T17:03:02.627366Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### nominal_bitrate Count plot","metadata":{}},{"cell_type":"code","source":"bar_plot(audio_meta[\"nominal_bitrate\"],\"nominal_bitrate\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:03:02.629327Z","iopub.execute_input":"2022-03-14T17:03:02.63003Z","iopub.status.idle":"2022-03-14T17:03:02.856937Z","shell.execute_reply.started":"2022-03-14T17:03:02.629983Z","shell.execute_reply":"2022-03-14T17:03:02.856152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### channels Count plot:\nSeems ike mostly are 2 channel audio files.","metadata":{}},{"cell_type":"code","source":"bar_plot(audio_meta[\"channels\"],\"channels\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-14T17:03:02.858246Z","iopub.execute_input":"2022-03-14T17:03:02.859039Z","iopub.status.idle":"2022-03-14T17:03:03.058335Z","shell.execute_reply.started":"2022-03-14T17:03:02.858989Z","shell.execute_reply":"2022-03-14T17:03:03.057245Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ✂Model on Metadata:","metadata":{}},{"cell_type":"code","source":"Y = audio_meta['genre']\nX = audio_meta.drop(['genre'],axis = 1)","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:03.060033Z","iopub.execute_input":"2022-03-14T17:03:03.060367Z","iopub.status.idle":"2022-03-14T17:03:03.065756Z","shell.execute_reply.started":"2022-03-14T17:03:03.060324Z","shell.execute_reply":"2022-03-14T17:03:03.065205Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"SEED = 42\nfrom sklearn.model_selection import train_test_split\n\nx_train,x_val,y_train,y_val = train_test_split(X,Y,test_size = 0.2,random_state = SEED)","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:03.066741Z","iopub.execute_input":"2022-03-14T17:03:03.067469Z","iopub.status.idle":"2022-03-14T17:03:03.084493Z","shell.execute_reply.started":"2022-03-14T17:03:03.067431Z","shell.execute_reply":"2022-03-14T17:03:03.083563Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.ensemble import RandomForestClassifier\nfrom sklearn.model_selection import RandomizedSearchCV,GridSearchCV\n\nclassifier_rf = RandomForestClassifier(random_state=SEED)\nclassifier_rf1 = RandomForestClassifier(max_depth=70, min_samples_leaf=4, min_samples_split=5,\n n_estimators=180, random_state=42)\nclassifier_rf1.fit(X,Y)","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:03.085964Z","iopub.execute_input":"2022-03-14T17:03:03.086349Z","iopub.status.idle":"2022-03-14T17:03:07.782947Z","shell.execute_reply.started":"2022-03-14T17:03:03.086309Z","shell.execute_reply":"2022-03-14T17:03:07.782203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df = pd.read_csv(\"../input/kaggle-pog-series-s01e02/test.csv\")\ntest_data = {\n \"song_id\": [],\n \"filesize\":[],\n \"duration\": [],\n \"channels\": [],\n \"bitrate\": [],\n \"max_bitrate\": [],\n \"min_bitrate\": [],\n \"nominal_bitrate\": [],\n \"sample_rate\": [],\n}\n\nmissing_file_list_test = []\n\nfor item in tqdm(range(len(test_df)), total=len(test_df)):\n\n file_path = test_df[\"filepath\"].iloc[item]\n\n try:\n meta = audio_metadata.load(f\"../input/kaggle-pog-series-s01e02/{file_path}\")\n test_meta_list = [test_df[\"song_id\"].iloc[item],meta[\"filesize\"],meta[\"streaminfo\"][\"duration\"],\n meta[\"streaminfo\"][\"channels\"],meta[\"streaminfo\"][\"bitrate\"],meta[\"streaminfo\"][\"max_bitrate\"],\n meta[\"streaminfo\"][\"min_bitrate\"],meta[\"streaminfo\"][\"nominal_bitrate\"],\n meta[\"streaminfo\"][\"sample_rate\"]]\n for num in range(len(test_data.keys())):\n test_data[list(test_data.keys())[num]].append(test_meta_list[num])\n except:\n missing_file_list_test.append(file_path)\n print(f\"could not found: {file_path}\")\naudio_meta_test = pd.DataFrame(test_data)\naudio_meta_test.head() # banie","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:07.78414Z","iopub.execute_input":"2022-03-14T17:03:07.784537Z","iopub.status.idle":"2022-03-14T17:03:47.947578Z","shell.execute_reply.started":"2022-03-14T17:03:07.784502Z","shell.execute_reply":"2022-03-14T17:03:47.946667Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"audio_meta_test.drop([\"max_bitrate\",\"min_bitrate\"],axis=1,inplace=True)\naudio_meta_test.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:47.950481Z","iopub.execute_input":"2022-03-14T17:03:47.951031Z","iopub.status.idle":"2022-03-14T17:03:47.967899Z","shell.execute_reply.started":"2022-03-14T17:03:47.950983Z","shell.execute_reply":"2022-03-14T17:03:47.966843Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"audio_meta_test1 = audio_meta_test.drop([\"song_id\"],axis=1)\naudio_meta_test1.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:47.96946Z","iopub.execute_input":"2022-03-14T17:03:47.969791Z","iopub.status.idle":"2022-03-14T17:03:47.984251Z","shell.execute_reply.started":"2022-03-14T17:03:47.96975Z","shell.execute_reply":"2022-03-14T17:03:47.983269Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred = classifier_rf1.predict(audio_meta_test1)\ny_pred","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:47.985816Z","iopub.execute_input":"2022-03-14T17:03:47.986088Z","iopub.status.idle":"2022-03-14T17:03:48.242217Z","shell.execute_reply.started":"2022-03-14T17:03:47.986058Z","shell.execute_reply":"2022-03-14T17:03:48.241332Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"audio_meta_test[\"predictions\"] = pd.DataFrame(y_pred)\naudio_meta_test.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:48.243996Z","iopub.execute_input":"2022-03-14T17:03:48.244331Z","iopub.status.idle":"2022-03-14T17:03:48.258941Z","shell.execute_reply.started":"2022-03-14T17:03:48.244287Z","shell.execute_reply":"2022-03-14T17:03:48.257924Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_df = pd.read_csv(\"../input/kaggle-pog-series-s01e02/sample_submission.csv\")\nsub_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:48.260623Z","iopub.execute_input":"2022-03-14T17:03:48.260939Z","iopub.status.idle":"2022-03-14T17:03:48.286087Z","shell.execute_reply.started":"2022-03-14T17:03:48.260893Z","shell.execute_reply":"2022-03-14T17:03:48.285239Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"list_preds = audio_meta_test[\"predictions\"].to_list()\nprint(len(list_preds))\nlist_preds.insert(3546,1)\nlist_preds.insert(4249,0)\nprint(len(list_preds))","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:48.287469Z","iopub.execute_input":"2022-03-14T17:03:48.28769Z","iopub.status.idle":"2022-03-14T17:03:48.29444Z","shell.execute_reply.started":"2022-03-14T17:03:48.287664Z","shell.execute_reply":"2022-03-14T17:03:48.293532Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_genre_id = sub_df[\"genre_id\"].to_list()\nfor i in range(len(sub_genre_id)):\n sub_genre_id[i] = list_preds[i]","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:04:35.802483Z","iopub.execute_input":"2022-03-14T17:04:35.803037Z","iopub.status.idle":"2022-03-14T17:04:35.809377Z","shell.execute_reply.started":"2022-03-14T17:04:35.802981Z","shell.execute_reply":"2022-03-14T17:04:35.808663Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_df[\"genre_id\"] = pd.DataFrame(sub_genre_id)\nsub_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:04:37.506775Z","iopub.execute_input":"2022-03-14T17:04:37.507096Z","iopub.status.idle":"2022-03-14T17:04:37.520102Z","shell.execute_reply.started":"2022-03-14T17:04:37.507051Z","shell.execute_reply":"2022-03-14T17:04:37.519247Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub_df.to_csv(\"submissions.csv\",index=False)\npd.read_csv(\"./submissions.csv\").head()","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:04:39.878695Z","iopub.execute_input":"2022-03-14T17:04:39.87899Z","iopub.status.idle":"2022-03-14T17:04:39.905801Z","shell.execute_reply.started":"2022-03-14T17:04:39.878961Z","shell.execute_reply":"2022-03-14T17:04:39.90523Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# inspiration: https://www.kaggle.com/tatamikenn/birdclef2022-train-metadata-with-audio-metadata","metadata":{"execution":{"iopub.status.busy":"2022-03-14T17:03:48.394003Z","iopub.status.idle":"2022-03-14T17:03:48.394414Z","shell.execute_reply.started":"2022-03-14T17:03:48.394204Z","shell.execute_reply":"2022-03-14T17:03:48.394228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> # ⭕ WORK IN PROGRESS ! ! !\n

\n\n

","metadata":{}}]}