{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":21669,"databundleVersionId":1692278,"sourceType":"competition"}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"papermill":{"default_parameters":{},"duration":4179.363865,"end_time":"2023-11-29T19:47:43.525575","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2023-11-29T18:38:04.161710","version":"2.4.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import torch.nn as nn\nimport numpy as np\nimport librosa\nimport os\n\nfrom torch.utils.data import Dataset, DataLoader\nfrom sklearn.model_selection import KFold","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.execute_input":"2023-11-29T18:38:07.466537Z","iopub.status.busy":"2023-11-29T18:38:07.466163Z","iopub.status.idle":"2023-11-29T18:38:11.380932Z","shell.execute_reply":"2023-11-29T18:38:11.380117Z"},"papermill":{"duration":3.923495,"end_time":"2023-11-29T18:38:11.383251","exception":false,"start_time":"2023-11-29T18:38:07.459756","status":"completed"},"tags":[],"ExecuteTime":{"end_time":"2023-12-09T11:44:47.552596900Z","start_time":"2023-12-09T11:44:39.411786400Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\n\nnum_labels = 24\n\ndevice = 'cuda:0' if torch.cuda.is_available() else 'cpu'\nprint(device)","metadata":{"execution":{"iopub.execute_input":"2023-11-29T18:38:11.395361Z","iopub.status.busy":"2023-11-29T18:38:11.394987Z","iopub.status.idle":"2023-11-29T18:38:11.450572Z","shell.execute_reply":"2023-11-29T18:38:11.449682Z"},"papermill":{"duration":0.063281,"end_time":"2023-11-29T18:38:11.452595","exception":false,"start_time":"2023-11-29T18:38:11.389314","status":"completed"},"tags":[],"ExecuteTime":{"end_time":"2023-12-09T11:44:47.570934Z","start_time":"2023-12-09T11:44:47.544640200Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Определяем функции для работы со спектрограммами","metadata":{"papermill":{"duration":0.004292,"end_time":"2023-11-29T18:38:11.461568","exception":false,"start_time":"2023-11-29T18:38:11.457276","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from skimage.transform import resize\nfrom skimage.filters import gaussian\nfrom skimage.color import rgb2gray\nfrom skimage import exposure, util\n\n\ndef horizontal_flip(image: np.ndarray) -> np.ndarray:\n    horizontal_flipped_img = image[:, ::-1]\n    return add_channels(horizontal_flipped_img)\n\ndef vertical_flip(image: np.ndarray) -> np.ndarray:\n    vertical_flipped_img = image[::-1, :]\n    return add_channels(vertical_flipped_img)\n\ndef add_noisy(image: np.ndarray) -> np.ndarray:\n    noised_img = util.random_noise(image)\n    return add_channels(noised_img)\n\ndef contrast_stretching(image: np.ndarray) -> np.ndarray:\n    contrasted_img = exposure.rescale_intensity(image)\n    return add_channels(contrasted_img)\n\ndef random_gaussian(image: np.ndarray) -> np.ndarray:\n    gaussian_img = gaussian(image)\n    return add_channels(gaussian_img)\n\ndef gray_scale(image: np.ndarray) -> np.ndarray:\n    gray_img = rgb2gray(image)\n    return add_channels(gray_img)\n\ndef random_gamma(image: np.ndarray) -> np.ndarray:\n    img_gamma = exposure.adjust_gamma(image)\n    return add_channels(img_gamma)\n\ndef add_channels(image: np.ndarray):\n    return np.stack((image, image, image))\n\ndef spec_to_image(spec: np.ndarray) -> np.ndarray:\n    spec = resize(spec, (224, 400))\n    eps=1e-6\n    mean = spec.mean()\n    std = spec.std()\n    spec_norm = (spec - mean) / (std + eps)\n    spec_min, spec_max = spec_norm.min(), spec_norm.max()\n    spec_scaled = 255 * (spec_norm - spec_min) / (spec_max - spec_min)\n    spec_scaled = spec_scaled.astype(np.uint8)\n    spec_scaled = np.asarray(spec_scaled)\n    return spec_scaled","metadata":{"execution":{"iopub.execute_input":"2023-11-29T18:38:11.471707Z","iopub.status.busy":"2023-11-29T18:38:11.471397Z","iopub.status.idle":"2023-11-29T18:38:11.727741Z","shell.execute_reply":"2023-11-29T18:38:11.726801Z"},"papermill":{"duration":0.264034,"end_time":"2023-11-29T18:38:11.730119","exception":false,"start_time":"2023-11-29T18:38:11.466085","status":"completed"},"tags":[],"ExecuteTime":{"end_time":"2023-12-09T11:44:47.849170400Z","start_time":"2023-12-09T11:44:47.563917700Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Считывание входных данных для обучения","metadata":{"papermill":{"duration":0.004601,"end_time":"2023-11-29T18:38:11.739623","exception":false,"start_time":"2023-11-29T18:38:11.735022","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import pandas as pd\n\nsr = 48000\nlength = 10 * sr\ndata = pd.read_csv(\"../input/rfcx-species-audio-detection/train_tp.csv\")\n\nfmin = sr / 2\nfmax = 0\nfor i in range(0, len(data)):\n    if fmin > float(data.iloc[i]['f_min']):\n        fmin = float(data.iloc[i]['f_min'])\n    if fmax < float(data.iloc[i]['f_max']):\n        fmax = float(data.iloc[i]['f_max'])\n        \nfmin = int(fmin * 0.9)\nfmax = int(fmax * 1.1)\n\nlabel_list = []\ndata_list = []\naudio_data = {}\nfor i in range(0, len(data)):\n    recording_id = data.recording_id.values[i]\n    species_id = int(data.species_id.values[i])\n    data_list.append(recording_id)\n    label_list.append(species_id)\n\n    audio, sr = librosa.load('../input/rfcx-species-audio-detection/train/' + recording_id + '.flac', sr=None)\n    t_min = float(data.t_min.values[i]) * sr\n    t_max = float(data.t_max.values[i]) * sr\n    center = np.round((t_min + t_max) / 2)\n    beginning = center - length / 2\n    if beginning < 0:\n        beginning = 0\n    ending = beginning + length\n    if ending > len(audio):\n        ending = len(audio)\n        beginning = ending - length\n    cropped_audio = audio[int(beginning):int(ending)]\n    \n    spec = librosa.feature.melspectrogram(y=cropped_audio, sr=sr, fmin=fmin, fmax=fmax)\n    spec_db = librosa.power_to_db(spec, top_db=80)\n    \n    img = spec_to_image(spec_db)\n    \n    audio_data[recording_id] = img","metadata":{"execution":{"iopub.execute_input":"2023-11-29T18:38:11.750003Z","iopub.status.busy":"2023-11-29T18:38:11.749699Z","iopub.status.idle":"2023-11-29T18:43:14.709323Z","shell.execute_reply":"2023-11-29T18:43:14.707930Z"},"papermill":{"duration":302.969224,"end_time":"2023-11-29T18:43:14.713337","exception":false,"start_time":"2023-11-29T18:38:11.744113","status":"completed"},"tags":[],"ExecuteTime":{"end_time":"2023-12-09T11:47:01.663877300Z","start_time":"2023-12-09T11:44:47.843120400Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Обучение","metadata":{"papermill":{"duration":0.009538,"end_time":"2023-11-29T18:43:14.733228","exception":false,"start_time":"2023-11-29T18:43:14.723690","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import copy\nfrom tqdm import tqdm\n\nlearning_rate = 1e-4\nepochs = 20\nloss_fn = nn.CrossEntropyLoss()\n\ndef train(model, loss_fn, train_loader, valid_loader, epochs, optimizer, scheduler):\n    best_model_wts = copy.deepcopy(model.state_dict())\n    best_acc = 0.0\n    train_losses = []\n    valid_losses = []\n    \n    for epoch in tqdm(range(1,epochs+1)):\n        model.train()\n        batch_losses=[]\n        for _, data in enumerate(train_loader):\n            x, y = data\n            optimizer.zero_grad()\n            x = x.to(device, dtype=torch.float32)\n            y = y.to(device, dtype=torch.long)\n            y_hat = model(x)\n            loss = loss_fn(y_hat, y)\n            loss.backward()\n            batch_losses.append(loss.item())\n            optimizer.step()\n        train_losses.append(batch_losses)\n\n        model.eval()\n        batch_losses=[]\n        trace_y = []\n        trace_yhat = []\n        \n        for _, data in enumerate(valid_loader):\n            x, y = data\n            x = x.to(device, dtype=torch.float32)\n            y = y.to(device, dtype=torch.long)\n            y_hat = model(x)\n            loss = loss_fn(y_hat, y)\n            trace_y.append(y.cpu().detach().numpy())\n            trace_yhat.append(y_hat.cpu().detach().numpy())      \n            batch_losses.append(loss.item())\n        valid_losses.append(batch_losses)\n        trace_y = np.concatenate(trace_y)\n        trace_yhat = np.concatenate(trace_yhat)\n        accuracy = np.mean(trace_yhat.argmax(axis=1)==trace_y)\n        \n        print(\"epoch = %d, train_loss = %.5f, val_loss = %.5f, val_accuracy = %.5f\" % (epoch, np.mean(train_losses[-1]), np.mean(valid_losses[-1]), accuracy))\n\n        scheduler.step(np.mean(valid_losses[-1]))\n        if accuracy > best_acc:\n            best_acc = accuracy\n            best_model_wts = copy.deepcopy(model.state_dict())\n\n    model.load_state_dict(best_model_wts)\n    return model","metadata":{"execution":{"iopub.execute_input":"2023-11-29T18:43:14.755785Z","iopub.status.busy":"2023-11-29T18:43:14.754743Z","iopub.status.idle":"2023-11-29T18:43:14.780760Z","shell.execute_reply":"2023-11-29T18:43:14.779570Z"},"papermill":{"duration":0.041107,"end_time":"2023-11-29T18:43:14.784055","exception":false,"start_time":"2023-11-29T18:43:14.742948","status":"completed"},"tags":[],"ExecuteTime":{"end_time":"2023-12-09T11:47:01.711321200Z","start_time":"2023-12-09T11:47:01.667907600Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import random\n\nclass AudioData(Dataset):\n    def __init__(self, X, y, data_type):\n        self.data = []\n        self.labels = []\n        self.hindrances = [\n            add_noisy, contrast_stretching, random_gaussian,random_gamma, \n            vertical_flip, horizontal_flip, add_channels\n        ]\n        self.data_type=data_type\n        for i in range(0, len(X)):\n            recording_id = X[i]\n            label = y[i]\n            mel_spec = audio_data[recording_id]\n            self.data.append(mel_spec)\n            self.labels.append(label)\n                \n    def __len__(self):\n        return len(self.data)\n\n    def __getitem__(self, idx):\n        if self.data_type == \"train\":\n            aug= random.choice(self.hindrances)\n            data = aug(self.data[idx])\n        else:\n            data = add_channels(self.data[idx])\n        return data, self.labels[idx]","metadata":{"execution":{"iopub.execute_input":"2023-11-29T18:43:14.805710Z","iopub.status.busy":"2023-11-29T18:43:14.805178Z","iopub.status.idle":"2023-11-29T18:43:14.819150Z","shell.execute_reply":"2023-11-29T18:43:14.817991Z"},"papermill":{"duration":0.028405,"end_time":"2023-11-29T18:43:14.822318","exception":false,"start_time":"2023-11-29T18:43:14.793913","status":"completed"},"tags":[],"ExecuteTime":{"end_time":"2023-12-09T11:47:01.726879200Z","start_time":"2023-12-09T11:47:01.701799100Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"В качетсве модели была использована resnet101. Выбор ResNet-101 обусловлен его способностью извлекать сложные пространственные и временные зависимости из изображений, что является важным при анализе спектрограмм, представляющих звуковые данные.","metadata":{"papermill":{"duration":0.004411,"end_time":"2023-11-29T18:43:14.834021","exception":false,"start_time":"2023-11-29T18:43:14.829610","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from torchvision.models import resnet101\n\ndef get_model():\n    model = resnet101(pretrained=True)\n    num_ftrs = model.fc.in_features\n    model.fc = nn.Linear(num_ftrs, num_labels)\n    model = model.to(device)\n    return model","metadata":{"execution":{"iopub.execute_input":"2023-11-29T18:43:14.845083Z","iopub.status.busy":"2023-11-29T18:43:14.844273Z","iopub.status.idle":"2023-11-29T18:43:15.150426Z","shell.execute_reply":"2023-11-29T18:43:15.149476Z"},"papermill":{"duration":0.31421,"end_time":"2023-11-29T18:43:15.152869","exception":false,"start_time":"2023-11-29T18:43:14.838659","status":"completed"},"tags":[],"ExecuteTime":{"end_time":"2023-12-09T11:47:03.370150800Z","start_time":"2023-12-09T11:47:01.725881900Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fold_num = 4\nskf = KFold(n_splits=fold_num, shuffle=True, random_state=32)\n\nfor fold_id, (train_index, val_index) in enumerate(skf.split(data_list, label_list)):\n    X_train = np.take(data_list, train_index)\n    y_train = np.take(label_list, train_index, axis = 0)\n    X_val = np.take(data_list, val_index)\n    y_val = np.take(label_list, val_index, axis = 0)\n\n    train_data = AudioData(X_train, y_train, \"train\")\n    valid_data = AudioData(X_val, y_val, \"valid\")\n    train_loader = DataLoader(train_data, batch_size=8, shuffle=True, drop_last=True)\n    valid_loader = DataLoader(valid_data, batch_size=8, shuffle=True, drop_last=True)\n\n    model = get_model()\n    optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)\n    scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, 'min', patience=3)\n    model = train(model, loss_fn, train_loader, valid_loader, epochs, optimizer, scheduler)\n    torch.save(model.state_dict(), \"./model\" + str(fold_id) + \".pt\")\n    \n    del train_data, valid_data, train_loader, valid_loader, model, X_train, X_val, y_train, y_val","metadata":{"execution":{"iopub.execute_input":"2023-11-29T18:43:15.164076Z","iopub.status.busy":"2023-11-29T18:43:15.163441Z","iopub.status.idle":"2023-11-29T19:09:54.001395Z","shell.execute_reply":"2023-11-29T19:09:54.000358Z"},"papermill":{"duration":1598.846465,"end_time":"2023-11-29T19:09:54.004269","exception":false,"start_time":"2023-11-29T18:43:15.157804","status":"completed"},"tags":[],"is_executing":true,"ExecuteTime":{"start_time":"2023-12-09T11:47:03.371151500Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_test_file(f):\n    audio, sr = librosa.load('../input/rfcx-species-audio-detection/test/' + f, sr=None)\n\n    segments = len(audio) / length\n    segments = int(np.ceil(segments))\n    \n    mel_array = []\n    \n    for i in range(0, segments):\n        if (i + 1) * length > len(audio):\n            cropped_audio = audio[len(audio) - length:len(audio)]\n        else:\n            cropped_audio = audio[i * length:(i + 1) * length]\n        \n        spec = librosa.feature.melspectrogram(y=cropped_audio, sr=sr, fmin=fmin, fmax=fmax)\n        spec_db = librosa.power_to_db(spec,top_db=80)\n\n        img = spec_to_image(spec_db)\n        mel_spec = np.stack((img, img, img))\n        mel_array.append(mel_spec)\n    \n    return mel_array","metadata":{"execution":{"iopub.execute_input":"2023-11-29T19:09:54.046179Z","iopub.status.busy":"2023-11-29T19:09:54.045342Z","iopub.status.idle":"2023-11-29T19:09:54.053076Z","shell.execute_reply":"2023-11-29T19:09:54.052331Z"},"papermill":{"duration":0.030554,"end_time":"2023-11-29T19:09:54.055029","exception":false,"start_time":"2023-11-29T19:09:54.024475","status":"completed"},"tags":[],"is_executing":true,"ExecuteTime":{"start_time":"2023-12-09T13:59:37.680122200Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"members = []\nfor i in range(fold_num):\n    model = get_model()\n    model.load_state_dict(torch.load('./model'+str(i)+'.pt'))\n    model.eval()\n    members.append(model)\n    \nos.remove('./model0.pt') \nos.remove('./model1.pt')\nos.remove('./model2.pt') \nos.remove('./model3.pt')","metadata":{"execution":{"iopub.execute_input":"2023-11-29T19:09:54.096274Z","iopub.status.busy":"2023-11-29T19:09:54.095990Z","iopub.status.idle":"2023-11-29T19:09:58.321214Z","shell.execute_reply":"2023-11-29T19:09:58.320199Z"},"papermill":{"duration":4.2489,"end_time":"2023-11-29T19:09:58.323727","exception":false,"start_time":"2023-11-29T19:09:54.074827","status":"completed"},"tags":[],"is_executing":true,"ExecuteTime":{"start_time":"2023-12-09T13:59:37.691821500Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import csv\n\nwith open('submission.csv', 'w', newline='') as csvfile:\n    submission_writer = csv.writer(csvfile, delimiter=',')\n    submission_writer.writerow(['recording_id','s0','s1','s2','s3','s4','s5','s6','s7','s8','s9','s10','s11',\n                               's12','s13','s14','s15','s16','s17','s18','s19','s20','s21','s22','s23'])\n    \n    test_files = os.listdir('../input/rfcx-species-audio-detection/test/')\n    \n    for i in range(0, len(test_files)):\n        data = load_test_file(test_files[i])\n        data = torch.tensor(data)\n        data = data.float()\n        if torch.cuda.is_available():\n            data = data.cuda()\n\n        output_list = []\n        for m in members:\n            output = m(data)\n            maxed_output = torch.max(output, dim=0)[0]\n            maxed_output = maxed_output.cpu().detach()\n            output_list.append(maxed_output)\n        avg_maxed_output = torch.mean(torch.stack(output_list), dim=0)\n        \n        file_id = str.split(test_files[i], '.')[0]\n        write_array = [file_id]\n        \n        for out in avg_maxed_output:\n            write_array.append(out.item())\n    \n        submission_writer.writerow(write_array)\n","metadata":{"execution":{"iopub.execute_input":"2023-11-29T19:09:58.365265Z","iopub.status.busy":"2023-11-29T19:09:58.364938Z","iopub.status.idle":"2023-11-29T19:47:41.142652Z","shell.execute_reply":"2023-11-29T19:47:41.141827Z"},"papermill":{"duration":2262.801279,"end_time":"2023-11-29T19:47:41.145213","exception":false,"start_time":"2023-11-29T19:09:58.343934","status":"completed"},"tags":[],"is_executing":true,"ExecuteTime":{"start_time":"2023-12-09T13:59:37.692854300Z"}},"execution_count":null,"outputs":[]}]}