{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Update(2020/08/06)\nThis notebook has fatal problem.\nWe test run is successful use birdcall-check dataset.  \nbut, public score don't move at 0.544.  \nI try debug use try-except.    \nApparently, this notebook gives an error where it calculates the prediction.  \nif you solve this error, please comment me.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# About\n\nThis notebook is write about simple audio classification for beginner.    \nPreprocessing use torchaudio, and cnn written by pytorch.   \nI would be happy if even a little help beginners to try.\n\n__Attention__  \nThis notebook author use so so bad english.  \nI would be happy if you could read it easily :)  \n\n__Thank__  \nI would like thank for some great Data Scientist.  \n\n- [@radek1](https://www.kaggle.com/radek1)  \n  I uesd [dataset](https://www.kaggle.com/c/birdsong-recognition/discussion/160222) for train that he processed to convert 32kHz.  \n  thank you for your great job!\n  \n- [@hidehisaarai1213](https://www.kaggle.com/hidehisaarai1213)  \n  This notebook is enormous inspiration from he sheard [notebook](https://www.kaggle.com/hidehisaarai1213/inference-pytorch-birdcall-resnet-baseline).    \n  And, several code use for predict in this note.  \n  thank you for sheard notebook!\n \n- [@shonenkov](https://www.kaggle.com/shonenkov)  \n  When I codeing predict code, I used dataset that create by shonenkov.\n  thank you for your great job!\n  \n- [@stefankahl](https://www.kaggle.com/stefankahl),\n  [@tomdenton](https://www.kaggle.com/tomdenton),\n  [@sohier](https://www.kaggle.com/sohier)  \n  thank you for hosting a really interesting and meaningfull competition!!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Audio data is what talk it?\nWhat do you think about sound? What is sound?  \nsound is wave that vibration by air.\n\ntherfore, we can visualize vibration.","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import torchaudio\nimport matplotlib.pyplot as plt\n\nwaveform, sample_rate = torchaudio.load(\"../input/birdsong-recognition/train_audio/aldfly/XC134874.mp3\")\n\nplt.plot(waveform.t().numpy())\nplt.xlabel(\"time\")\nplt.ylabel(\"signal\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Raw sound data is Time series.  \nThis competiton is classification problem.  \nTherefore we can think classification by time series like text classification. \n\nhowever sound data is big tensor.  \nbecuse sound is sampling about 8M to 32MHz.  \nThis is say that take 32000 point in one second.  \nIt will be __160,000__ point that sampling only 5 second!  \n\nTherefore, we take picture that convert from sound.\n\n\n### spectrogram  \nSpectrogram is visualize by frequency and signal strength.  \nFrequency is calculated by FFT（fast Fourier transform）  \nIn this notebook, training and predict use mel spectrogram.  \n\nmel spectrogram is convert mel scale from frequency.  \nIf you want know detailed mel spectrogrm, Please see this page.  \n[→Getting to Know the Mel Spectrogram](Detailed)  \nIt's more humorous and understandable than I explain！","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"mel_specgram = torchaudio.transforms.MelSpectrogram(sample_rate, n_fft=2**11, f_max=8000)(waveform)\nmel_specgram= torchaudio.transforms.AmplitudeToDB(top_db=80)(mel_specgram)\n\nplt.figure()\nplt.title(\"exsample mel Spectrogram\")\nplt.imshow(mel_specgram[0].detach().numpy()[::-1], cmap='magma',aspect=5);\nplt.xlabel(\"time\")\nplt.ylabel(\"mel scale\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We cloud convert picture from sound.  \nIf picture classification, it is commonplace problem isn't it?\n\nIn this notebook use by simple cnn to classification.\n\nLet's use spectrogram to find out who is talking！！","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# code","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import logging\nimport os\nimport random\nimport time\nimport warnings\nfrom typing import Optional\nfrom fastprogress import progress_bar\nfrom contextlib import contextmanager\n\nimport torch\nimport torchaudio\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\nfrom pathlib import Path\n\nfrom sklearn.model_selection import train_test_split\n\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.optim import Adam\nfrom torch.distributions import Uniform\nfrom torch.utils.data import DataLoader, Dataset\n\nfrom torchaudio.transforms import Spectrogram, MelSpectrogram\nfrom torchaudio.transforms import TimeStretch, AmplitudeToDB, ComplexNorm, Resample\nfrom torchaudio.transforms import FrequencyMasking, TimeMasking","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Utilities\ncode write by @hidehisaarai1213 thamks!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def set_seed(seed: int = 42):\n    random.seed(seed)\n    np.random.seed(seed)\n    os.environ[\"PYTHONHASHSEED\"] = str(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)  # type: ignore\n    torch.backends.cudnn.deterministic = True  # type: ignore\n    torch.backends.cudnn.benchmark = True  # type: ignore\n    \n    \ndef get_logger(out_file=None):\n    logger = logging.getLogger()\n    formatter = logging.Formatter(\"%(asctime)s - %(levelname)s - %(message)s\")\n    logger.handlers = []\n    logger.setLevel(logging.INFO)\n\n    handler = logging.StreamHandler()\n    handler.setFormatter(formatter)\n    handler.setLevel(logging.INFO)\n    logger.addHandler(handler)\n\n    if out_file is not None:\n        fh = logging.FileHandler(out_file)\n        fh.setFormatter(formatter)\n        fh.setLevel(logging.INFO)\n        logger.addHandler(fh)\n    logger.info(\"logger set up\")\n    return logger\n\n@contextmanager\ndef timer(name: str, logger: Optional[logging.Logger] = None):\n    t0 = time.time()\n    msg = f\"[{name}] start\"\n    if logger is None:\n        print(msg)\n    else:\n        logger.info(msg)\n    yield\n\n    msg = f\"[{name}] done in {time.time() - t0:.2f} s\"\n    if logger is None:\n        print(msg)\n    else:\n        logger.info(msg)\n\nlogger = get_logger(\"main.log\");\nset_seed(1213);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Data Loding","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"TARGET_SR = 32000\nTEST = Path(\"../input/birdsong-recognition/test_audio\").exists()\n\nif TEST:\n    DATA_DIR = Path(\"../input/birdsong-recognition/\")\nelse:\n    # dataset created by @shonenkov, thanks!\n    DATA_DIR = Path(\"../input/birdcall-check/\")\n\n\ntest = pd.read_csv(DATA_DIR / \"test.csv\")\ntest_audio = DATA_DIR / \"test_audio\"\n\nMODE_DIR = Path(\"../input/birdcallfirstmodelcnn/\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#sub = pd.read_csv(\"../input/birdsong-recognition/sample_submission.csv\")\n#sub.to_csv(\"submission.csv\", index=False)  # this will be overwritten if everything goes well","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BIRD_CODE = {\n    'aldfly': 0, 'ameavo': 1, 'amebit': 2, 'amecro': 3, 'amegfi': 4,\n    'amekes': 5, 'amepip': 6, 'amered': 7, 'amerob': 8, 'amewig': 9,\n    'amewoo': 10, 'amtspa': 11, 'annhum': 12, 'astfly': 13, 'baisan': 14,\n    'baleag': 15, 'balori': 16, 'banswa': 17, 'barswa': 18, 'bawwar': 19,\n    'belkin1': 20, 'belspa2': 21, 'bewwre': 22, 'bkbcuc': 23, 'bkbmag1': 24,\n    'bkbwar': 25, 'bkcchi': 26, 'bkchum': 27, 'bkhgro': 28, 'bkpwar': 29,\n    'bktspa': 30, 'blkpho': 31, 'blugrb1': 32, 'blujay': 33, 'bnhcow': 34,\n    'boboli': 35, 'bongul': 36, 'brdowl': 37, 'brebla': 38, 'brespa': 39,\n    'brncre': 40, 'brnthr': 41, 'brthum': 42, 'brwhaw': 43, 'btbwar': 44,\n    'btnwar': 45, 'btywar': 46, 'buffle': 47, 'buggna': 48, 'buhvir': 49,\n    'bulori': 50, 'bushti': 51, 'buwtea': 52, 'buwwar': 53, 'cacwre': 54,\n    'calgul': 55, 'calqua': 56, 'camwar': 57, 'cangoo': 58, 'canwar': 59,\n    'canwre': 60, 'carwre': 61, 'casfin': 62, 'caster1': 63, 'casvir': 64,\n    'cedwax': 65, 'chispa': 66, 'chiswi': 67, 'chswar': 68, 'chukar': 69,\n    'clanut': 70, 'cliswa': 71, 'comgol': 72, 'comgra': 73, 'comloo': 74,\n    'commer': 75, 'comnig': 76, 'comrav': 77, 'comred': 78, 'comter': 79,\n    'comyel': 80, 'coohaw': 81, 'coshum': 82, 'cowscj1': 83, 'daejun': 84,\n    'doccor': 85, 'dowwoo': 86, 'dusfly': 87, 'eargre': 88, 'easblu': 89,\n    'easkin': 90, 'easmea': 91, 'easpho': 92, 'eastow': 93, 'eawpew': 94,\n    'eucdov': 95, 'eursta': 96, 'evegro': 97, 'fiespa': 98, 'fiscro': 99,\n    'foxspa': 100, 'gadwal': 101, 'gcrfin': 102, 'gnttow': 103, 'gnwtea': 104,\n    'gockin': 105, 'gocspa': 106, 'goleag': 107, 'grbher3': 108, 'grcfly': 109,\n    'greegr': 110, 'greroa': 111, 'greyel': 112, 'grhowl': 113, 'grnher': 114,\n    'grtgra': 115, 'grycat': 116, 'gryfly': 117, 'haiwoo': 118, 'hamfly': 119,\n    'hergul': 120, 'herthr': 121, 'hoomer': 122, 'hoowar': 123, 'horgre': 124,\n    'horlar': 125, 'houfin': 126, 'houspa': 127, 'houwre': 128, 'indbun': 129,\n    'juntit1': 130, 'killde': 131, 'labwoo': 132, 'larspa': 133, 'lazbun': 134,\n    'leabit': 135, 'leafly': 136, 'leasan': 137, 'lecthr': 138, 'lesgol': 139,\n    'lesnig': 140, 'lesyel': 141, 'lewwoo': 142, 'linspa': 143, 'lobcur': 144,\n    'lobdow': 145, 'logshr': 146, 'lotduc': 147, 'louwat': 148, 'macwar': 149,\n    'magwar': 150, 'mallar3': 151, 'marwre': 152, 'merlin': 153, 'moublu': 154,\n    'mouchi': 155, 'moudov': 156, 'norcar': 157, 'norfli': 158, 'norhar2': 159,\n    'normoc': 160, 'norpar': 161, 'norpin': 162, 'norsho': 163, 'norwat': 164,\n    'nrwswa': 165, 'nutwoo': 166, 'olsfly': 167, 'orcwar': 168, 'osprey': 169,\n    'ovenbi1': 170, 'palwar': 171, 'pasfly': 172, 'pecsan': 173, 'perfal': 174,\n    'phaino': 175, 'pibgre': 176, 'pilwoo': 177, 'pingro': 178, 'pinjay': 179,\n    'pinsis': 180, 'pinwar': 181, 'plsvir': 182, 'prawar': 183, 'purfin': 184,\n    'pygnut': 185, 'rebmer': 186, 'rebnut': 187, 'rebsap': 188, 'rebwoo': 189,\n    'redcro': 190, 'redhea': 191, 'reevir1': 192, 'renpha': 193, 'reshaw': 194,\n    'rethaw': 195, 'rewbla': 196, 'ribgul': 197, 'rinduc': 198, 'robgro': 199,\n    'rocpig': 200, 'rocwre': 201, 'rthhum': 202, 'ruckin': 203, 'rudduc': 204,\n    'rufgro': 205, 'rufhum': 206, 'rusbla': 207, 'sagspa1': 208, 'sagthr': 209,\n    'savspa': 210, 'saypho': 211, 'scatan': 212, 'scoori': 213, 'semplo': 214,\n    'semsan': 215, 'sheowl': 216, 'shshaw': 217, 'snobun': 218, 'snogoo': 219,\n    'solsan': 220, 'sonspa': 221, 'sora': 222, 'sposan': 223, 'spotow': 224,\n    'stejay': 225, 'swahaw': 226, 'swaspa': 227, 'swathr': 228, 'treswa': 229,\n    'truswa': 230, 'tuftit': 231, 'tunswa': 232, 'veery': 233, 'vesspa': 234,\n    'vigswa': 235, 'warvir': 236, 'wesblu': 237, 'wesgre': 238, 'weskin': 239,\n    'wesmea': 240, 'wessan': 241, 'westan': 242, 'wewpew': 243, 'whbnut': 244,\n    'whcspa': 245, 'whfibi': 246, 'whtspa': 247, 'whtswi': 248, 'wilfly': 249,\n    'wilsni1': 250, 'wiltur': 251, 'winwre3': 252, 'wlswar': 253, 'wooduc': 254,\n    'wooscj2': 255, 'woothr': 256, 'y00475': 257, 'yebfly': 258, 'yebsap': 259,\n    'yehbla': 260, 'yelwar': 261, 'yerwar': 262, 'yetvir': 263\n}\n\nINV_BIRD_CODE = {v: k for k, v in BIRD_CODE.items()}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model\n\nCNN has 3 layer that simple conv2D and some Norm layer.  \nInput data has one chanel, because spectrogram is gray scale picture.\n\nTo assume data is 5sec sound.  \nspectrogram is calculation at in the model.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class RondomStretchMelSpectrogram(nn.Module):\n    def __init__(self, sample_rate, n_fft, top_db, max_perc):\n        super().__init__()\n        self.time_stretch = TimeStretch(hop_length=None, n_freq=n_fft//2+1)\n        self.stft = Spectrogram(n_fft=n_fft, power=None)\n        self.com_norm = ComplexNorm(power=2.)\n        self.mel_specgram = MelSpectrogram(sample_rate, n_fft=n_fft, f_max=8000)\n        self.AtoDB= AmplitudeToDB(top_db=top_db)\n    \n    def forward(self, x):\n        x = self.stft(x)\n        x = self.com_norm(x)\n        x = self.mel_specgram.mel_scale(x)\n        x = self.AtoDB(x)\n\n        return x","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class cnn_audio(nn.Module):\n    def __init__(self, \n                 output_class=264,\n                 d_size=256,\n                 sample_rate=32000, \n                 n_fft=2**11, \n                 top_db=80,\n                 max_perc=0.4):\n        \n        super().__init__()\n        self.mel = RondomStretchMelSpectrogram(sample_rate, n_fft, top_db, max_perc)\n\n        self.conv1 = nn.Conv2d(1, 64, kernel_size=3, stride=(1, 1))\n        self.bn1 = nn.BatchNorm2d(64)\n        self.relu = nn.ReLU(0.1)\n        self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2)\n        self.dropout = nn.Dropout(0.1)\n\n        self.conv2 = nn.Conv2d(64, 128, kernel_size=3, stride=(1, 1))\n        self.bn2 = nn.BatchNorm2d(128)\n        self.relu2 = nn.ReLU(0.1)\n        self.maxpool2 = nn.MaxPool2d(kernel_size=3, stride=2)\n        self.dropout2 = nn.Dropout(0.1)\n        \n        self.conv3 = nn.Conv2d(128, 256, kernel_size=3, stride=(1, 1))\n        self.bn3 = nn.BatchNorm2d(256)\n        self.relu3 = nn.ReLU(0.1)\n        self.maxpool3 = nn.MaxPool2d(kernel_size=3, stride=3)\n        self.dropout3 = nn.Dropout(0.1)\n        \n        self.conv4 = nn.Conv2d(256, 512, kernel_size=3, stride=(1, 1))\n        self.bn4 = nn.BatchNorm2d(512)\n        self.relu4 = nn.ReLU(0.1)\n        self.maxpool4 = nn.MaxPool2d(kernel_size=3, stride=3)\n        self.dropout4 = nn.Dropout(0.1)\n        \n        self.lstm = nn.LSTM(6, 512, 2, batch_first=True)\n        self.dropout_lstm = nn.Dropout(0.3)\n        self.bn_lstm = nn.BatchNorm1d(512)\n        \n        self.output = nn.Linear(512, output_class)\n    \n    def forward(self, x):\n        x = self.mel(x)\n        #x = self.norm_db(x)\n        \n        x = self.conv1(x)\n        x = self.bn1(x)\n        x = self.relu(x)\n        x = self.maxpool(x)\n        x = self.dropout(x)\n        \n        x = self.conv2(x)\n        x = self.bn2(x)\n        x = self.relu2(x)\n        x = self.maxpool2(x)\n        x = self.dropout2(x)\n        \n        x = self.conv3(x)\n        x = self.bn3(x)\n        x = self.relu3(x)\n        x = self.maxpool3(x)\n        x = self.dropout3(x)\n        \n        x = self.conv4(x)\n        x = self.bn4(x)\n        x = self.relu4(x)\n        x = self.maxpool4(x)\n        x = self.dropout4(x)\n        \n        x, _ = self.lstm(x.view(x.size(0), 512, 6), None)\n        x = self.dropout_lstm(x[:, -1, :])\n        x = self.bn_lstm(x)\n        \n        x = x.view(-1, 512)\n        x = self.output(x)\n        \n        return x","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data sets\ndata is loding by torch's dataLoder.  \nIf the site is 1 or 2, cut at the period specified by test.csv.  \nIf the site is 3, cut at the several section per 5 sec.  \nThe data loader finally returns 5 seconds of sound data as torch tensor.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class TestDataset(Dataset):\n    def __init__(self, df: pd.DataFrame, clip):\n        self.df = df\n        self.clip = clip\n        \n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, idx: int):\n        SR = 32000\n        sample = self.df.loc[idx, :]\n        site = sample.site\n        row_id = sample.row_id\n        \n        if site == \"site_3\":\n            len_y = self.clip.size()[1]\n            start = 0\n            end = SR * 5\n            waveforms = []\n            while len_y > start:\n                waveform = self.clip[:, start:end]\n                if waveform.size(1) != (SR * 5):\n                    break\n                start = end\n                end = end + SR * 5\n                \n                waveforms.append(waveform.numpy())\n            waveforms = torch.tensor(waveforms)\n            return waveforms, row_id, site\n        else:\n            end_seconds = int(sample.seconds)\n            start_seconds = int(end_seconds - 5)\n            \n            start_index = SR * start_seconds\n            end_index = SR * end_seconds\n            \n            waveform = self.clip[:, start_index:end_index]\n\n            return waveform, row_id, site","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def predicter(test_df: pd.DataFrame, \n                        clip: np.ndarray, \n                        model,\n                        threshold=0.5):\n    \n    dataset = TestDataset(df=test_df, \n                          clip=clip)\n    loader = DataLoader(dataset, batch_size=1, shuffle=False)\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    \n    model = model.to(device)\n    model.eval()\n    prediction_dict = {}\n    for waveform, row_id, site in progress_bar(loader):\n        site = site[0]\n        row_id = row_id[0]\n        if site in {\"site_1\", \"site_2\"}:\n            waveform = waveform.to(device)\n\n            with torch.no_grad():\n                prediction = model(waveform)\n                prediction = prediction.detach().cpu().numpy().reshape(-1)\n                \n            event = prediction> threshold\n            labels = np.argwhere(event).reshape(-1).tolist()\n                \n        else:\n            # to avoid prediction on large batch\n            waveform = waveform.squeeze(0)\n            batch_size = 16\n            whole_size = waveform.size()[0]\n            if whole_size % batch_size == 0:\n                n_iter = whole_size // batch_size\n            else:\n                n_iter = whole_size // batch_size + 1\n                \n            all_events = set()\n            for batch_i in range(n_iter):\n                batch = waveform[batch_i * batch_size:(batch_i + 1) * batch_size, :, :]\n\n                batch = batch.to(device)\n                with torch.no_grad():\n                    prediction = model(batch)\n                    proba = prediction.detach().cpu().numpy()\n                    \n                global g\n                g = proba\n                \n                events = proba >= threshold\n                for i in range(len(events)):\n                    event = events[i, :]\n                    labels = np.argwhere(event).reshape(-1).tolist()\n                    for label in labels:\n                        all_events.add(label)\n                        \n            labels = list(all_events)\n        if len(labels) == 0:\n            prediction_dict[row_id] = \"nocall\"\n        else:\n            labels_str_list = list(map(lambda x: INV_BIRD_CODE[x], labels))\n            label_string = \" \".join(labels_str_list)\n            prediction_dict[row_id] = label_string\n    return prediction_dict\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def prediction(test_df: pd.DataFrame,\n               test_audio: Path,\n               model,\n               threshold=0.5):\n    \n    unique_audio_id = test_df.audio_id.unique()\n\n    prediction_dfs = []\n    for audio_id in unique_audio_id:\n        clip, _ = torchaudio.load(test_audio / (audio_id + \".mp3\"), normalization=True)\n        test_df_for_audio_id = test_df.query(\n            f\"audio_id == '{audio_id}'\").reset_index(drop=True)\n        with timer(f\"Prediction on {audio_id}\", logger):\n            prediction_dict = predicter(test_df_for_audio_id,\n                                                  clip=clip[0].unsqueeze(0),\n                                                  model=model,\n                                                  threshold=threshold)\n            \n        row_id = list(prediction_dict.keys())\n        birds = list(prediction_dict.values())\n        prediction_df = pd.DataFrame({\n            \"row_id\": row_id,\n            \"birds\": birds\n        })\n        prediction_dfs.append(prediction_df)\n    \n    prediction_df = pd.concat(prediction_dfs, axis=0, sort=False).reset_index(drop=True)\n    return prediction_df\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = cnn_audio()\ncheckpoint = torch.load(MODE_DIR / \"crnn_2o.model\")\nmodel.load_state_dict(checkpoint['model_state_dict'])\n\nsubmission = prediction(test_df=test,\n                        test_audio=test_audio,\n                        model=model,\n                        threshold=0.0)\nsubmission.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Afterword\n\nthank you for read you to end!  \nIf you fond mistake or question, please comment it!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}