{"cells":[{"metadata":{},"cell_type":"markdown","source":"Let me share the experiments performed with it's LB score and the visualization on deciding the final submission for now.\n\n* [Experiments](#section-one)\n    - [Alexnet with input image size as 224x224](#subsection-one)\n    - [VGG19 wit input image size as 224x224](#subsection-two)\n* [Outlier Detection](#section-two)\n* [Experiments after outlier detection](#section-three)\n    - [VGG19 with input image size as 224x224](#subsection-oneinthree)\n    - [ResNext with input image size as 512x512](#subsection-twointhree)\n* [Conclusion](#section-four)"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import torch\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom collections import OrderedDict\nimport torchvision.models as models\nimport torch.utils.model_zoo as model_zoo\nimport cv2\nfrom torch import nn\nimport torchvision\nimport os\nfrom sklearn.manifold import TSNE\nfrom albumentations import (Compose, Normalize, Resize)\nimport time\nfrom albumentations.pytorch import ToTensor\nfrom tqdm.notebook import tqdm\nimport json","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"class Load_Dataset(torch.utils.data.Dataset):\n    def __init__(self,df,image_size):\n        self.image_paths = df['image_id']\n        self.labels = df['label']\n        self.default_transform = Compose([\n            Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n            Resize(image_size,image_size),\n            ToTensor()\n            \n        ])\n        \n    def __len__(self):\n        return self.image_paths.shape[0]\n    \n    def __getitem__(self,i):\n        image_name = self.image_paths[i]\n        img_path = os.path.join('../input/cassava-leaf-disease-classification/train_images',image_name)\n        image = cv2.cvtColor(cv2.imread(img_path),cv2.COLOR_BGR2RGB)\n        image = self.default_transform(image=image)['image']\n        label = torch.tensor(self.labels[i])\n\n        return image,label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/cassava-leaf-disease-classification/train.csv')\ntrain","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import json\n#maping the class labels mentioned in json file wiht its respective disease name\ndisease_names = open('../input/cassava-leaf-disease-classification/'+'label_num_to_disease_map.json')\ndisease_names = json.load(disease_names)\ndisease_names","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train['disease_name'] = train['label'].apply(lambda x: disease_names[str(x)])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data = train['disease_name']\np = plt.hist(data)\nplt.xticks(rotation='vertical')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"section-one\"></a>\n# Experiments"},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-one\"></a>\n## Alexnet with input image size as 224x224"},{"metadata":{"trusted":true},"cell_type":"code","source":"num_classes=5\nmodel= model = models.__dict__['alexnet'](num_classes=num_classes)\nmodel.load_state_dict(torch.load('../input/pretrained-model-plant-disease/alexnet_epoch_4'))\nmodel =model.cuda()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Load dataset\nimg_size=224\ntrain_data = Load_Dataset(train,img_size)\ntrain_loader = torch.utils.data.DataLoader(train_data,batch_size=128)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.classifier = nn.Sequential(*list(model.classifier.children())[:-2])\noutput_descriptor = np.zeros((1,4096))\noutput_label = np.zeros((1))\ndevice='cuda:0'\n#evaluating the image with pretrained model\nmodel.eval()\nwith torch.no_grad():\n    for _, (images,labels) in tqdm(enumerate(train_loader)):\n        images,labels = images.to(device),labels.to(device)\n        pred = model(images)\n        #concatenating all the outputs and labels as a batch of 128 and store in a variable\n        output_descriptor =np.concatenate((output_descriptor,pred.cpu().numpy().squeeze()),0)\n        output_label = np.concatenate((output_label,labels.cpu().numpy()))\noutput_descriptor = output_descriptor[1:]\noutput_label = output_label[1:]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"st_time = time.time()\nt_sne = TSNE(random_state=2020)\nt_sne_tr = t_sne.fit_transform(output_descriptor)\nprint('TNSE done; Time take {} seconds'.format(time.time()-st_time))\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"##T-SNE df\ntsne_tr = pd.DataFrame()\nfor idx in range(t_sne_tr.shape[1]):\n    tsne_tr['t_sne'+str(idx+1)] = t_sne_tr[:,idx]\ntsne_tr['label'] = output_label.astype(int)\ntsne_tr['disease_name'] = tsne_tr['label'].apply(lambda x: disease_names[str(x)])\ntsne_tr.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import plotly.graph_objects as go\nfig = go.Figure()\ncolors = ['rgb(243, 247, 15)','rgb(13, 160, 200)','rgb(190, 81, 249)','rgb(248, 104, 73)','rgb(0,255,0)']\nfor idx,dn in enumerate(tsne_tr['disease_name'].unique()):\n    df = tsne_tr[tsne_tr['disease_name'] == dn]\n    fig.add_trace(go.Scatter(x=df['t_sne1'],y=df['t_sne2'],mode='markers',marker_color = colors[idx],name=dn))\nfig.update_layout(title='Trained Alexnet model performance')\nfig.update_xaxes(title_text=\"TSNE_1\")\nfig.update_yaxes(title_text=\"TSNE_2\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Almost all classes are in the same cluster, the only difference we can see is that the class CMD is separated from all other classes but that again is due to the imbalance in the data points."},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-two\"></a>\n## VGG19 with input image size as 224x224"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nmodel_path='../input/pretrained-model-plant-disease/vgg19_epoch_20'\nmodel= model = models.__dict__['vgg19'](num_classes=num_classes)\npretrained_state = torch.load(model_path)\nnew_pretrained_state= OrderedDict()\nfor k, v in pretrained_state.items():\n    layer_name = k.replace(\"module.\", \"\")\n    new_pretrained_state[layer_name] = v\nmodel.load_state_dict(new_pretrained_state)\nmodel= model.cuda()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Load dataset\nimg_size=224\ntrain_data = Load_Dataset(train,img_size)\ntrain_loader = torch.utils.data.DataLoader(train_data,batch_size=128)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#taking only the first layer from classifier (4096)\nmodel.classifier = torch.nn.Sequential(model.classifier[0])\noutput_descriptor = np.zeros((1,4096))\noutput_label = np.zeros((1))\ndevice='cuda:0'\n#evaluating the image with pretrained model\nmodel.eval()\nwith torch.no_grad():\n    for _, (images,labels) in tqdm(enumerate(train_loader)):\n        images,labels = images.to(device),labels.to(device)\n        \n        pred = model(images)\n        #concatenating all the outputs and labels as a batch of 128 and store in a variable\n        output_descriptor =np.concatenate((output_descriptor,pred.cpu().numpy().squeeze()),0)\n        output_label = np.concatenate((output_label,labels.cpu().numpy()))\noutput_descriptor = output_descriptor[1:]\noutput_label = output_label[1:]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"st_time = time.time()\nt_sne = TSNE(random_state=2020)\nt_sne_tr = t_sne.fit_transform(output_descriptor)\nprint('TNSE done; Time take {} seconds'.format(time.time()-st_time))\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"##T-SNE df\ntsne_tr = pd.DataFrame()\nfor idx in range(t_sne_tr.shape[1]):\n    tsne_tr['t_sne'+str(idx+1)] = t_sne_tr[:,idx]\ntsne_tr['label'] = output_label.astype(int)\ntsne_tr['disease_name'] = tsne_tr['label'].apply(lambda x: disease_names[str(x)])\ntsne_tr.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import plotly.graph_objects as go\nfig = go.Figure()\ncolors = ['rgb(243, 247, 15)','rgb(13, 160, 200)','rgb(190, 81, 249)','rgb(248, 104, 73)','rgb(0,255,0)']\nfor idx,dn in enumerate(tsne_tr['disease_name'].unique()):\n    df = tsne_tr[tsne_tr['disease_name'] == dn]\n    fig.add_trace(go.Scatter(x=df['t_sne1'],y=df['t_sne2'],mode='markers',marker_color = colors[idx],name=dn))\nfig.update_layout(title='VGG TSNE 1 Vs TSNE 2')\nfig.update_xaxes(title_text=\"TSNE_1\")\nfig.update_yaxes(title_text=\"TSNE_2\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From the plot, the trained model can identify the class CMD better (due to the imablance in the class data) as CMD has almost 50% data higher than any class in the dataset. If you infer more, we can find the model finds it difficult to differentiate between Healthy and CBB .The CBB class is completely hidden behind the healthy class."},{"metadata":{},"cell_type":"markdown","source":"<a id=\"section-two\"></a>\n# Outlier Detection"},{"metadata":{},"cell_type":"markdown","source":"Notebook from the below link, gives an outstanding way on detecting outliers from the dataset.\n\nhttps://www.kaggle.com/ramjib/cassava-leaf-disease-eda-and-outliers\n\nChosing the outlier detection using normalized mean distribution of the images, around 40 images from the dataset are found to be outliers. The following is the way to find the outliers based on normalized mean of the images"},{"metadata":{"trusted":true},"cell_type":"code","source":"for idx in tqdm(train.index):\n    img_name = train.loc[idx,'image_id']\n    #reading the image and converting BGR color space to RGB\n    img = cv2.cvtColor(cv2.imread('../input/cassava-leaf-disease-classification/train_images/'+img_name), cv2.COLOR_BGR2RGB)\n    \n    #normalize the image in the range [0,1]\n    norm_image = cv2.normalize(img, None, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)\n    \n    \n    #calculate mean for each normalized image\n    train.loc[idx,'Normalized_Mean'] = norm_image.mean()\nfig = go.Figure()\ncolors = ['rgb(243, 247, 15)','rgb(13, 160, 200)','rgb(190, 81, 249)','rgb(248, 104, 73)','rgb(0,255,0)']\nfor idx,class_name in enumerate(train['disease_name'].unique()):\n    fig.add_trace(go.Box(y=train[train['disease_name'] == class_name]['Normalized_Mean'],\n                        name=class_name,showlegend=False,\n                        marker_color=colors[idx]))\nfig.update_layout(title='Outlier Detection - Box Plot')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Found the following range as outlier in each class of the dataset."},{"metadata":{"trusted":true},"cell_type":"code","source":"outlier_mean ={}\noutlier_mean['Cassava Bacterial Blight (CBB)'] = [0.17,0.58]\noutlier_mean['Cassava Mosaic Disease (CMD)']=[0.19,0.64]\noutlier_mean['Cassava Brown Streak Disease (CBSD)'] =[0.158,0.64]\noutlier_mean['Cassava Green Mottle (CGM)']=[0.2,0.64]\noutlier_mean['Healthy']=[0.175,0.64]\noutlier_mean","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"outliers=[]\n#filtering only   'Healthy' class from original data\nfor disease in train['disease_name'].unique():\n    dis  = train[train['disease_name'] ==    disease]\n    print('Number of Images in '+disease+ ' Class: '+str(len(dis)))\n\n    #filtering the CMD data for which Mean is between 0.175 and 0.64 (observation from box plot)\n    outliers_dis = dis[dis['Normalized_Mean'].between(outlier_mean[disease][0],outlier_mean[disease][1],inclusive=True)]\n    # filter only the rows which are not in above list of images\n    outliers_dis = dis[~dis['image_id'].isin(outliers_dis['image_id'])]\n    print('Number of Outlier Images in '+disease+ ' Class: '+str(len(outliers_dis))+'\\n')\n    print('../input/cassava-leaf-disease-classification/train_images/'+outliers_dis['image_id'].values)\n    outliers.extend(list(outliers_dis['image_id'].values))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"indices = train[train['image_id'].isin(outliers)].index\ntrain.drop(indices,axis=0,inplace=True)\ntrain.reset_index(inplace=True)\nprint('Number of Train data after removing outliers: ',len(train))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#visulaizing the performance for all the train images\ntrain = pd.read_csv('../input/cassava-leaf-disease-classification/train.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"section-three\"></a>\n# Experiments after removing outliers"},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-oneinthree\"></a>\n## VGG19 with input image size as 224x224"},{"metadata":{"trusted":true},"cell_type":"code","source":"num_classes=5\nmodel_path='../input/pretrained-model-plant-disease/vgg19_epoch_20_oulier_romoved'\nmodel= model = models.__dict__['vgg19'](num_classes=num_classes)\npretrained_state = torch.load(model_path)\nnew_pretrained_state= OrderedDict()\nfor k, v in pretrained_state.items():\n    layer_name = k.replace(\"module.\", \"\")\n    new_pretrained_state[layer_name] = v\nmodel.load_state_dict(new_pretrained_state)\nmodel= model.cuda()\n\n#Load dataset\nimg_size=224\ntrain_data = Load_Dataset(train,img_size)\ntrain_loader = torch.utils.data.DataLoader(train_data,batch_size=128)\n\n\n#taking only the first layer from classifier (4096)\nmodel.classifier = torch.nn.Sequential(model.classifier[0])\noutput_descriptor = np.zeros((1,4096))\noutput_label = np.zeros((1))\ndevice='cuda:0'\n#evaluating the image with pretrained model\nmodel.eval()\nwith torch.no_grad():\n    for _, (images,labels) in tqdm(enumerate(train_loader)):\n        images,labels = images.to(device),labels.to(device)\n        \n        pred = model(images)\n        #concatenating all the outputs and labels as a batch of 128 and store in a variable\n        output_descriptor =np.concatenate((output_descriptor,pred.cpu().numpy().squeeze()),0)\n        output_label = np.concatenate((output_label,labels.cpu().numpy()))\noutput_descriptor = output_descriptor[1:]\noutput_label = output_label[1:]\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"st_time = time.time()\nt_sne = TSNE(random_state=2020)\nt_sne_tr = t_sne.fit_transform(output_descriptor)\nprint('TNSE done; Time take {} seconds'.format(time.time()-st_time))\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\ntsne_tr = pd.DataFrame()\nfor idx in range(t_sne_tr.shape[1]):\n    tsne_tr['t_sne'+str(idx+1)] = t_sne_tr[:,idx]\ntsne_tr['label'] = output_label.astype(int)\ntsne_tr['disease_name'] = tsne_tr['label'].apply(lambda x: disease_names[str(x)])\ntsne_tr.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import plotly.graph_objects as go\nfig = go.Figure()\ncolors = ['rgb(243, 247, 15)','rgb(13, 160, 200)','rgb(190, 81, 249)','rgb(248, 104, 73)','rgb(0,255,0)']\nfor idx,dn in enumerate(tsne_tr['disease_name'].unique()):\n    df = tsne_tr[tsne_tr['disease_name'] == dn]\n    fig.add_trace(go.Scatter(x=df['t_sne1'],y=df['t_sne2'],mode='markers',marker_color = colors[idx],name=dn))\nfig.update_layout(title='VGG after outliers removal TSNE 1 Vs TSNE 2')\nfig.update_xaxes(title_text=\"TSNE_1\")\nfig.update_yaxes(title_text=\"TSNE_2\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The effect in removal of outliers is evident with the class CBSD. I found the false classification with CBSD class is lesser comparitively. But still the model cannot separate the Healthy and CBB class"},{"metadata":{},"cell_type":"markdown","source":"Frequent discussions in this competition gave the following insights\n1. Image size of 512x512 works better\n2. Smaller architectures (like VGG,Resnet) underfits and bigger architectures (like EffNetb7,b5) overfits\n\n\nSo,I experimented on ResNext with image size 512x512"},{"metadata":{},"cell_type":"markdown","source":"<a id=\"subsection-twointhree\"></a>\n## ResNext with input image size as 512x512"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nmodel_path='../input/pretrained-model-plant-disease/best-model-resnext_outliers_removal_epoch_20'\nmodel = torchvision.models.resnext50_32x4d(pretrained=False)\n    \n#     for parameter in model.parameters():\n#         parameter.requires_grad = False\nfc_inputs = model.fc.in_features\nmodel.fc = nn.Sequential(\n    nn.Linear(fc_inputs, 5),\n    nn.Softmax(dim=1) # For using NLLLoss()\n)\nuse_gpu = torch.cuda.is_available()\npretrained_state = torch.load(model_path,map_location=torch.device('cpu'))\nnew_pretrained_state= OrderedDict()\n   \nfor k, v in pretrained_state.items():\n    layer_name = k.replace(\"module.\", \"\")\n    new_pretrained_state[layer_name] = v\nmodel.load_state_dict(new_pretrained_state)\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.fc = model.fc[0]\nfc_inputs = model.fc.in_features\nmodel.fc = nn.Sequential(nn.Linear(fc_inputs,fc_inputs))\nmodel","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Load dataset\nimg_size=512\ntrain_data = Load_Dataset(train,img_size)\ntrain_loader = torch.utils.data.DataLoader(train_data,batch_size=128)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\noutput_descriptor = np.zeros((1,2048))\noutput_label = np.zeros((1))\ndevice='cuda:0'\n#evaluating the image with pretrained model'\nmodel=model.cuda()\nmodel.eval()\nwith torch.no_grad():\n    for _, (images,labels) in tqdm(enumerate(train_loader)):\n        images,labels = images.to(device),labels.to(device)\n        \n        pred = model(images)\n\n\n        #concatenating all the outputs and labels as a batch of 128 and store in a variable\n        output_descriptor =np.concatenate((output_descriptor,pred.cpu().numpy().squeeze()),0)\n        output_label = np.concatenate((output_label,labels.cpu().numpy()))\noutput_descriptor = output_descriptor[1:]\noutput_label = output_label[1:]\n\n\n\nst_time = time.time()\nt_sne = TSNE(random_state=2020)\nt_sne_tr = t_sne.fit_transform(output_descriptor)\nprint('TNSE done; Time take {} seconds'.format(time.time()-st_time))\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\ntsne_tr = pd.DataFrame()\nfor idx in range(t_sne_tr.shape[1]):\n    tsne_tr['t_sne'+str(idx+1)] = t_sne_tr[:,idx]\ntsne_tr['label'] = output_label.astype(int)\ntsne_tr['disease_name'] = tsne_tr['label'].apply(lambda x: disease_names[str(x)])\ntsne_tr.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import plotly.graph_objects as go\nfig = go.Figure()\ncolors = ['rgb(243, 247, 15)','rgb(13, 160, 200)','rgb(190, 81, 249)','rgb(248, 104, 73)','rgb(0,255,0)']\nfor idx,dn in enumerate(tsne_tr['disease_name'].unique()):\n    df = tsne_tr[tsne_tr['disease_name'] == dn]\n    fig.add_trace(go.Scatter(x=df['t_sne1'],y=df['t_sne2'],mode='markers',marker_color = colors[idx],name=dn))\nfig.update_layout(title='Resnext after outliers removal TSNE 1 Vs TSNE 2')\nfig.update_xaxes(title_text=\"TSNE_1\")\nfig.update_yaxes(title_text=\"TSNE_2\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Though the CBB and Healthy not yet separated, the other classes are more separated than bove experiments. The overlap between the classes CGM and Healthy is less and also between the classes CMD and CGM."},{"metadata":{},"cell_type":"markdown","source":"**Insights from the visualizations:**\nComparatively the RexNext experiment gave better separation of classes among all the above experiments.\n\nBut the LB board results says the opposite. Below are the LB results of all the experiments"},{"metadata":{"trusted":true},"cell_type":"code","source":"models =['Alexnet' ,'VGG19', 'VGG19', 'ResNext']\nimage_size= ['224x224','224x224', '224x224','512x512']\nout_det = ['Not performed','Not performed','Performed','Performed']\nlb_score =[0.784, 0.839, 0.824,0.818]\n\nfig = go.Figure()\nfig.add_trace(\n        go.Table(\n            header=dict(\n                values=[\"Models\",\"Image size\", \"Outlier detection\",\"LB score\"],\n                font=dict(size=10),\n                align=\"center\"\n            ),\n            cells=dict(\n                values=[models,image_size,out_det,lb_score],\n                align = \"center\")\n        ))\n\n\nfig.update_layout(title='Results in LB')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"section-four\"></a>\n# Conclusion\n\nThe public LB may stand correct and may not, always go with the validation performance and the result visualizations of the models. We all know the reason-the public LB has only 31% of hidden test data; A small visulaization and proof for this fact is performed in this kernel.\n\nPlease Upvote!!!"},{"metadata":{},"cell_type":"markdown","source":"---"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}