{"cells":[{"metadata":{},"cell_type":"markdown","source":"## OSIC Pulmonary fibrosis Progression.⚕️\n\nPlumonary fibrosis is a lung disease which occurs due to damage in tissue
\nand damage causes tissue to thicken and patient becomes short of breath.\n\nThe problem with plumonary fibrosis it can be caused by various number of
\nfactors and it is difficult for doctors to find out what is causing the problem
\nand condition in which doctors can not find out the cause is called **idiopathic plumonary fibrosis.**\n\n\n\n\n\n### Symptoms\n* Shortness of breath\n* A dry cough\n* Fatigue\n* Unexplained Weight Loss\n* Aching muscles and joints\n* Widning and Rounding of tips of the fingers or toes.\n\nTo read more about plumonary fibrosis click [here](https://www.mayoclinic.org/diseases-conditions/pulmonary-fibrosis/symptoms-causes/syc-20353690)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 1. What is the competion about ?💡\n\n* So task here is simply to determine how much will patient's current
\nlung health will decline using images from CT(computerised tomography)
\nmetadata and current health of lungs. The health or current state of lungs
\ncan be deterimined using spirometer which is an instrument which measures how
\nmuch and how rapid a person breath. Spirometer gives a value called FVC which
\nstands for **Forces vital Capacity**.\n\n* ### 1.1 what we need to predict ?\nWe will predict a decline in the lung function of patient in
final three FVC measures and their confidence.\nactually we need to make prediction for all the weeks given in the test set but only final three would be considered for evaluation.\nHere you will find negative weeks in the dataset and explanation is a patient took a CT scan at week zero but he/she might have took
\ntest for FVC before or after CT scan so week 0 is baseline but sometimes patient do not get FVC score on the same day as CT scan so there
\nmight be no FVC score for week=0.\n\n**Note: Let me know if any information is wrong**.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 2.Metrics: Laplace Log Likelihood 📏\n\n\n\n* ### what is Laplace Log Likelihood ?\n\nHere σclipped is the confidence score of the and it is clipped
so value is less than or equal to 70.\n\nand delta is abs difference in FVC precited and true which is also clipped with max value of 1000.\n\n\n\nActully I know what laplace distribution is and what log likelihood is but I did't find any article on laplace loglikelihood.
\nso please let me know If you find any good article on it.\n\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Importing Libraries 📘","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import os\nimport sys\nimport glob\nfrom os import listdir\nimport glob\nimport tqdm\nfrom typing import Dict\nimport cv2\nimport pydicom as dicom\n\n\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nplt.style.use('fivethirtyeight')\n\n#plotly\n# !pip install chart_studio\nimport plotly.express as px\n# import chart_studio.plotly as py\nimport plotly.graph_objs as go\nfrom plotly.offline import iplot\nimport plotly.figure_factory as ff\nimport cufflinks\ncufflinks.go_offline()\ncufflinks.set_config_file(world_readable=True,theme='pearl')\n\n#pydicom\nimport pydicom\n\n#supress warnings\nimport warnings\nwarnings.filterwarnings('ignore')\n\n\n# from bokeh.layouts import row, column\n# from bokeh.models import ColumnDataSource, CustomJS, Label,Range1d,Slider,Span\n# from bokeh.plotting import figure, output_notebook, show\n\n#used for changing color of text in print statement\nfrom colorama import Fore, Back, Style\ny_ = Fore.YELLOW\nr_ = Fore.RED\ng_ = Fore.GREEN\nb_ = Fore.BLUE\nm_ = Fore.MAGENTA\nsr_ = Style.RESET_ALL\n\n# output_notebook()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Getting data💽","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"folder_path = '../input/osic-pulmonary-fibrosis-progression'\ntrain_csv = folder_path + '/train.csv'\ntest_csv = folder_path+ '/test.csv'\nsample_csv = folder_path + '/sample_submission.csv'\n\ntrain_data = pd.read_csv(train_csv)\ntest_data = pd.read_csv(test_csv)\nsample = pd.read_csv(sample_csv)\n\nprint(f\"{y_}Number of rows in train data: {r_}{train_data.shape[0]}\\n{y_}Number of columns in train data: {r_}{train_data.shape[1]}\")\nprint(f\"{g_}Number of rows in test data: {r_}{test_data.shape[0]}\\n{g_}Number of columns in test data: {r_}{test_data.shape[1]}\")\nprint(f\"{b_}Number of rows in submission data: {r_}{sample.shape[0]}\\n{b_}Number of columns in submission data:{r_}{sample.shape[1]}\")\n\ntrain_data.head().style.applymap(lambda x: 'background-color:lightgreen')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 3. EDA\n\n### 3.1 FVC Distribution📈\n","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def distribution(feature, color):\n plt.figure(dpi=100)\n sns.distplot(train_data[feature],color=color)\n print(\"{}Max value of {} is: {} {:.2f} \\n{}Min value of {} is: {} {:.2f}\\n{}Mean of {} is: {}{:.2f}\\n{}Standard Deviation of {} is:{}{:.2f}\"\\\n .format(y_,feature,r_,train_data[feature].max(),g_,feature,r_,train_data[feature].min(),b_,feature,r_,train_data[feature].mean(),m_,feature,r_,train_data[feature].std()))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"distribution(\"FVC\",\"blue\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.2 Distribution of Age 👶🧒🧑🧓","execution_count":null},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"distribution(\"Age\",\"brown\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.3 Distribution of Percent📈","execution_count":null},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"distribution(\"Percent\",\"blue\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.4 Distribution of weeks📅","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"distribution(\"Weeks\",\"yellow\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" ### 3.5 Number of smokers sex 🚬","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.figure(dpi=100)\nsns.countplot(data=train_data,x='SmokingStatus',hue='Sex');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.6 Distribution of FVC based on smoking status and Sex","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def distribution2(feature):\n plt.figure(figsize=(15,7))\n plt.subplot(121)\n for i in train_data.Sex.unique():\n sns.distplot(train_data[train_data['Sex']==i][feature],label=i)\n plt.title(f\"Distribution of {feature} based on Sex\")\n plt.legend()\n\n plt.subplot(122)\n for i in train_data.SmokingStatus.unique():\n sns.distplot(train_data[train_data['SmokingStatus']==i][feature],label=i)\n plt.title(f\"Distribution of {feature} based on Smoking Status\")\n plt.legend()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"distribution2(\"FVC\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.7 Distribution of Percent based on Sex and SmokingStatus","execution_count":null},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"distribution2(\"Percent\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" ### 3.8 Distribution of Age based on SmokingStatus and Sex","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"distribution2(\"Age\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.9 Distribution of Weeks based on SmokingStatus and Sex","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"distribution2(\"Weeks\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.10 FVC vs Percent \n**Note:now will make graphs in plotly just for change**","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def vs(feature1,feature2,color=None):\n fig = px.scatter(train_data,x=feature1,y=feature2,color=color)\n fig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"vs(\"FVC\",\"Percent\",'SmokingStatus')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.11 FVC vs Age","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"vs(\"FVC\",\"Age\",'SmokingStatus')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.12 FVC vs Weeks","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"vs(\"FVC\",\"Weeks\",\"SmokingStatus\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n### 3.13 FVC vs weeks of 20 random patient 🎲","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"rn = np.random.randint(0,train_data.Patient.nunique()-20,1)[0]\npatients_ids = train_data.Patient.unique()[rn:rn+20]\nfig =go.Figure()\n\nfor patient in patients_ids:\n df = train_data[train_data[\"Patient\"] == patient]\n fig.add_trace(go.Scatter(x=df.Weeks,y=df.FVC,\n mode='lines',\n name=str(patient)))\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.14 count of weeks of each patient ⌛","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(f\"{y_}Number of unique patient is {r_}{train_data.Patient.nunique()}\")\n\ndf = train_data.Patient.value_counts()\nfig = px.bar(x=[f\"Patient {i}\" for i in range(len(df.index))],y=df.values)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.15 Box plot of FVC bases on sex and smokingstatus 👀","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def box(feature1,feature2,color=None):\n fig = px.box(train_data,x=feature2,y=feature1,color=color)\n fig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"box(\"FVC\",\"Sex\",\"SmokingStatus\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.16 Box plot of Percent based on sex and smoking status","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"box(\"Percent\",\"Sex\",\"SmokingStatus\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.16 Box plot of Age based on sex and smoking status","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"box(\"Age\",\"Sex\",\"SmokingStatus\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3.17 Correlation matrix of train data ","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.figure(dpi=100)\nsns.heatmap(train_data.corr(),annot=True);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 4. Visulizing Images 🖼️\n\n* What is DICOM image ?\n**DICOM(Digital Image and Communication in Medecine)** is a standard developed and maintained by **National Electrical Manufacturers Association (NEMA)** for storing and transfering the medical images like CT(computerised Tomography), Magnetic resonanse image(MRI) and other types of medical images.\n\nDICOM is very good protocol and intresting to read further click [here](https://en.wikipedia.org/wiki/DICOM)\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### 4.1 Single Image","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train_image_path = folder_path + '/train/'\ntest_image_path = folder_path + '/test/'\n\ntrain_images = os.listdir(train_image_path)\ntest_images = os.listdir(test_image_path)\n\nimage = train_image_path+train_images[0]+\"/1.dcm\"\n\ndef show_image(image):\n print(f\"{y_} Image {r_}{image}\")\n image = dicom.dcmread(image)\n image = image.pixel_array \n plt.figure(figsize=(7,7))\n plt.imshow(image,cmap='gray')\n plt.axis('off')\n plt.show()\n\nshow_image(image)\n\n ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 4.2 Grid of sorted images of some random patient","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def show_grid(cmap='gray'):\n rn = np.random.randint(0,len(train_images),1)[0]\n path= train_image_path+train_images[rn]\n images = [dicom.read_file(path+\"/\"+img) for img in os.listdir(path)]\n images.sort(key=lambda x: float(x.ImagePositionPatient[2]))\n plt.figure(figsize=(10,10))\n for i,image in enumerate(images[:100]):\n plt.subplot(10,10,i+1)\n plt.imshow(image.pixel_array,cmap=cmap)\n plt.axis('off')\n plt.show()\n\nshow_grid()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"show_grid(cmap='jet')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"show_grid(cmap='RdYlBu')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 4.3 Animation","execution_count":null},{"metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import matplotlib.animation as animation\nfrom IPython.display import HTML\n\ndef show_animation():\n rn = np.random.randint(0,len(train_images),1)[0]\n fig = plt.figure()\n path= train_image_path+train_images[0]\n images = [dicom.read_file(path+\"/\"+img) for img in os.listdir(path)]\n images.sort(key=lambda x: float(x.ImagePositionPatient[2]))\n ims = list()\n for image in images:\n image = plt.imshow(image.pixel_array,cmap='gray',animated=True)\n plt.axis('off')\n ims.append([image])\n ani = animation.ArtistAnimation(fig,ims,interval=100,blit=False,repeat_delay=1000)\n return ani\n\nani = show_animation()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"HTML(ani.to_jshtml())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 4.4 3-D reconstruction.\n\nThis code for converting 2-D slices to a 3-D image is written by [mrbean-bremen](https://github.com/mrbean-bremen).\n**Orignal Code** [here](https://github.com/pydicom/pydicom/blob/master/examples/image_processing/reslice.py).","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# load the DICOM files\ndef reconstruct():\n files = []\n path= train_image_path+train_images[0]\n\n for fname in os.listdir(path):\n # print(\"loading: {}\".format(fname))\n files.append(pydicom.dcmread(path+\"/\"+fname))\n\n print(\"file count: {}\".format(len(files)))\n\n # skip files with no SliceLocation (eg scout views)\n slices = []\n skipcount = 0\n for f in files:\n if hasattr(f, 'SliceLocation'):\n slices.append(f)\n else:\n skipcount = skipcount + 1\n\n print(\"skipped, no SliceLocation: {}\".format(skipcount))\n\n # ensure they are in the correct order\n slices = sorted(slices, key=lambda s: s.SliceLocation)\n\n # pixel aspects, assuming all slices are the same\n ps = slices[0].PixelSpacing\n ss = slices[0].SliceThickness\n ax_aspect = ps[1]/ps[0]\n sag_aspect = ps[1]/ss\n cor_aspect = ss/ps[0]\n\n # create 3D array\n img_shape = list(slices[0].pixel_array.shape)\n img_shape.append(len(slices))\n img3d = np.zeros(img_shape)\n\n # fill 3D array with the images from the files\n for i, s in enumerate(slices):\n img2d = s.pixel_array\n img3d[:, :, i] = img2d\n\n # plot 3 orthogonal slices\n plt.figure(figsize=(15,7))\n a1 = plt.subplot(1,3,1)\n plt.imshow(img3d[:, :, img_shape[2]//2])\n a1.set_aspect(ax_aspect)\n plt.axis('off')\n\n\n a2 = plt.subplot(1, 3, 2)\n plt.imshow(img3d[:, img_shape[1]//2, :])\n a2.set_aspect(sag_aspect)\n plt.axis('off')\n\n\n a3 = plt.subplot(1, 3, 3)\n plt.imshow(img3d[img_shape[0]//2, :, :].T)\n a3.set_aspect(cor_aspect)\n plt.axis('off')\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"reconstruct()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 5. Preprocessing Data\n\nLet us create a baseline pytorch model.\n\n### 5.1 Importing some more libraries 📘","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import StandardScaler,LabelEncoder,OneHotEncoder,PowerTransformer\nfrom sklearn.model_selection import train_test_split, cross_val_score,cross_validate, KFold,GroupKFold\nfrom sklearn.metrics import make_scorer\nfrom sklearn.compose import ColumnTransformer\nfrom sklearn.pipeline import Pipeline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#getting base week for patient\ndef get_baseline_week(data):\n df = data.copy()\n df['Weeks'] = df['Weeks'].astype(int)\n df['min_week'] = df.groupby('Patient')['Weeks'].transform('min')\n df['baseline_week'] = df['Weeks'] - df['min_week']\n return df\n\n#getting FVC for base week and setting it as base_FVC of patient\ndef get_base_FVC(data):\n df = data.copy()\n base = df.loc[df.Weeks == df.min_week][['Patient','FVC']].copy()\n base.columns = ['Patient','base_FVC']\n \n base['nb']=1\n base['nb'] = base.groupby('Patient')['nb'].transform('cumsum')\n \n base = base[base.nb==1]\n base.drop('nb',axis =1,inplace=True)\n df = df.merge(base,on=\"Patient\",how='left')\n df.drop(['min_week'], axis = 1)\n return df ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.drop_duplicates(keep=False,inplace=True,subset=['Patient','Weeks'])\ntrain_data = get_baseline_week(train_data)\ntrain_data = get_base_FVC(train_data)\n\nsample = pd.read_csv(sample_csv)\nsample.drop(\"FVC\",axis=1,inplace=True)\nsample[[\"Patient\",\"Weeks\"]] = sample[\"Patient_Week\"].str.split(\"_\",expand=True) \nsample = sample.merge(test_data.drop(\"Weeks\",axis=1),on=\"Patient\",how=\"left\")\n\n#we have to predict for all weeks \nsample[\"min_Weeks\"] = np.nan\nsample = get_baseline_week(sample)\nsample = get_base_FVC(sample)\n\ntrain_columns = ['baseline_week','base_FVC','Percent','Age','Sex','SmokingStatus']\ntrain_label = ['FVC']\nsub_columns = ['Patient_Week','FVC','Confidence']\n\ntrain = train_data[train_columns]\ntest = sample[train_columns]\n#Preprocessing\ntransformer = ColumnTransformer([('s',StandardScaler(),[0,1,2,3]),('o',OneHotEncoder(),[4,5])])\ntarget = train_data[train_label].values\ntrain = transformer.fit_transform(train)\ntest = transformer.transform(test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"distribution(\"baseline_week\",'green');","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## 6 Simple Pytorch Model\n\n### 6.1 importing pytorch libraries 📘","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nfrom torch.utils.data import TensorDataset, DataLoader","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class Model(nn.Module):\n def __init__(self,n):\n super(Model,self).__init__()\n self.layer1 = nn.Linear(n,200)\n self.layer2 = nn.Linear(200,100)\n \n self.out1 = nn.Linear(100,3)\n self.relu3 = nn.ReLU()\n self.out2 = nn.Linear(100,3)\n \n def forward(self,xb):\n x1 = F.leaky_relu(self.layer1(xb))\n x1 = F.leaky_relu(self.layer2(x1))\n \n o1 = self.out1(x1)\n o2 = F.relu(self.out2(x1))\n return o1 + torch.cumsum(o2,dim=1)\n ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 6.2 Training Model","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def run():\n \n def score(outputs,target):\n confidence = outputs[:,2] - outputs[:,0]\n clip = torch.clamp(confidence,min=70)\n target=torch.reshape(target,outputs[:,1].shape)\n delta = torch.abs(outputs[:,1] - target)\n delta = torch.clamp(delta,max=1000)\n sqrt_2 = torch.sqrt(torch.tensor([2.])).to(device)\n metrics = (delta*sqrt_2/clip) + torch.log(clip*sqrt_2)\n return torch.mean(metrics)\n \n def qloss(outputs,target):\n qs = [0.25,0.5,0.75]\n qs = torch.tensor(qs,dtype=torch.float).to(device)\n e = target - outputs\n e.to(device)\n v = torch.max(qs*e,(qs-1)*e)\n v = torch.sum(v,dim=1)\n return torch.mean(v)\n \n def loss_fn(outputs,target,l):\n return l * qloss(outputs,target) + (1- l) * score(outputs,target)\n \n def train_loop(train_loader,model,loss_fn,device,optimizer,lr_scheduler=None):\n model.train()\n losses = list()\n metrics = list()\n for i, (inputs, labels) in enumerate(train_loader):\n inputs = inputs.to(device)\n labels = labels.to(device)\n \n optimizer.zero_grad()\n with torch.set_grad_enabled(True): \n outputs = model(inputs) \n metric = score(outputs,labels)\n\n loss = loss_fn(outputs,labels,0.8)\n metrics.append(metric.cpu().detach().numpy())\n losses.append(loss.cpu().detach().numpy())\n\n loss.backward()\n\n optimizer.step()\n if lr_scheduler != None:\n lr_scheduler.step()\n \n return losses,metrics\n \n def valid_loop(valid_loader,model,loss_fn,device):\n model.eval()\n losses = list()\n metrics = list()\n for i, (inputs, labels) in enumerate(valid_loader):\n inputs = inputs.to(device)\n labels = labels.to(device)\n \n outputs = model(inputs) \n metric = score(outputs,labels)\n \n loss = loss_fn(outputs,labels,0.8)\n metrics.append(metric.cpu().detach().numpy())\n losses.append(loss.cpu().detach().numpy())\n \n return losses,metrics \n\n NFOLDS =5\n kfold = KFold(NFOLDS,shuffle=True,random_state=42)\n \n #kfold\n for k , (train_idx,valid_idx) in enumerate(kfold.split(train)):\n batch_size = 64\n epochs = 50\n device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n print(f\"{device} is used\")\n x_train,x_valid,y_train,y_valid = train[train_idx,:],train[valid_idx,:],target[train_idx],target[valid_idx]\n n = x_train.shape[1]\n model = Model(n)\n model.to(device)\n lr = 0.1\n optimizer = optim.Adam(model.parameters(),lr=lr)\n lr_scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=20, gamma=0.5)\n\n train_tensor = torch.tensor(x_train,dtype=torch.float)\n y_train_tensor = torch.tensor(y_train,dtype=torch.float)\n\n train_ds = TensorDataset(train_tensor,y_train_tensor)\n train_dl = DataLoader(train_ds,\n batch_size = batch_size,\n num_workers=4,\n shuffle=True\n )\n\n valid_tensor = torch.tensor(x_valid,dtype=torch.float)\n y_valid_tensor = torch.tensor(y_valid,dtype=torch.float)\n\n valid_ds = TensorDataset(valid_tensor,y_valid_tensor)\n valid_dl = DataLoader(valid_ds,\n batch_size = batch_size,\n num_workers=4,\n shuffle=False\n )\n \n print(f\"Fold {k}\")\n for i in range(epochs):\n losses,metrics = train_loop(train_dl,model,loss_fn,device,optimizer,lr_scheduler)\n valid_losses,valid_metrics = valid_loop(valid_dl,model,loss_fn,device)\n if (i+1)%5==0:\n print(f\"epoch:{i} Training | loss:{np.mean(losses)} score: {np.mean(metrics)}| \\n Validation | loss:{np.mean(valid_losses)} score:{np.mean(valid_metrics)}|\")\n torch.save(model.state_dict(),f'model{k}.bin')\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"run()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 6.3 Inference","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def inference():\n device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n nfold = 5\n all_prediction = np.zeros((test.shape[0],3))\n \n for i in range(nfold):\n n = train.shape[1]\n \n model = Model(n)\n model.load_state_dict(torch.load(f\"model{i}.bin\"))\n predictions = list()\n model.to(device)\n test_tensor = torch.tensor(test,dtype=torch.float)\n test_dl = DataLoader(test_tensor,\n batch_size=64,\n num_workers=2,\n shuffle=False)\n \n with torch.no_grad():\n for i, inputs in enumerate(test_dl):\n inputs = inputs.to(device, dtype=torch.float)\n outputs= model(inputs) \n predictions.extend(outputs.cpu().detach().numpy())\n\n all_prediction += np.array(predictions)/nfold\n \n return all_prediction ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 6.4 submission","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"prediction = inference()\nsample[\"Confidence\"] = np.abs(prediction[:,2] - prediction[:,0])\nsample[\"FVC\"] = prediction[:,1]\nsub = sample[sub_columns]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.to_csv(\"submission.csv\",index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(15,7))\nplt.subplot(121)\nsns.distplot(sub.Confidence)\nplt.subplot(122)\nsns.distplot(sub.FVC);","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(sub.shape)\nsub.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.to_csv(\"submission.csv\",index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" Please tell me in the comments if anything is wrong or if you did not understand something.🤗
\n Please upvote if you found it useful.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## Todo \n\n* How to use image to make prediction.","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}