{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Notebook Focusing on Data Pre-processing for Brain Tumor Classification using 3D Images and 3D Convolutional Neural Networks \n\n⚠️ Note that this notebook is finished after the actual competition deadline ended. Work and personal life came in-between.\n\n⚠️ Note that the results of the competition showed very poor performance - possibly indicating issues with the data or insignificant number of samples.  \nRef: https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/284024  \nRef: https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/279820\n\nHowever, this is my first time implementing a 3D neural network so I thought it would be a good idea to document the outcome even though I stopped trying to get good results after I saw the outcome of the competition. Also the notebook proposed a method of extracting N number of slices in a consistent way from the largely varying 3D scan data. \n\n**Main points of this notebook** \n* I am interested in applying ML/neural networks in applications that can help people - Brain tumor classification is a prime example of that.\n* Implementation of a 3D CNN using Keras\n* The structure of MRI scan data had a huge variation in for instance: number of segments, size, intensities. This notebook implements an algorithm to extract N slices from the data in a consistent way. \n\n\n**Note:**\n* Each sample comes with four different scan data:\n    * Fluid Attenuated Inversion Recovery (FLAIR)\n    * T1-weighted pre-contrast (T1w)\n    * T1-weighted post-contrast (T1Gd)\n    * T2-weighted (T2)\n    \n \n* This notebook uses the FLAIR type\n* The intended approach (if the competition data would have been OK) would have been to train four different 3D-CNNs on the 4 different data types. Then apply a weighted average on the output of the four 3D-CNNs for the final classification. \n","metadata":{}},{"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport os\nimport glob\nimport re\nimport cv2\n\nimport pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\n\nfrom matplotlib import pyplot as plt\nimport math\nimport keras\nimport tensorflow as tf\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras import Input\nfrom tensorflow.keras.models import Model\nfrom tensorflow.keras.layers import GlobalAveragePooling3D,MaxPool3D,MaxPooling3D,AveragePooling3D,Dense, Flatten, Dropout, GlobalAveragePooling2D, Flatten, BatchNormalization, Conv3D\nfrom random import shuffle\nfrom sklearn.model_selection import train_test_split\n\nfrom keras import backend as K\nfrom textwrap import wrap","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-11-06T18:42:06.547152Z","iopub.execute_input":"2021-11-06T18:42:06.547488Z","iopub.status.idle":"2021-11-06T18:42:12.446804Z","shell.execute_reply.started":"2021-11-06T18:42:06.547409Z","shell.execute_reply":"2021-11-06T18:42:12.446004Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Settings \nclass settings:\n    size = 128#64\n    num_images =24\n    learning_rate = 1e-4\n    batchSize = 16\n    numEpochs = 75\n    debug = False\n    debug_Nsamples = 30\n    batch_size = 6","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:46:22.504747Z","iopub.execute_input":"2021-11-06T13:46:22.504999Z","iopub.status.idle":"2021-11-06T13:46:22.510444Z","shell.execute_reply.started":"2021-11-06T13:46:22.50497Z","shell.execute_reply":"2021-11-06T13:46:22.50899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Prepare the data:**","metadata":{}},{"cell_type":"code","source":"#Prepare the training dataframe\ndf = pd.read_csv(\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/train_labels.csv\")\ntestSub = pd.read_csv(\"../input/rsna-miccai-brain-tumor-radiogenomic-classification/sample_submission.csv\")\ndf['BraTS21ID5'] = [format(x, '05d') for x in df.BraTS21ID]\ndata_path = '../input/rsna-miccai-brain-tumor-radiogenomic-classification'\ntrain_folder = os.path.join(data_path, 'train')\ndf['imfolder'] = ['{:05d}'.format(s) for s in df['BraTS21ID']]\ndf['path'] = [os.path.join(train_folder, s) for s in df['imfolder']]\n\n#Drop these four samples sice organizers pointed out they were not valid:\ndf.drop(df.index[df['BraTS21ID5'] == \"00109\"], inplace = True)\ndf.drop(df.index[df['BraTS21ID5'] == \"00123\"], inplace = True)\ndf.drop(df.index[df['BraTS21ID5'] == \"00709\"], inplace = True)\n\nif settings.debug:\n    df = df[0:settings.debug_Nsamples]\n\n#Split into train and test set:\ntrain_df, test_df = train_test_split(df, test_size=0.25)\ntrain_df.iloc[0:5]","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:43:13.929523Z","iopub.execute_input":"2021-11-06T13:43:13.930018Z","iopub.status.idle":"2021-11-06T13:43:13.999799Z","shell.execute_reply.started":"2021-11-06T13:43:13.929983Z","shell.execute_reply":"2021-11-06T13:43:13.999062Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Define function to rotate images:**","metadata":{}},{"cell_type":"code","source":"# Define function to rotate images\ndef rotate_image(image, angle):\n    image_center = tuple(np.array(image.shape[1::-1]) / 2)\n    rot_mat = cv2.getRotationMatrix2D(image_center, angle, 1.0)\n    result = cv2.warpAffine(image, rot_mat, image.shape[1::-1], flags=cv2.INTER_LINEAR)\n    return result","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:43:14.008997Z","iopub.execute_input":"2021-11-06T13:43:14.009948Z","iopub.status.idle":"2021-11-06T13:43:14.016267Z","shell.execute_reply.started":"2021-11-06T13:43:14.00991Z","shell.execute_reply":"2021-11-06T13:43:14.015426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Function that loads the dicom images. Note that this function will read the rotation angle from the dicom and reverse the rotation.**","metadata":{}},{"cell_type":"code","source":"# Function to load dicom images\ndef load_dicom_image(path, img_size=settings.size, voi_lut=False, rotate_img=True):\n    dicom = pydicom.read_file(path)\n    data = dicom.pixel_array\n    \n    if not hasattr(dicom,'FlipAngle'):\n        rotate=0\n    else:\n        rotate=np.int(dicom.FlipAngle)    \n    #May apply \"Value if Interest Look Up Table\", ref: https://www.medicalconnections.co.uk/kb/lookup-tables/\n    if voi_lut:\n        data = apply_voi_lut(dicom.pixel_array, dicom)\n    else:\n        data = dicom.pixel_array\n    if rotate_img:    \n        if rotate > 0:\n            data = rotate_image(data,-rotate)\n\n    data = cv2.resize(data, (img_size, img_size))\n    return data","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:56:22.024232Z","iopub.execute_input":"2021-11-06T13:56:22.024614Z","iopub.status.idle":"2021-11-06T13:56:22.031199Z","shell.execute_reply.started":"2021-11-06T13:56:22.024573Z","shell.execute_reply":"2021-11-06T13:56:22.030238Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Function to stack images into 3D arrays from the dicom data. The different steps will be discussed and visualized below.**","metadata":{}},{"cell_type":"code","source":"def stack3Dimages(mri_type, df=df, plotExample = False):\n    pick_idx = pd.DataFrame()\n    stat_nFiles = np.zeros(len(df))\n    stat_Intensity = []\n    for k,d in enumerate(df[\"imfolder\"]):\n        if settings.debug:\n            print('starting:' + str(d))\n            print('k = ' + str(k))\n\n        split=\"train\"\n        scan_id = str(d)\n        files = sorted(glob.glob(f\"{data_path}/{split}/{scan_id}/{mri_type}/*.dcm\"), \n               key=lambda var:[int(x) if x.isdigit() else x for x in re.findall(r'[^0-9]|[0-9]+', var)])\n        \n        #img3d=np.zeros((0,0,0))\n        img3d=[]\n        for f in files:\n            if len(img3d)==0:\n                img3d = np.expand_dims(load_dicom_image(f,rotate_img=True),-1)\n            else:\n                img3d = np.append(img3d,np.expand_dims(load_dicom_image(f,rotate_img=True),-1),axis=-1)\n                \n        if settings.debug:\n            print('shape img3d')\n            print(np.shape(img3d))\n            \n        stat_nFiles[k] = np.shape(img3d)[2]\n        if settings.debug:\n            print(\"Len Files:\")\n            print(stat_nFiles[k])\n        \n        #Get the total intensity of each image\n        img_int = np.zeros(np.shape(img3d)[2])\n        for kk in range(np.shape(img3d)[2]):\n            img_int[kk] = np.sum(np.abs(img3d[:,:,kk]))\n            \n        stat_Intensity.append(img_int)\n\n        #plt.plot(img_int)\n        idx_margin = 7 #Dont choose the first/last non-zero image\n        idx = np.argmax(img_int)\n        zero_idx = np.nonzero(img_int==0)[0]\n        if np.sum(zero_idx<idx) == 0:\n            start_idx = np.array([0])\n        else:\n            start_idx = zero_idx[zero_idx<idx]   \n\n        start_idx = start_idx[-1]+idx_margin\n\n        if np.sum(zero_idx>=idx) == 0:\n            end_idx = np.array([len(img_int)])\n        else:\n            end_idx = zero_idx[zero_idx>=idx]\n        end_idx = end_idx[0]-idx_margin\n        idx_array = np.linspace(start_idx,end_idx,settings.num_images,dtype=int)\n\n        if plotExample:\n            plt.plot(img_int)\n            plt.plot(idx_array,img_int[idx_array],'o')\n\n        img_out = img3d[:,:,idx_array] #only works when single image is called, i.e. for illustration \n        pick_idx[scan_id] = idx_array\n        \n    return pick_idx, stat_nFiles, stat_Intensity, img_out\n    #np.shape(img_out)","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:56:26.413245Z","iopub.execute_input":"2021-11-06T13:56:26.413813Z","iopub.status.idle":"2021-11-06T13:56:26.428522Z","shell.execute_reply.started":"2021-11-06T13:56:26.413776Z","shell.execute_reply":"2021-11-06T13:56:26.427804Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Loop through the data, gather statistics, save the slice indices per sample (all explained below):**","metadata":{}},{"cell_type":"code","source":"pick_idx, stat_nFiles, stat_Intensity, img_out = stack3Dimages(\"FLAIR\", df=df)","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:43:14.05183Z","iopub.execute_input":"2021-11-06T13:43:14.052174Z","iopub.status.idle":"2021-11-06T13:44:35.658282Z","shell.execute_reply.started":"2021-11-06T13:43:14.052135Z","shell.execute_reply":"2021-11-06T13:44:35.657533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**We can first look at the number of images available per sample. As seen, there is a huge spread in the number of images.**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(8, 5))\nplt.hist(stat_nFiles,100)\nplt.xlabel('# if images')\nplt.ylabel('Counts')\n#plt.plot(stat_Intensity)   ","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:44:35.659799Z","iopub.execute_input":"2021-11-06T13:44:35.660304Z","iopub.status.idle":"2021-11-06T13:44:36.04957Z","shell.execute_reply.started":"2021-11-06T13:44:35.660264Z","shell.execute_reply":"2021-11-06T13:44:36.048887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**We can also look at the distribution of the intensity per image, i.e. by taking the sum of the intensity over all pixels per image. \nFirst, the intensities as a function of image number are plotted for all the samples. However, this is quite messy so also the sum of all intensities over all samples are plotted to get an envelope of the distribution of the intensities.**","metadata":{}},{"cell_type":"code","source":"#pick_idx, stat_nFiles, stat_Intensity \n\nfig,axes = plt.subplots(figsize=(8, 5))\nfor intstat in stat_Intensity:\n    plt.plot(intstat)\nplt.grid('minor')\nplt.xlabel('Image #')\nplt.ylabel('Sum of Intensity per Image #')\nplt.title('Intensity as a function of image number for all samples')\n\nmaxLen=0\nfor kk in stat_Intensity:\n    if len(kk)>maxLen:\n        maxLen=len(kk)\nintSum = np.zeros(maxLen)\n\nfor kk in stat_Intensity:\n    intSum[0:len(kk)] += kk \n    \nfig,axes = plt.subplots(figsize=(8, 5))\naxes.plot(intSum)\nplt.grid('minor')\nplt.xlabel('Image #')\nplt.ylabel('Sum of Intensity per Image #')\n#plt.title(\ntitle = axes.set_title(\"\\n\".join(wrap(\"Envelope of the Intensity as a fucntion of image number for all samples\", 60)))\n\n","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:44:36.050991Z","iopub.execute_input":"2021-11-06T13:44:36.051476Z","iopub.status.idle":"2021-11-06T13:44:36.545953Z","shell.execute_reply.started":"2021-11-06T13:44:36.051438Z","shell.execute_reply":"2021-11-06T13:44:36.54524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**It is clear that a method to pre-process the data into 3D data with the size of NxNxM is needed. The approach taken here is to first find the sum of the pixel intensities per image. Then the first and last non-zero intensity images are found. A margin of, in this example 7 images, is also added since the images in the beginning and end of the sequences seem to contain very little information. In between these two values, N number of images are chosen with evenly distributed distances. See the examples below where the blue lines indicate the intensity per image and the orange dots indicate the selected images. Note that the x-axis (showing the number of images) is greatly varying.**","metadata":{}},{"cell_type":"code","source":"fig = plt.figure(figsize=(8, 5))\nplt.subplot(221)\n_,_,_,img0 = stack3Dimages('FLAIR', df=df.iloc[0:1], plotExample = True)\nplt.xlabel('Image #')\nplt.ylabel('Sum of Intenisties')\nplt.title(\"Img0\")\nplt.subplot(222)\n_,_,_,img1 = stack3Dimages('FLAIR', df=df.iloc[2:3], plotExample = True)\nplt.xlabel('Image #')\nplt.ylabel('Sum of Intenisties')\nplt.title(\"Img1\")\nplt.subplot(223)\n_,_,_,img2 = stack3Dimages('FLAIR', df=df.iloc[4:5], plotExample = True)\nplt.xlabel('Image #')\nplt.ylabel('Sum of Intenisties')\nplt.title(\"Img2\")\nplt.subplot(224)\n_,_,_,img3 = stack3Dimages('FLAIR', df=df.iloc[6:7], plotExample = True)\nplt.xlabel('Image #')\nplt.ylabel('Sum of Intenisties')\nplt.title(\"Img3\")\nfig.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:56:31.625005Z","iopub.execute_input":"2021-11-06T13:56:31.625521Z","iopub.status.idle":"2021-11-06T13:56:38.119615Z","shell.execute_reply.started":"2021-11-06T13:56:31.625483Z","shell.execute_reply":"2021-11-06T13:56:38.118948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**The output image sequences illustrated for the four examples above:**","metadata":{}},{"cell_type":"code","source":"for k_img in range(4):\n    fig = plt.figure(figsize=(35, 7))\n    for k in range(settings.num_images):\n        plt.subplot(2,int(settings.num_images/2),k+1)\n        plt.imshow(eval(\"img\"+str(k_img) +\"[:,:,k]\"))\n        plt.title(\"sub image #: \" +str(k))\n    fig.suptitle(\"Image \" +str(k_img), fontsize=35)\n    fig.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:56:46.955381Z","iopub.execute_input":"2021-11-06T13:56:46.956062Z","iopub.status.idle":"2021-11-06T13:57:00.067605Z","shell.execute_reply.started":"2021-11-06T13:56:46.956007Z","shell.execute_reply":"2021-11-06T13:57:00.066979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**We now define a new image loader that uses the method described above to load images. Note that the there is not enough memory to store constructed images. Instead, the found indices are stored and used when loading new images to avoid having to run the image stacking algorithm over and over again.**","metadata":{}},{"cell_type":"code","source":"def load_dicom_images_3d_new(scan_id, num_imgs=settings.num_images, img_size=settings.size, mri_type=\"FLAIR\", split=\"train\", rotate_img=True,idx_array=pick_idx):\n\n    files = sorted(glob.glob(f\"{data_path}/{split}/{scan_id}/{mri_type}/*.dcm\"), \n       key=lambda var:[int(x) if x.isdigit() else x for x in re.findall(r'[^0-9]|[0-9]+', var)])\n    img3d=[]\n    \n    if scan_id in pick_idx: \n        #print('happens')\n        img_out = np.stack([load_dicom_image(f, rotate_img=rotate_img) for f in [files[kk] for kk in pick_idx[scan_id]]]).T    \n    else:\n        for f in files:\n            if len(img3d)==0:\n                img3d = np.expand_dims(load_dicom_image(f,rotate_img=rotate_img),-1)\n            else:\n                img3d = np.append(img3d,np.expand_dims(load_dicom_image(f,rotate_img=rotate_img),-1),axis=-1)\n\n        img_int = np.zeros(np.shape(img3d)[2])\n        for k in range(np.shape(img3d)[2]):\n            img_int[k] = np.sum(np.abs(img3d[:,:,k]))\n\n        idx_margin = 4 #Dont choose the first/last non-zero image\n        idx = np.argmax(img_int)\n        zero_idx = np.nonzero(img_int==0)[0]\n        if np.sum(zero_idx<idx) == 0:\n            start_idx = np.array([0])\n        else:\n            start_idx = zero_idx[zero_idx<idx]   \n\n        start_idx = start_idx[-1]+idx_margin\n\n        if np.sum(zero_idx>=idx) == 0:\n            end_idx = np.array([len(img_int)])\n        else:\n            end_idx = zero_idx[zero_idx>=idx]\n        end_idx = end_idx[0]-idx_margin\n        idx_array = np.linspace(start_idx,end_idx,NUM_IMAGES,dtype=int)\n\n        img3d = np.array(img3d,dtype=float)\n        img_out = img3d[:,:,idx_array]\n    \n    for k in range(np.shape(img_out)[2]):\n        img_out[:,:,k] = img_out[:,:,k] - np.min(img_out[:,:,k])\n        img_out[:,:,k] = img_out[:,:,k] / np.max(img_out[:,:,k])\n    img_out = np.expand_dims(img_out,0)\n    return(img_out)","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:44:55.958781Z","iopub.execute_input":"2021-11-06T13:44:55.959436Z","iopub.status.idle":"2021-11-06T13:44:55.976824Z","shell.execute_reply.started":"2021-11-06T13:44:55.95939Z","shell.execute_reply":"2021-11-06T13:44:55.976164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**We then need to define a custom image loader to use the methods described above to load images.**","metadata":{}},{"cell_type":"code","source":"class customImageGenerator(tf.keras.utils.Sequence):\n    def __init__(self,df,is_train=True,batch_size=settings.batch_size,shuffle=True):\n        self.idx = df[\"BraTS21ID\"].values\n        self.paths = df[\"BraTS21ID5\"].values\n        self.y =  df[\"MGMT_value\"].values\n        self.is_train = is_train\n        self.batch_size = batch_size\n        self.shuffle = shuffle\n    def __len__(self):\n        return math.ceil(len(self.idx)/self.batch_size)\n   \n    def __getitem__(self,ids):\n        id_path= self.paths[ids]\n        batch_paths = self.paths[ids * self.batch_size:(ids + 1) * self.batch_size]\n        \n        if self.y is not None:\n            batch_y = self.y[ids * self.batch_size: (ids + 1) * self.batch_size]\n        \n        if self.is_train:\n            list_x =  [load_dicom_images_3d_new(x,split=\"train\",idx_array=pick_idx) \n                       for x in batch_paths]       \n            batch_X = np.stack(list_x, axis=0)\n            batch_X =np.reshape(batch_X,(np.shape(batch_X)[0],settings.size,settings.size,settings.num_images,1))\n            return batch_X,batch_y\n        else:\n            list_x =  [load_dicom_images_3d_new(x,split=\"test\",idx_array=pick_idx) \n                       for x in batch_paths]\n            batch_X = np.stack(list_x, axis=0)\n            batch_X =np.reshape(batch_X,(np.shape(batch_X)[0],settings.size,settings.size,settings.num_images,1))\n            return batch_X\n    \n    def on_epoch_end(self):\n        if self.shuffle and self.is_train:\n            shuffle(self.idx)","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:49:49.478809Z","iopub.execute_input":"2021-11-06T13:49:49.479652Z","iopub.status.idle":"2021-11-06T13:49:49.492687Z","shell.execute_reply.started":"2021-11-06T13:49:49.479614Z","shell.execute_reply":"2021-11-06T13:49:49.491473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Next, let's define the 3D CNN to be used for classification:**\n\nNote: I played around a lot with different regularization methods. This seems to behave differently compared to conventional 2D CNNs. For instance, in some cases I saw issues with convergence with batch normalization. ","metadata":{}},{"cell_type":"code","source":"l2 = tf.keras.regularizers.l2(l2=0.01)\ndef return_model():\n    model = Sequential()\n    model.add(Input((settings.size,settings.size,settings.num_images,1)))\n    model.add(Conv3D(64, kernel_size=(3,3, 3), strides=(1,1,1), activation='relu', kernel_initializer='he_uniform', name='conv3d_1',kernel_regularizer=l2))\n    model.add(BatchNormalization(center=True, scale=True))\n    model.add(Dropout(0.1))\n    model.add(MaxPooling3D(pool_size=(2, 2, 1)))    \n    model.add(Conv3D(64, kernel_size=(3, 3, 3), strides=(1,1,1), activation='relu', kernel_initializer='he_uniform',name='conv3d_2',kernel_regularizer=l2))\n    model.add(BatchNormalization(center=True, scale=True)) \n    model.add(Dropout(0.1))\n    model.add(MaxPooling3D(pool_size=(2, 2, 2)))    \n    model.add(Conv3D(64, kernel_size=(3, 3, 3), strides=(1,1,1), activation='relu', kernel_initializer='he_uniform',name='conv3d_3',kernel_regularizer=l2))  \n\n    model.add(GlobalAveragePooling3D())\n    model.add(Dropout(0.25))\n    model.add(Dense(512, activation='relu', kernel_initializer='he_uniform'))\n    model.add(Dense(1, activation='sigmoid'))\n    return model","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:46:44.394377Z","iopub.execute_input":"2021-11-06T13:46:44.394971Z","iopub.status.idle":"2021-11-06T13:46:44.404297Z","shell.execute_reply.started":"2021-11-06T13:46:44.394934Z","shell.execute_reply":"2021-11-06T13:46:44.403587Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = return_model()\nmodel.compile(loss='binary_crossentropy', optimizer=tf.keras.optimizers.Adam(learning_rate=settings.learning_rate), metrics=[\"accuracy\"])\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:46:46.08913Z","iopub.execute_input":"2021-11-06T13:46:46.089814Z","iopub.status.idle":"2021-11-06T13:46:48.529509Z","shell.execute_reply.started":"2021-11-06T13:46:46.089777Z","shell.execute_reply":"2021-11-06T13:46:48.528832Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Training the 3D CNN:**","metadata":{}},{"cell_type":"code","source":"#Currently not used, but kept if adaptive LR should be re-implemented:\nclass CustomCallback(keras.callbacks.Callback):\n    def on_epoch_end(self, epoch, logs=None):\n        LR = K.get_value(model.optimizer.learning_rate)   \n        print(\"LR used previous epoch:\" +str(LR))\n\nmy_callbacks = [ CustomCallback() ]\n\nhistory = model.fit(\n    customImageGenerator(train_df,batch_size=settings.batch_size, shuffle = True),\n    validation_data=customImageGenerator(test_df,batch_size=settings.batch_size,shuffle = False),\n    epochs=settings.numEpochs,\n    shuffle=True,\n    verbose=1,\n    callbacks =[CustomCallback()]\n)","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:49:52.025452Z","iopub.execute_input":"2021-11-06T13:49:52.026128Z","iopub.status.idle":"2021-11-06T13:51:24.04491Z","shell.execute_reply.started":"2021-11-06T13:49:52.026079Z","shell.execute_reply":"2021-11-06T13:51:24.044221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Plotting the loss and the accuracy:**","metadata":{}},{"cell_type":"code","source":"plt.figure()\nplt.plot(history.history['loss'],'-',label='loss')\nplt.plot(history.history['val_loss'],'--',label='val_loss')\nplt.legend()\nplt.xlabel('Epoch #')\nplt.ylabel('Loss')\n\nplt.figure()\nplt.plot(history.history['accuracy'],'-',label='accuracy')\nplt.plot(history.history['val_accuracy'],'--',label='val_accuracy')\nplt.legend()\nplt.xlabel('Epoch #')\nplt.ylabel('Accuracy')","metadata":{"execution":{"iopub.status.busy":"2021-11-06T13:52:57.820628Z","iopub.execute_input":"2021-11-06T13:52:57.820884Z","iopub.status.idle":"2021-11-06T13:52:58.373352Z","shell.execute_reply.started":"2021-11-06T13:52:57.820857Z","shell.execute_reply":"2021-11-06T13:52:58.372633Z"},"trusted":true},"execution_count":null,"outputs":[]}]}