{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Fastai2 DICOM starter","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"[Fastai2](https://github.com/fastai/fastai2) starter code using DICOMs.  DICOM(Digital Imaging and COmmunications in Medicine) is the de-facto standard that establishes rules that allow medical images(X-Ray, MRI, CT) and associated information to be exchanged between imaging equipment from different vendors, computers, and hospitals.\n\nDICOM files typically have a `.dcm` extension and provides a means of storing data in separate 'tags' such as patient information as well as image/pixel data. A DICOM file consists of a header and image data sets packed into a single file. The information within the header is organized as a constant and standardized series of tags. \n\nBy extracting data from these tags one can access important information regarding the patient demographics, study parameters, etc\n\n![Parts of a DICOM](https://asvcode.github.io/MedicalImaging/images/copied_from_nb/my_icons/dicom_.PNG)\n\nYou can find out more about medical imaging by viewing this [blog](https://asvcode.github.io/MedicalImaging/)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install fastai2 -q","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Load the dependancies","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"In order to load DICOMs in fastai2 we need to all load the `fastai2.medical.imaging` module.  However we will not be able to use the full functionality of the medical imaging module because these DICOM images are saved as `XC` format which stands for `External-camera Photography` hence these images are restricted to pixel values between `0` and `255`.  This is way limited to say 16 bit DICOM images that could have values ranging from `-32768` to `32768`.\n\n`Pydicom` is a python package for parsing DICOM files and makes it easy to covert DICOM files into pythonic structures for easier manipulation. Files are opened using pydicom.dcmread\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#Load the dependancies\nfrom fastai2.basics import *\nfrom fastai2.callback.all import *\nfrom fastai2.vision.all import *\nfrom fastai2.medical.imaging import *\n\nimport pydicom\nimport seaborn as sns\n\nimport numpy as np\nimport pandas as pd\nimport os","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Specify the source","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"source = Path(\"../input/siim-isic-melanoma-classification\")\nfiles = os.listdir(source)\nprint(files)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Specify the folder that contains the training images `train` and use `fastai2`s method of accessing the DICOM files by using `get_dicom_files`","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train = source/'train'\ntrain_files = get_dicom_files(train)\ntrain_files","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets see what information is contained within each DICOM file","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"patient1 = train_files[7]\ndimg = dcmread(patient1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"You can now view all the information of the DICOM file. Explanation of each element is beyond the scope of this notebook but [this](http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.7.6.3.html#sect_C.7.6.3.1.4) site has some excellent information about each of the entries. Information is listed by the DICOM tag (eg: 0008, 0005) or DICOM keyword (eg: Specific Character Set)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dimg","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Create a function that will display an image and show choosen tags within the head of the DICOM, in this case `PatientName`, `PatientID`, `PatientSex`, `BodyPartExamined` and we can use `Tranform` from `fastai2` that conveniently allows us to resize the image","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def show_one_patient(file):\n    \"\"\" function to view patient image and choosen tags within the head of the DICOM\"\"\"\n    pat = dcmread(file)\n    print(f'patient Name: {pat.PatientName}')\n    print(f'Patient ID: {pat.PatientID}')\n    print(f'Patient age: {pat.PatientAge}')\n    print(f'Patient Sex: {pat.PatientSex}')\n    print(f'Body part: {pat.BodyPartExamined}')\n    trans = Transform(Resize(256))\n    dicom_create = PILDicom.create(file)\n    dicom_transform = trans(dicom_create)\n    return show_image(dicom_transform)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"show_one_patient(patient1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**But why does the image look so unnatural?**","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"This is because these images are stored in `YBR_FULL_422` color space and this is stated in the following tag:\n\n`(0028, 0004) Photometric Interpretation CS: 'YBR_FULL_422'`\n\nTo view the images as they are intended the color space needs to be converted from `YBR_FULL_422` to `RGB`.  `Pydicom` provides a means of converting from one color space to another by using `convert_color_space` where it takes the (pixel array, current color space, desired color space) as attributes.  This is done by acessing the `pixel_array` and then converting to the desired color space","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from pydicom.pixel_data_handlers.util import convert_color_space","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"arr = dimg.pixel_array\nconvert = convert_color_space(arr, 'YBR_FULL_422', 'RGB')\nshow_image(convert)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That looks better!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Pixel Distribution","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We can also view the pixel distribution of the image.  For this competition this is not really that important but can be as shown in this kernel [Understanding Dicoms](https://www.kaggle.com/avirdee/understanding-dicoms)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"px = dimg.pixels.flatten()\nplt.hist(px, color='c')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# EDA","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Load in the csv file","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df = pd.read_csv(source/'train.csv')\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can now explore the distribution of the data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#Plot 3 comparisons\ndef plot_comparison3(df, feature, feature1, feature2):\n    \"Plot 3 comparisons from a dataframe\"\n    fig, (ax1, ax2, ax3) = plt.subplots(1,3, figsize = (16, 4))\n    s1 = sns.countplot(df[feature], ax=ax1)\n    s1.set_title(feature)\n    s2 = sns.countplot(df[feature1], ax=ax2)\n    s2.set_title(feature1)\n    s3 = sns.countplot(df[feature2], ax=ax3)\n    s3.set_title(feature2)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_comparison3(df, 'sex', 'age_approx', 'benign_malignant')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Plot 1 comparisons\ndef plot_comparison1(df, feature):\n    \"Plot 1 comparisons from a dataframe\"\n    fig, (ax1) = plt.subplots(1,1, figsize = (16, 4))\n    s1 = sns.countplot(df[feature], ax=ax1)\n    s1.set_title(feature)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_comparison1(df, 'diagnosis')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_comparison1(df, 'target')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_comparison1(df, 'anatom_site_general_challenge')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets create a dataframe with a few features we want to explore more","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"eda_df = df[['sex','age_approx','anatom_site_general_challenge','diagnosis','target']]\neda_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(eda_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There are a number of `Nan` values within each column that we want to get rid off","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sex_count = eda_df['sex'].isna().sum(); age_count = eda_df['age_approx'].isna().sum(); anatom_count = eda_df['anatom_site_general_challenge'].isna().sum()\nprint(f'Nan values in sex column: {sex_count}, age column: {age_count}, anatom count: {anatom_count}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_drop = eda_df.dropna()\nlen(df_drop)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For some more EDA we need to convert the categorical features in the dataframe into numeric values. `LabelEncoder` encode labels with a value between 0 and n_classes-1 where `n` is the number of distinct labels","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nle = LabelEncoder()\nedaa_df = eda_df.apply(lambda col: le.fit_transform(col.astype(str)), axis=0, result_type='expand')\nedaa_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"lets set some seaborn parameters","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.set(style=\"whitegrid\")\nsns.set_context(\"paper\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.pairplot(eda_df, hue=\"target\", height=5, aspect=2, palette='gist_rainbow_r')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.pairplot(eda_df, hue=\"age_approx\", height=6, aspect=3, diag_kws={'bw':'0.05'})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The plot above shows the distribution between age and target, here it is clear to see the differences between age in patients with target `0` and target `1`","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.set(style=\"whitegrid\")\nsns.set_context(\"poster\")\nsns.pairplot(edaa_df, hue=\"target\", height=6, palette='gist_rainbow', diag_kws={'bw':'0.05'})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Getting the data ready for training","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Lets first specify the `x` or input.  In this case we can create a `lambda` function that will get the image files from the `train` folder","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"get_x = lambda x:source/'train'/f'{x[0]}.dcm'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We now specify the 'y' or output using `ColReader` and specify the `target` column in the csv file which in this case `0` denotes benign and `1` denotes malignant.  ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"get_y=ColReader('target')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Getting some quick `batch_tfms`","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"batch_tfms = aug_transforms(flip_vert=True, max_lighting=0.1, max_zoom=1.05, max_warp=0.)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`fastai2` provides a convenient way of using `blocks`, in this case because we are specifying an `x` and a `y` we can now specify that the `x` will be `PILDicom` image and the `y` will be a `CategoryBlock` because we want the target to be either benign, `0` or malignant `1`","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"However before we can create the `DataBlock` so that the images look 'real' we need to create a new method so that `PILBase` takes into consideration the `Photometric Interpretation`.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class PILDicom2(PILBase):\n    _open_args,_tensor_cls,_show_args = {},TensorDicom,TensorDicom._show_args\n    @classmethod\n    def create(cls, fn:(Path,str,bytes), mode=None)->None:\n        \"Open a `DICOM file` from path `fn` or bytes `fn` and load it as a `PIL Image`\"\n        dimg = dcmread(fn)\n        arr = dimg.pixel_array; convert = convert_color_space(arr,'YBR_FULL_422', 'RGB')\n        im = Image.fromarray(convert)\n        im.load()\n        im = im._new(im.im)\n        return cls(im.convert(mode) if mode else im)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"blocks = (ImageBlock(cls=PILDicom2), CategoryBlock)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can now easily collate all the data into a `DataBlock` and use `fastai2`s inbuilt `splitter` function that will split the data into `train` and `valid` sets.  `Resize` ensure all the images are the same size when we feed it to the model.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"melanoma = DataBlock(blocks=blocks,\n                   get_x=get_x,\n                   splitter=RandomSplitter(),\n                   item_tfms=Resize(128),\n                   get_y=ColReader('target'),\n                   batch_tfms=batch_tfms)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As the dataset is huge, we can test the model by just training with `100` samples from the `train` dataset","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dls = melanoma.dataloaders(df.sample(100), bs=2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dls = dls.cuda()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Viewing a batch","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dls.show_batch(max_n=12, nrows=2, ncols=6)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Specify the evaluation metric and check how many labels there are","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"roc = RocAuc()\ndls.c","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Specify the architecure to be used.  In this case we ensure that the the output of the model is either `0` or `1` or 2 classes.  `dls.c` is a convenient way to specify that the output of the model will be 2.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"model = xresnet18_deeper(n_out=dls.c)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"set_seed(77)\nlearn = Learner(dls, model, \n                opt_func=ranger,\n                loss_func=LabelSmoothingCrossEntropy(),\n                metrics=[accuracy, roc],\n                cbs = ShowGraphCallback())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.freeze()\nlearn.fit_one_cycle(1, 5e-2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.save('xresnet18_stg1')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.unfreeze()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fit_flat_cos(2,slice(1e-6,1e-4))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.save('xresnet18_stg2')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"interp = Interpretation.from_learner(learn)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can look at the top losses","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"interp.plot_top_losses(12)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Loading the `test` set","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"tst = source/'test'\ntest_set = get_dicom_files(tst)\ntest_set","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For testing purposes we will only use the first 100 images in the test set","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"test_set = test_set[:100]\ntest_set","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Specify a test patient ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"test_patient = test_set[1]\ntest_patient","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.load('xresnet18_stg2')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets look at a prediction for the `test_patient`","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"_ = learn.predict(test_patient)\n_","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The `predict` function displays the predicted class, in this case `0`, the tensor class `tensor(0)` and the probabilites of the each of the classes.  In this dataset there are 2 classes `0` and `1` and the the probabilites are predicted for each class so in this case the probablility that the `test_patient` is `benign` or class `0` is `0.9923` and the probability that the `test_patient` is `malignant` or `1` is `0.0317`","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"The evalution requirement in this competiton is that for each image_name in the test set, you must predict the probability (target) that the sample is malignant.  So we need to get the probability of class `1`\n\nWe can use the code below to get the probability of class `1`:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"_ = learn.predict(test_patient)\nprint(_[2][1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Load the `sample_submisson`","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_sub = pd.read_csv(source/'sample_submission.csv')\nsample_sub = sample_sub[:100]\nsample_sub","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can delete the `target` column as we will be populating this with the probabilites","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"del sample_sub['target']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Get the probabilites for the test set and create a list of the probabilites for each image in the test set and convert the probabilty to a float","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_list = []\nfor i in test_set:\n    pre = learn.predict(i)\n    l = float(pre[2][1])\n    sample_list.append(l)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_list","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub = sample_sub.assign(target=sample_list)\nsub.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub = pd.read_csv('submission.csv')\nsub","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Next steps:\n\n- Experiment with various models and augmentations\n","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}