{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction\n**Running out of TPU quota? Want to improve your model?**\n\nDon't worry. There is always something to work with in your model (until it is not ideal). This notebook shows LIME method applied to SIIM melanoma competition data. Although LIME (Local Interpretable Model-agnostic Explanations) is a great tool to explain what machine learning classifiers (or models) are doing, there is a link between citrus consumption and melanoma (link in refferences)- incredible!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"![lime](https://www.cancertherapyadvisor.com/wp-content/uploads/sites/12/2018/12/citrusmelanomariskskincanc_794020.jpg)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Refferences:\n* https://towardsdatascience.com/understanding-model-predictions-with-lime-a582fdff3a3b - LIME described\n* https://towardsdatascience.com/decrypting-your-machine-learning-model-using-lime-5adc035109b5 - also LIME described\n* https://github.com/marcotcr/lime - package lime on github\n* https://scikit-image.org/docs/stable/auto_examples/segmentation/plot_segmentations.html segmentation algorithms\n* https://www.cancertherapyadvisor.com/home/cancer-topics/skin-cancer/link-between-citrus-consumption-and-melanoma/ - additional article\n\n## Versions:\n* V1-V4: Testing and choosing segmentation algorithm\n* V5: Added predictions from csv \n* V6: Added case comparison vs SHAP explanations\n* V7: Model with microscope transform applied to training data (wow!)\n* V9: Re-run with DEFAULT_NUM_SAMPLES = 100 (checking stability of results)\n* V10: Re-run with DEFAULT_NUM_SAMPLES = 500 (checking stability of results)\n* V11: Added transform view\n* V12: Bug fixed\n* V13: Investigating @cdeotte model fold 0 from notebook: https://www.kaggle.com/cdeotte/triple-stratified-kfold-with-tfrecords","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Loading packages, initial config, model, dataset and predictions","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"!pip install -q efficientnet","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import math, re, os\nimport tensorflow as tf, tensorflow.keras.backend as K\nimport numpy as np\nimport pandas as pd\nfrom matplotlib import pyplot as plt\nfrom kaggle_datasets import KaggleDatasets\nimport efficientnet.tfkeras as efn\nimport cv2\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\nfrom matplotlib import pyplot as plt\nfrom sklearn import metrics\nfrom skimage.segmentation import mark_boundaries\nfrom sklearn.metrics import confusion_matrix\nimport time\n\nAUTO = tf.data.experimental.AUTOTUNE\nGCS_PATH = KaggleDatasets().get_gcs_path('melanoma-384x384')\nGCS_PATH2 = KaggleDatasets().get_gcs_path('isic2019-384x384')\nIMAGE_SIZE = [384, 384]\nBATCH_SIZE = 128\n#VALIDATION_FILENAMES = [tf.io.gfile.glob(GCS_PATH + '/train*.tfrec')[fi] for fi in [ 0,  3, 12, 15, 21, 25]]\nDEFAULT_NUM_SAMPLES = 100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Model\nModel used in this notebook is loaded from great @cdeotte notebook https://www.kaggle.com/cdeotte/triple-stratified-kfold-with-tfrecords","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"!pip install -q efficientnet >> /dev/null","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"EFNS = [efn.EfficientNetB0, efn.EfficientNetB1, efn.EfficientNetB2, efn.EfficientNetB3, \n        efn.EfficientNetB4, efn.EfficientNetB5, efn.EfficientNetB6]\nimport efficientnet.tfkeras as efn\n\ndef build_model(dim=128, ef=0):\n    inp = tf.keras.layers.Input(shape=(dim,dim,3))\n    base = EFNS[ef](input_shape=(dim,dim,3),weights='imagenet',include_top=False)\n    x = base(inp)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    x = tf.keras.layers.Dense(1,activation='sigmoid')(x)\n    model = tf.keras.Model(inputs=inp,outputs=x)\n    opt = tf.keras.optimizers.Adam(learning_rate=0.001)\n    loss = tf.keras.losses.BinaryCrossentropy(label_smoothing=0.05) \n    model.compile(optimizer=opt,loss=loss,metrics=['AUC'])\n    return model\n\nmodel = build_model(dim=IMAGE_SIZE[0],ef=6)\nmodel.load_weights('/kaggle/input/triple-stratified-kfold-with-tfrecords/fold-0.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Dataset pipelines","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"ROT_ = 180.0\nSHR_ = 2.0\nHZOOM_ = 8.0\nWZOOM_ = 8.0\nHSHIFT_ = 8.0\nWSHIFT_ = 8.0\n\ndef get_mat(rotation, shear, height_zoom, width_zoom, height_shift, width_shift):\n    # returns 3x3 transformmatrix which transforms indicies\n        \n    # CONVERT DEGREES TO RADIANS\n    rotation = math.pi * rotation / 180.\n    shear    = math.pi * shear    / 180.\n\n    def get_3x3_mat(lst):\n        return tf.reshape(tf.concat([lst],axis=0), [3,3])\n    \n    # ROTATION MATRIX\n    c1   = tf.math.cos(rotation)\n    s1   = tf.math.sin(rotation)\n    one  = tf.constant([1],dtype='float32')\n    zero = tf.constant([0],dtype='float32')\n    \n    rotation_matrix = get_3x3_mat([c1,   s1,   zero, \n                                   -s1,  c1,   zero, \n                                   zero, zero, one])    \n    # SHEAR MATRIX\n    c2 = tf.math.cos(shear)\n    s2 = tf.math.sin(shear)    \n    \n    shear_matrix = get_3x3_mat([one,  s2,   zero, \n                                zero, c2,   zero, \n                                zero, zero, one])        \n    # ZOOM MATRIX\n    zoom_matrix = get_3x3_mat([one/height_zoom, zero,           zero, \n                               zero,            one/width_zoom, zero, \n                               zero,            zero,           one])    \n    # SHIFT MATRIX\n    shift_matrix = get_3x3_mat([one,  zero, height_shift, \n                                zero, one,  width_shift, \n                                zero, zero, one])\n    \n    return K.dot(K.dot(rotation_matrix, shear_matrix), \n                 K.dot(zoom_matrix,     shift_matrix))\n\n\ndef transform(image, DIM=256):    \n    # input image - is one image of size [dim,dim,3] not a batch of [b,dim,dim,3]\n    # output - image randomly rotated, sheared, zoomed, and shifted\n    XDIM = DIM%2 #fix for size 331\n    \n    rot = ROT_ * tf.random.normal([1], dtype='float32')\n    shr = SHR_ * tf.random.normal([1], dtype='float32') \n    h_zoom = 1.0 + tf.random.normal([1], dtype='float32') / HZOOM_\n    w_zoom = 1.0 + tf.random.normal([1], dtype='float32') / WZOOM_\n    h_shift = HSHIFT_ * tf.random.normal([1], dtype='float32') \n    w_shift = WSHIFT_ * tf.random.normal([1], dtype='float32') \n\n    # GET TRANSFORMATION MATRIX\n    m = get_mat(rot,shr,h_zoom,w_zoom,h_shift,w_shift) \n\n    # LIST DESTINATION PIXEL INDICES\n    x   = tf.repeat(tf.range(DIM//2, -DIM//2,-1), DIM)\n    y   = tf.tile(tf.range(-DIM//2, DIM//2), [DIM])\n    z   = tf.ones([DIM*DIM], dtype='int32')\n    idx = tf.stack( [x,y,z] )\n    \n    # ROTATE DESTINATION PIXELS ONTO ORIGIN PIXELS\n    idx2 = K.dot(m, tf.cast(idx, dtype='float32'))\n    idx2 = K.cast(idx2, dtype='int32')\n    idx2 = K.clip(idx2, -DIM//2+XDIM+1, DIM//2)\n    \n    # FIND ORIGIN PIXEL VALUES           \n    idx3 = tf.stack([DIM//2-idx2[0,], DIM//2-1+idx2[1,]])\n    d    = tf.gather_nd(image, tf.transpose(idx3))\n        \n    return tf.reshape(d,[DIM, DIM,3])\n\ndef read_labeled_tfrecord(example):\n    tfrec_format = {\n        'image'                        : tf.io.FixedLenFeature([], tf.string),\n        'image_name'                   : tf.io.FixedLenFeature([], tf.string),\n        'patient_id'                   : tf.io.FixedLenFeature([], tf.int64),\n        'sex'                          : tf.io.FixedLenFeature([], tf.int64),\n        'age_approx'                   : tf.io.FixedLenFeature([], tf.int64),\n        'anatom_site_general_challenge': tf.io.FixedLenFeature([], tf.int64),\n        'diagnosis'                    : tf.io.FixedLenFeature([], tf.int64),\n        'target'                       : tf.io.FixedLenFeature([], tf.int64)\n    }           \n    example = tf.io.parse_single_example(example, tfrec_format)\n    return example['image'], example['target']\n\n\ndef read_unlabeled_tfrecord(example, return_image_name):\n    tfrec_format = {\n        'image'                        : tf.io.FixedLenFeature([], tf.string),\n        'image_name'                   : tf.io.FixedLenFeature([], tf.string),\n    }\n    example = tf.io.parse_single_example(example, tfrec_format)\n    return example['image'], example['image_name'] if return_image_name else 0\n\n \ndef prepare_image(img, augment=True, dim=256):    \n    img = tf.image.decode_jpeg(img, channels=3)\n    img = tf.cast(img, tf.float32) / 255.0\n    \n    if augment:\n        img = transform(img,DIM=dim)\n        img = tf.image.random_flip_left_right(img)\n        #img = tf.image.random_hue(img, 0.01)\n        img = tf.image.random_saturation(img, 0.7, 1.3)\n        img = tf.image.random_contrast(img, 0.8, 1.2)\n        img = tf.image.random_brightness(img, 0.1)\n                      \n    img = tf.reshape(img, [dim,dim, 3])\n            \n    return img\n\ndef count_data_items(filenames):\n    n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) \n         for filename in filenames]\n    return np.sum(n)\n\ndef get_dataset(files, augment = False, shuffle = False, repeat = False, \n                labeled=True, return_image_names=True, batch_size=16, dim=256):\n    \n    ds = tf.data.TFRecordDataset(files, num_parallel_reads=AUTO)\n    ds = ds.cache()\n    \n    if repeat:\n        ds = ds.repeat()\n    \n    if shuffle: \n        ds = ds.shuffle(1024*8)\n        opt = tf.data.Options()\n        opt.experimental_deterministic = False\n        ds = ds.with_options(opt)\n        \n    if labeled: \n        ds = ds.map(read_labeled_tfrecord, num_parallel_calls=AUTO)\n    else:\n        ds = ds.map(lambda example: read_unlabeled_tfrecord(example, return_image_names), \n                    num_parallel_calls=AUTO)      \n    \n    ds = ds.map(lambda img, imgname_or_label: (prepare_image(img, augment=augment, dim=dim), \n                                               imgname_or_label), \n                num_parallel_calls=AUTO)\n    \n    ds = ds.batch(batch_size * REPLICAS)\n    ds = ds.prefetch(AUTO)\n    return ds\n\nSEED = 42\nFOLDS = 5\nREPLICAS = 1\nTTA = 11\nfrom sklearn.model_selection import KFold\nskf = KFold(n_splits=FOLDS,shuffle=True,random_state=SEED)\n\nfor fold,(idxT,idxV) in enumerate(skf.split(np.arange(15))):\n    if fold == 0:\n        files_valid = tf.io.gfile.glob([GCS_PATH + '/train%.2i*.tfrec'%x for x in idxV])\n\n\nNUM_VALIDATION_IMAGES = count_data_items(files_valid)\nprint('Dataset: {} validation images'.format(NUM_VALIDATION_IMAGES))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Predictions\nThis notebook runs without TPU, so doing predictions could take a long time. Therefore we will previously writed predictions to csv on validation dataset (after training model).","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-output":true,"_kg_hide-input":true},"cell_type":"code","source":"#%%time\n#N_SAMPLE = 2000\n#ds = get_dataset().unbatch().batch(N_SAMPLE)\n#predictions = model.predict(next(iter(ds)))\n\n#ds = get_dataset().unbatch().batch(N_SAMPLE).map(lambda image,image_name, target: image_name)\n#image_names = next(iter(ds)).numpy().astype('U')\n#ds = get_dataset().unbatch().batch(N_SAMPLE).map(lambda image,image_name, target: target)\n#targets = next(iter(ds)).numpy().astype('int32')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#df = pd.DataFrame({\"image_name\": image_names, \"prob\": np.concatenate(predictions),  \"prediction\": np.concatenate(np.round(predictions)), \"target\": targets})\ndf = pd.read_csv('/kaggle/input/triple-stratified-kfold-with-tfrecords/oof.csv')\ndf = df[df['fold'].isin([0])]\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## LIME\nWe will use latest version of package directly from github (although older version is available on Kaggle and via pip install):\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install -q git+https://github.com/marcotcr/lime.git","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from lime import lime_image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Segmentation algorithms\nBefore applying LIME, note that one of the major parts of this method is doing segmentation (the division of the image into smaller parts called **superpixels**). Let's see how most common algorithms segments our SIIM example image:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"img = next(iter(get_dataset(files_valid, augment=False, repeat=False, dim=IMAGE_SIZE[0],\n        labeled=True, return_image_names=True).unbatch().batch(1).map(lambda image, image_name: image))).numpy().squeeze()\nplt.imshow(img)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from skimage.color import rgb2gray\nfrom skimage.filters import sobel\nfrom skimage.segmentation import felzenszwalb, slic, quickshift, watershed\nfrom skimage.segmentation import mark_boundaries\nfrom skimage.util import img_as_float\n\nsegments_fz = felzenszwalb(img, scale=200, sigma=0.5, min_size=50)\nsegments_slic = slic(img, n_segments=50, compactness=10, sigma=1)\nsegments_quick = quickshift(img, kernel_size=2, max_dist=50, ratio=0.5)\ngradient = sobel(rgb2gray(img))\nsegments_watershed = watershed(gradient, markers=50, compactness=0.001)\n\nprint(f\"Felzenszwalb number of segments: {len(np.unique(segments_fz))}\")\nprint(f\"SLIC number of segments: {len(np.unique(segments_slic))}\")\nprint(f\"Quickshift number of segments: {len(np.unique(segments_quick))}\")\nprint(f\"Watershed number of segments: {len(np.unique(segments_watershed))}\")\nfig, ax = plt.subplots(2, 2, figsize=(10, 10), sharex=True, sharey=True)\n\nax[0, 0].imshow(mark_boundaries(img, segments_fz))\nax[0, 0].set_title(\"Felzenszwalbs's method\")\nax[0, 1].imshow(mark_boundaries(img, segments_slic))\nax[0, 1].set_title('SLIC')\nax[1, 0].imshow(mark_boundaries(img, segments_quick))\nax[1, 0].set_title('Quickshift')\nax[1, 1].imshow(mark_boundaries(img, segments_watershed))\nax[1, 1].set_title('Compact watershed')\n\nfor a in ax.ravel():\n    a.set_axis_off()\n\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Looks like Felzenszwalb method doesn't handle well hairs, so we will use SLIC segmenation with custom parameters. I prepared wrapper for standard lime functions, so explanation using LIME of single SIIM image looks like this:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def segment_fn(image):\n    return slic(image, n_segments=50, compactness=10, sigma=1)\n\nDIM = IMAGE_SIZE[0]\ncircle = 1-cv2.circle((np.ones([DIM, DIM, 3])).astype(np.uint8),(DIM//2, DIM//2),np.random.randint(DIM//2 - 3, DIM//2 + 5),\n                    (0, 0, 0),-1)\ndef get_explanations(image_names, num_samples=  DEFAULT_NUM_SAMPLES, random_state = 0, progress_bar = False):\n\n    n_img = len(image_names)\n    id_img = 1\n    \n    for image_name in image_names:\n        explainer = lime_image.LimeImageExplainer(random_state = random_state)\n        \n        img = cv2.imread('/kaggle/input/melanoma-merged-external-data-512x512-jpeg/512x512-dataset-melanoma/512x512-dataset-melanoma/'+image_name+'.jpg', cv2.IMREAD_UNCHANGED)\n        width = IMAGE_SIZE[0]\n        height = IMAGE_SIZE[1]\n        dim = (width, height)\n        img = cv2.resize(img, dim)\n        img = img/255.0\n        \n        if df[df['image_name'].isin([image_name])].shape[0]==0:# if case wasnt in validation data\n            prob = np.concatenate(model.predict(tf.expand_dims(img,axis = 0)))\n            prd = np.round(prob)\n            df2  = pd.read_csv(\"/kaggle/input/melanoma-merged-external-data-512x512-jpeg/marking.csv\")\n            target =df2[df2['image_id'].isin([image_name])][['target']].values[0][0]\n        else:\n            prd = df[df['image_name'].isin([image_name])][['pred']].values[0][0]\n            target = df[df['image_name'].isin([image_name])][['target']].values[0][0]\n        \n        plot_num_cols = 4\n        sp = plt.subplot(n_img, plot_num_cols,id_img)\n        id_img+=1\n        title = 'original (target = '+str(target)+')'\n        sp.set_title(title)\n        sp.set_ylabel(image_name+'.jpg')\n        sp.imshow(img)\n\n        #sp = plt.subplot(n_img, plot_num_cols,id_img)\n        #id_img+=1\n        #title = 'transformed (microscope)'\n        #img*=circle\n        #sp.set_title(title)\n        #sp.imshow(img)        \n\n        sp = plt.subplot(n_img, plot_num_cols,id_img)\n        id_img+=1\n        title = 'segmentation (prediction = '+str(prd)+')'\n        \n        sp.set_title(title)\n        sp.imshow(mark_boundaries(img, segment_fn(img)))\n        \n        explanation = explainer.explain_instance(img, model.predict, num_samples=num_samples, segmentation_fn = segment_fn,progress_bar=progress_bar)\n        temp, mask = explanation.get_image_and_mask(0, positive_only=False,  num_features=5, hide_rest=False, min_weight=0.0)\n        sp = plt.subplot(n_img, plot_num_cols,id_img)\n        sp.set_title('positive and negative regions')\n        id_img+=1\n        sp.imshow(mark_boundaries(temp, mask))\n        \n        temp, mask = explanation.get_image_and_mask(0, positive_only=True if round(prd) == 1 else False, negative_only = True if round(prd) == 0 else False,  num_features=1, hide_rest=False, min_weight=0.0)\n        sp = plt.subplot(n_img, plot_num_cols,id_img)\n        sp.set_title('top '+ ('positive' if round(prd) == 1 else 'negative') + ' region')\n        id_img+=1\n        sp.imshow(mark_boundaries(temp, mask))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"img_list = ['ISIC_2637011']\nplt.rcParams['figure.figsize'] = [18, 5*len(img_list)]\nget_explanations(img_list, num_samples=  DEFAULT_NUM_SAMPLES, random_state = 0, progress_bar = True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Starting from left, we have:\n* original image (described with actual target), \n* segmented image (described with actual prediction), \n* top 5 **positive** and **negative** regions. We have only one class to explain (target) so: positive regions (marked as green) are these superpixels, which, when activated, are increasing the probability in model.predict() output. Respectively  negative regions (marked as red) are these superpixels, which, when activated, are decreasing the probability in model.predict() output.\n* and top decisive region (positive if prediction = 1, negative if prediction = 0)\n\nLet's go further and find out, how our model is working on another examples from dataset! Note that, **num_samples** is one of crucial parameter for this method. More samples, better explanations. In fact, maximum number of samples could be 2^num_superpixels. We will use num_samples = 100.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Results diagnosis\nModel used in this notebook, scores about 0.896 auc on validation dataset:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"y = df['target']\nprob = df['pred']\nfpr, tpr, thresholds = metrics.roc_curve(y, prob, pos_label=1)\nmetrics.auc(fpr, tpr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There are 6397 true negatives, 23 true positives, 23 false positives and 93 (!!) false negatives:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"tn, fp, fn, tp = confusion_matrix(df['target'], round(df['pred'])).ravel()\n(tn, fp, fn, tp)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## True negatives","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"tn = df[df['target'].isin([0]) & round(df['pred']).isin([0])]\ntn.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"%%time\nsuspects = tn.head()['image_name']\nplt.rcParams['figure.figsize'] = [18, 5*len(suspects)]\nget_explanations(suspects)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## True positives ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"tp = df[df['target'].isin([1]) & round(df['pred']).isin([1])]\ntp.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"%%time \nsuspects = tp.head(20)['image_name']\nplt.rcParams['figure.figsize'] = [18, 5*len(suspects)]\nget_explanations(suspects)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## False positives","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fp = df[df['target'].isin([0]) & round(df['pred']).isin([1])]\nfp","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"%%time\nsuspects = fp.head(20)['image_name']\nplt.rcParams['figure.figsize'] = [18, 5*len(suspects)]\nget_explanations(suspects)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## False negatives","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fn = df[df['target'].isin([1]) & round(df['pred']).isin([0])]\nfn","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"suspects = fn.head(20)['image_name']\nplt.rcParams['figure.figsize'] = [18, 5*len(suspects)]\nget_explanations(suspects)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"At the first glance, very similar story to FP. So many cases, where skin colors are decideable!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Case compared to SHAP\nAs an exercise, I want to compare LIME results with SHAP with some case described here:\nhttps://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/163425","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"img_list = ['ISIC_0232101']\nplt.rcParams['figure.figsize'] = [18, 5*len(img_list)]\nget_explanations(img_list, num_samples=  DEFAULT_NUM_SAMPLES, random_state = 0, progress_bar = True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Conclusions\nFirst of all, this example shows that there are many ideas to try with this competiton:\n* microscope transformation \n* unmicroscope augmentation (fill dark corners with average color)\n* batch normalization,\n* hair augmentation\n\nAnd many others that maybe I didn't see!\n\nSecond thing, it's worth to save your model_weights or model after training. Thank you for reading this notebook!","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}