{"cells":[{"metadata":{},"cell_type":"markdown","source":"<h1 style='color:white; background:#0A0502; border:0'><center>RANZCR: Xception TPU</center></h1>\n\n![](https://storage.googleapis.com/kaggle-competitions/kaggle/23870/logos/header.png?t=2020-12-01-04-28-05)\n\nSerious complications can occur as a result of malpositioned lines and tubes in patients. Doctors and nurses frequently use checklists for placement of lifesaving equipment to ensure they follow protocol in managing patients. Yet, these steps can be time consuming and are still prone to human error, especially in stressful situations when hospitals are at capacity.\n\nHospital patients can have catheters and lines inserted during the course of their admission and serious complications can arise if they are positioned incorrectly. Nasogastric tube malpositioning into the airways has been reported in up to 3% of cases, with up to 40% of these cases demonstrating complications [1-3]. Airway tube malposition in adult patients intubated outside the operating room is seen in up to 25% of cases [4,5]. The likelihood of complication is directly related to both the experience level and specialty of the proceduralist. Early recognition of malpositioned tubes is the key to preventing risky complications (even death), even more so now that millions of COVID-19 patients are in need of these tubes and lines.\n\n<a id=\"start\"></a>\n\nThe gold standard for the confirmation of line and tube positions are chest radiographs. However, a physician or radiologist must manually check these chest x-rays to verify that the lines and tubes are in the optimal position. Not only does this leave room for human error, but delays are also common as radiologists can be busy reporting other scans. Deep learning algorithms may be able to automatically detect malpositioned catheters and lines. Once alerted, clinicians can reposition or remove them to avoid life-threatening complications.\n\n<h2 style='color:#0A0502; background:white; border:2px solid #0A0502'><center>Table of contents:</center></h2>\n\n* [**Fast look at the data**](#1)\n* [**Preparation for prediction**](#2)\n* [**Visualization of CNN intermediate activations**](#3)\n* [**Prediction**](#4)\n\n## The first part of this notebook: [RANZCR: Xception TPU](https://www.kaggle.com/maksymshkliarevskyi/ranzcr-xception-tpu)"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom sklearn.model_selection import train_test_split\nimport tensorflow as tf\nfrom tensorflow.keras import models, layers\nfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau\nfrom tensorflow.keras.applications import Xception\nfrom tensorflow.keras.optimizers import Adam\nimport tensorflow_addons as tfa\n\n# ignoring warnings\nimport warnings\nwarnings.simplefilter(\"ignore\")\n\nimport os, cv2","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# TPU or GPU detection\n# Detect hardware, return appropriate distribution strategy\n# try:\n#     tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n#     print(f'Running on TPU {tpu.master()}')\n# except ValueError:\n#     tpu = None\n\n# if tpu:\n#     tf.config.experimental_connect_to_cluster(tpu)\n#     tf.tpu.experimental.initialize_tpu_system(tpu)\n#     strategy = tf.distribute.experimental.TPUStrategy(tpu)\n# else:\n#     strategy = tf.distribute.get_strategy()\n\n# AUTO = tf.data.experimental.AUTOTUNE\n# REPLICAS = strategy.num_replicas_in_sync\n# print(f'REPLICAS: {REPLICAS}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<h3 style='color:white; background:#0A0502; border:0'><center>Work directory</center></h3>"},{"metadata":{"trusted":true},"cell_type":"code","source":"WORK_DIR = '../input/ranzcr-clip-catheter-line-classification'\nos.listdir(WORK_DIR)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"1\"></a>\n<h1 style='color:white; background:#0A0502; border:0'><center>Fast look at the data</center></h1>\n\n[**Back to the table of contents**](#start)"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print('Train images: %d' %len(os.listdir(os.path.join(WORK_DIR, \"train\"))))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"# Data\ntrain = pd.read_csv(os.path.join(WORK_DIR, \"train.csv\"))\ntrain_images = WORK_DIR + \"/train/\" + train['StudyInstanceUID'] + '.jpg'\n\nss = pd.read_csv(os.path.join(WORK_DIR, 'sample_submission.csv'))\ntest_images = WORK_DIR + \"/test/\" + ss['StudyInstanceUID'] + '.jpg'\n\nlabel_cols = ss.columns[1:]\nlabels = train[label_cols].values\n\ntrain_annot = pd.read_csv(os.path.join(WORK_DIR, \"train_annotations.csv\"))\n\nprint('Labels:\\n', '*'*20, '\\n', label_cols.values)\nprint('*'*50)\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"sns.set_style(\"whitegrid\")\nfig = plt.figure(figsize = (15, 12), dpi = 300)\nplt.suptitle('Labels count', fontfamily = 'serif', size = 15)\n\nfor ind, i in enumerate(label_cols):\n    fig.add_subplot(4, 3, ind + 1)\n\n    sns.countplot(train[i], edgecolor = 'black',\n                  palette = reversed(sns.color_palette('viridis', 2)))\n    \n    plt.xlabel('')\n    plt.ylabel('')\n    plt.xticks(fontfamily = 'serif', size = 10)\n    plt.yticks(fontfamily = 'serif', size = 10)\n    plt.title(i, fontfamily = 'serif', size = 10)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"sample = train.sample(9)\nplt.figure(figsize=(10, 7), dpi = 300)\nfor ind, image_id in enumerate(sample.StudyInstanceUID):\n    plt.subplot(3, 3, ind + 1)\n    image = image_id + '.jpg'\n    img = cv2.imread(os.path.join(WORK_DIR, \"train\", image))\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    plt.imshow(img)\n    plt.title('Shape: {}'.format(img.shape[:2]))\n    plt.axis(\"off\")\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"2\"></a>\n<h1 style='color:white; background:#0A0502; border:0'><center>Preparation for prediction</center></h1>\n\n[**Back to the table of contents**](#start)"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Main parameters\nBATCH_SIZE = 8 * 1\nSTEPS_PER_EPOCH = len(train) * 0.85 / BATCH_SIZE\nVALIDATION_STEPS = len(train) * 0.15 / BATCH_SIZE\nEPOCHS = 30\nTARGET_SIZE = 750","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# https://www.kaggle.com/xhlulu/ranzcr-efficientnet-b3-gpu-starter\n\ndef build_decoder(with_labels = True,\n                  target_size = (TARGET_SIZE, TARGET_SIZE), \n                  ext = 'jpg'):\n    def decode(path):\n        file_bytes = tf.io.read_file(path)\n        if ext == 'png':\n            img = tf.image.decode_png(file_bytes, channels = 3)\n        elif ext in ['jpg', 'jpeg']:\n            img = tf.image.decode_jpeg(file_bytes, channels = 3)\n        else:\n            raise ValueError(\"Image extension not supported\")\n\n        img = tf.cast(img, tf.float32) / 255.0\n        img = tf.image.resize(img, target_size)\n\n        return img\n    \n    def decode_with_labels(path, label):\n        return decode(path), label\n    \n    return decode_with_labels if with_labels else decode\n\n\ndef build_augmenter(with_labels = True):\n    def augment(img):\n        img = tf.image.random_flip_left_right(img)\n        img = tf.image.random_flip_up_down(img)\n        img = tf.image.adjust_brightness(img, 0.1)\n        \n#         rotate = tf.random.uniform([], 0, 1.0, dtype = tf.float32)      \n#         if rotate > .75:\n#             img = tf.image.rot90(img, k = 3)\n#         elif rotate > .5:\n#             img = tf.image.rot90(img, k = 2)\n#         elif rotate > .25:\n#             img = tf.image.rot90(img, k = 1)\n        \n#         saturation = tf.random.uniform([], 0, 1.0, dtype = tf.float32)\n#         if saturation >= .5:\n#             img = tf.image.random_saturation(img, lower = 0.9, upper = 1.1)\n        \n#         contrast = tf.random.uniform([], 0, 1.0, dtype = tf.float32)\n#         if contrast >= .5:\n#             img = tf.image.random_contrast(img, lower = 0.9, upper = 1.1)\n        \n#         brightness = tf.random.uniform([], 0, 1.0, dtype = tf.float32)\n#         if brightness >= .5:\n#             img = tf.image.random_brightness(img, max_delta = 0.1)\n        \n        return img\n    \n    def augment_with_labels(img, label):\n        return augment(img), label\n    \n    return augment_with_labels if with_labels else augment\n\n\ndef build_dataset(paths, labels = None, bsize = 32, cache = True,\n                  decode_fn = None, augment_fn = None,\n                  augment = True, repeat = True, shuffle = 1024, \n                  cache_dir = \"\"):\n    if cache_dir != \"\" and cache is True:\n        os.makedirs(cache_dir, exist_ok=True)\n    \n    if decode_fn is None:\n        decode_fn = build_decoder(labels is not None)\n    \n    if augment_fn is None:\n        augment_fn = build_augmenter(labels is not None)\n    \n    AUTO = tf.data.experimental.AUTOTUNE\n    slices = paths if labels is None else (paths, labels)\n    \n    dset = tf.data.Dataset.from_tensor_slices(slices)\n    dset = dset.map(decode_fn, num_parallel_calls = AUTO)\n    dset = dset.cache(cache_dir) if cache else dset\n    dset = dset.map(augment_fn, num_parallel_calls = AUTO) if augment else dset\n    dset = dset.repeat() if repeat else dset\n    dset = dset.shuffle(shuffle) if shuffle else dset\n    dset = dset.batch(bsize).prefetch(AUTO)\n    \n    return dset","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Train test split\n# (train_img, valid_img, \n#  train_labels, valid_labels) = train_test_split(train_images, labels, \n#                                                 train_size = 0.85, \n#                                                 random_state = 0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Tensorflow datasets\n# train_df = build_dataset(\n#     train_img, tf.cast(train_labels, tf.float32), bsize = BATCH_SIZE, \n#     cache = True)\n\n# valid_df = build_dataset(\n#     valid_img, tf.cast(valid_labels, tf.float32), bsize = BATCH_SIZE, \n#     repeat = False, shuffle = False, augment = False, \n#     cache = True)\n\ntest_df = build_dataset(\n    test_images, bsize = BATCH_SIZE, repeat = False, \n    shuffle = False, augment = False, cache = False)\n\ntest_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# def create_model():\n#     conv_base = Xception(include_top = False, weights = 'imagenet',\n#                          input_shape = (TARGET_SIZE, TARGET_SIZE, 3))\n#     model = conv_base.output\n#     model = layers.GlobalAveragePooling2D()(model)\n#     model = layers.Dropout(0.3)(model)\n#     model = layers.Dense(11, activation = \"sigmoid\")(model)\n#     model = models.Model(conv_base.input, model)\n\n#     model.compile(optimizer = Adam(lr = 0.001),\n#                   loss = tfa.losses.SigmoidFocalCrossEntropy(alpha = 0.5, gamma = 2),\n#                   metrics = [\"AUC\"])\n#     return model","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"model = models.load_model('../input/ranzcr-xception-tpu-baseline/Xception_750_TPU.h5')\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print('Our Xception CNN has %d layers' %len(model.layers))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"3\"></a>\n<h1 style='color:white; background:#0A0502; border:0'><center>Visualization of CNN intermediate activations</center></h1>\n\n[**Back to the table of contents**](#start)"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def activation_layer_vis(img, activation_layer = 0, layers = 10):\n    layer_outputs = [layer.output for layer in model.layers[:layers]]\n    activation_model = models.Model(inputs = model.input, outputs = layer_outputs)\n    activations = activation_model.predict(img)\n    \n    rows = int(activations[activation_layer].shape[3] / 3)\n    cols = int(activations[activation_layer].shape[3] / rows)\n    fig, axes = plt.subplots(rows, cols, figsize = (15, 15 * cols))\n    axes = axes.flatten()\n    \n    for i, ax in zip(range(activations[activation_layer].shape[3]), axes):\n        ax.matshow(activations[activation_layer][0, :, :, i], cmap = 'viridis')\n        ax.axis('off')\n    plt.tight_layout()\n    plt.show()\n\ndef all_activations_vis(img, layers = 10):\n    layer_outputs = [layer.output for layer in model.layers[:layers]]\n    activation_model = models.Model(inputs = model.input, outputs = layer_outputs)\n    activations = activation_model.predict(img)\n    \n    layer_names = []\n    for layer in model.layers[:layers]: \n        layer_names.append(layer.name) \n\n    images_per_row = 3\n    for layer_name, layer_activation in zip(layer_names, activations): \n        n_features = layer_activation.shape[-1] \n\n        size = layer_activation.shape[1] \n\n        n_cols = n_features // images_per_row \n        display_grid = np.zeros((size * n_cols, images_per_row * size)) \n\n        for col in range(n_cols): \n            for row in range(images_per_row): \n                channel_image = layer_activation[0, :, :, col * images_per_row + row] \n                channel_image -= channel_image.mean() \n                channel_image /= channel_image.std() \n                channel_image *= 64 \n                channel_image += 128 \n                channel_image = np.clip(channel_image, 0, 255).astype('uint8') \n                display_grid[col * size : (col + 1) * size, \n                             row * size : (row + 1) * size] = channel_image \n        scale = 1. / size \n        plt.figure(figsize=(scale * 5 * display_grid.shape[1], \n                            scale * 5 * display_grid.shape[0])) \n        plt.title(layer_name) \n        plt.grid(False)\n        plt.axis('off')\n        plt.imshow(display_grid, aspect = 'auto', cmap = 'viridis')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<h3 style='color:white; background:#0A0502; border:0'><center>Visualization of the first 3 layers</center></h3>"},{"metadata":{"trusted":true},"cell_type":"code","source":"img_tensor = build_dataset(\n    pd.Series(test_images[0]), bsize = 1,repeat = False, \n    shuffle = False, augment = False, cache = False)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"all_activations_vis(img_tensor, 3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"4\"></a>\n<h1 style='color:white; background:#0A0502; border:0'><center>Prediction</center></h1>\n\n[**Back to the table of contents**](#start)"},{"metadata":{"trusted":true},"cell_type":"code","source":"ss[label_cols] = model.predict(test_df, verbose = 1)\nss.to_csv('submission.csv', index = False)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"ss.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## The first part of this notebook: [RANZCR: Xception TPU](https://www.kaggle.com/maksymshkliarevskyi/ranzcr-xception-tpu)"},{"metadata":{},"cell_type":"markdown","source":"<h1 style='color:white; background:#0A0502; border:0'><center>WORK IN PROGRESS...</center></h1>"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}