{"cells":[{"metadata":{},"cell_type":"markdown","source":"## What are we predicting?\n\nIn this competition, you’ll detect the presence and position of catheters and lines on chest x-rays. Use machine learning to train and test your model on 40,000 images to categorize a tube that is poorly placed.\n\n## Evaluation criteria?\n\nSubmissions are evaluated on area under the ROC curve between the predicted probability and the observed target.\nTo calculate the final score, AUC is calculated for each of the 11 labels, then averaged. The score is then the average of the individual AUCs of each predicted column.\n\n## Train vs Test?\n\nA code-only competition so there is a hidden test set (approximately 4x larger, with ~14k images) as well.\n\ntrain.csv contains image IDs, binary labels, and patient IDs.\n\nTFRecords are available for both train and test. (They are also available for the hidden test set.)\n\ntrain_annotations.csv includes segmentation annotations for training samples that have them as solely additional information.\n\n## Similar Dataset & Competitions?\n\n[RSNA Pneumonia Detection Challenge](\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/notebooks\")\n\n[SIIM-ACR Pneumothorax Segmentation](\"https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation\")\n\n[XRay Lung Segmentation](\"https://www.kaggle.com/c/xray-lung-segmentation/data\")\n\n[RSNA Pneumonia Detection Challenge](\"https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/notebooks\")\n\nHave more?"},{"metadata":{"trusted":true},"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow.keras.applications.densenet import DenseNet121\nfrom tensorflow.keras.applications.inception_v3 import InceptionV3\nfrom tensorflow.keras.applications.xception import Xception\nfrom tensorflow.keras.applications.nasnet import NASNetMobile, NASNetLarge\nfrom tensorflow.keras.applications.inception_resnet_v2 import InceptionResNetV2\n\n# base_model = InceptionResNetV2(include_top=False, weights='imagenet', input_shape=(256, 256, 3))\n# x = base_model.output\n# x = tf.keras.layers.GlobalAveragePooling2D()(x)\n# output = tf.keras.layers.Dense(len(labels), activation=\"sigmoid\")(x)\n# model = tf.keras.Model(base_model.input, output)\n# model.compile(optimizer=tf.keras.optimizers.RMSprop(), loss='binary_crossentropy', metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\n\n# import efficientnet.tfkeras as efn\nimport numpy as np\nimport pandas as pd\nfrom kaggle_datasets import KaggleDatasets\nfrom sklearn.model_selection import train_test_split\nimport tensorflow as tf","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Helper functions and configurations"},{"metadata":{},"cell_type":"markdown","source":"The following functions are defined below (unhide to see):\n```python\nauto_select_accelerator()\n\nbuild_decoder(with_labels=True, target_size=(256, 256), ext='jpg')\n\nbuild_augmenter(with_labels=True)\n\nbuild_dataset(paths, labels=None, bsize=32, cache=True,\n              decode_fn=None, augment_fn=None,\n              augment=True, repeat=True, shuffle=1024, \n              cache_dir=\"\")\n```"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def auto_select_accelerator():\n    try:\n        tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n        tf.config.experimental_connect_to_cluster(tpu)\n        tf.tpu.experimental.initialize_tpu_system(tpu)\n        strategy = tf.distribute.experimental.TPUStrategy(tpu)\n        print(\"Running on TPU:\", tpu.master())\n    except ValueError:\n        strategy = tf.distribute.get_strategy()\n    print(f\"Running on {strategy.num_replicas_in_sync} replicas\")\n    \n    return strategy\n\n\ndef build_decoder(with_labels=True, target_size=(256, 256), ext='jpg'):\n    def decode(path):\n        file_bytes = tf.io.read_file(path)\n        if ext == 'png':\n            img = tf.image.decode_png(file_bytes, channels=3)\n        elif ext in ['jpg', 'jpeg']:\n            img = tf.image.decode_jpeg(file_bytes, channels=3)\n        else:\n            raise ValueError(\"Image extension not supported\")\n\n        img = tf.cast(img, tf.float32) / 255.0\n        img = tf.image.resize(img, target_size)\n\n        return img\n    \n    def decode_with_labels(path, label):\n        return decode(path), label\n    \n    return decode_with_labels if with_labels else decode\n\n\ndef build_augmenter(with_labels=True):\n    def augment(img):\n        img = tf.image.random_flip_left_right(img)\n        img = tf.image.random_flip_up_down(img)\n        return img\n    \n    def augment_with_labels(img, label):\n        return augment(img), label\n    \n    return augment_with_labels if with_labels else augment\n\n\ndef build_dataset(paths, labels=None, bsize=32, cache=True,\n                  decode_fn=None, augment_fn=None,\n                  augment=True, repeat=True, shuffle=1024, \n                  cache_dir=\"\"):\n    if cache_dir != \"\" and cache is True:\n        os.makedirs(cache_dir, exist_ok=True)\n    \n    if decode_fn is None:\n        decode_fn = build_decoder(labels is not None)\n    \n    if augment_fn is None:\n        augment_fn = build_augmenter(labels is not None)\n    \n    AUTO = tf.data.experimental.AUTOTUNE\n    slices = paths if labels is None else (paths, labels)\n    \n    dset = tf.data.Dataset.from_tensor_slices(slices)\n    dset = dset.map(decode_fn, num_parallel_calls=AUTO)\n    dset = dset.cache(cache_dir) if cache else dset\n    dset = dset.map(augment_fn, num_parallel_calls=AUTO) if augment else dset\n    dset = dset.repeat() if repeat else dset\n    dset = dset.shuffle(shuffle) if shuffle else dset\n    dset = dset.batch(bsize).prefetch(AUTO)\n    \n    return dset","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"COMPETITION_NAME = \"ranzcr-clip-catheter-line-classification\"\nstrategy = auto_select_accelerator()\nBATCH_SIZE = strategy.num_replicas_in_sync * 16\nGCS_DS_PATH = KaggleDatasets().get_gcs_path(COMPETITION_NAME)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Preparing dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"load_dir = f\"/kaggle/input/{COMPETITION_NAME}/\"\ndf = pd.read_csv(load_dir + 'train.csv')\n\n# paths = load_dir + \"train/\" + df['StudyInstanceUID'] + '.jpg'\npaths = GCS_DS_PATH + \"/train/\" + df['StudyInstanceUID'] + '.jpg'\n\nsub_df = pd.read_csv(load_dir + 'sample_submission.csv')\n\n# test_paths = load_dir + \"test/\" + sub_df['StudyInstanceUID'] + '.jpg'\ntest_paths = GCS_DS_PATH + \"/test/\" + sub_df['StudyInstanceUID'] + '.jpg'\n\n# Get the multi-labels\nlabel_cols = sub_df.columns[1:]\nlabels = df[label_cols].values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Train test split\n(train_paths, valid_paths, train_labels, valid_labels) = train_test_split(paths, labels, test_size=0.2, random_state=42)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Build the tensorflow datasets\nIMSIZE = (224, 240, 260, 300, 380, 456, 528, 600)\n\ndecoder = build_decoder(with_labels=True, target_size=(IMSIZE[6], IMSIZE[6]))\ntest_decoder = build_decoder(with_labels=False, target_size=(IMSIZE[6], IMSIZE[6]))\n\ntrain_dataset = build_dataset(\n    train_paths, train_labels, bsize=BATCH_SIZE, decode_fn=decoder\n)\n\nvalid_dataset = build_dataset(\n    valid_paths, valid_labels, bsize=BATCH_SIZE, decode_fn=decoder,\n    repeat=False, shuffle=False, augment=False\n)\n\ntest_dataset = build_dataset(\n    test_paths, cache=False, bsize=BATCH_SIZE, decode_fn=test_decoder,\n    repeat=False, shuffle=False, augment=False\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Modeling"},{"metadata":{"trusted":true},"cell_type":"code","source":"n_labels = labels.shape[1]\n\nwith strategy.scope():\n    model = tf.keras.Sequential([\n        InceptionResNetV2(\n            input_shape=(IMSIZE[6], IMSIZE[6], 3),\n            weights='imagenet',\n            include_top=False),\n        tf.keras.layers.GlobalAveragePooling2D(),\n        tf.keras.layers.Dense(n_labels, activation='sigmoid')\n    ])\n    model.compile(\n        optimizer='adam',\n        loss='binary_crossentropy',\n        metrics=[tf.keras.metrics.AUC(multi_label=True)])\n    model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"steps_per_epoch = train_paths.shape[0] // BATCH_SIZE\ncheckpoint = tf.keras.callbacks.ModelCheckpoint(\n    'model.h5', save_best_only=True, monitor='val_auc', mode='max')\nlr_reducer = tf.keras.callbacks.ReduceLROnPlateau(\n    monitor=\"val_auc\", patience=3, min_lr=1e-6, mode='max')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit(\n    train_dataset, \n    epochs=30,\n    verbose=2,\n    callbacks=[checkpoint, lr_reducer],\n    steps_per_epoch=steps_per_epoch,\n    validation_data=valid_dataset)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"hist_df = pd.DataFrame(history.history)\nhist_df.to_csv('history.csv')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}