{"cells":[{"metadata":{},"cell_type":"markdown","source":"### **Problem Statement:**\n\nSkin cancer is the most prevalent type of cancer. Melanoma, specifically, is responsible for 75% of skin cancer deaths, despite being the least common skin cancer, but if caught early, most melanomas can be cured with minor surgery. In this competition, the task is to use images of skin lesions within the same patient and determine which are likely to represent a melanoma.\n\n### **Dataset:**\nData is provided in the following formats:\n1. DICOM (Digital Imaging and Communications in Medicine) contains both image and metadata\n2. JPEG (Images in original size)\n3. TFRecord (Images resized to 1024x1024)\n4. Metadata in CSV\n\n### **Contents:**\n<b><a href=\"#one\">1. Reading data</a><br></b>\n&emsp;&emsp;<a href=\"#one.one\">1.1. DICOM format</a><br>\n&emsp;&emsp;<a href=\"#one.two\">1.2. JPEG format</a><br>\n&emsp;&emsp;<a href=\"#one.three\">1.3. TFRecord format</a><br>\n<b><a href=\"#two\">2. Quality comparison</a><br></b>\n&emsp;&emsp;<a href=\"#two.one\">2.1. DICOM vs JPEG</a><br>\n&emsp;&emsp;<a href=\"#two.two\">2.2. Source of TFRecord</a><br>\n<b><a href=\"#three\">3. DICOM to PNG/JPEG</a><br></b>\n<b><a href=\"#four\">4. Conclusions</a><br></b>\n<b><a href=\"#five\">5. Next steps</a><br></b>\n\n### **Motivation:**\n1. Learning how to handle of dicom files.\n2. Compare the quality of the images between the formats provided. \n3. Create dataset with the images of highest available quality resized to 512x512, 256x256 etc.,\n\nThis notebook is inspired by this [discussion](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/155579#875036) and this [post](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/155415).\n\n### **Credits:**\n1. Thanks [@cdeotte](https://www.kaggle.com/cdeotte) for this [notebook](https://www.kaggle.com/cdeotte/how-to-create-tfrecords/notebook) on working with TFRecords.\n2. Thanks [@Abhishek](https://www.kaggle.com/abhishek) for this [notebook](https://www.kaggle.com/abhishek/convert-to-png-on-steroids-with-actual-data/notebook) on converting images from DICOM to PNG in parallel.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import pydicom\nfrom pydicom.pixel_data_handlers.util import convert_color_space\n\nfrom PIL import Image\nfrom skimage import io\nimport cv2\n\nimport tensorflow as tf\n\nimport albumentations \n\nimport numpy as np\nimport pandas as pd\n\nfrom joblib import Parallel, delayed\nfrom tqdm.notebook import tqdm\n\nimport matplotlib.pyplot as plt\nimport matplotlib\nmatplotlib.rcParams['font.size'] = 15\n\nfrom pathlib import Path\n\nINPUT_PATH = Path(\"/kaggle/input/siim-isic-melanoma-classification\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**<h1 id=\"one\" >1. Reading data </h1>**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_metadata = pd.read_csv(INPUT_PATH/\"train.csv\");print(f\"Train shape: {train_metadata.shape}\")\ntrain_metadata.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Select a random image for analysis\nimage_name = np.random.choice(train_metadata.image_name)\nimage_name","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**<h2 id=\"one.one\" >1.1. DICOM format </h2>**","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(f\"Reading {image_name} DICOM file\")\nds = pydicom.dcmread(INPUT_PATH/f\"train/{image_name}.dcm\")\nprint(f\"Contents of {image_name} DICOM file\")\nds","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* DICOM file processing is discussed [here](https://www.kaggle.com/avirdee/fastai2-dicom-starter#Fastai2-DICOM-starter) and more details on the format are available [here].(http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.7.6.3.html#sect_C.7.6.3.1.4)\n* Key part in order to process the image for this competition is **(0028, 0004) Photometric Interpretation**.\n* I have verified that all the DICOM files of train and test have images in **YBR_FULL_422** color scale. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dicom_arr = convert_color_space(ds.pixel_array, \"YBR_FULL_422\", \"RGB\")\ndicom_img = Image.fromarray(dicom_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**<h2 id=\"one.two\" >1.2. JPEG format </h2>**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"jpeg_img = Image.open(f\"{INPUT_PATH}/jpeg/train/{image_name}.jpg\")\njpeg_arr = np.asarray(jpeg_img)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**<h2 id=\"one.three\" >1.3. TFRecord format </h2>**","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def mapping_func(serialized_example):\n    TFREC_FORMAT = {\n                    \"image_name\": tf.io.FixedLenFeature([], tf.string),\n                    \"image\": tf.io.FixedLenFeature([], tf.string)\n                    }\n    example = tf.io.parse_single_example(serialized_example, features=TFREC_FORMAT)\n    return example\n\ndef filter_tfrecords(dataset, image_name):\n    record_dataset = dataset.filter(lambda example: tf.equal(example[\"image_name\"], image_name))\n    example = next(iter(record_dataset))\n    arr = tf.image.decode_jpeg(example['image'], channels=3).numpy()\n    return arr","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dataset = tf.data.TFRecordDataset(tf.io.gfile.glob('/kaggle/input/siim-isic-melanoma-classification/tfrecords/train*.tfrec'))\ndataset = dataset.map(mapping_func)\n\ntfrecord_arr = filter_tfrecords(dataset, image_name)\ntfrecord_img = Image.fromarray(tfrecord_arr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f\"DICOM array shape: {dicom_arr.shape}\")\nprint(f\"JPEG array shape: {jpeg_arr.shape}\")\nprint(f\"TFRecord array shape: {tfrecord_arr.shape}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see how they look!","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(20, 8))\nplt.subplot(131), io.imshow(dicom_arr)\nplt.title('DICOM Format') \n\nplt.subplot(132), io.imshow(jpeg_arr)\nplt.title('JPEG Format') \n\nplt.subplot(133), io.imshow(tfrecord_arr)\nplt.title('TFRecord Format') \n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### **Observations:**\n\n1. Both the image from DICOM file and JPEG are of same size.\n2. Image from TFRecord seems to be cropped and resized to uniform dimensions of 1024x1024 as mentioned in the [data description](https://www.kaggle.com/c/siim-isic-melanoma-classification/data).","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**<h1 id=\"two\" >2. Quality comparison </h1>**","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"The differences between the images are hard to be noticed by a human eye. I am exploring the following approaches to compare two images:\n1. Exact match\n2. Mean squared error\n\nFeel free to suggest other approaches to compare two images with respect to the quality. I'd be happy to learn and add it to this analysis","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"is_same_image = lambda a, b: np.allclose(a, b)\nmse = lambda a, b: np.square(np.subtract(a, b)).mean()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**<h2 id=\"two.one\" >2.1. DICOM vs JPEG </h2>**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"is_same_image(dicom_arr, jpeg_arr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mse(dicom_arr, jpeg_arr) #Mean Squared Error","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Although DICOM image and JPEG image are of same size, they are not one and the same.\n* MSE only shows if two images are different and more analysis is needed in order to understand their differences in quality.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Write DICOM image to JPEG at 75% quality\ndicom_img.save(f\"dicom2jpeg_75q_{image_name}.jpg\", \"JPEG\", quality=75)\n\n# Read the image\ndicom2jpeg_75q_img = Image.open(f\"dicom2jpeg_75q_{image_name}.jpg\")\ndicom2jpeg_75q_arr = np.asarray(dicom2jpeg_75q_img)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"is_same_image(jpeg_arr, dicom2jpeg_75q_arr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mse(jpeg_arr, dicom2jpeg_75q_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## That's interesting! Let's check this for randomly selected 100 images.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def compare(image_name):\n    ds = pydicom.dcmread(INPUT_PATH/f\"train/{image_name}.dcm\")\n    dicom_arr = convert_color_space(ds.pixel_array, \"YBR_FULL_422\", \"RGB\")\n    dicom_img = Image.fromarray(dicom_arr)\n    \n    jpeg_img = Image.open(f\"{INPUT_PATH}/jpeg/train/{image_name}.jpg\")\n    jpeg_arr = np.asarray(jpeg_img)\n    \n    # Write DICOM image to JPEG at 75% quality\n    dicom_img.save(f\"dicom2jpeg_75q_{image_name}.jpg\", \"JPEG\", quality=75)\n    # Read the image\n    dicom2jpeg_75q_img = Image.open(f\"dicom2jpeg_75q_{image_name}.jpg\")\n    dicom2jpeg_75q_arr = np.asarray(dicom2jpeg_75q_img)\n    Path(f\"dicom2jpeg_75q_{image_name}.jpg\").unlink()\n    \n    comparison = {\n        \"image_name\": image_name,\n        \"dicom_vs_jpeg_is_same\" : is_same_image(dicom_arr, jpeg_arr),\n        \"dicom_vs_jpeg_mse\": mse(dicom_arr, jpeg_arr),\n        \"jpeg_vs_dicom2jpeg_75q_is_same\": is_same_image(jpeg_arr, dicom2jpeg_75q_arr),\n        \"jpeg_vs_dicom2jpeg_75q_mse\": mse(jpeg_arr, dicom2jpeg_75q_arr)\n    }\n    return comparison","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"image_names = np.random.choice(train_metadata.image_name, size=100, replace=False)\ncomparisons = Parallel(n_jobs=8, backend='threading')(delayed(\n    compare)(image_name) for image_name in tqdm(image_names, total=len(image_names)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"comparisons_df = pd.DataFrame(comparisons)\ncomparisons_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(np.sum(~comparisons_df[\"jpeg_vs_dicom2jpeg_75q_is_same\"]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(np.sum(comparisons_df[\"jpeg_vs_dicom2jpeg_75q_mse\"]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Wow! JPEG images seem be saved at 75% quality compared to DICOM images","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**<h2 id=\"two.two\" >2.2. Source of TFRecord </h2>**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"resize_and_crop = albumentations.Compose([\n    albumentations.SmallestMaxSize(max_size=1024, interpolation=cv2.INTER_LINEAR),\n    albumentations.CenterCrop(1024, 1024)\n])\nresized_cropped_dicom_arr = resize_and_crop(image=dicom_arr)[\"image\"]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(20, 45))\n\nplt.subplot(121), io.imshow(tfrecord_arr)\nplt.title('TFRecord Format') \n\nplt.subplot(122), io.imshow(resized_cropped_dicom_arr)\nplt.title('Resized and Cropped DICOM Format') \n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"is_same_image(tfrecord_arr, resized_cropped_dicom_arr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mse(tfrecord_arr, resized_cropped_dicom_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The images in TFRecord are still different after resizing holding the aspect ratio same as original image. Following are the possible reasons:\n* The images in TFRecord are processed a bit more than just resize and crop.\n* There might be slight loss of information while encoding the image in TFRecord. \n\nNote: I am not an expert in Computer Vision and I might very well be wrong here. I look forward to comments/suggestions to experiment and learn.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**<h1 id=\"three\" >3. DICOM to PNG/JPEG </h1>**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Write DICOM image to JPEG at 100% quality\ndicom_img.save(f\"dicom2jpeg_100q_{image_name}.jpg\", \"JPEG\", quality=100)\n\n# Read the image\ndicom2jpeg_100q_img = Image.open(f\"dicom2jpeg_100q_{image_name}.jpg\")\ndicom2jpeg_100q_arr = np.asarray(dicom2jpeg_100q_img)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"is_same_image(dicom_arr, dicom2jpeg_100q_arr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mse(dicom_arr, dicom2jpeg_100q_arr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Write DICOM image to PNG at 100% quality\ndicom_img.save(f\"dicom2png_100q_{image_name}.png\", \"PNG\", quality=100)\n\n# Read the image\ndicom2png_100q_img = Image.open(f\"dicom2png_100q_{image_name}.png\")\ndicom2png_100q_arr = np.asarray(dicom2png_100q_img)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"is_same_image(dicom_arr, dicom2png_100q_arr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mse(dicom_arr, dicom2png_100q_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**<h1 id=\"four\" >4. Conclusions </h1>**\n* JPEG images are at 75% quality compared to images in DICOM files\n* There is a minimal loss of information while converting DICOM image to JPEG with 100 quality.\n* There is zero loss of information while converting DICOM image to PNG with 100 quality.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**<h1 id=\"five\" >5. Next steps </h1>**\n* Explore how important is maintaining aspect ratio while resizing the images to different dimensions (512x512, 1024x1024 etc.,).\n* Use images from DICOM files to resize and convert in to PNG format for modelling.\n* Build models at different qualities and different sizes to understand the performance.","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}