{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"This kernel can be used to visualize the bounding boxes interactively using W&B and will walk you through the process of logging bounding boxes. \n\nThis kernel can be considered a simple EDA kernel with the focus on deriving insights by interactively playing with the bounding boxes for each images.\n\n![img](https://i.imgur.com/3vk2h3j.gif)\n\n### Credits\n\n* I am using [xhlulu's](https://www.kaggle.com/xhlulu) resized dataset. Check out his amazing [kernel](https://www.kaggle.com/xhlulu/siim-covid-19-convert-to-jpg-256px) that can be used to generate resized images with different resolution. The uploaded 256x256 Kaggle dataset is [here](https://www.kaggle.com/xhlulu/siim-covid19-resized-to-256px-jpg).","metadata":{}},{"cell_type":"markdown","source":"# Imports and Setups","metadata":{}},{"cell_type":"code","source":"!pip install -q --upgrade wandb\n\nimport wandb\nwandb.login()","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:44:59.295235Z","iopub.execute_input":"2021-05-19T23:44:59.295641Z","iopub.status.idle":"2021-05-19T23:45:25.063399Z","shell.execute_reply.started":"2021-05-19T23:44:59.295558Z","shell.execute_reply":"2021-05-19T23:45:25.062443Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport gc\nimport cv2\nimport ast\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\nimport matplotlib.pyplot as plt\n%matplotlib inline","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-05-19T23:45:32.874021Z","iopub.execute_input":"2021-05-19T23:45:32.874358Z","iopub.status.idle":"2021-05-19T23:45:33.003508Z","shell.execute_reply.started":"2021-05-19T23:45:32.874326Z","shell.execute_reply":"2021-05-19T23:45:33.002756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Hyperparams","metadata":{}},{"cell_type":"code","source":"TRAIN_PATH = '../input/siim-covid19-resized-to-256px-jpg/train/'\nIMG_SIZE = 256\nNUM_SAMPLES_TO_VIZ = 32","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:34.319670Z","iopub.execute_input":"2021-05-19T23:45:34.320004Z","iopub.status.idle":"2021-05-19T23:45:34.325884Z","shell.execute_reply.started":"2021-05-19T23:45:34.319977Z","shell.execute_reply":"2021-05-19T23:45:34.324007Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Dataset\n\nThis competition is unique because of how the data is presented and thus the problem statement. The DICOM formatted radiograms of chest scans are available in this directory structure `study/series/images`. What's `study` and `image`?\n \nThere are 6334 unique chest scans or `images` while 6054 unique `study` directories. This means that in some study directory there are more than one images. \n\nIn this competition, the task is to provide image level predictions - `none` vs `opacity` as well as study level predictions - `negative`, `typical`, `atypical`, `indeterminate`. Thus we will build two separate classifiers - one for study level prediction and another for image level prediction. Then when bounding boxes or localization?\n\nFor images with `opacity` label i.e, image-level label, we need to train an object detector for localization. \n\n* `train_study_level.csv`: Study level labels.\n\n* `train_image_level.csv`: Image level labels.\n\nIn this kernel we will try to understand the relationship better.","metadata":{}},{"cell_type":"code","source":"# Load image level csv file\ndf = pd.read_csv('../input/siim-covid19-detection/train_image_level.csv')\n# Load study level csv file\nlabel_df = pd.read_csv('../input/siim-covid19-detection/train_study_level.csv')\n\n# Modify values in the id column\ndf['id'] = df.apply(lambda row: row.id.split('_')[0], axis=1)\n# Add absolute path\ndf['path'] = df.apply(lambda row: TRAIN_PATH+row.id+'.jpg', axis=1)\n# Get image level labels\ndf['image_level'] = df.apply(lambda row: row.label.split(' ')[0], axis=1)\n\n# Modify values in the id column\nlabel_df['id'] = label_df.apply(lambda row: row.id.split('_')[0], axis=1)\n# Rename the column id with StudyInstanceUID\nlabel_df.columns = ['StudyInstanceUID', 'Negative for Pneumonia', 'Typical Appearance', 'Indeterminate Appearance', 'Atypical Appearance']\n\n# Merge both dataframes\ndf = df.merge(label_df, on='StudyInstanceUID',how=\"left\")\ndf.head(2)","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:35.930028Z","iopub.execute_input":"2021-05-19T23:45:35.930341Z","iopub.status.idle":"2021-05-19T23:45:36.349476Z","shell.execute_reply.started":"2021-05-19T23:45:35.930312Z","shell.execute_reply":"2021-05-19T23:45:36.348535Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'Number of unique image in training dataset: {len(df)}')\n\nbbox_nan_num = df['boxes'].isna().sum()\nprint(f'Number of images without any bbox annotation: {bbox_nan_num}')","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:37.281584Z","iopub.execute_input":"2021-05-19T23:45:37.281909Z","iopub.status.idle":"2021-05-19T23:45:37.290308Z","shell.execute_reply.started":"2021-05-19T23:45:37.281875Z","shell.execute_reply":"2021-05-19T23:45:37.288860Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Label encode study-level labels\nlabels = df[['Negative for Pneumonia','Typical Appearance','Indeterminate Appearance','Atypical Appearance']].values\nlabels = np.argmax(labels, axis=1)\n\ndf['study_level'] = labels\ndf.head(2)","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:38.031650Z","iopub.execute_input":"2021-05-19T23:45:38.031985Z","iopub.status.idle":"2021-05-19T23:45:38.051835Z","shell.execute_reply.started":"2021-05-19T23:45:38.031956Z","shell.execute_reply":"2021-05-19T23:45:38.051024Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class_label_to_id = {\n    'Negative for Pneumonia': 0,\n    'Typical Appearance': 1,\n    'Indeterminate Appearance': 2,\n    'Atypical Appearance': 3\n}\n\nclass_id_to_label = {val: key for key, val in class_label_to_id.items()}","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:39.058279Z","iopub.execute_input":"2021-05-19T23:45:39.058617Z","iopub.status.idle":"2021-05-19T23:45:39.063173Z","shell.execute_reply.started":"2021-05-19T23:45:39.058586Z","shell.execute_reply":"2021-05-19T23:45:39.062212Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load meta.csv file\nmeta_df = pd.read_csv('../input/siim-covid19-resized-to-256px-jpg/meta.csv')\ntrain_meta_df = meta_df.loc[meta_df.split == 'train']\ntrain_meta_df.columns = ['id', 'dim0', 'dim1', 'split']\ntrain_meta_df.head(2)","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:39.854140Z","iopub.execute_input":"2021-05-19T23:45:39.854463Z","iopub.status.idle":"2021-05-19T23:45:39.884995Z","shell.execute_reply.started":"2021-05-19T23:45:39.854433Z","shell.execute_reply":"2021-05-19T23:45:39.884021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = df.merge(train_meta_df, on='id',how=\"left\")\ndf.head(5)","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:41.034960Z","iopub.execute_input":"2021-05-19T23:45:41.035275Z","iopub.status.idle":"2021-05-19T23:45:41.062459Z","shell.execute_reply.started":"2021-05-19T23:45:41.035246Z","shell.execute_reply":"2021-05-19T23:45:41.061511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualize Bounding Boxes\n\nThis section will walk you through the steps required to use W&B to log bounding boxes. \n\nNotes: \n* Since some of the images don't have bounding box coordinates we will drop those rows. \n* The steps below can be used as it is with a training pipeline with littleeee modification. \n\nNote: Even though the `true_label` is `opacity` or `none` but I have logged the study-level labels for more insight. Every image with bounding box coordinates belong to `opacity` label.","metadata":{}},{"cell_type":"code","source":"# Since there are over 2000 rows without bounding box coordinates.\nopacity_df = df.dropna(subset = [\"boxes\"], inplace=False)\nopacity_df = opacity_df.reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:42.610045Z","iopub.execute_input":"2021-05-19T23:45:42.610403Z","iopub.status.idle":"2021-05-19T23:45:42.636611Z","shell.execute_reply.started":"2021-05-19T23:45:42.610370Z","shell.execute_reply":"2021-05-19T23:45:42.635694Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the raw bounding box \n# Ref: https://www.kaggle.com/yujiariyasu/plot-3positive-classes\ndef get_bbox(row):\n    bboxes = []\n    bbox = []\n    for i, l in enumerate(row.label.split(' ')):\n        if (i % 6 == 0) | (i % 6 == 1):\n            continue\n        bbox.append(float(l))\n        if i % 6 == 5:\n            bboxes.append(bbox)\n            bbox = []  \n            \n    return bboxes\n\n# Scale the bounding boxes.\ndef scale_bbox(row, bboxes):\n    # Get scaling factor\n    scale_x = IMG_SIZE/row.dim1\n    scale_y = IMG_SIZE/row.dim0\n    \n    scaled_bboxes = []\n    for bbox in bboxes:\n        x = int(np.round(bbox[0]*scale_x, 4))\n        y = int(np.round(bbox[1]*scale_y, 4))\n        x1 = int(np.round(bbox[2]*(scale_x), 4))\n        y1= int(np.round(bbox[3]*scale_y, 4))\n\n        scaled_bboxes.append([x, y, x1, y1]) # xmin, ymin, xmax, ymax\n        \n    return scaled_bboxes\n\n# To log a bounding box, you'll need to provide a dictionary with \n# the following keys and values to the boxes keyword argument of wandb.Image.\ndef wandb_bbox(image, bboxes, true_label, class_id_to_label):\n    all_boxes = []\n    for bbox in bboxes:\n        box_data = {\"position\": {\n                        \"minX\": bbox[0],\n                        \"minY\": bbox[1],\n                        \"maxX\": bbox[2],\n                        \"maxY\": bbox[3]\n                    },\n                     \"class_id\" : int(true_label),\n                     \"box_caption\": class_id_to_label[true_label],\n                     \"domain\" : \"pixel\"}\n        all_boxes.append(box_data)\n    \n\n    return wandb.Image(image, boxes={\n        \"ground_truth\": {\n            \"box_data\": all_boxes,\n          \"class_labels\": class_id_to_label\n        }\n    })","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:43.800478Z","iopub.execute_input":"2021-05-19T23:45:43.800881Z","iopub.status.idle":"2021-05-19T23:45:43.811373Z","shell.execute_reply.started":"2021-05-19T23:45:43.800846Z","shell.execute_reply":"2021-05-19T23:45:43.810452Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sampled_df = opacity_df.sample(NUM_SAMPLES_TO_VIZ).reset_index(drop=True)\n\nrun = wandb.init(project='kaggle-covid', \n                 config={'competition': 'siim-fisabio-rsna', '_wandb_kernel': 'ayut'},\n                 job_type='visualize_sample_bbox')\n\nwandb_bbox_list = []\nfor i in tqdm(range(NUM_SAMPLES_TO_VIZ)):\n    row = sampled_df.loc[i]\n    # Load image\n    image = cv2.imread(row.path)\n    # Get bboxes\n    bboxes = get_bbox(row)\n    # Scale bounding boxes\n    scale_bboxes = scale_bbox(row, bboxes)\n    # Get ground truth label\n    true_label = row.study_level\n    \n    wandb_bbox_list.append(wandb_bbox(image, \n                                      scale_bboxes, \n                                      true_label, \n                                      class_id_to_label))\n    \nwandb.log({\"radiograph\": wandb_bbox_list})\n\nrun.finish()\n\nrun","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:45:47.240982Z","iopub.execute_input":"2021-05-19T23:45:47.241316Z","iopub.status.idle":"2021-05-19T23:46:02.428707Z","shell.execute_reply.started":"2021-05-19T23:45:47.241284Z","shell.execute_reply":"2021-05-19T23:46:02.427700Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> 📌 Click on the [run page](https://wandb.ai/ayush-thakur/kaggle-covid/runs/3vxougc9?workspace=user-ayush-thakur) to interactively play with the bounding box coordinates. \n\n> 📌 Click on the ⚙️ icon to interact with the UI.","metadata":{}},{"cell_type":"markdown","source":"# Better Data Understanding using W&B Tables","metadata":{}},{"cell_type":"code","source":"# W&B image\ndef wandb_bbox(image, bboxes, true_label, class_id_to_label, class_set):\n    all_boxes = []\n    for bbox in bboxes:\n        box_data = {\"position\": {\n                        \"minX\": bbox[0],\n                        \"minY\": bbox[1],\n                        \"maxX\": bbox[2],\n                        \"maxY\": bbox[3]\n                    },\n                     \"class_id\" : int(true_label),\n                     \"box_caption\": class_id_to_label[true_label],\n                     \"domain\" : \"pixel\"}\n        all_boxes.append(box_data)\n    \n\n    return wandb.Image(image, boxes={\n        \"ground_truth\": {\n            \"box_data\": all_boxes,\n          \"class_labels\": class_id_to_label\n        }\n    }, classes=class_set)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-05-19T23:49:07.675555Z","iopub.execute_input":"2021-05-19T23:49:07.675893Z","iopub.status.idle":"2021-05-19T23:49:07.686040Z","shell.execute_reply.started":"2021-05-19T23:49:07.675864Z","shell.execute_reply":"2021-05-19T23:49:07.685190Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"run = wandb.init(project='kaggle-covid', \n                 config={'competition': 'siim-fisabio-rsna', '_wandb_kernel': 'ayut'},\n                 job_type='visualize-everything')\n\nclass_set = wandb.Classes([{'id': id, 'name': name} for id, name in class_id_to_label.items()])\n\n\ntable = wandb.Table(columns=['ImageID', 'StudyInstanceUID', 'Radiogram', 'image-label', 'study-label',\n                             'Negative', 'Typical', 'Indeterminate', 'Atypical',\n                             'ori_dim0', 'ori_dim1', 'split'])\n\n# create an artifact for all the raw data\nviz_at = wandb.Artifact('eda', type=\"basic-eda\")\n\nfor i in tqdm(range(len(df))):\n    row = df.loc[i]\n    # Load image\n    image = cv2.imread(row.path)\n    # Get bboxes\n    bboxes = get_bbox(row)\n    # Scale bounding boxes\n    scale_bboxes = scale_bbox(row, bboxes)\n    # Get ground truth label\n    true_label = row.study_level\n    # Get image with bounding boxes\n    wandb_img = wandb_bbox(image, \n                           scale_bboxes, \n                           true_label, \n                           class_id_to_label,\n                           class_set)\n    \n    # Add info in the table as new row\n    table.add_data(row.id, row.StudyInstanceUID, wandb_img, row.image_level, row.study_level,\n                   row['Negative for Pneumonia'], row['Typical Appearance'], row['Indeterminate Appearance'], row['Atypical Appearance'],\n                   row.dim0, row.dim1, row.split)\n    \n    del row, wandb_img\n    _ = gc.collect()\n    \n# wandb.log({'radiogram_eda': table})\nviz_at.add(table, \"Radiogram EDA\")\nrun.log_artifact(viz_at)\nrun.finish()","metadata":{"execution":{"iopub.status.busy":"2021-05-19T23:49:11.206765Z","iopub.execute_input":"2021-05-19T23:49:11.207082Z","iopub.status.idle":"2021-05-20T00:04:04.814723Z","shell.execute_reply.started":"2021-05-19T23:49:11.207053Z","shell.execute_reply":"2021-05-20T00:04:04.813741Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n## [Check out the Table $\\rightarrow$](https://wandb.ai/ayush-thakur/kaggle-covid/artifacts/basic-eda/eda/c17893f765f142a4acbf/files/Radiogram%20EDA.table.json)\n\n![img](https://i.imgur.com/zdRSgtn.gif)","metadata":{}},{"cell_type":"markdown","source":"# Work In Progress\n\nUpcoming:\n* ~Visualize the entire training dataset using W&B Tables.~\n* ~Visualize Metadata~.\n* Get insight from the tables.","metadata":{}}]}