{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Shrink Wasted Space 87% and Increase Data Pts 4x\n\n## Motivation\n\nThis notebook is based on [previous work](https://www.kaggle.com/dannellyz/tissue-detect-scaling-bounding-boxes-4xfaster) I have done to isolate key areas in medical imaging. From experience in other competitions, I certainly agree with [@Loulou](https://www.kaggle.com/louise2001) in the [post](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/203356) on previous work. Finding the most import areas of the images and shrinking the learning area is a must. As the first seciton shows there is a ton of area in each image that is currently un-labeled when bounding just the annotated sections. Additionally, when investigating I found large inconsistencies on the number of data points per annotation as well as the spacing between observations. I worked to help add data points in order to normalize the labeling.\n\n## Review on Unused Space in Train Images\n\nTo begin lets look at the spaces in each of the images that is not used in traning. Lets find some features of each image to help show this:"},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"#Install and update as needed\n!pip install --upgrade seaborn","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"#import needed libs\nimport pandas as pd\nimport numpy as np\nimport cv2\nimport ast\nimport itertools\nfrom PIL import Image\nimport matplotlib.pyplot as plt\nfrom math import hypot\nimport seaborn as sns \n\n#Set up Globals and get Annotaions\nBASE_DIR = \"/kaggle/input/ranzcr-clip-catheter-line-classification/\"\nTRAIN_IMG_DIR = BASE_DIR + \"train/\"\nTESTING = False\ntrain_anno = pd.read_csv(BASE_DIR + \"train_annotations.csv\")\n\n#Sample for Testing\nif TESTING:\n    train_anno = train_anno.sample(int(len(train_anno) * .1))\n    \ntrain_anno.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Features of Each Image ()\n* data - nested list of all points that make up the points along the catheter insertion\n* data_pt_cnt - total number of data points in a image\n* data_pt_len - the total length of the placement annotation\n* pt_lev_avg - the average length between points in an annotation\n* img_sz - (height, width) touple of image size\n* img_area - simple product of img_sz touple\n* min_rect - (x,y,w,h) produced from cv2.boundingRect(data)\n* min_area - area from the min rectangle bounding the data points\n* extra_area - difference from img_area - min_arae\n* extra_pct - difference in percentage terms"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def get_img_loc(UID):\n    return f\"{TRAIN_IMG_DIR}{UID}.jpg\"\n\n\ndef get_img_features(row):\n    return_cols = []\n    \n    # Transform string data into nested list and get number\n    data = ast.literal_eval(row.data)\n    data_pt_cnt = len(data)\n    \n    # Length of the annotation and average per points\n    data_pt_len = sum(hypot(x1 - x2, y1 - y2) for (x1, y1), (x2, y2) in zip(data, data[1:]))\n    pt_lev_avg = data_pt_len / data_pt_cnt\n    \n    #image Size with PIL...much faster than cv2\n    fname = get_img_loc(row[\"StudyInstanceUID\"])\n    im = Image.open(fname)\n    img_sz = im.size\n    img_area = np.prod(img_sz)\n    \n    # min_rect = x,y,w,h - bounded around all data points\n    data_array = np.float32(data)\n    min_rect = cv2.boundingRect(data_array)\n    \n    #Get areas and extra\n    min_area = np.prod(min_rect[2:])\n    extra_area = img_area - min_area\n    pct_extra = extra_area / img_area\n    return pd.Series([data, data_pt_cnt, data_pt_len, pt_lev_avg, img_sz, img_area, min_rect, min_area, extra_area, pct_extra])\n\nimg_features = [\"data\", \"data_pt_cnt\", \"data_pt_len\", \"pt_lev_avg\", \"img_sz\", \"img_area\", \"min_rect\", \"min_area\", \"extra_area\", \"pct_extra\"]\ntrain_anno[img_features] = pd.DataFrame(train_anno.apply(get_img_features, axis=1))\n\n#Add number of placement types\ntrain_df = pd.read_csv(BASE_DIR + \"train.csv\")\ntrain_df[\"placement_cnt\"] = train_df.sum(axis=1)\ntrain_anno = train_anno.merge(train_df[[\"StudyInstanceUID\",\"placement_cnt\"]])\n\ntrain_anno.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"avg_extra = train_anno[\"pct_extra\"].mean()\nprint(f\"The average percent of unused space in each image, extra_pct, is: {avg_extra}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Mean Extra Space 87%!**\n\nGiven this there must be ways to cut down images to hone in on the parts that contain the data."},{"metadata":{},"cell_type":"markdown","source":"### Correlation of Extra Space to data points or placement types\n\nI started exploring this extra space by seeing if the images that had more types of placements or more annotated points used less space. This would be the idea that as you increase the amount of data there should be less space:"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"data_extra = train_anno[\"data_pt_cnt\"].corr(train_anno[\"pct_extra\"])\nplacement_extra = train_anno[\"placement_cnt\"].corr(train_anno[\"pct_extra\"])\nplacement_data = train_anno[\"placement_cnt\"].corr(train_anno[\"data_pt_cnt\"])\nprint(f\"1: {data_extra:.2} = data_pt_cnt coor pct_extra | strong negative | more data points less extra space\")\nprint(f\"2: {placement_extra:.2} = placement_cnt to pct_extra | weak positive | more placements somewhat more extra space\")\nprint(f\"3: {placement_data:.2} = placement_cnt to data_pt_cnt | weak negative | more placements somewhat less data points\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"1. The first finding makes sense that as you add data points there is less extra space\n2. The second makes less sense? One would assume that as you add types of placements there would be less extra space? Potentially the x-rays are tighter in cases where there are more types of placements.\n3. This also does not make sense, but does square with the above. As there are more types of placements there are less data points? Have added a [discussion](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/206294) to explore this. \n\nIn that discussion it was the thought of the competiton host that:\n> @jarrelscy: My guess would be that because labelers label the whole image at one time, when there are many lines and tubes they may be trying to finish it faster and hence putting less points.\n\nI think this makes sense so I wanted to explore more that relationship and also look into iterpolating data points on those with less labels"},{"metadata":{},"cell_type":"markdown","source":"# Discovery 1: Data Point Inconsistency Per Annotattion\n\n\nThe chat below is a histogram of the count of data points per x-ray, `data_pt_cnt`. As you can see it fluctuates quite a bit and is not consistent across the number of placement annotations."},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.displot(train_anno, x=\"data_pt_cnt\", hue=\"placement_cnt\", multiple=\"stack\", palette=\"Set2\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Furthermore when looking at the average length between annotation points, `pt_lev_avg`, one can see a lower skew towards those with fewer placements. "},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.displot(train_anno, x=\"pt_lev_avg\", hue=\"placement_cnt\", multiple=\"stack\", palette=\"Set2\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Looking at these two findings as box plots you can see the downward skew of data points in relation to number of placement annotations as well as the upward skew in the average length of an annotation over the same x vector."},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.boxplot(x=\"placement_cnt\", y=\"pt_lev_avg\", data=train_anno)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.boxplot(x=\"placement_cnt\", y=\"data_pt_cnt\", data=train_anno)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Finding 1: The more annotations the less provided data\n\nGiven this, the x-rays with more types of placements receive less attention when labeling form the experts. From a human based perspective this would make sense. On more crowded, full slides the expert would need to move quicker and have less time to detail each individual placement. For those with 1 or only two types of placement they could add much more detail in the same period of time."},{"metadata":{},"cell_type":"markdown","source":"# Solution 1: a) Interpolate or b) Drop Outliers.\n\n## a) Interpolating\n\nInterpolating seeks to take a vector and add additional data points based on its shape. As seen below this works well for convex lines but concave gives it problems. "},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np\nimport scipy as sp\nimport scipy.interpolate\nimport matplotlib.pyplot as plt\n\n#https://stackoverflow.com/questions/4072844/add-more-sample-points-to-data\n\n# Generate some random data\ny = (np.random.random(10) - 0.5).cumsum()\nx = np.arange(y.size)\n\n# Interpolate the data using a cubic spline to \"new_length\" samples\nnew_length = 50\nnew_x = np.linspace(x.min(), x.max(), new_length)\nnew_y = sp.interpolate.interp1d(x, y, kind='cubic')(new_x)\n\n# Plot the results\nplt.figure()\nplt.subplot(2,1,1)\nplt.plot(x, y, 'bo-')\nplt.title('Using 1D Cubic Spline Interpolation')\n\nplt.subplot(2,1,2)\nplt.plot(new_x, new_y, 'ro-')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since many of the annotations can be quite odd we need to interpolate in-between each point rather than over the line as a whole.\n\n<span style=\"color:blue;\"> Blue == Base Annotation </span>\n\n<span style=\"color:crimson;\"> Red  == Interpolation </span>"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"#sample_img = train_anno.sample(1)\n#sample_loc = sample_img[\"StudyInstanceUID\"].values[0]\n#sample_data = sample_img[\"data\"].values[0]\nsample_data = train_anno[train_anno[\"StudyInstanceUID\"] == '1.2.826.0.1.3680043.8.498.13137786603668786022908361036269592497'][\"data\"].values[0]\nx_test = np.array([x[0] for x in sample_data])\ny_test = np.array([x[1] for x in sample_data])\n\nfor point in range(1, len(x_test) + 1):\n    # Interpolate the data using a cubic spline to \"new_length\" samples\n    new_length = 50\n    new_x_test = np.linspace(x_test.min(), x_test.max(), new_length)\n    new_y_test = sp.interpolate.interp1d(x_test, y_test, kind='linear')(new_x_test)\n\n# Plot the results\nplt.figure()\nplt.subplot(2,1,1)\nplt.plot(x_test, y_test, 'bo-')\nplt.title('Using 1D Cubic Spline Interpolation')\n\nplt.subplot(2,1,2)\nplt.plot(new_x_test, new_y_test, 'ro-')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The fix is to interpolate iterativly between each pair of lines as shown below."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"x_test = np.array([x[0] for x in sample_data])\ny_test = np.array([x[1] for x in sample_data])\n\nall_xs = []\nall_ys = []\n\nfor i in range(1, len(x_test)):\n    xs = np.array([x_test[i-1],  x_test[i]])\n    ys = np.array([y_test[i-1],  y_test[i]])\n    # Interpolate the data using a cubic spline to \"new_length\" samples\n    new_length = 6\n    new_xs = np.linspace(xs.min(), xs.max(), new_length)\n    new_ys = sp.interpolate.interp1d(xs, ys, kind='linear')(new_xs)\n    \n    #print(xs, ys)\n    #print(new_xs, new_ys)\n    #print(\"-----\")\n    all_xs.extend(new_xs)\n    all_ys.extend(new_ys)\n#print(all_xs)\n#print(all_ys)\n# Plot the results\nplt.figure()\nplt.subplot(2,1,1)\nplt.plot(x_test, y_test, 'bo-')\nplt.title('Using 1D Cubic Spline Interpolation')\n\nplt.subplot(2,1,2)\nplt.plot(all_xs, all_ys, 'ro-')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"sample1 = '1.2.826.0.1.3680043.8.498.13137786603668786022908361036269592497'\nsample2 = '1.2.826.0.1.3680043.8.498.80893755100097324352087763503154307486'\nsample3 = '1.2.826.0.1.3680043.8.498.87570237940815582849204532832440641266'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## More to come..."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}