{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Cropped Crown of Thorns Dataset Builder\n\nThis notebook builds a dataset of just cropped COTS images (contents of each bounding box) that may be useful for training/data augmentation purposes. It also shows how to work with the data at a basic level. For example showing images and drawing bounding boxes. Code is written in a more readable format for beginners, efficiency is not taken into serious consideration.\n\n**Plan**\n 1. Extract all COTS images from bounding box regions and be able save to new img files.\n 2. Create new dataset of all these COTS images - could be used for augmentation purposes or other training\n \n## 😅 If you use the cropped pics upvote the notebook and/or dataset. Lot of people just copying code/forking on Kaggle these days. 👀\n\n* In the next notebook I'll build an augmented dataset for easy use.\n* Check back here for the link:\n\n## FINAL DATASET: [COTS v NotCOTS Cropped Crown of Thorns Dataset](https://www.kaggle.com/alexteboul/binary-cropped-crown-of-thorns-dataset)","metadata":{}},{"cell_type":"markdown","source":"![COTS BANNER](https://storage.googleapis.com/kaggle-datasets-images/1912529/3140514/88cd069275dfbb414e8b94783a781450/data-original.png?t=2022-02-05-01-08-03)","metadata":{}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport cv2\nimport time\nimport ast","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:31.586077Z","iopub.execute_input":"2022-02-04T23:38:31.586463Z","iopub.status.idle":"2022-02-04T23:38:31.930586Z","shell.execute_reply.started":"2022-02-04T23:38:31.586360Z","shell.execute_reply":"2022-02-04T23:38:31.929947Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#create a directory to save cropped images to. Cropped images are going to be just what is inside the bounding boxes.\nos.mkdir('cots_crops')\nos.mkdir('notcots_crops')\nos.listdir()","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:31.952438Z","iopub.execute_input":"2022-02-04T23:38:31.953002Z","iopub.status.idle":"2022-02-04T23:38:31.960984Z","shell.execute_reply.started":"2022-02-04T23:38:31.952959Z","shell.execute_reply":"2022-02-04T23:38:31.960442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#make it so the whole df.head() column width is shown.\npd.set_option('display.max_colwidth', None)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:36.564939Z","iopub.execute_input":"2022-02-04T23:38:36.565366Z","iopub.status.idle":"2022-02-04T23:38:36.569417Z","shell.execute_reply.started":"2022-02-04T23:38:36.565335Z","shell.execute_reply":"2022-02-04T23:38:36.568726Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 1. Extract COTS images from bounding boxes","metadata":{}},{"cell_type":"markdown","source":"## 1.1 Get the data","metadata":{}},{"cell_type":"code","source":"#get data\npath = '/kaggle/input/tensorflow-great-barrier-reef/'\ntrain = pd.read_csv(path + 'train.csv')\ntest = pd.read_csv(path+ 'test.csv')","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:37.520384Z","iopub.execute_input":"2022-02-04T23:38:37.520654Z","iopub.status.idle":"2022-02-04T23:38:37.584828Z","shell.execute_reply.started":"2022-02-04T23:38:37.520625Z","shell.execute_reply":"2022-02-04T23:38:37.584000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:37.692905Z","iopub.execute_input":"2022-02-04T23:38:37.693156Z","iopub.status.idle":"2022-02-04T23:38:37.711224Z","shell.execute_reply.started":"2022-02-04T23:38:37.693128Z","shell.execute_reply":"2022-02-04T23:38:37.710453Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:37.892749Z","iopub.execute_input":"2022-02-04T23:38:37.893235Z","iopub.status.idle":"2022-02-04T23:38:37.904370Z","shell.execute_reply.started":"2022-02-04T23:38:37.893188Z","shell.execute_reply":"2022-02-04T23:38:37.903639Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.dtypes","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:38.072858Z","iopub.execute_input":"2022-02-04T23:38:38.073155Z","iopub.status.idle":"2022-02-04T23:38:38.081318Z","shell.execute_reply.started":"2022-02-04T23:38:38.073123Z","shell.execute_reply":"2022-02-04T23:38:38.080507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.2 add a column for the img_path","metadata":{}},{"cell_type":"code","source":"#the file path is a combination of the video_id and video_frame columns\ntrain['img_path'] = '/kaggle/input/tensorflow-great-barrier-reef/train_images/video_'+train['video_id'].astype(str)+'/'+train['video_frame'].astype(str)+'.jpg'\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:39.416646Z","iopub.execute_input":"2022-02-04T23:38:39.417114Z","iopub.status.idle":"2022-02-04T23:38:39.501138Z","shell.execute_reply.started":"2022-02-04T23:38:39.417066Z","shell.execute_reply":"2022-02-04T23:38:39.500604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#what do annotations look like\ntrain['annotations'].iloc[35]","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:40.064452Z","iopub.execute_input":"2022-02-04T23:38:40.064853Z","iopub.status.idle":"2022-02-04T23:38:40.071119Z","shell.execute_reply.started":"2022-02-04T23:38:40.064823Z","shell.execute_reply":"2022-02-04T23:38:40.070616Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.3 separate out the images that have annotations from those that do not","metadata":{}},{"cell_type":"code","source":"#grab just the rows that have annotations (aka they have cots in them)\ntrain_onlycots = train[train['annotations'] != '[]']\ntrain_nocots = train[train['annotations'] == '[]']","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:41.164448Z","iopub.execute_input":"2022-02-04T23:38:41.164906Z","iopub.status.idle":"2022-02-04T23:38:41.183473Z","shell.execute_reply.started":"2022-02-04T23:38:41.164869Z","shell.execute_reply":"2022-02-04T23:38:41.182622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#all the rows with annotations\ntrain_onlycots.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:42.149152Z","iopub.execute_input":"2022-02-04T23:38:42.149402Z","iopub.status.idle":"2022-02-04T23:38:42.162762Z","shell.execute_reply.started":"2022-02-04T23:38:42.149374Z","shell.execute_reply":"2022-02-04T23:38:42.161920Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#all the rows without annotations\ntrain_nocots.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:43.541548Z","iopub.execute_input":"2022-02-04T23:38:43.541828Z","iopub.status.idle":"2022-02-04T23:38:43.553447Z","shell.execute_reply.started":"2022-02-04T23:38:43.541798Z","shell.execute_reply":"2022-02-04T23:38:43.552705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#interesting so we have \nprint(f'original:{train.shape}\\ntrain_onlycots:{train_onlycots.shape}\\ntrain_nocots:{train_nocots.shape}')\nprint(f'percentage of images with annotations/cots in them: {round(train_onlycots.shape[0]/train.shape[0]*100)}%')","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:43.705091Z","iopub.execute_input":"2022-02-04T23:38:43.705500Z","iopub.status.idle":"2022-02-04T23:38:43.710541Z","shell.execute_reply.started":"2022-02-04T23:38:43.705468Z","shell.execute_reply":"2022-02-04T23:38:43.709878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.4 try viewing some images","metadata":{}},{"cell_type":"code","source":"#example pic without COTS\nex_nocots = train_nocots['img_path'].iloc[0]\nprint(ex_nocots)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:45.252998Z","iopub.execute_input":"2022-02-04T23:38:45.253253Z","iopub.status.idle":"2022-02-04T23:38:45.258609Z","shell.execute_reply.started":"2022-02-04T23:38:45.253227Z","shell.execute_reply":"2022-02-04T23:38:45.258032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#read the image in as a numpy array with cv2\nstart_time = time.time()\nex_nocots_img = cv2.imread(ex_nocots)\nprint(\"--- %s seconds ---\" % (time.time() - start_time))\nprint(type(ex_nocots_img))","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:45.892763Z","iopub.execute_input":"2022-02-04T23:38:45.893307Z","iopub.status.idle":"2022-02-04T23:38:45.949389Z","shell.execute_reply.started":"2022-02-04T23:38:45.893272Z","shell.execute_reply":"2022-02-04T23:38:45.948603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#notice how the color looks off - this is because it displays the color channels as BGR not RGB\nplt.figure(figsize=(18, 18))\nplt.imshow(ex_nocots_img)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:46.608682Z","iopub.execute_input":"2022-02-04T23:38:46.609337Z","iopub.status.idle":"2022-02-04T23:38:47.440458Z","shell.execute_reply.started":"2022-02-04T23:38:46.609301Z","shell.execute_reply":"2022-02-04T23:38:47.439667Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#adding [:,:,::-1] will flip it around to display the RGB colors - and note that it doesn't slow you down to do so\nstart_time = time.time()\nex_nocots_img = cv2.imread(ex_nocots)[:,:,::-1]\nprint(\"--- %s seconds ---\" % (time.time() - start_time))\nprint(type(ex_nocots_img))","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:47.441946Z","iopub.execute_input":"2022-02-04T23:38:47.442333Z","iopub.status.idle":"2022-02-04T23:38:47.472256Z","shell.execute_reply.started":"2022-02-04T23:38:47.442289Z","shell.execute_reply":"2022-02-04T23:38:47.471399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(18, 18))\nplt.imshow(ex_nocots_img)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:48.148985Z","iopub.execute_input":"2022-02-04T23:38:48.149265Z","iopub.status.idle":"2022-02-04T23:38:49.004140Z","shell.execute_reply.started":"2022-02-04T23:38:48.149232Z","shell.execute_reply":"2022-02-04T23:38:49.002574Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#example pic with COTS\nex_yescots = train_onlycots['img_path'].iloc[28]\nprint(ex_yescots)\nprint(type(ex_yescots))","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:50.129165Z","iopub.execute_input":"2022-02-04T23:38:50.129871Z","iopub.status.idle":"2022-02-04T23:38:50.136459Z","shell.execute_reply.started":"2022-02-04T23:38:50.129833Z","shell.execute_reply":"2022-02-04T23:38:50.135540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ex_yescots_img = cv2.imread(ex_yescots)[:,:,::-1]\nplt.figure(figsize=(18, 18))\nplt.imshow(ex_yescots_img)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:50.316594Z","iopub.execute_input":"2022-02-04T23:38:50.317076Z","iopub.status.idle":"2022-02-04T23:38:51.075425Z","shell.execute_reply.started":"2022-02-04T23:38:50.317040Z","shell.execute_reply":"2022-02-04T23:38:51.074252Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.5 let's add a bounding box to the annotated image because I can't see the COT starfish 😅","metadata":{}},{"cell_type":"code","source":"#check out the annotation - this is the bounding box area\n#note that there is only 1 bounding box atm\n#also not that it appears as a list\nex_annotation = train_onlycots['annotations'].iloc[0]\nprint(ex_annotation, type(ex_annotation))\n#note that we have to turn this into a list to work with it","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:55.065950Z","iopub.execute_input":"2022-02-04T23:38:55.066250Z","iopub.status.idle":"2022-02-04T23:38:55.071895Z","shell.execute_reply.started":"2022-02-04T23:38:55.066216Z","shell.execute_reply":"2022-02-04T23:38:55.071110Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#lowkey kindof dangerous to do this this way, but this is Kaggle and going for readability\n#turns the stringified list to a normal one\nprint(ast.literal_eval(ex_annotation))\nprint(type(ast.literal_eval(ex_annotation)))","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:55.500977Z","iopub.execute_input":"2022-02-04T23:38:55.501796Z","iopub.status.idle":"2022-02-04T23:38:55.506386Z","shell.execute_reply.started":"2022-02-04T23:38:55.501753Z","shell.execute_reply":"2022-02-04T23:38:55.505834Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def bbox_drawer(img_path, annotation):\n    '''Accepts an image path as a string and an annotation as a stringified list of dictionaries. \n    Outputs the image in RGB and has bounding boxes drawn on the image'''\n    #box parameters\n    #window_name = 'COTS'\n    color = (0, 0, 255) #as (B,G,R) this means this color is Red\n    thickness = 2\n    \n    #get img from url\n    img = cv2.imread(img_path)#[:,:,::-1]\n    \n    #fix stringified list\n    annotation_fixed = ast.literal_eval(annotation)\n\n    #loop through the list of annotations and draw the box on the image for each\n    #start_point is the top left coordinate as a tuple\n    #end_point is the bottom right coordinate as a tuple\n    for ann in annotation_fixed:\n        start_point,end_point = (ann['x'], ann['y']) , (ann['x'] + ann['width'], ann['y'] + ann['height'])\n        #print(start_point,end_point)\n        img = cv2.rectangle(img, start_point, end_point, color, thickness)\n    img = img[:,:,::-1]\n    plt.figure(figsize=(18, 18))\n    plt.imshow(img)\n    return","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:56.080992Z","iopub.execute_input":"2022-02-04T23:38:56.081456Z","iopub.status.idle":"2022-02-04T23:38:56.090011Z","shell.execute_reply.started":"2022-02-04T23:38:56.081406Z","shell.execute_reply":"2022-02-04T23:38:56.089451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_index_selector = 28 #change this number to see different images\nbbox_drawer(train_onlycots['img_path'].iloc[image_index_selector],train_onlycots['annotations'].iloc[image_index_selector])","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:56.796783Z","iopub.execute_input":"2022-02-04T23:38:56.797243Z","iopub.status.idle":"2022-02-04T23:38:57.593759Z","shell.execute_reply.started":"2022-02-04T23:38:56.797179Z","shell.execute_reply":"2022-02-04T23:38:57.593093Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_index_selector = 888 #change this number to see different images\nbbox_drawer(train_onlycots['img_path'].iloc[image_index_selector],train_onlycots['annotations'].iloc[image_index_selector])","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:38:57.594909Z","iopub.execute_input":"2022-02-04T23:38:57.595510Z","iopub.status.idle":"2022-02-04T23:38:58.435040Z","shell.execute_reply.started":"2022-02-04T23:38:57.595480Z","shell.execute_reply":"2022-02-04T23:38:58.434260Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* So we are now able to visualize the images and bounding boxes\n* It also works for any number of COTS in the images\n* Next I'm going to save the bounding box regions as their own .jpg files. \n* This way we can end up with a folder that just has pictures of COTS.","metadata":{}},{"cell_type":"markdown","source":"## 1.6 lets extract just the COT starfish from the annotated image and save it","metadata":{}},{"cell_type":"code","source":"def img_bb_cropper(img_path, annotation):\n    '''Accepts an image path as a string and an annotation as a stringified list of dictionaries.\n    output is saving the file to the /'''\n    #turn img_path to just the 'video_id-video_frame'\n    #example '/kaggle/input/tensorflow-great-barrier-reef/train_images/video_0/16.jpg' --> 'video_0-16.jpg'\n    img_name = img_path[57:-4].replace('/','-')\n    \n    #get img from url\n    img = cv2.imread(img_path)  #[:,:,::-1]\n    \n    #fix stringified list annotation\n    annotation_fixed = ast.literal_eval(annotation)\n\n    #loop through the list of annotations and draw the box on the image for each\n    #in each loop grab the x, y, width, and height\n    #use numpy array slicing to grab the bounding box area and save it in img\n    #put it in RGB too\n    #the ann_counters is just a counter that gets thrown on the image name because each pic can have multiple annotations\n    ann_counter = 0\n    for ann in annotation_fixed:\n        x,y,w,h = ann['x'], ann['y'], ann['width'], ann['height']\n        cropped_img = img[y:y+h,x:x+w]\n        cv2.imwrite(f'cots_crops/cotscrop-{img_name}-{ann_counter}.jpg',cropped_img)\n        cropped_img = cropped_img[:,:,::-1]\n        ann_counter+=1\n    plt.figure(figsize=(6, 6))\n    plt.imshow(cropped_img)\n    return #cropped_img","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:39:06.905270Z","iopub.execute_input":"2022-02-04T23:39:06.905955Z","iopub.status.idle":"2022-02-04T23:39:06.914079Z","shell.execute_reply.started":"2022-02-04T23:39:06.905913Z","shell.execute_reply":"2022-02-04T23:39:06.913313Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_index_selector = 888 #change this number to see different images\nimg_bb_cropper(train_onlycots['img_path'].iloc[image_index_selector],train_onlycots['annotations'].iloc[image_index_selector])","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:39:07.912975Z","iopub.execute_input":"2022-02-04T23:39:07.913468Z","iopub.status.idle":"2022-02-04T23:39:08.159400Z","shell.execute_reply.started":"2022-02-04T23:39:07.913432Z","shell.execute_reply":"2022-02-04T23:39:08.158578Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Pretty low res image gets output. Makes sense, made up a tiny portion of the total image and this is just getting those pixels out. \n* But it works at least!","metadata":{}},{"cell_type":"code","source":"#check that a cropped pic saved correctly to the crops folder\ndiditwork = cv2.imread('cots_crops/cotscrop-video_0-4703-0.jpg')[:,:,::-1]\nplt.imshow(diditwork)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:39:10.496443Z","iopub.execute_input":"2022-02-04T23:39:10.497231Z","iopub.status.idle":"2022-02-04T23:39:10.693668Z","shell.execute_reply.started":"2022-02-04T23:39:10.497173Z","shell.execute_reply":"2022-02-04T23:39:10.693031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Create new dataset of all these COTS only images\nFor now lets just make a dataset of only cots images from the bounding boxes. Next notebook will be an augmented dataset in the format that will make it more conducive to train an actual object detection model.\n\nWe currently have a function called 'img_bb_cropper' that accepts an 'img_path' and a 'annotation' and can save those crops to the '/crops/' folder. Now lets apply that to the whole dataframe of only cots images 'train_onlycots'.","metadata":{}},{"cell_type":"code","source":"#lets modify the function a bit so it doesn't show the pics each time. will speed it up a bit. Also no need to flip the BGR to RGB in the loop anymore.\ndef img_bb_cropper_saver(img_path, annotation):\n    '''Accepts an image path as a string and an annotation as a stringified list of dictionaries.\n    output is saving the file to the /'''\n    #get image name from the path\n    img_name = img_path[57:-4].replace('/','-')\n    \n    #get img from url\n    img = cv2.imread(img_path)  #[:,:,::-1]\n    \n    #fix stringified list annotation\n    annotation_fixed = ast.literal_eval(annotation)\n\n    #save the cots image from each annotated bounding box to the crops folder\n    ann_counter = 0\n    for ann in annotation_fixed:\n        x,y,w,h = ann['x'], ann['y'], ann['width'], ann['height']\n        cropped_img = img[y:y+h,x:x+w]\n        cv2.imwrite(f'cots_crops/cotscrop-{img_name}-{ann_counter}.jpg',cropped_img)\n        ann_counter+=1\n    return ","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:39:14.221025Z","iopub.execute_input":"2022-02-04T23:39:14.221767Z","iopub.status.idle":"2022-02-04T23:39:14.229069Z","shell.execute_reply.started":"2022-02-04T23:39:14.221713Z","shell.execute_reply":"2022-02-04T23:39:14.228233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#this is ugly but is fast enough for this use case I suppose. Roughly 120-160 seconds to get all the cots images from the bounding boxes saved into the new crops folder\nstart_time = time.time()\nrun_it = train_onlycots.apply(lambda row: img_bb_cropper_saver(row['img_path'], row['annotations']), axis=1)\nprint(\"--- %s seconds ---\" % (time.time() - start_time))","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:39:17.800595Z","iopub.execute_input":"2022-02-04T23:39:17.801444Z","iopub.status.idle":"2022-02-04T23:41:45.576038Z","shell.execute_reply.started":"2022-02-04T23:39:17.801388Z","shell.execute_reply":"2022-02-04T23:41:45.575173Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#check to see the first time\nprint(os.listdir('cots_crops')[:5])\nprint(len(os.listdir('cots_crops')))","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:41:54.285160Z","iopub.execute_input":"2022-02-04T23:41:54.286310Z","iopub.status.idle":"2022-02-04T23:41:54.308406Z","shell.execute_reply.started":"2022-02-04T23:41:54.286262Z","shell.execute_reply":"2022-02-04T23:41:54.307823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Awesome so we have currently 11,898 images of cots in the /crops folder","metadata":{}},{"cell_type":"markdown","source":"# 3. Let's also grab some not cots images in case anyone wants to build a binary cots-not cots classifier\n* to do this we can try grabbing the bounding box regions from the images that actually don't have cots","metadata":{}},{"cell_type":"code","source":"print(train_onlycots.shape)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:42:28.949393Z","iopub.execute_input":"2022-02-04T23:42:28.949691Z","iopub.status.idle":"2022-02-04T23:42:28.954294Z","shell.execute_reply.started":"2022-02-04T23:42:28.949660Z","shell.execute_reply":"2022-02-04T23:42:28.953430Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_nocots_2 = train_nocots[:4919]\ntrain_nocots_2.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:42:36.529405Z","iopub.execute_input":"2022-02-04T23:42:36.530040Z","iopub.status.idle":"2022-02-04T23:42:36.544654Z","shell.execute_reply.started":"2022-02-04T23:42:36.529997Z","shell.execute_reply":"2022-02-04T23:42:36.543822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#we want equal number of rows here\ntrain_onlycots_2 = train_onlycots\nprint(train_nocots_2.shape)\nprint(train_onlycots_2.shape)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:42:50.813704Z","iopub.execute_input":"2022-02-04T23:42:50.813983Z","iopub.status.idle":"2022-02-04T23:42:50.820155Z","shell.execute_reply.started":"2022-02-04T23:42:50.813956Z","shell.execute_reply":"2022-02-04T23:42:50.819610Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#dumb but reset index values\ntrain_onlycots_2 = train_onlycots_2.reset_index(drop=True)\ntrain_nocots_2 = train_nocots_2.reset_index(drop=True)\n","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:43:02.317702Z","iopub.execute_input":"2022-02-04T23:43:02.318284Z","iopub.status.idle":"2022-02-04T23:43:02.324512Z","shell.execute_reply.started":"2022-02-04T23:43:02.318241Z","shell.execute_reply":"2022-02-04T23:43:02.323939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#now lets put in the annotations from the images in train_only cots in train_nocots_2\ntrain_nocots_2['annotations2'] = train_onlycots_2['annotations'].values\ntrain_nocots_2.head(20)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:43:22.129896Z","iopub.execute_input":"2022-02-04T23:43:22.130653Z","iopub.status.idle":"2022-02-04T23:43:22.147792Z","shell.execute_reply.started":"2022-02-04T23:43:22.130613Z","shell.execute_reply":"2022-02-04T23:43:22.147266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_nocots_2.tail()","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:44:58.072213Z","iopub.execute_input":"2022-02-04T23:44:58.072496Z","iopub.status.idle":"2022-02-04T23:44:58.087807Z","shell.execute_reply.started":"2022-02-04T23:44:58.072467Z","shell.execute_reply":"2022-02-04T23:44:58.086940Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#lets modify the function so that it grabs an equal sized to the bounding box right above the actual bounding box. Just flip the y+h to y-h\ndef img_notcots_cropper_saver(img_path, annotation):\n    '''Accepts an image path as a string and an annotation as a stringified list of dictionaries.\n    output is saving the file to the /'''\n    #get image name from the path\n    img_name = img_path[57:-4].replace('/','-')\n    \n    #get img from url\n    img = cv2.imread(img_path)  #[:,:,::-1]\n    \n    #fix stringified list annotation\n    annotation_fixed = ast.literal_eval(annotation)\n\n    #save the cots image from each annotated bounding box to the crops folder\n    ann_counter = 0\n    for ann in annotation_fixed:\n        x,y,w,h = ann['x'], ann['y'], ann['width'], ann['height']\n        cropped_img = img[y:y+h,x:x+w]\n        cv2.imwrite(f'notcots_crops/notcotscrop-{img_name}-{ann_counter}.jpg',cropped_img)\n        ann_counter+=1\n    return ","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:46:20.157734Z","iopub.execute_input":"2022-02-04T23:46:20.158566Z","iopub.status.idle":"2022-02-04T23:46:20.165942Z","shell.execute_reply.started":"2022-02-04T23:46:20.158515Z","shell.execute_reply":"2022-02-04T23:46:20.165018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#this is ugly but is fast enough for this use case I suppose. Roughly 120-160 seconds to get all the cots images from the bounding boxes saved into the new crops folder\nstart_time = time.time()\nrun_it_again = train_nocots_2.apply(lambda row: img_notcots_cropper_saver(row['img_path'], row['annotations2']), axis=1)\nprint(\"--- %s seconds ---\" % (time.time() - start_time))","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:46:22.477843Z","iopub.execute_input":"2022-02-04T23:46:22.478323Z","iopub.status.idle":"2022-02-04T23:48:40.540068Z","shell.execute_reply.started":"2022-02-04T23:46:22.478273Z","shell.execute_reply":"2022-02-04T23:48:40.539125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#check to see the first time\nprint(os.listdir('notcots_crops')[:5])\nprint(len(os.listdir('notcots_crops')))","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:48:46.914198Z","iopub.execute_input":"2022-02-04T23:48:46.914470Z","iopub.status.idle":"2022-02-04T23:48:46.937155Z","shell.execute_reply.started":"2022-02-04T23:48:46.914437Z","shell.execute_reply":"2022-02-04T23:48:46.936303Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#check that a cropped pic saved correctly to the crops folder\ndiditwork2 = cv2.imread('notcots_crops/notcotscrop-video_0-11553-5.jpg')[:,:,::-1]\nplt.imshow(diditwork2)","metadata":{"execution":{"iopub.status.busy":"2022-02-04T23:49:40.477532Z","iopub.execute_input":"2022-02-04T23:49:40.477841Z","iopub.status.idle":"2022-02-04T23:49:40.684576Z","shell.execute_reply.started":"2022-02-04T23:49:40.477806Z","shell.execute_reply":"2022-02-04T23:49:40.683760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Cool so now we have an equal number of cots and not cots images saved, 11,898 of each. We did this by using the bounding boxes from the annotations, but applied to images without any bounding boxes (images that didn't have COTS).\n* There are now a couple ways we can try using this for data augmentation purposes or probably directly if you want to stack a binary cots/not cots classifier onto object detector","metadata":{}},{"cell_type":"markdown","source":"### Save both sets to zip files","metadata":{}},{"cell_type":"code","source":"!ls","metadata":{"execution":{"iopub.status.busy":"2022-02-05T00:04:40.625345Z","iopub.execute_input":"2022-02-05T00:04:40.625683Z","iopub.status.idle":"2022-02-05T00:04:41.398662Z","shell.execute_reply.started":"2022-02-05T00:04:40.625643Z","shell.execute_reply":"2022-02-05T00:04:41.397651Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%capture\n!zip -r cots.zip cots_crops","metadata":{"execution":{"iopub.status.busy":"2022-02-05T00:04:36.765639Z","iopub.execute_input":"2022-02-05T00:04:36.765952Z","iopub.status.idle":"2022-02-05T00:04:39.546722Z","shell.execute_reply.started":"2022-02-05T00:04:36.765917Z","shell.execute_reply":"2022-02-05T00:04:39.545498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%capture\n!zip -r notcots.zip notcots_crops","metadata":{"execution":{"iopub.status.busy":"2022-02-05T00:04:30.886297Z","iopub.execute_input":"2022-02-05T00:04:30.886611Z","iopub.status.idle":"2022-02-05T00:04:33.485248Z","shell.execute_reply.started":"2022-02-05T00:04:30.886572Z","shell.execute_reply":"2022-02-05T00:04:33.484294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls","metadata":{},"execution_count":null,"outputs":[]}]}