{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **The \"Help Protect the Great Barrier Reef\" Code Explained by Dino** #","metadata":{}},{"cell_type":"markdown","source":"## Intro ##\nThe Great Barrier Reef is the world's largest coral reef species in the world. However, the Great Barrier Reef is at a critical tipping point that will determine its long-term survival. Coral bleaching as a result of global warming is a key reason for the reef's decline. Not only that, the infamous starfishes overpopulate too. Like sea urchins devouring kelp in California (the place where I live), the starfishes devoured every coral, day by day. Because of that, scientists are worried about the corals being eaten up by starfishes, so the Great Barrier Reef Foundation and Google teamed up together to create a one full competition on Kaggle. Today, we are explaining the solution about the \"Help Protect the Great Barrier Reef\" contest.","metadata":{}},{"cell_type":"markdown","source":"## Imports and Modules ##\nBefore classifying the images whether there is a seastar or not, we have to import the neccessary libraries. Importing them can make the Python code classify more and more of seastars well, for, at least.","metadata":{}},{"cell_type":"code","source":"import os\nimport cv2\nimport ast\nimport numpy as np\nimport pandas as pd\nimport matplotlib\nimport matplotlib.pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.428761Z","iopub.execute_input":"2021-12-09T10:57:32.4297Z","iopub.status.idle":"2021-12-09T10:57:32.435809Z","shell.execute_reply.started":"2021-12-09T10:57:32.429651Z","shell.execute_reply":"2021-12-09T10:57:32.435086Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Variables setup ##\nWell, well, we had made our good progress with that imports, let's now define variables! First, define the \"path\" variable, which defines the path of the files toward the \"tensorflow-great-barrier-reef' folder, at least. From there, you can find all 6 files inside listed in the listdir thing.","metadata":{}},{"cell_type":"code","source":"path = '/kaggle/input/tensorflow-great-barrier-reef/'\nos.listdir(path)","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.437311Z","iopub.execute_input":"2021-12-09T10:57:32.438134Z","iopub.status.idle":"2021-12-09T10:57:32.45681Z","shell.execute_reply.started":"2021-12-09T10:57:32.438095Z","shell.execute_reply":"2021-12-09T10:57:32.455789Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And... We have to set up train_data, test_data, and even samp_subm! How? We use pandas to read the csvs with the path variable concatenated with the strings: train.csv, test.csv, and example_sample_submission.csv.","metadata":{}},{"cell_type":"code","source":"train_data = pd.read_csv(path+'train.csv')\ntest_data = pd.read_csv(path+'test.csv')\nsamp_subm = pd.read_csv(path+'example_sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.458215Z","iopub.execute_input":"2021-12-09T10:57:32.458871Z","iopub.status.idle":"2021-12-09T10:57:32.506293Z","shell.execute_reply.started":"2021-12-09T10:57:32.458836Z","shell.execute_reply":"2021-12-09T10:57:32.50537Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data.head()","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.508258Z","iopub.execute_input":"2021-12-09T10:57:32.508508Z","iopub.status.idle":"2021-12-09T10:57:32.520149Z","shell.execute_reply.started":"2021-12-09T10:57:32.508478Z","shell.execute_reply":"2021-12-09T10:57:32.519289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Box Setup Classification of a Crown-of-Thorns Starfish ##\nNow we defined all the variables, then we should know that we have to define the video_frame first, then concatenate the video_frame variable to the '.jpg' string. Then, we can call train_data down towards the train_data key inside, video_frame, and set equal to the video frame variable.","metadata":{}},{"cell_type":"code","source":"video_frame = 621\nfile_name = str(video_frame)+'.jpg'\ntrain_data[train_data['video_frame']==video_frame]","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.521676Z","iopub.execute_input":"2021-12-09T10:57:32.522173Z","iopub.status.idle":"2021-12-09T10:57:32.539686Z","shell.execute_reply.started":"2021-12-09T10:57:32.522136Z","shell.execute_reply":"2021-12-09T10:57:32.538948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The train_data dictionary returned two results, one of which contained annotations about where the sea stars prowl. Since we know that video_id 1 has annotations, we can assign one of the image_folder variables, which each made the OpenCV library read an image file from 3 different paths, from the video_0 path to the video_2 path.","metadata":{}},{"cell_type":"code","source":"image_folder_0 = cv2.imread(path+'train_images/video_0/'+file_name)\nimage_folder_1 = cv2.imread(path+'train_images/video_1/'+file_name)\nimage_folder_2 = cv2.imread(path+'train_images/video_2/'+file_name)","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.540605Z","iopub.execute_input":"2021-12-09T10:57:32.540854Z","iopub.status.idle":"2021-12-09T10:57:32.611662Z","shell.execute_reply.started":"2021-12-09T10:57:32.540824Z","shell.execute_reply":"2021-12-09T10:57:32.608685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we have to call out the train_data dictionary variable again, but this time, we combine the two key terms: the one with the video_frame key equals to the video frame variable and the one with the video_id key equals to the number which contained the annotations. If the number in the video_id has the annotation, then, the row is displayed, that contains the coordinates.","metadata":{}},{"cell_type":"code","source":"train_data[(train_data['video_frame']==video_frame)&(train_data['video_id']==1)]['annotations']","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.614106Z","iopub.execute_input":"2021-12-09T10:57:32.616776Z","iopub.status.idle":"2021-12-09T10:57:32.62575Z","shell.execute_reply.started":"2021-12-09T10:57:32.616721Z","shell.execute_reply":"2021-12-09T10:57:32.624755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since row 7329 contains the annotations, we want to set the variable row to the row where it contained the annotations.","metadata":{}},{"cell_type":"code","source":"row = 7329","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.626948Z","iopub.execute_input":"2021-12-09T10:57:32.627347Z","iopub.status.idle":"2021-12-09T10:57:32.635635Z","shell.execute_reply.started":"2021-12-09T10:57:32.627309Z","shell.execute_reply":"2021-12-09T10:57:32.634838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, we can assign the boxes variable to the ast module over the literal_eval function, over the .loc dataframe over the locations of the annotations by the row variable.","metadata":{}},{"cell_type":"code","source":"boxes = ast.literal_eval(train_data.loc[row, 'annotations'])","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.636771Z","iopub.execute_input":"2021-12-09T10:57:32.637081Z","iopub.status.idle":"2021-12-09T10:57:32.648681Z","shell.execute_reply.started":"2021-12-09T10:57:32.637036Z","shell.execute_reply":"2021-12-09T10:57:32.647929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Now let's plot! ##","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(1, 1, figsize=(20, 8))\nax.imshow(image_folder_1)\nfor box in boxes:\n    p = matplotlib.patches.Rectangle((box['x'], box['y']), box['width'], box['height'],\n                                     ec='r', fc='none', lw=2.)\n    ax.add_patch(p)\nax.set_xticklabels([])\nax.set_yticklabels([])\nplt.savefig(\"oh_no_one_reef_eater.png\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:32.651669Z","iopub.execute_input":"2021-12-09T10:57:32.65214Z","iopub.status.idle":"2021-12-09T10:57:33.518412Z","shell.execute_reply.started":"2021-12-09T10:57:32.652102Z","shell.execute_reply":"2021-12-09T10:57:33.51707Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Alas, we see at least one crown-of-thorns starfish looking for coral to devour. But if there are more crown-of-thorns starfish lurking around the coral reef, how does the machine detect more of those? Well, let's find out ways to tackle the reef-eater problem.","metadata":{}},{"cell_type":"markdown","source":"## Multi-Box Object Detection over more \"Reef-Eaters\" ##\nAfter we detect one sea-star, we are going detect more of that! First of all, we now define row again first, then define file_name again, but this time, locate the row with the video_frame key in train_data and concatenate it with .jpg inside the string. And then, we are now define video_folder variable with the string of video_ concatenated with the string of the location of train_data over the video_id key and the row variable. Lastly, we can again assign the boxes variable to the ast module over the literal_eval function, over the .loc dataframe over the locations of the annotations by the row variable.","metadata":{}},{"cell_type":"code","source":"row = 2845\nfile_name = str(train_data.loc[row, 'video_frame'])+'.jpg'\nvideo_folder = 'video_'+str(train_data.loc[row, 'video_id'])\nboxes = ast.literal_eval(train_data.loc[row, 'annotations'])","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:33.519571Z","iopub.execute_input":"2021-12-09T10:57:33.51981Z","iopub.status.idle":"2021-12-09T10:57:33.525335Z","shell.execute_reply.started":"2021-12-09T10:57:33.51978Z","shell.execute_reply":"2021-12-09T10:57:33.524472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('video folder:', video_folder)\nprint('file name:', file_name)","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:33.526708Z","iopub.execute_input":"2021-12-09T10:57:33.527471Z","iopub.status.idle":"2021-12-09T10:57:33.541048Z","shell.execute_reply.started":"2021-12-09T10:57:33.527414Z","shell.execute_reply":"2021-12-09T10:57:33.540047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, we can define the image variable with OpenCV using concatenation under the imread attribute. If we find out the shape of the image variable, it will return the tuple containing three numbers of the entire path.","metadata":{}},{"cell_type":"code","source":"image = cv2.imread(path+'train_images/'+video_folder+'/'+file_name)\nimage.shape","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:33.54251Z","iopub.execute_input":"2021-12-09T10:57:33.543551Z","iopub.status.idle":"2021-12-09T10:57:33.583877Z","shell.execute_reply.started":"2021-12-09T10:57:33.543499Z","shell.execute_reply":"2021-12-09T10:57:33.583074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## And now, let's plot again! ##","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(1, 1, figsize=(20, 8))\nax.imshow(image)\nfor box in boxes:\n    p = matplotlib.patches.Rectangle((box['x'], box['y']), box['width'], box['height'],\n                                     ec='r', fc='none', lw=2.)\n    ax.add_patch(p)\nax.set_xticklabels([])\nax.set_yticklabels([])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:33.585443Z","iopub.execute_input":"2021-12-09T10:57:33.585847Z","iopub.status.idle":"2021-12-09T10:57:34.140401Z","shell.execute_reply.started":"2021-12-09T10:57:33.585814Z","shell.execute_reply":"2021-12-09T10:57:34.1398Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Our model now captured three reef-eating sea-stars in this image. After we detected all of the seastars in some images, now it's time to submit predictions.","metadata":{}},{"cell_type":"markdown","source":"## API Usage ##\nSince we detected all of the seastars in all of the images, let's use the api afterall. In order to use the api, we have to follow directions of [this notebook made by Sohier](https://www.kaggle.com/sohier/great-barrier-reef-api-tutorial/notebook).","metadata":{}},{"cell_type":"code","source":"import PIL.Image\n\n# these sys calls aren't actually necesarry in Kaggle notebooks, but you may need to add the data directory to your pythonpath to run the sample API off of Kaggle\nimport sys\nsys.path.append('../input/tensorflow-great-barrier-reef')   \n\nimport greatbarrierreef\nenv = greatbarrierreef.make_env()   # initialize the environment\niter_test = env.iter_test() ","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:34.141527Z","iopub.execute_input":"2021-12-09T10:57:34.141898Z","iopub.status.idle":"2021-12-09T10:57:34.224808Z","shell.execute_reply.started":"2021-12-09T10:57:34.141869Z","shell.execute_reply":"2021-12-09T10:57:34.223454Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pixel_array, sample_prediction_df = next(iter_test)\npixel_array","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:34.226427Z","iopub.status.idle":"2021-12-09T10:57:34.22697Z","shell.execute_reply.started":"2021-12-09T10:57:34.226692Z","shell.execute_reply":"2021-12-09T10:57:34.226721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PIL.Image.fromarray(pixel_array)","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:34.228326Z","iopub.status.idle":"2021-12-09T10:57:34.228829Z","shell.execute_reply.started":"2021-12-09T10:57:34.228558Z","shell.execute_reply":"2021-12-09T10:57:34.228584Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## And finally, let's submit the data!!!!! ##","metadata":{}},{"cell_type":"code","source":"samp_subm.to_csv('submission.csv', index=False) # Remember: name your submission, submission.csv!","metadata":{"execution":{"iopub.status.busy":"2021-12-09T10:57:34.230456Z","iopub.status.idle":"2021-12-09T10:57:34.230954Z","shell.execute_reply.started":"2021-12-09T10:57:34.230695Z","shell.execute_reply":"2021-12-09T10:57:34.230722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Conclusion ##\nJob well done, we just detect a lot of starfishes in a pile of images. Overall, we will know that a lot of environmental conservationists and scientists will inject vinegar over to the infamous reef-eaters and kill them up for once and for all! When there are no crown-of-thorn seastars lurking through the Great Barrier Reef, the corals in there would thrive happily ever after... since I can make up as a fairy tale story in this notebook hehe.","metadata":{}},{"cell_type":"markdown","source":"## Acknowledgements ##\nSpecial thanks to DrCapa, who provided the starter notebook as an example to some people who are new to computer coding competition like me. Details over here: \nhttps://www.kaggle.com/drcapa/great-barrier-reef-starter","metadata":{}}]}