{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Overview\n\n## Discription\n\nResearchers manually track marine life by the shape and markings on their tails, dorsal fins, heads and other body parts. Identification by natural markings via photographs—known as photo-ID—is a powerful tool for marine mammal science. It allows individual animals to be tracked over time and enables assessments of population status and trends. To automate whale and dolphin photo-ID, researchers can reduce image identification times by over 99%. More efficient identification could enable a scale of study previously unaffordable or impossible.\n\nThousands of hours go into manual matching, which involves staring at photos to compare one individual to another, finding matches, and identifying new individuals. Manual matching limits the scope and reach.\n\nIn this competition, you’ll develop a model to match individual whales and dolphins by unique characteristics of their natural markings. You'll pay particular attention to dorsal fins and lateral body views in image sets from a multi-species dataset. The best submissions will suggest photo-ID solutions that are fast and accurate.\n\n## Evaluate\n\nSubmissions are evaluated according to the Mean Average Precision @ 5 (MAP@5):\n\n__$\nMAP@5 = \\cfrac{1}{U}\\displaystyle\\sum^{U}_{u=1}\\displaystyle\\sum^{min(n, 5)}_{k=1}P(k) \\times rel(k)\n$__\n\nWhere $U$ is the number of images, $P(k)$ is the precision at cutoff $k$, $n$ is the number predictions per image, and $rel(k)$ is an indicator function equaling 1 if the item at rank $k$ is a relevant (correct) label, zero otherwise.\n\nOnce a correct label has been scored for an observation, that label is no longer considered relevant for that observation, and additional predictions of that label are skipped in the calculation. For example, if the correct label is A for an observation, the following predictions all score an average precision of 1.0.","metadata":{}},{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"%%time\nimport os\nimport cv2\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport tensorflow as tf\n\nfrom tensorflow.keras import layers\n\ntf.config.list_physical_devices()","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:06:49.874269Z","iopub.execute_input":"2022-02-28T00:06:49.874953Z","iopub.status.idle":"2022-02-28T00:06:57.030835Z","shell.execute_reply.started":"2022-02-28T00:06:49.874830Z","shell.execute_reply":"2022-02-28T00:06:57.029977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Consts","metadata":{}},{"cell_type":"code","source":"ROOT_PATH = \"../input/happy-whale-and-dolphin/\"\n\nTRAIN_CSV = ROOT_PATH + \"train.csv\"\nTRAIN_DIR = ROOT_PATH + \"train_images/\"\nTEST_DIR = ROOT_PATH + \"test_images/\"\n\nTRAIN_IMAGES = os.listdir(TRAIN_DIR)\nTEST_IMAGES = os.listdir(TEST_DIR)\n\nSAMPLE_SUBMITION_CSV = ROOT_PATH + \"sample_submission.csv\"","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:06:57.032902Z","iopub.execute_input":"2022-02-28T00:06:57.033206Z","iopub.status.idle":"2022-02-28T00:06:58.383571Z","shell.execute_reply.started":"2022-02-28T00:06:57.033166Z","shell.execute_reply":"2022-02-28T00:06:58.382944Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Whales and dolphins in this dataset can be identified by shapes, features and markings (some natural, some acquired) of dorsal fins, backs, heads and flanks. Individuals have been manually identified and given an __individual_id__ by marine researches, and your task is to correctly identify these individuals in images.\n\nAn important note about data quality: Bringing together this dataset from many different research organization posed a number of practical challenges.\n\n## Files\n- __train.csv__ - provides the species and the individual_id for each of the training images\n- __train_images/__ - a folder containing the training images\n- __test_images/__ - a folder containing the test images; for each image, your task is to predict the __individual_id__; no species information is given for the test data; there are individuals in the test data that are not observed in the training data, which should be predicted as __new_individual__.\n- __sample_submission.csv__ - a sample submission file in the correct format","metadata":{}},{"cell_type":"markdown","source":"# EDA","metadata":{}},{"cell_type":"markdown","source":"## Dataframe","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(TRAIN_CSV)\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:06:58.385055Z","iopub.execute_input":"2022-02-28T00:06:58.385761Z","iopub.status.idle":"2022-02-28T00:06:58.493501Z","shell.execute_reply.started":"2022-02-28T00:06:58.385704Z","shell.execute_reply":"2022-02-28T00:06:58.492657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.info()","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:06:58.496264Z","iopub.execute_input":"2022-02-28T00:06:58.496667Z","iopub.status.idle":"2022-02-28T00:06:58.538973Z","shell.execute_reply.started":"2022-02-28T00:06:58.496623Z","shell.execute_reply":"2022-02-28T00:06:58.538332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.describe()","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:06:58.541070Z","iopub.execute_input":"2022-02-28T00:06:58.541928Z","iopub.status.idle":"2022-02-28T00:06:58.627608Z","shell.execute_reply.started":"2022-02-28T00:06:58.541863Z","shell.execute_reply":"2022-02-28T00:06:58.626636Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- All images are unique\n- Total 30 species\n- Total 15587 individuals","metadata":{}},{"cell_type":"code","source":"51_033 / 15_587","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:06:58.628655Z","iopub.execute_input":"2022-02-28T00:06:58.628894Z","iopub.status.idle":"2022-02-28T00:06:58.634683Z","shell.execute_reply.started":"2022-02-28T00:06:58.628866Z","shell.execute_reply":"2022-02-28T00:06:58.633930Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For each image need to collect 5 individual_id. This means that individual_id must appear on different images.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(14, 8))\n\nplt.title(\"species distribution\")\nsns.countplot(data=train_df, y='species')\n\nplt.xticks(np.linspace(0, 10_000, 11))\nplt.grid(axis='x')","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:06:58.635591Z","iopub.execute_input":"2022-02-28T00:06:58.635784Z","iopub.status.idle":"2022-02-28T00:06:59.200754Z","shell.execute_reply.started":"2022-02-28T00:06:58.635759Z","shell.execute_reply":"2022-02-28T00:06:59.199954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It is clear that some species are not balanced: \n- These species should be deleted or duplicated.","metadata":{}},{"cell_type":"markdown","source":"## Images","metadata":{}},{"cell_type":"code","source":"class PrepImgs:\n    \n    def __init__(self, root: str, file: str):\n        \"\"\"Allows to manipulate of image\n        Methods:\n            info(): image description\n            prep(): preprocess for this competition\n            save(): image save\n        Paramns:\n            root: image folder\n            file: image name with extension\n        \"\"\"\n        self.root = root\n        self.file = file\n        self.path = root + file\n        \n        self.array = cv2.imread(self.path)\n        self.array = self.array[:, :, ::-1]\n        \n    def save(self, to: str = \"./new_image/\"):\n        \"\"\"Image save to path\n        Params:\n            to: where save image\n        \"\"\"\n        array = self.array[:, :, ::-1]\n        array = array if array.max() > 1 else array * 255\n        array = array.astype(np.uint8)\n        \n        cv2.imwrite(to + self.file, array)\n        \n    def prep(self, size: int = 256):\n        \"\"\"Preprocess image for this competition\n        Params:\n            size: size for new image\n        \"\"\"\n        self.array = layers.Resizing(size, size)(self.array) # equate and lowering\n        self.array = self.array.numpy() # tf in numpy\n        self.array = self.array.astype(np.uint8)\n        \n        return self.array\n        \n    def info(self, show: bool = True):\n        \"\"\"Description of the image: shows it, shows the path, min, max and shape of image\n        Params:\n            show: flag for view images\n        \"\"\"\n        if show:\n            plt.figure(figsize=(2, 2))\n            plt.imshow(self.array)\n            plt.axis(\"off\")\n            plt.show()\n        \n        print(f\"path: {self.path}\",\n              f\"min: {self.array.min()}\",\n              f\"max: {self.array.max()}\",\n              f\"shape: {self.array.shape}\", sep=\"\\n\")","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:06:59.202143Z","iopub.execute_input":"2022-02-28T00:06:59.202615Z","iopub.status.idle":"2022-02-28T00:06:59.214125Z","shell.execute_reply.started":"2022-02-28T00:06:59.202580Z","shell.execute_reply":"2022-02-28T00:06:59.213402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PrepImgs(TRAIN_DIR, TRAIN_IMAGES[0]).info()\nPrepImgs(TEST_DIR, TEST_IMAGES[0]).info()","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:06:59.215497Z","iopub.execute_input":"2022-02-28T00:06:59.216036Z","iopub.status.idle":"2022-02-28T00:07:01.233464Z","shell.execute_reply.started":"2022-02-28T00:06:59.216006Z","shell.execute_reply":"2022-02-28T00:07:01.232594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Img encoding in RGB 0 - 255:\n- Need to be rescaling in RGB 0 - 1","metadata":{}},{"cell_type":"code","source":"def view_images(directory, files, titles_from=[], name=\"\", rows=2, cols=3, figsize=(13, 7)):\n    \"\"\"Show sample of images in grid\n    Params:\n        directory:  path where has images\n        files:      names with extensions\n        title_from: where take titles for images\n        name:       title for all sample\n        rows:       rows of grid\n        cols:       cols of grid\n        figsize:    size of all grid\n    \"\"\"\n\n    fig, ax = plt.subplots(rows, cols, figsize=figsize)\n    \n    fig.suptitle(name, fontsize=16)\n    \n    for r in range(rows):\n        for c in range(cols):\n            \n            file = files[c + r*3]\n            img = PrepImgs(directory, file).array\n            \n            if len(titles_from):\n                title = titles_from.loc[titles_from[\"image\"] == file, \"species\"]\n                title = title.values[0]\n                \n                ax[r, c].set_title(title)\n                \n            ax[r, c].imshow(img)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:07:01.235720Z","iopub.execute_input":"2022-02-28T00:07:01.236072Z","iopub.status.idle":"2022-02-28T00:07:01.243815Z","shell.execute_reply.started":"2022-02-28T00:07:01.236042Z","shell.execute_reply":"2022-02-28T00:07:01.243051Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"view_images(TRAIN_DIR, TRAIN_IMAGES, train_df, name=\"TRAIN IMAGES\")","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:07:01.245119Z","iopub.execute_input":"2022-02-28T00:07:01.245702Z","iopub.status.idle":"2022-02-28T00:07:06.586482Z","shell.execute_reply.started":"2022-02-28T00:07:01.245655Z","shell.execute_reply":"2022-02-28T00:07:06.585877Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"view_images(TEST_DIR, TEST_IMAGES, name=\"TEST IMAGES\")","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:07:06.587641Z","iopub.execute_input":"2022-02-28T00:07:06.588028Z","iopub.status.idle":"2022-02-28T00:07:12.155656Z","shell.execute_reply.started":"2022-02-28T00:07:06.587982Z","shell.execute_reply":"2022-02-28T00:07:12.154888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"All images have different shapes and too big size for NN:\n- Need to lowering size this images\n- And equate image sizes","metadata":{}},{"cell_type":"markdown","source":"Since need to predict 5 individual_id for each image, it would be interesting to look at different images of the same animal.","metadata":{}},{"cell_type":"code","source":"individual = \"60008f293a2b\"\nexample = train_df[train_df[\"individual_id\"] == individual]\nexample = list(example[\"image\"])\n\nview_images(TRAIN_DIR, example, name=individual)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:07:12.156972Z","iopub.execute_input":"2022-02-28T00:07:12.157788Z","iopub.status.idle":"2022-02-28T00:07:16.249565Z","shell.execute_reply.started":"2022-02-28T00:07:12.157742Z","shell.execute_reply":"2022-02-28T00:07:16.248963Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Resume\n\nIn process part need to do next actions:\n1. Need to labeling for each individual_id\n2. Equate image sizes\n3. Need to lowering sizes\n4. Rescaling image from 0 - 255 in 0 - 1\n5. Decide what to do with unbalanses species","metadata":{}},{"cell_type":"markdown","source":"# Preproces","metadata":{}},{"cell_type":"markdown","source":"## Labeling","metadata":{}},{"cell_type":"code","source":"labels = train_df[\"individual_id\"].unique()\nlabels = {ind: idx for idx, ind in enumerate(labels)}\n\ntrain_df[\"label\"] = train_df[\"individual_id\"].map(labels)\ntrain_df[\"label\"] = train_df['label'].astype(np.uint)\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:07:16.250742Z","iopub.execute_input":"2022-02-28T00:07:16.251133Z","iopub.status.idle":"2022-02-28T00:07:16.285358Z","shell.execute_reply.started":"2022-02-28T00:07:16.251088Z","shell.execute_reply":"2022-02-28T00:07:16.284474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Images\n\nI want to resize images and save them, for speed up of loading later","metadata":{}},{"cell_type":"code","source":"def save_new_images(path, to=\"./new_images/\", count=None):\n    \"\"\"Resaves images in new path\n    Params:\n        path: where to get files from\n        to:   path to save files\n    \"\"\"\n    try:\n        os.mkdir(to)\n    except FileExistsError as e:\n        print(e)\n    \n    files = os.listdir(path)\n    if count:\n        files = files[:count]\n    \n    for file in files:\n        img = PrepImgs(path, file)\n        img.prep()\n        img.save(to)\n\n# new image dirs\nNEW_TRAIN_DIR = \"./new_train_images/\"\nNEW_TEST_DIR = \"./new_test_images/\"\n        \nsave_new_images(TRAIN_DIR, NEW_TRAIN_DIR, 100)\n# save_new_images(TEST_DIR, NEW_TEST_DIR)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:07:16.286875Z","iopub.execute_input":"2022-02-28T00:07:16.287390Z","iopub.status.idle":"2022-02-28T00:07:25.905252Z","shell.execute_reply.started":"2022-02-28T00:07:16.287344Z","shell.execute_reply":"2022-02-28T00:07:25.904442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"view_images(NEW_TRAIN_DIR, TRAIN_IMAGES, train_df, \"TRAIN IMAGES\", cols=4)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T00:07:25.906294Z","iopub.execute_input":"2022-02-28T00:07:25.906519Z","iopub.status.idle":"2022-02-28T00:07:27.121001Z","shell.execute_reply.started":"2022-02-28T00:07:25.906492Z","shell.execute_reply":"2022-02-28T00:07:27.120369Z"},"trusted":true},"execution_count":null,"outputs":[]}]}