{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Dataset derived from [iWildCam 2021 - Starter Notebook](https://www.kaggle.com/nayuts/iwildcam-2021-starter-notebook#Explanation-for-metadata-file).","metadata":{}},{"cell_type":"markdown","source":"I created a 256x256 image dataset for prototyping. Using the MegaDetector detection results for each image, I cropped out the location of the animal. Images with no detection results were resized as is. After the cropping process, the cropped out images were indexed for easy use in learning. A summary of the process is shown below.","metadata":{}},{"cell_type":"markdown","source":"Use case is like this. \n\n<img src=\"https://raw.githubusercontent.com/tasotasoso/kaggle_media/main/iwildcam2021/model_image.png\" width=\"***300***\">\n\n1. First we crop the image based on the bbox detected by MegaDetector.\n2. In the training data, the correct answer labels are given as annotations, so we can use them to train the model.\n3. Classify the cropped images of the test data with the trained model.\n4. We choose the animal species and their counts of the image with the highest count among the images in the same image burst.\n\nI published notebook [here](https://www.kaggle.com/nayuts/efficientnet-with-undersampling). There is no need to vote on the linked note, as the notebooks are only separated due to execution time if you like both notebook.","metadata":{}},{"cell_type":"markdown","source":"### Note\n\nI created this dataset and benchmark to make it easier for newcomers to submit to the competition, but it did not provide much accuracy. Instance masks are released [Data](https://www.kaggle.com/c/iwildcam2021-fgvc8/data), so I think we may good performance if we solve this probrem using MOTS. I posted some resource for MOTS at [Multi-object tracking papers](https://www.kaggle.com/c/iwildcam2021-fgvc8/discussion/236338).","metadata":{}},{"cell_type":"code","source":"from IPython.display import YouTubeVideo\nYouTubeVideo('K38_pZw_P9s')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---------------------------","metadata":{}},{"cell_type":"markdown","source":"# Preperation","metadata":{}},{"cell_type":"code","source":"import glob \nimport json\nimport os\nimport pickle\n\nimport cv2\nimport numpy as np \nimport pandas as pd \nfrom PIL import Image, ImageFile, ImageFont, ImageDraw","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TRAIN_PATH = \"../input/iwildcam2021-fgvc8/train/\"\nTEST_PATH = \"../input/iwildcam2021-fgvc8/test/\"\nANNOTATIONS_PATH = \"../input/iwildcam2021-fgvc8/metadata/\"\n\nCROPED_TRAIN_PATH = \"./croped_images_train/\"\nCROPED_TEST_PATH = \"./croped_images_test/\"\n\nthreshould = 0.9","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.mkdir(CROPED_TRAIN_PATH)\nos.mkdir(CROPED_TEST_PATH)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(ANNOTATIONS_PATH + 'iwildcam2021_megadetector_results.json', encoding='utf-8') as json_file:\n    megadetector_results =json.load(json_file)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open('../input/iwildcam2021-fgvc8/metadata/iwildcam2021_train_annotations.json', encoding='utf-8') as json_file:\n    train_annotations =json.load(json_file)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"annotations = megadetector_results[\"images\"]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_crop_area(bbox, image_size):\n    x1, y1,w_box, h_box = bbox\n    ymin,xmin,ymax, xmax = y1, x1, y1 + h_box, x1 + w_box\n    area = (xmin * image_size[0], ymin * image_size[1], \n            xmax * image_size[0], ymax * image_size[1])\n    return area","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_ids_train = []\nimg_idx_train = []\nimg_ids_test = []\nimg_idx_test = []\n\nx_tot_list,x2_tot_list = [],[]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def save_image(img, img_id, idx, is_train):  \n    if is_train:\n        img.save( CROPED_TRAIN_PATH + f\"{img_id}_{idx}.jpg\", format=\"jpeg\")\n        img_ids_train.append(f\"{img_id}\")\n        img_idx_train.append(idx)\n    else:\n        img.save( CROPED_TEST_PATH + f\"{img_id}_{idx}.jpg\", format=\"jpeg\")\n        img_ids_test.append(f\"{img_id}\")\n        img_idx_test.append(idx)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def calc_x_and_x2_tot(img):\n    \n    img = np.array(img, dtype=np.uint8)\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    img = np.transpose(img,(2,0,1))\n            \n    return (img/255.0).reshape(-1,3).mean(0), ((img/255.0)**2).reshape(-1,3).mean(0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## main processing module↓","metadata":{}},{"cell_type":"code","source":"def converet_images(annotation):\n\n    size = (256,256)\n    img_id = annotation[\"id\"]\n    is_train = True \n    \n    try:\n        detections = annotation[\"detections\"]\n    except:\n        print(f\"Passed {img_id}. There are no detection data.\")\n        return\n    \n    path_for_train = TRAIN_PATH + annotation[\"id\"] + \".jpg\"\n    path_for_test = TEST_PATH + annotation[\"id\"] + \".jpg\"\n    \n    if os.path.exists(path_for_train):\n        file_path = path_for_train\n    elif os.path.exists(path_for_test):\n        file_path = path_for_test\n        is_train = False\n    else:\n        print(f\"Passed {img_id}. There are no data.\")\n        return\n    \n    \n    try:      \n        img = Image.open(file_path)\n    except:\n        print(f\"Passed {img_id}. Fail to open image.\")\n        print(f\"pass {file_path}.\")\n        return\n    \n    for i, detection in enumerate(detections, 1):\n        \n        if detection[\"conf\"] < threshould:\n            continue\n\n        if detection[\"category\"] != \"1\":\n            continue\n            \n        if len(detection) == 0:\n            img = img.resize(size)\n            save_image(img, img_id, 0, is_train)\n            \n            x_tot, x2_tot = calc_mean_and_var(img)\n            x_tot_list.append(x_tot)\n            x2_tot_list.append(x2_tot)\n            \n        else:\n            crop_area = get_crop_area(detection[\"bbox\"], img.size)\n            img_croped = img.crop(crop_area).resize(size)\n            save_image(img_croped, img_id, i, is_train)\n        \n            x_tot, x2_tot = calc_x_and_x2_tot(img_croped)\n            x_tot_list.append(x_tot)\n            x2_tot_list.append(x2_tot)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Cropping\n\nCrop by calling converet_images fuction.","metadata":{}},{"cell_type":"code","source":"for annotation in annotations:\n    converet_images(annotation)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The mean and variance for standardization.","metadata":{}},{"cell_type":"code","source":"#image stats\nimg_avr =  np.array(x_tot_list).mean(0)\nimg_std =  np.sqrt(np.array(x2_tot_list).mean(0) - img_avr**2)\nprint('mean:',img_avr, ', std:', img_std)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## zipping","metadata":{}},{"cell_type":"code","source":"!zip croped_images_train -r ./croped_images_train\n!zip croped_images_test -r ./croped_images_test","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!rm -r ./croped_images_train\n!rm -r ./croped_images_test","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Creating csv file for reference","metadata":{}},{"cell_type":"code","source":"croped_train = {\"id\": img_ids_train, \"idx\":img_idx_train}\ndf_croped_train = pd.DataFrame(croped_train)\ndf_train_annotation = pd.DataFrame(train_annotations[\"annotations\"])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_croped_train = df_croped_train.merge(\n    df_train_annotation[[\"image_id\", \"category_id\"]], \n    left_on='id', right_on='image_id')[[\"id\", \"idx\", \"category_id\"]]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_croped_train.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"croped_test = {\"id\": img_ids_test, \"idx\":img_idx_test}\ndf_croped_test = pd.DataFrame(croped_test)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_croped_test.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_croped_train.to_csv(\"./croped_train.csv\", index=False)\ndf_croped_test.to_csv(\"./croped_test.csv\", index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}