{"cells":[{"metadata":{},"cell_type":"markdown","source":"# <div class=\"h1\">iWildCam 2020 - FGVC7</div>\n\n## Let's detect wild animals in new places!"},{"metadata":{},"cell_type":"markdown","source":"### <u>Note</u>\n\n**Update information**\n\nUpdate1(Commit5): I added visualization of data distribution.\n\nUpdate2(Commit9): I organized graphs for easy comparison of datas, and add comments.\n\nUpdate3(Commit15): Added analysis of populations appearing in photos. And edit appendix.\n\nUpdate4(Commit19): Added EfficientNet-B3 example. \n\nUpdate5(Commit21): Added analysis of timelly and locationaly apparence in animals."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"%%HTML\n<style type=\"text/css\">\ndiv.h1 {\n    background-color:#e17b34; \n    color: white; \n    padding: 8px; \n    padding-right: 300px; \n    font-size: 35px; \n    max-width: 1500px; \n    margin: auto; \n    margin-top: 50px;\n}\n\ndiv.h2 {\n    background-color:#83ccd2; \n    color: white; \n    padding: 8px; \n    padding-right: 300px; \n    font-size: 35px; \n    max-width: 1500px; \n    margin: auto; \n    margin-top: 50px;\n}\n</style>","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## About this kernel\n\nI want to give you an overview of the prior knowledge and data that might be needed in this challenge!\n\nThis contest aims to detect wildlife in trapped picture at new monitoring locations. Now, we can get great insights about wildlife by camera traps. Camera trap is popular method and there are so many data in the world. But due to the so large number of data, it seems that the data was not always effectively accessed and utilized. \n\nIn this competition, we aim to develop a model that effectively classifies animals taken at different observation points　in the world. This challenge. This challenge will surely bring great insights. "},{"metadata":{},"cell_type":"markdown","source":"### We can deal great wildlife picures!\n\n### Let's take a look!!!😺"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from PIL import Image\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nfig = plt.figure(figsize=(20, 12))\nim = Image.open('../input/iwildcam-2020-fgvc7/train/939e25fc-21bc-11ea-a13a-137349068a90.jpg')\nplt.imshow(im)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"How cute!😻"},{"metadata":{},"cell_type":"markdown","source":"# <div class=\"h2\">knowladge</div>"},{"metadata":{},"cell_type":"markdown","source":"## How can we take wildlife pictures and use?\n\nWe can take wildlife pictures by camera trap. Camera traps have infrared sensor or motion sensor, so they can detect animals. When animals come near, camera take their pictures.  Using camera traps, we can monitor wildlifes continuously ,at several point and  at the same time. So we can understand how animals run their life in the area researchers interest.[1]\n\nFor example, in Kaen Krachan National Park in Thailand, indian sinatra are directlly observed. So it was pointed out that there is no longer any possibility. But by using camera trap, we could confirm thir existence. [2]  \n\n\nTraditionally, camera traps have been considered an excellent method for investigating ecological information about wildlife in a certain area.　Data are used for population estimation, calculating population index, 24hours monitorling and so on.[3]\n\nResently there has been a movement to make effective use of photos from camera traps using machine learning. One of the example is [4]. Google successed to access so much wildlife photo knowledge. Following Wilflife Insights is the platform of taking the initiative."},{"metadata":{"trusted":true},"cell_type":"code","source":"from IPython.display import YouTubeVideo\nYouTubeVideo('qKgRbkCkRFY')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Thus, creating a model for effectively classifying wild animals is a very significant effort."},{"metadata":{},"cell_type":"markdown","source":"### Referece\n\n[1]https://www.wwf.org.uk/project/conservationtechnology/camera-trap\n\n[2]https://www.wwf.or.jp/campaign/2015_camera/\n\n[3]https://en.wikipedia.org/wiki/Camera_trap\n\n[4]https://www.blog.google/products/earth/ai-finds-where-the-wild-things-are/"},{"metadata":{},"cell_type":"markdown","source":"# <div class=\"h2\">Data Overview</div>"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from PIL import Image, ImageDraw\nimport collections\nimport glob \nfrom datetime import datetime as dt\nimport gc\nimport json\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## <u>Imagination of submission</u>"},{"metadata":{},"cell_type":"markdown","source":"From https://www.kaggle.com/c/iwildcam-2020-fgvc7/overview/evaluation:\n> The Id column corresponds to the test image id. The Category is an integer value that indicates the class of the animal, or 0 to represent the absence of an animal."},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.read_csv(\"../input/iwildcam-2020-fgvc7/sample_submission.csv\").head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## <u>Overview of train & test data</u>\n\nThere are color data and gray data.  There seems to be photos taken while it was bright and photos taken at night."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_jpeg = glob.glob('../input/iwildcam-2020-fgvc7/train/*')\ntest_jpeg = glob.glob('../input/iwildcam-2020-fgvc7/test/*')\n\nprint(\"number of train jpeg data:\", len(train_jpeg))\nprint(\"number of test jpeg data:\", len(test_jpeg))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Train data"},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(25, 16))\nfor i,im_path in enumerate(train_jpeg[:16]):\n    ax = fig.add_subplot(4, 4, i+1, xticks=[], yticks=[])\n    im = Image.open(im_path)\n    im = im.resize((480,270))\n    plt.imshow(im)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(25, 16))\nfor i,im_path in enumerate(train_jpeg[16:32]):\n    ax = fig.add_subplot(4, 4, i+1, xticks=[], yticks=[])\n    im = Image.open(im_path)\n    im = im.resize((480,270))\n    plt.imshow(im)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Test data"},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(25, 16))\nfor i,im_path in enumerate(test_jpeg[:16]):\n    ax = fig.add_subplot(4, 4, i+1, xticks=[], yticks=[])\n    im = Image.open(im_path)\n    im = im.resize((480,270))\n    plt.imshow(im)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## <u>Checking iwildcam2020_megadetector_results.json</u>\n\nThis json contains the detection result of the training data by megadetector.\n\nMegadetector is the pre-trained model to detect animals, people, and vehicles in camera trap images.\n\nhttps://github.com/microsoft/CameraTraps/blob/master/megadetector.md#our-ask-to-megadetector-users"},{"metadata":{"trusted":true},"cell_type":"code","source":"with open('../input/iwildcam-2020-fgvc7/iwildcam2020_megadetector_results.json', encoding='utf-8') as json_file:\n    megadetector_results =json.load(json_file)\n    \nmegadetector_results.keys()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"megadetector_results_df = pd.DataFrame(megadetector_results[\"images\"])\nmegadetector_results_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Refered: https://www.kaggle.com/qinhui1999/how-to-use-bbox-for-iwildcam-2020 \n\ndef draw_bboxs(detections_list, im):\n    \"\"\"\n    detections_list: list of set includes bbox.\n    im: image read by Pillow.\n    \"\"\"\n    \n    for detection in detections_list:\n        x1, y1,w_box, h_box = detection[\"bbox\"]\n        ymin,xmin,ymax, xmax=y1, x1, y1 + h_box, x1 + w_box\n        draw = ImageDraw.Draw(im)\n        \n        imageWidth=im.size[0]\n        imageHeight= im.size[1]\n        (left, right, top, bottom) = (xmin * imageWidth, xmax * imageWidth,\n                                      ymin * imageHeight, ymax * imageHeight)\n        \n        draw.line([(left, top), (left, bottom), (right, bottom),\n               (right, top), (left, top)], width=4, fill='Red')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_num = 8857\nim = Image.open(\"../input/iwildcam-2020-fgvc7/train/\" + megadetector_results_df.loc[data_num]['id'] + \".jpg\")\nim = im.resize((480,270))\ndraw_bboxs(megadetector_results_df.loc[data_num]['detections'], im)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(im)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can also know how many aminals detected by megadetector in each pictures. "},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_data(x):\n    if x == []:\n        return 0\n    return len(x)\n\nmegadetector_results_df[\"detected_num\"] = megadetector_results_df.loc[:, \"detections\"].map(get_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(15, 4))\nax = sns.countplot(x=\"detected_num\", data=megadetector_results_df)\nax.set(ylabel='count')\n#ax.set(ylim=(0,80000))\nplt.title('distribution of count per animals each data of train')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can select training data depending on the number of animals in the picture."},{"metadata":{"trusted":true},"cell_type":"code","source":"megadetector_results_df = megadetector_results_df[megadetector_results_df['detected_num'] < 2]\nmegadetector_results_df = megadetector_results_df.rename(columns={'id': 'image_id'})\nmegadetector_results_df.head()\n#We can get filtered data of following df_train_file_cat\n#pd.merge(megadetector_results_df, df_train_file_cat, on='image_id', how='inner')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## <u>Data format conversion</u>\n\nI think, maybe you are interested in data such as category, location and datetime.\n\nThese data are provided as json data\n\nFor comvenience, I convert json data to DataFrame."},{"metadata":{},"cell_type":"markdown","source":"### Train data"},{"metadata":{"trusted":true},"cell_type":"code","source":"with open('/kaggle/input/iwildcam-2020-fgvc7/iwildcam2020_train_annotations.json') as json_file:\n    train_annotations_json = json.load(json_file)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_annotations_json.keys()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_annotations = pd.DataFrame(train_annotations_json[\"annotations\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_annotations.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_images = pd.DataFrame(train_annotations_json[\"images\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_images.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_categories = pd.DataFrame(train_annotations_json[\"categories\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_categories.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f\"There are {len(df_categories) - 1} species\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# I export DataFrame as CSV for convenience. \ndf_annotations.to_csv(\"iwildcam2020_train_annotations_annotations.csv\", index=False)\ndf_images.to_csv(\"iwildcam2020_train_annotations_images.csv\", index=False)\ndf_categories.to_csv(\"iwildcam2020_train_annotations_categories.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Test data"},{"metadata":{"trusted":true},"cell_type":"code","source":"with open('/kaggle/input/iwildcam-2020-fgvc7/iwildcam2020_test_information.json') as json_file:\n    test_information_json = json.load(json_file)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_information_json.keys()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_images_test = pd.DataFrame(test_information_json[\"images\"])\ndf_images_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_categories_test = pd.DataFrame(test_information_json[\"categories\"])\ndf_categories_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# I export DataFrame as CSV for convenience. \ndf_images_test.to_csv(\"iwildcam2020_train_annotations_images_test.csv\", index=False)\ndf_categories_test.to_csv(\"iwildcam2020_train_annotations_categories_test.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## <u>Data distributions</u>\n\nI plot train and test data in following perspective:\n\n* Time point\n\n* Category ID\n\n* Location"},{"metadata":{},"cell_type":"markdown","source":"### When did data taken?\n\nBecause animals can change their activity from time to time, we want to understand how data is distributed over time."},{"metadata":{},"cell_type":"markdown","source":"**Monthly perspective**"},{"metadata":{"trusted":true},"cell_type":"code","source":"month_year = df_images['datetime'].map(lambda str: str[2:7])\nlabels_month_year = sorted(list(set(month_year)))\n\nmonth_year_test = df_images_test['datetime'].map(lambda str: str[2:7])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize=(30,7))\nax = plt.subplot(1,2,1)\nax = plt.title('Count of train data per month & year')\nax = sns.countplot(month_year, order=labels_month_year)\nax.set(xlabel='YY-mm', ylabel='count')\nax.set(ylim=(0,50000))\n\nax = plt.subplot(1,2,2)\nax = plt.title('Count of test data per month & year')\n\nax = sns.countplot(month_year_test, order=labels_month_year)\nax.set(xlabel='YY-mm', ylabel='count')\nax.set(ylim=(0,50000))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Data starts 2013-01 but there seems be some lacks. For example, train data between 2013-11 to 2014-02 are missing. \n\nAlso we can find that train data in between 2013-01 to 2013-07 are rich than other time point.\n\nTrain data covers test data in perspective of time point."},{"metadata":{"trusted":true},"cell_type":"code","source":"labels_month = sorted(list(set(df_images['datetime'].map(lambda str: str[5:7]))))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize=(20,7))\nax = plt.subplot(1,2,1)\nplt.title('Count of train data per month')\nax = sns.countplot(df_images['datetime'].map(lambda str: str[5:7] ), order=labels_month)\nax.set(xlabel='mm', ylabel='count')\nax.set(ylim=(0,55000))\n\nax = plt.subplot(1,2,2)\nplt.title('Count of test data per month')\nax = sns.countplot(df_images_test['datetime'].map(lambda str: str[5:7] ), order=labels_month)\nax.set(xlabel='mm', ylabel='count')\nax.set(ylim=(0,55000))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Train data are bias. In February, March, June and July, data are rich than other months. \n\nTrain data covers test data in perspective of month.\n\nData for November and December are missing. Do animals hibernate?"},{"metadata":{},"cell_type":"markdown","source":"**Hourly perspective**"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_taken_hour = df_images['datetime'].map(lambda x: dt.strptime(x, '%Y-%m-%d %H:%M:%S.%f').hour)\ntest_taken_hour = df_images_test['datetime'].map(lambda x: dt.strptime(x, '%Y-%m-%d %H:%M:%S.%f').hour)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize=(20,7))\nax = plt.subplot(1,2,1)\nplt.title('Count of train data per hour')\nax = sns.countplot(train_taken_hour)\nax.set(xlabel='hour', ylabel='count')\nax.set(ylim=(0,20000))\n\nax = plt.subplot(1,2,2)\nplt.title('Count of test data per hour')\nax = sns.countplot(test_taken_hour)\nax.set(xlabel='hour', ylabel='count')\nax.set(ylim=(0,20000))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If we decide arbitrarily during daytime and at night, we can also calculate diurnal and nocturnal data counts.\n\nFor example, we define \"during daytime\" is \"6~17 O'clock\" and \"at night\" is \"18~5 O'clock\","},{"metadata":{"trusted":true},"cell_type":"code","source":"train_taken_phase = train_taken_hour.map(lambda x: \"daytime\" if x >= 6 and x < 18 else \"night\")\ntest_taken_phase = test_taken_hour.map(lambda x: \"daytime\" if x >= 6 and x < 18 else \"night\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize=(20,7))\nax = plt.subplot(1,2,1)\nplt.title('Count of train data per phase')\nax = sns.countplot(train_taken_phase, order=[\"daytime\", \"night\"])\nax.set(xlabel='phase', ylabel='count')\nax.set(ylim=(0,200000))\n\nax = plt.subplot(1,2,2)\nplt.title('Count of test data per phase')\nax = sns.countplot(test_taken_phase, order=[\"daytime\", \"night\"])\nax.set(xlabel='phase', ylabel='count')\nax.set(ylim=(0,200000))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Memory savings\ndel train_taken_phase\ndel test_taken_phase\ndel train_taken_hour\ndel test_taken_hour\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### How many data are there per animal category Id?\n\nThere are a lot of categories in dataset. To confirme how many data are there in each categories, I plot barplot.  "},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig = plt.figure(figsize=(30, 4))\nlabels_id = sorted(list(set(df_categories[\"id\"])))\nax = sns.barplot(x=\"id\", y=\"count\",data=df_categories, order=labels_id)\nax.set(ylabel='count')\nax.set(ylim=(0,80000))\nplt.title('distribution of count per id in train')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(30, 4))\nlabels_id = sorted(list(set(df_categories[\"id\"])))\nax = sns.barplot(x=\"id\", y=\"count\",data=df_categories_test, order=labels_id)\nax.set(ylabel='count')\nax.set(ylim=(0,80000))\nplt.title('distribution of count per id in test')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Frequency of data in each id is similar for train and test data."},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(15, 9))\nax = sns.distplot(df_categories['count'][1:])\nax.set(ylim=(0,0.00005))\nax.set(xlabel='count')\nplt.title('distribution of number of data per id zoomed')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Almost data is less than 5000 pictures. \n\nBut we can find that the number of specific IDs is very large and the data is biased."},{"metadata":{},"cell_type":"markdown","source":"### How many data per location?\n\nWe are required to detect photographs taken at different locations, but how distribute are data  in perspect of  location?"},{"metadata":{"trusted":true},"cell_type":"code","source":"labels_location_train = sorted(list(set(df_images['location'])))\nlabels_location_test = sorted(list(set(df_images_test['location'])))\nlabels_location = labels_location_train + labels_location_test","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig = plt.figure(figsize=(30, 4))\nax = sns.countplot(df_images['location'], order=labels_location)\nax.set(xlabel='location', ylabel='count')\nplt.title('Count of train data per location')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig = plt.figure(figsize=(30, 4))\nax = sns.countplot(df_images_test['location'], order=labels_location)\nax.set(xlabel='location', ylabel='count')\nplt.title('Count of test data per location')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Train data and test data seems be completelly taken in different locations.\n\nNumber of pictures are greatly differend by location."},{"metadata":{},"cell_type":"markdown","source":"### Are different animal kinds photographed in different places?\n\nWe have location data and animal category data, so we can investigate wheathrer different animal kinds are photographed in different places."},{"metadata":{"trusted":true},"cell_type":"code","source":"loc_cat_df_test = pd.merge(df_images, df_annotations, left_on='id', right_on='image_id', how = \"inner\").loc[:,[\"location\", \"category_id\", \"image_id\"]]\nloc_cat_df_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"loc_cat_dict = {}\nfor loc in set(loc_cat_df_test[\"location\"]):\n    loc_cat_dict[loc] = list(set(loc_cat_df_test[loc_cat_df_test[\"location\"] == loc][\"category_id\"]))   \nloc_cat_matrix = np.zeros([loc_cat_df_test[\"location\"].max()+1, loc_cat_df_test[\"category_id\"].max()+1])\nloc_cat_matrix.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for loc in loc_cat_dict.keys():\n    for cat in loc_cat_dict[loc]:\n        loc_cat_matrix[loc, cat] = 1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(15, 15))\nax = sns.heatmap(loc_cat_matrix)\nax.set(xlabel='category', ylabel='location')\nplt.title('Relation between animal categories and locations')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It's a bit hard to see, but columns are the categories of animals and rows are the locations. Since similar patterns appear in the vertical direction, it seems that similar species may appear even in different places."},{"metadata":{},"cell_type":"markdown","source":"### How many animals in each photograph sequence?\n\nWe can know how many animals each photograph sequence (continuous frame of camera trap from startup to taking) by annotations value of train_annotations_json. Count key seems represent number of animals appear in each photograph sequence. \n\nThree Pan troglodytes appear in folloing frames."},{"metadata":{"trusted":true},"cell_type":"code","source":"df_annotations[2429:2440]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(25, 16))\n#for i,im_path in enumerate(df_annotations[df_annotations.loc[:,\"count\"] == 3].loc[:,\"image_id\"][2429:2440]):\nfor i,im_path in enumerate(df_annotations.loc[:,\"image_id\"][2429:2440]):\n    ax = fig.add_subplot(4, 4, i+1, xticks=[], yticks=[])\n    im = Image.open(\"../input/iwildcam-2020-fgvc7/train/\" + im_path + \".jpg\")\n    im = im.resize((480,270))\n    plt.imshow(im)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Does the number of individuals that appear at one time vary depending on the animal species?"},{"metadata":{"trusted":true},"cell_type":"code","source":"c = collections.Counter(df_annotations.loc[:,\"count\"])\nanimals_in_pict = list(c.keys())\nfreq = list(c.values())\nk = zip(animals_in_pict,freq)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"animals_in_pict_df = pd.DataFrame(sorted(k),columns=['num_of_animals','freq'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"animals_in_pict_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Only a limited number of animals are grouped together. 266 species are taken in train set, but only 14 species were photographed, with more than 10 animals at a time. It turns out that there is a relationship between the animal species and the number of individuals reflected.　"},{"metadata":{"trusted":true},"cell_type":"code","source":"species_morethan_10 = list(set(df_annotations[df_annotations.loc[:,\"count\"] >= 10].loc[:,\"category_id\"]))\ndf_categories[df_categories.loc[:,\"id\"].isin(species_morethan_10)].loc[:,[\"id\", \"name\"]]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Species that shows more than 30 individuals at a time in a photo is  only a cow and a boar."},{"metadata":{"trusted":true},"cell_type":"code","source":"species_morethan_30 = list(set(df_annotations[df_annotations.loc[:,\"count\"] >= 30].loc[:,\"category_id\"]))\ndf_categories[df_categories.loc[:,\"id\"].isin(species_morethan_30)].loc[:,[\"id\", \"name\"]]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_annotations[(df_annotations.loc[:,\"category_id\"] == 2) & (df_annotations.loc[:,\"count\"] >= 30)].head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(25, 16))\n#for i,im_path in enumerate(df_annotations[df_annotations.loc[:,\"count\"] == 3].loc[:,\"image_id\"][2429:2440]):\nfor i,im_path in enumerate(df_annotations.loc[:,\"image_id\"][149674:149678]):\n    ax = fig.add_subplot(4, 4, i+1, xticks=[], yticks=[])\n    im = Image.open(\"../input/iwildcam-2020-fgvc7/train/\" + im_path + \".jpg\")\n    im = im.resize((480,270))\n    plt.imshow(im)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_annotations[(df_annotations.loc[:,\"category_id\"] == 71) & (df_annotations.loc[:,\"count\"] >= 30)].head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"jpeg ='../input/iwildcam-2020-fgvc7/train/8897a9f8-21bc-11ea-a13a-137349068a90.jpg'\nim = Image.open(jpeg)\nim = im.resize((480,270))\nplt.imshow(im)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model example"},{"metadata":{},"cell_type":"markdown","source":"I prepare EfficientNet example. The point of this model is that I choose photos which one or less　animal is taken for training.\n\nI refered following kernels.\n\nhttps://www.kaggle.com/ateplyuk/inat2019-starter-keras-efficientnet/data\n\nhttps://www.kaggle.com/mobassir/keras-efficientnetb2-for-classifying-cloud"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install tensorflow-gpu==1.14.0\n!pip install keras==2.2.4\n!pip install git+https://github.com/qubvel/efficientnet\n\nimport cv2\nfrom copy import deepcopy\nimport efficientnet.keras as efn \nimport glob\nfrom IPython.display import Image\nimport json\nimport keras\nimport keras.backend as K\nfrom keras.optimizers import Adam\nfrom keras.applications.densenet import DenseNet201\nfrom keras.layers import Dense, Flatten, Activation, Dropout, GlobalAveragePooling2D\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras import optimizers, applications\nfrom keras.models import Model, load_model\nfrom keras.callbacks import Callback, ModelCheckpoint, LearningRateScheduler, TensorBoard, EarlyStopping\nfrom keras.utils import Sequence\nimport matplotlib.pyplot as plt\nimport multiprocessing\nimport numpy as np\nfrom numpy.random import seed\nimport os\nimport random\nfrom sklearn.metrics import precision_recall_curve, auc\nfrom sklearn.model_selection import train_test_split\nfrom tqdm import tqdm_notebook as tqdm\n\nseed(10)\n\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_anns_df = df_annotations[['image_id','category_id']]\ntrain_img_df = df_images[['id', 'file_name']].rename(columns={'id':'image_id'})\ndf_train_file_cat = pd.merge(train_img_df, train_anns_df, on='image_id')\ndf_train_file_cat['category_id']=df_train_file_cat['category_id'].astype(str)\ndf_train_file_cat.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#megadetector_results_df was done the following operation in the previous cell.\n#megadetector_results_df = megadetector_results_df[megadetector_results_df['detected_num'] < 2]\n\ndf_train_file_cat = pd.merge(megadetector_results_df, df_train_file_cat, on='image_id', how='inner')\n\nfig = plt.figure(figsize=(15, 4))\nax = sns.countplot(x=\"detected_num\", data=megadetector_results_df)\nax.set(ylabel='count')\n#ax.set(ylim=(0,80000))\nplt.title('distribution of count per animals each data of train')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Prepare data and model"},{"metadata":{"trusted":true},"cell_type":"code","source":"#Parameters\n\nbatch_size = 256\nimg_size = 96\nlr = 0.001 \nnb_classes = 267\nnb_epochs = 6","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\ntrain_datagen=ImageDataGenerator(rescale=1./255, \n    validation_split=0.25,\n    horizontal_flip = True,    \n    zoom_range = 0.3,\n    width_shift_range = 0.3,\n    height_shift_range=0.3\n    )\n\ntrain_generator=train_datagen.flow_from_dataframe(    \n    dataframe=df_train_file_cat[:50000],    \n    directory=\"../input/iwildcam-2020-fgvc7/train\",\n    x_col=\"file_name\",\n    y_col=\"category_id\",\n    batch_size=batch_size,\n    shuffle=True,\n    classes = [ str(i) for i in range(nb_classes)],\n    class_mode=\"categorical\",    \n    target_size=(img_size,img_size))\n\ntest_datagen = ImageDataGenerator(rescale=1./255)\n\nvalid_generator=test_datagen.flow_from_dataframe(    \n    dataframe=df_train_file_cat[50000:],    \n    directory=\"../input/iwildcam-2020-fgvc7/train\",\n    x_col=\"file_name\",\n    y_col=\"category_id\",\n    batch_size=batch_size,\n    shuffle=True,\n    classes = [ str(i) for i in range(nb_classes)],\n    class_mode=\"categorical\",  \n    target_size=(img_size,img_size))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_model():\n    K.clear_session()\n    base_model =  efn.EfficientNetB3(weights='imagenet', include_top=False, pooling='avg', input_shape=(img_size, img_size, 3))\n    x = base_model.output\n    predictions = Dense(nb_classes, activation=\"softmax\")(x)\n    return Model(inputs=base_model.input, outputs=predictions)\n\nmodel = get_model()\nmodel.compile(optimizers.Adam(lr=lr, decay=1e-6),loss='categorical_crossentropy',metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"early = EarlyStopping(monitor='val_loss', min_delta=0, patience=3, verbose=1, mode='auto')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nhistory = model.fit_generator(generator=train_generator,  \n                                    steps_per_epoch=5,\n                                    validation_data=valid_generator, \n                                    validation_steps=2,\n                                    epochs=nb_epochs,\n                                    callbacks = [early],\n                                    verbose=2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history_df = pd.DataFrame(history.history)\nhistory_df[['loss', 'val_loss']].plot()\nhistory_df[['acc', 'val_acc']].plot()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import gc\ndel train_datagen, train_generator\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Prediction"},{"metadata":{"trusted":true},"cell_type":"code","source":"sam_sub_df = pd.read_csv('../input/iwildcam-2020-fgvc7/sample_submission.csv')\nsam_sub_df[\"file_name\"] = sam_sub_df[\"Id\"].map(lambda str : str + \".jpg\")\nsam_sub_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\ntest_generator = test_datagen.flow_from_dataframe(      \n    \n        dataframe=sam_sub_df,    \n    \n        directory = \"../input/iwildcam-2020-fgvc7/test\",    \n        x_col=\"file_name\",\n        target_size = (img_size,img_size),\n        batch_size = 1,\n        classes = [ str(i) for i in range(nb_classes)],\n        shuffle = False,\n        class_mode = None\n        )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntest_generator.reset()\npredict=model.predict_generator(test_generator, steps = len(test_generator.filenames))\npredicted_class_indices=np.argmax(predict,axis=1)\nsam_sub_df[\"Category\"] = predicted_class_indices\nsam_sub_df = sam_sub_df.loc[:,[\"Id\", \"Category\"]]\nsam_sub_df.to_csv(\"submission.csv\",index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The orginal training set contains pictures of many animals at one time. I thought they have a bad impact to learning. By eliminating these, it seems that learning has converged well. Instead, the predictions are not very good. As one of the improvement plans I think, it may be effective to cut out the part of data where the animal is reflected."},{"metadata":{},"cell_type":"markdown","source":"## [WIP] Appendix"},{"metadata":{},"cell_type":"markdown","source":"### About Landsat8\n\nWe are provided Landsat-8 multispectral imagery.\n\nThe Landsat 8 satellite takes pictures of the Earth using different wavelengths of light. Each of these are called 'bands'.  We reassemble them to tell us understand about Earth.\n\nFeatures of Landsat8:\n\n- The satellite circle the globe every 99 minutes.\n- The satellite takes 16 days to passed over every spot.\n- recorded data in 11 different wavelength.\n\nRefer following videos!\n"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"YouTubeVideo('mqVKR9OnqqA')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"YouTubeVideo('A6WzAc1FTeA')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"※Note that the data for second video was collected by Landsat 5 on November 10, 2011.\n\nWe can get Landsat-8 multispectral imagery in officially announced following URL.\n\nhttps://github.com/visipedia/iwildcam_comp#Data\n\nWe can get three type of files:\n\n- multispectral.npy\n- pixelqa.npy\n- radsatqa.npy\n\nDataset seems include multispectral imageryies for each location and certain period. For example, there are data between 2013/6/11 to 2019/12/5 for location 003."},{"metadata":{},"cell_type":"markdown","source":" Thank you for your reading!"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}