{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### Disclaimer: WORK IN PROGRESS\n\n#### Check back soon for more! Work in progress. This aims to be the ultimate starter notebook to this competition for beginners. :) ","metadata":{}},{"cell_type":"markdown","source":"### 😃 Author: Roshan Ram \n### 👀 LinkedIn: https://linkedin.com/in/roshanr11 ","metadata":{"execution":{"iopub.status.busy":"2022-02-15T19:59:08.748204Z","iopub.execute_input":"2022-02-15T19:59:08.749156Z","iopub.status.idle":"2022-02-15T19:59:08.754806Z","shell.execute_reply.started":"2022-02-15T19:59:08.749097Z","shell.execute_reply":"2022-02-15T19:59:08.753876Z"}}},{"cell_type":"markdown","source":"# 🔥 Herbarium 🌱 2022: Complete Starter NB 🔥","metadata":{}},{"cell_type":"markdown","source":"# Sources:\n- Modeled notebook UI/Markdown structure off of https://www.kaggle.com/ruchi798/and-identification-eda-augmentation by @ruchi798\n- Adapted starter EDA from https://www.kaggle.com/vad13irt/herbarium-2022-fast-exploratory-data-analysis by @vad13irt\n- https://www.kaggle.com/jirkaborovec/herbarium-eda\n- https://www.kaggle.com/venkatkumar001/herbarium-22-fgvc9-baseline\n- https://www.kaggle.com/drcapa/herbarium-2022-starter\n- https://www.kaggle.com/semack/herbaploration\n- https://www.kaggle.com/odins0n/json-pandas-herbarium-2022\n- https://www.kaggle.com/hamzaghanmi/welcome-herbarium-2022","metadata":{}},{"cell_type":"markdown","source":"# TL;DR (Jump Right In!) 🏃‍♂️ ⏩  \n\nThe Herbarium 2022: Flora of North America is a part of a project of the New York Botanical Garden funded by the National Science Foundation to build tools to identify novel plant species around the world. The dataset strives to represent all known vascular plant taxa in North America, using images gathered from 60 different botanical institutions around the world. For each image Id, you should predict the corresponding image label (category_id) in the Predicted column.","metadata":{}},{"cell_type":"markdown","source":"# Background 🔎 ","metadata":{}},{"cell_type":"markdown","source":"## Herbarium 2021: FGVC9\n\nThe Herbarium 2022: Flora of North America is a part of a project of the [New York Botanical Garden](https://www.nybg.org/) funded by the [National Science Foundation](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2054684&HistoricalAwards=false) to build tools to identify novel plant species around the world. The dataset strives to represent all known vascular plant taxa in North America, using images gathered from 60 different botanical institutions around the world.\n\nThe Herbarium 2022: Flora of North America dataset comprises 1.05 M images of 15,500 vascular plants, which constitute more than 90% of the taxa documented in North America. We used the comprehensive Checklist of the [Vascular Plants of the Americas (VPA)](Vascular Plants of the Americas (VPA)) produced by [Missouri Botanical Garden](Missouri Botanical Garden) and aligned the taxonomic names to [The World Checklist of Vascular Plants (WCVP)](https://www.nature.com/articles/s41597-021-00997-6) from the [Royal Botanical Garden Kew](https://www.kew.org/). Our dataset is constrained to include only vascular land plants (lycophytes, ferns, gymnosperms, and flowering plants).\n\n<!-- Source: https://www.kaggle.com/c/herbarium-2022-fgvc9/overview -->\n\n<!-- The Herbarium 2021: Half-Earth Challenge is to identify vascular plant specimens provided by the [New York Botanical Garden (NY)](https://www.nybg.org/), [Bishop Museum (BPBM)](https://www.bishopmuseum.org/), [Naturalis Biodiversity Center (NL)](https://www.naturalis.nl/en), [Queensland Herbarium (BRI)](https://www.qld.gov.au/environment/plants-animals/plants/herbarium), and [Auckland War Memorial Museum (AK)](https://www.aucklandmuseum.com/).\n\n*The Herbarium 2021: Half-Earth Challenge* dataset includes more than 2.5M images representing nearly 65,000 species from the Americas and Oceania that have been aligned to a standardized plant list ([LCVP v1.0.2](https://www.nature.com/articles/s41597-020-00702-z)). -->","metadata":{}},{"cell_type":"markdown","source":"## What is a herbarium?\n\nA herbarium is a collection of preserved plants stored, catalogued and arranged systematically for study by both professional taxonomists (scientists who name and identify plants), botanists and amateurs.\n\nThe creation of a herbarium specimen involves the pressing and drying of plants between sheets of paper, a practice that has changed very little since the beginning, 500 years ago. Thanks to this simple technique, most of the characteristics of living plants are visible on the dried plant. The few that are not (e.g. flower colour, scent, height of a tree, vegetation type) are written on the collection label by the collector. Most importantly, the label should tell us where and when the specimen was collected.\n\nA working reference collection A herbarium acts like a plant library or vast catalogue with each of our three million specimens providing unique information – where it was found, when it flowered, what it looks like and it’s DNA, which remains intact for many years. DNA is now routinely extracted from herbarium specimens. The most important specimens are called 'types'. The type specimen, chosen by the author of the species name, becomes the physical reference for the new species.\n\nThis unique working reference collection brings species from all over the world together into one place to be discovered, described and compared. The work is disseminated through the writing of Floras (a description of all the plants in a country or region), monographs (a description of plants or fungi within a group, such as a family) and scientific papers. This fundamental research provides an essential baseline for other plant-based research and helps inform conservation practices.\n\n<!-- Source: https://www.kaggle.com/shaunthesheep/fgvc7-herbarium-2020-data-viz -->\n\n[Read more about this here!](https://www.rbge.org.uk/science-and-conservation/herbarium/)","metadata":{}},{"cell_type":"markdown","source":"<!-- # Conscious planet movement? -->","metadata":{}},{"cell_type":"markdown","source":"# Terminology 📝","metadata":{}},{"cell_type":"markdown","source":"If you're as new as I am to all this plant stuff, chances are that you are just as clueless as I am about what any of the terms used in the Description and Data mean.   \n\nLet's figure it out together.\n\nTerms:","metadata":{"execution":{"iopub.status.busy":"2022-02-15T19:30:41.767213Z","iopub.execute_input":"2022-02-15T19:30:41.767507Z","iopub.status.idle":"2022-02-15T19:30:41.773753Z","shell.execute_reply.started":"2022-02-15T19:30:41.767479Z","shell.execute_reply":"2022-02-15T19:30:41.772501Z"}}},{"cell_type":"code","source":"!pip install -q addict","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:16:22.155322Z","iopub.execute_input":"2022-02-15T19:16:22.156246Z","iopub.status.idle":"2022-02-15T19:16:31.554894Z","shell.execute_reply.started":"2022-02-15T19:16:22.156117Z","shell.execute_reply":"2022-02-15T19:16:31.553974Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Import libraries 📚","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nfrom addict import Dict\nimport json\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport os\nimport cv2\nimport random","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:16:31.556852Z","iopub.execute_input":"2022-02-15T19:16:31.55706Z","iopub.status.idle":"2022-02-15T19:16:32.776565Z","shell.execute_reply.started":"2022-02-15T19:16:31.557035Z","shell.execute_reply":"2022-02-15T19:16:32.775708Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Important Helper Functions 😉","metadata":{}},{"cell_type":"code","source":"def load_image(path, channels=cv2.COLOR_BGR2RGB):\n    if os.path.exists(path):\n        image = cv2.imread(path)\n        image = cv2.cvtColor(image, channels)\n        image = np.asarray(image)\n        return image\n    else:\n        raise Exception(f\"Path '{path}' doesn't exist.\")\n\n\ndef get_institutions_data_frame(institutions):\n    all_ids, all_codes = [], []\n    \n    for institution in institutions:\n        all_ids.append(institution[\"institution_id\"])\n        all_codes.append(institution[\"collectionCode\"])\n        \n    data_frame = pd.DataFrame({\n        \"institution_id\": all_ids,\n        \"institution\": all_codes,\n    })\n    \n    return data_frame\n\n\ndef get_genera_data_frame(genera):\n    all_ids, all_genuses = [], []\n    for _ in genera:\n        all_ids.append(_[\"genus_id\"])\n        all_genuses.append(_[\"genus\"])\n        \n    data_frame = pd.DataFrame({\n        \"genus_id\": all_ids,\n        \"genus\": all_genuses\n    })\n    \n    return data_frame\n\n\ndef get_categories_data_frame(categories):\n    all_ids, all_names, all_families, all_genuses, all_species, all_authors = [], [], [], [], [], []\n    \n    for category in categories:\n        all_ids.append(category[\"category_id\"])\n        all_names.append(category[\"scientificName\"])\n        all_families.append(category[\"family\"])\n        all_genuses.append(category[\"genus\"])\n        all_species.append(category[\"species\"])\n        all_authors.append(category[\"authors\"])\n        \n    data_frame = pd.DataFrame({\n        \"category_id\": all_ids,\n        \"category\": all_names,\n        \"family\": all_families,\n        \"genus\": all_genuses,\n        \"species\": all_species,\n        \"authors\": all_authors\n    })\n    \n    return data_frame\n\n\ndef get_annotations_data_frame(annotations):\n    all_genuses, all_institutions, all_categories, all_images = [], [], [], []\n    \n    for annotation in annotations:\n        all_genuses.append(annotation[\"genus_id\"])\n        all_institutions.append(annotation[\"institution_id\"])\n        all_categories.append(annotation[\"category_id\"])\n        all_images.append(annotation[\"image_id\"])\n        \n        \n    data_frame = pd.DataFrame({\n        \"genus\": all_genuses,\n        \"institution_id\": all_institutions,\n        \"category_id\": all_categories,\n        \"image_id\": all_images,\n    })\n    \n    \n    return data_frame\n\n\ndef get_images_data_frame(images):\n    all_ids, all_pathes = [], []\n    for image in images:\n        all_ids.append(image[\"image_id\"])\n        all_pathes.append(image[\"file_name\"])\n        \n    data_frame = pd.DataFrame({\n        \"image_id\": all_ids,\n        \"image_path\": all_pathes,\n    })\n\n    return data_frame\n    \n    \ndef get_meta_data_frame(meta):\n    annotations = get_annotations_data_frame(meta[\"annotations\"])\n    annotations = annotations.drop(\"genus\", axis=1)\n    images = get_images_data_frame(meta[\"images\"])\n    categories = get_categories_data_frame(meta[\"categories\"])\n    genera = get_genera_data_frame(meta[\"genera\"])\n    institutions = get_institutions_data_frame(meta[\"institutions\"])\n    \n    data_frame = annotations.merge(images, on=\"image_id\")\n    data_frame = data_frame.merge(categories, on=\"category_id\")\n    data_frame = data_frame.merge(institutions, on=\"institution_id\")\n    \n    data_frame = data_frame.drop([\"image_id\", \"category_id\", \"institution_id\"], axis=1)\n    \n    return data_frame\n\n\ndef create_submission(ids, predictions, path=\"submission.csv\"):\n    submission = pd.DataFrame({\n        \"Id\": ids,\n        \"Predicted\": predictions,\n    })\n    \n    submission.to_csv(path, index=False)\n    return submission\n\ndef read_json(path):\n    with open(path, \"r\", encoding=\"utf-8\") as file:\n        data = json.loads(file.read())\n    return data\n\n\ndef plot_category_images(data_frame, category, title=\"Title\", rows=1, columns=5, background_color=\"#fff\"):\n    fig = plt.figure(figsize=(columns*3, rows*5))\n    fig.set_facecolor(background_color)\n    data_frame = data_frame[data_frame[\"category\"] == category]\n    images = rows * columns\n    genuses = \", \".join(data_frame[\"genus\"].unique())\n    families = \", \".join(data_frame[\"family\"].unique())\n    species = \", \".join(data_frame[\"species\"].unique())\n    \n    for i in range(images):\n        index = random.randint(0, len(data_frame)-1)\n        sample = data_frame.iloc[index]\n        image_path = sample[\"image_path\"]\n        image = load_image(image_path)\n        filename = image_path.split(\"/\")[-1]\n        \n        description = f\"Families: {families}\\nGenuses: {genuses}\\nSpecies: {species}\"\n        \n        ax = fig.add_subplot(rows, columns, i+1)\n        ax.set_facecolor(background_color)\n        ax.imshow(image)\n        ax.set_xlabel(filename, color=\"#000\", fontsize=14, y=1)\n        ax.xaxis.set_tick_params(labelsize=0, size=0)\n        ax.yaxis.set_visible(False)\n        hide_spines(ax)\n    \n    fig.suptitle(title, fontsize=25, fontweight=\"bold\", fontfamily=\"serif\", horizontalalignment=\"left\", y=1.1, x=0.01)\n    fig.text(s=description, x=0.01, y=0.97, fontsize=17, fontfamily=\"serif\", horizontalalignment=\"left\")\n    fig.tight_layout(h_pad=5, w_pad=2)\n    fig.show()\n    \ndef hide_spines(ax, spines=[\"top\", \"right\", \"left\", \"bottom\"]):\n    for spine in spines:\n        ax.spines[spine].set_visible(False)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:17:09.007296Z","iopub.execute_input":"2022-02-15T19:17:09.007548Z","iopub.status.idle":"2022-02-15T19:17:09.039177Z","shell.execute_reply.started":"2022-02-15T19:17:09.007521Z","shell.execute_reply":"2022-02-15T19:17:09.038197Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pathes = Dict({\n    \"train_meta\": \"../input/herbarium-2022-fgvc9/train_metadata.json\",\n    \"train_images\": \"../input/herbarium-2022-fgvc9/train_images\",\n    \"test_meta\": \"../input/herbarium-2022-fgvc9/test_metadata.json\",\n    \"test_images\": \"../input/herbarium-2022-fgvc9/test_images\",\n    \"sample_submission\": \"../input/herbarium-2022-fgvc9/sample_submission.csv\",\n})","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:24:47.460822Z","iopub.execute_input":"2022-02-15T19:24:47.461096Z","iopub.status.idle":"2022-02-15T19:24:47.465973Z","shell.execute_reply.started":"2022-02-15T19:24:47.461066Z","shell.execute_reply":"2022-02-15T19:24:47.465123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Read in Data 📖","metadata":{}},{"cell_type":"code","source":"train_meta = read_json(pathes.train_meta)\ntrain = get_meta_data_frame(train_meta)\ntrain[\"image_path\"] = train[\"image_path\"].apply(lambda _: os.path.join(pathes.train_images, _))\n\n\ntest_meta = read_json(pathes.test_meta)\ntest = get_images_data_frame(test_meta)\ntest[\"image_path\"] = test[\"image_path\"].apply(lambda _: os.path.join(pathes.test_images, _))\n\n\nsample_submission = pd.read_csv(pathes.sample_submission)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:24:49.754455Z","iopub.execute_input":"2022-02-15T19:24:49.754754Z","iopub.status.idle":"2022-02-15T19:25:13.540981Z","shell.execute_reply.started":"2022-02-15T19:24:49.754722Z","shell.execute_reply":"2022-02-15T19:25:13.540244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Gentle EDA 📈","metadata":{}},{"cell_type":"code","source":"fig = plt.figure(figsize=(20, 35))\nfig.set_facecolor(\"#fff\")\nax = fig.add_subplot()\nax.set_facecolor(\"#fff\")\nax.grid(color=\"lightgrey\", alpha=0.7, axis=\"both\", zorder=0)\nsns.countplot(y=\"institution\", data=train, ax=ax, zorder=2)\nax.xaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.yaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.set_ylabel(\"Institution\", fontsize=15, fontweight=\"normal\", labelpad=10)\nax.set_xlabel(\"Count\", fontsize=15, fontweight=\"normal\", labelpad=10)\nhide_spines(ax)\n\nax.set_title(\"Institution Distribution\", fontsize=25, fontweight=\"bold\", fontfamily=\"serif\", loc=\"left\", y=1.02)\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:25:13.542707Z","iopub.execute_input":"2022-02-15T19:25:13.54293Z","iopub.status.idle":"2022-02-15T19:25:15.576932Z","shell.execute_reply.started":"2022-02-15T19:25:13.542904Z","shell.execute_reply":"2022-02-15T19:25:15.57612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"k = 50\ntopk_authors = train[\"authors\"].value_counts()[:k]\n\nfig = plt.figure(figsize=(10, 25))\nfig.set_facecolor(\"#fff\")\nax = fig.add_subplot()\nax.set_facecolor(\"#fff\")\nax.grid(color=\"lightgrey\", alpha=0.7, axis=\"both\", zorder=0)\nsns.barplot(x=topk_authors.values, y=topk_authors.index, ax=ax, zorder=2)\nax.xaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.yaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.set_ylabel(\"Author\", fontsize=15, fontweight=\"normal\", labelpad=10)\nax.set_xlabel(\"Author's works\", fontsize=15, fontweight=\"normal\", labelpad=10)\nhide_spines(ax)\n\nax.set_title(f\"Top {k} Authors Distribution\", fontsize=25, fontweight=\"bold\", fontfamily=\"serif\", loc=\"left\", y=1.02)\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:25:15.578519Z","iopub.execute_input":"2022-02-15T19:25:15.57895Z","iopub.status.idle":"2022-02-15T19:25:16.704026Z","shell.execute_reply.started":"2022-02-15T19:25:15.57891Z","shell.execute_reply":"2022-02-15T19:25:16.703073Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"k = 50\ntopk_authors = train[\"species\"].value_counts()[:k]\n\nfig = plt.figure(figsize=(10, 25))\nfig.set_facecolor(\"#fff\")\nax = fig.add_subplot()\nax.set_facecolor(\"#fff\")\nax.grid(color=\"lightgrey\", alpha=0.7, axis=\"both\", zorder=0)\nsns.barplot(x=topk_authors.values, y=topk_authors.index, ax=ax, zorder=2)\nax.xaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.yaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.set_ylabel(\"Specie\", fontsize=15, fontweight=\"normal\", labelpad=10)\nax.set_xlabel(\"Count\", fontsize=15, fontweight=\"normal\", labelpad=10)\nhide_spines(ax)\n\nax.set_title(f\"Top {k} Species Distribution\", fontsize=25, fontweight=\"bold\", fontfamily=\"serif\", loc=\"left\", y=1.02)\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:25:16.705693Z","iopub.execute_input":"2022-02-15T19:25:16.705894Z","iopub.status.idle":"2022-02-15T19:25:18.046208Z","shell.execute_reply.started":"2022-02-15T19:25:16.705869Z","shell.execute_reply":"2022-02-15T19:25:18.045204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"k = 50\ntopk_authors = train[\"genus\"].value_counts()[:k]\n\nfig = plt.figure(figsize=(10, 25))\nfig.set_facecolor(\"#fff\")\nax = fig.add_subplot()\nax.set_facecolor(\"#fff\")\nax.grid(color=\"lightgrey\", alpha=0.7, axis=\"both\", zorder=0)\nsns.barplot(x=topk_authors.values, y=topk_authors.index, ax=ax, zorder=2)\nax.xaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.yaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.set_ylabel(\"Genus\", fontsize=15, fontweight=\"normal\", labelpad=10)\nax.set_xlabel(\"Count\", fontsize=15, fontweight=\"normal\", labelpad=10)\nhide_spines(ax)\n\nax.set_title(f\"Top {k} Genuses Distribution\", fontsize=25, fontweight=\"bold\", fontfamily=\"serif\", loc=\"left\", y=1.02)\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:25:18.047422Z","iopub.execute_input":"2022-02-15T19:25:18.047649Z","iopub.status.idle":"2022-02-15T19:25:19.144513Z","shell.execute_reply.started":"2022-02-15T19:25:18.047621Z","shell.execute_reply":"2022-02-15T19:25:19.144002Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(20, 100))\nfig.set_facecolor(\"#fff\")\nax = fig.add_subplot()\nax.set_facecolor(\"#fff\")\nax.grid(color=\"lightgrey\", alpha=0.7, axis=\"both\", zorder=0)\nsns.countplot(y=\"family\", data=train, ax=ax, zorder=2)\nax.xaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.yaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.set_ylabel(\"Family\", fontsize=15, fontweight=\"normal\", labelpad=10)\nax.set_xlabel(\"Count\", fontsize=15, fontweight=\"normal\", labelpad=10)\nhide_spines(ax)\n\nax.set_title(\"Family Distribution\", fontsize=25, fontweight=\"bold\", fontfamily=\"serif\", loc=\"left\", y=1.02)\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:25:19.145498Z","iopub.execute_input":"2022-02-15T19:25:19.145769Z","iopub.status.idle":"2022-02-15T19:25:25.172052Z","shell.execute_reply.started":"2022-02-15T19:25:19.145744Z","shell.execute_reply":"2022-02-15T19:25:25.171361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"k = 50\ntopk_authors = train[\"category\"].value_counts()[:k]\n\nfig = plt.figure(figsize=(10, 25))\nfig.set_facecolor(\"#fff\")\nax = fig.add_subplot()\nax.set_facecolor(\"#fff\")\nax.grid(color=\"lightgrey\", alpha=0.7, axis=\"both\", zorder=0)\nsns.barplot(x=topk_authors.values, y=topk_authors.index, ax=ax, zorder=2)\nax.xaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.yaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.set_ylabel(\"Specie\", fontsize=15, fontweight=\"normal\", labelpad=10)\nax.set_xlabel(\"Count\", fontsize=15, fontweight=\"normal\", labelpad=10)\nhide_spines(ax)\n\nax.set_title(f\"Top {k} Categories Distribution\", fontsize=25, fontweight=\"bold\", fontfamily=\"serif\", loc=\"left\", y=1.02)\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:25:25.173182Z","iopub.execute_input":"2022-02-15T19:25:25.173807Z","iopub.status.idle":"2022-02-15T19:25:26.759274Z","shell.execute_reply.started":"2022-02-15T19:25:25.173768Z","shell.execute_reply":"2022-02-15T19:25:26.757722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"k = 50\ntopk_authors = train[\"category\"].value_counts()[-k:]\n\nfig = plt.figure(figsize=(10, 25))\nfig.set_facecolor(\"#fff\")\nax = fig.add_subplot()\nax.set_facecolor(\"#fff\")\nax.grid(color=\"lightgrey\", alpha=0.7, axis=\"both\", zorder=0)\nsns.barplot(x=topk_authors.values, y=topk_authors.index, ax=ax, zorder=2)\nax.xaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.yaxis.set_tick_params(labelsize=12, size=0, pad=5)\nax.set_ylabel(\"Specie\", fontsize=15, fontweight=\"normal\", labelpad=10)\nax.set_xlabel(\"Count\", fontsize=15, fontweight=\"normal\", labelpad=10)\nhide_spines(ax)\n\nax.set_title(f\"Bottom {k} Categories Distribution\", fontsize=25, fontweight=\"bold\", fontfamily=\"serif\", loc=\"left\", y=1.02)\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-15T19:25:26.761098Z","iopub.execute_input":"2022-02-15T19:25:26.761539Z","iopub.status.idle":"2022-02-15T19:25:28.482545Z","shell.execute_reply.started":"2022-02-15T19:25:26.7615Z","shell.execute_reply":"2022-02-15T19:25:28.481972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"categories = train[\"category\"].unique()\nfor category in categories[::2500]:\n    plot_category_images(train, category=category, rows=2, columns=5, title=category, background_color=\"#fff\")","metadata":{"execution":{"iopub.status.busy":"2022-02-15T19:25:28.483678Z","iopub.execute_input":"2022-02-15T19:25:28.483981Z","iopub.status.idle":"2022-02-15T19:25:47.806158Z","shell.execute_reply.started":"2022-02-15T19:25:28.483953Z","shell.execute_reply":"2022-02-15T19:25:47.805365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Check back soon for more! Work in progress. This aims to be the ultimate starter notebook to this competition for beginners. :) ","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}