{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\"Long-tailed learning, one of the most challenging problems in visual recognition, aims to train well-performing models from a large number of images that follow a long-tailed class distribution.\"\n\nhttps://paperswithcode.com/task/long-tail-learning\n\n

Taming The Long Tail in Machine Learning


\n\nHow to Tame the Long Tail in Machine Learning - By Sasha Harrison on June 29th, 2021\n\n\"Many AI systems rely on supervised learning methods in which neural networks train on labeled data. The challenge with supervised methods is getting models to perform well on examples not adequately represented in the training dataset.\n\n\"Typically, as the frequency of a particular category decreases, so does average model performance on this category. It is often difficult and costly to achieve strong performance on the rare edge cases that make up the long tail of a data distribution. In that blog post, the authors will take a deeper look at how sophisticated data curation tools can help machine learning teams target their experiments toward taming the long tail.\"\n\nhttps://scale.com/blog/taming-long-tail","metadata":{}},{"cell_type":"markdown","source":"![](https://mir-s3-cdn-cf.behance.net/projects/404/b1b3e9110740969.Y3JvcCwxMTkyLDkzMywxMDMsMA.jpg)behance.net","metadata":{}},{"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nimport cv2\nimport matplotlib.pyplot as plt\nimport json\nimport PIL\nimport skimage.io\nimport seaborn as sn\nfrom collections import Counter\nimport tensorflow as tf\nimport collections\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-02-20T22:49:34.096724Z","iopub.execute_input":"2022-02-20T22:49:34.097381Z","iopub.status.idle":"2022-02-20T22:49:41.800859Z","shell.execute_reply.started":"2022-02-20T22:49:34.097283Z","shell.execute_reply":"2022-02-20T22:49:41.799886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Deep Long-Tailed Learning: A Survey\n\n#Authors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n![](https://dingyue.ws.126.net/2021/1228/00f99036j00r4t86i002gd200om00jfg00om00jf.jpg)mln.news\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\nimport keras.backend as K\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.utils import to_categorical, Sequence\n#from keras.utils import to_categorical, Sequence #Deprecated!\nfrom keras.models import Sequential,Model\nfrom keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, Activation,BatchNormalization\n#from keras.optimizers import RMSprop,Adam #Deprecated!\nfrom tensorflow.keras.optimizers import RMSprop\n#from keras.applications import ResNet50 #Deprecated!\nfrom tensorflow.keras.applications import resnet50\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau, ModelCheckpoint, EarlyStopping\nfrom tensorflow.keras.optimizers import Adam","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:49:45.833483Z","iopub.execute_input":"2022-02-20T22:49:45.834009Z","iopub.status.idle":"2022-02-20T22:49:47.254554Z","shell.execute_reply.started":"2022-02-20T22:49:45.833923Z","shell.execute_reply":"2022-02-20T22:49:47.253798Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Long-tail learning via logit adjustment


\n\nLong-tail learning via logit adjustment \n\nAuthors: Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, Sanjiv Kumar\n28 Sept 2020 (modified: 10 Feb 2022)\n\n\"Real-world classification problems typically exhibit an imbalanced or long-tailed label distribution, wherein many labels have only a few associated samples. This poses a challenge for generalisation on such labels, and also makes naive learning biased towards dominant labels.\"\n\n\"In that paper, the authors presented a statistical framework that unifies and generalises several recent proposals to cope with these challenges. Their framework revisited the classic idea of logit adjustment based on the label frequencies, which encourages a large relative margin between logits of rare positive versus dominant negative labels. This yields two techniques for long-tail learning, where such adjustment is either applied post-hoc to a trained model, or enforced in the loss during training.\"\n\n\"Those techniques are statistically grounded, and practically effective on four real-world datasets with long-tailed label distributions.\"\n\nhttps://openreview.net/forum?id=37nvvqkCo5","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nPATH_BASE = \"../input/herbarium-2022-fgvc9/\"\nPATH_TRAIN = os.path.join(PATH_BASE)\nPATH_TRAIN_META = os.path.join(PATH_TRAIN, \"train_metadata.json\")\n\n\nwith open(PATH_TRAIN_META) as json_file:\n metadata = json.load(json_file)","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:49:57.614254Z","iopub.execute_input":"2022-02-20T22:49:57.614858Z","iopub.status.idle":"2022-02-20T22:50:13.292408Z","shell.execute_reply.started":"2022-02-20T22:49:57.614817Z","shell.execute_reply":"2022-02-20T22:50:13.291233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Type of Long-Tailed Learning Methods


\n\n\nTST: Two-Stage Training\n\nIS: Instance Sampling\n\nCBS: Class-Balanced Sampling\n\nCLW: Class-Level Weighting\n\nNC: Normalized Classifier\n\nENS: Ensemble\n\nDA: Data Augmentation","metadata":{}},{"cell_type":"markdown","source":"#First level elements","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nmetadata.keys()","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:50:17.712315Z","iopub.execute_input":"2022-02-20T22:50:17.712613Z","iopub.status.idle":"2022-02-20T22:50:17.721472Z","shell.execute_reply.started":"2022-02-20T22:50:17.712582Z","shell.execute_reply":"2022-02-20T22:50:17.720587Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Deep Long-Tailed Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\n\"Deep long-tailed learning, one of the most challenging problems in visual recognition, aims to train well-performing deep models from a large number of images that follow a long-tailed class distribution. In the last decade, deep learning has emerged as a powerful recognition model for learning high-quality image representations and has led to remarkable breakthroughs in generic visual recognition.\"\n\n\"However, long-tailed class imbalance, a common problem in practical visual recognition tasks, often limits the practicality of deep network based recognition models in real-world applications, since they can be easily biased towards dominant classes and perform poorly on tail classes.\"\n\n\"To address this problem, a large number of studies have been conducted in recent years, making promising progress in the field of deep long-tailed learning. Considering the rapid evolution of this field, this paper aims to provide a comprehensive survey on recent advances in deep long-tailed learning. To be specific, the authors grouped existing deep long-tailed learning studies into three main categories (i.e., class re-balancing, information augmentation and module improvement), and review these methods following this taxonomy in detail.\"\n\n\"Afterward, they empirically analyze several state-of-the-art methods by evaluating to what extent they address the issue of class imbalance via a newly proposed evaluation metric, i.e., relative accuracy. The authors concluded the survey by highlighting important applications of deep long-tailed learning and identifying several promising directions for future research.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"markdown","source":"#Check the number of images and their annotations","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nlen(metadata[\"annotations\"]), len(metadata[\"images\"])","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:50:22.032067Z","iopub.execute_input":"2022-02-20T22:50:22.033004Z","iopub.status.idle":"2022-02-20T22:50:22.038155Z","shell.execute_reply.started":"2022-02-20T22:50:22.032912Z","shell.execute_reply":"2022-02-20T22:50:22.037498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Datasets used in the Deep Long-Tailed Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\"In recent years, a variety of visual datasets have been released for long-tailed learning, differing in tasks, class numbers and sample numbers. The authors summarized nine visual datasets that are widely used in the deep long-tailed learning community.\"\n\n\"In long-tailed image classification, there are four benchmark datasets: ImageNet-LT , CIFAR100-LT , Places-LT , and iNaturalist 2018 . The previous three are sampled from ImageNet , CIFAR100 and Places365 following Pareto distributions, respectively, while iNaturalist is a real-world long-tailed dataset.\"\n\n\"The imbalance ratio of ImageNet-LT, Places-LT and iNaturalist are 256, 996 and 500, respectively; CIFAR100-LT has three variants with various imbalance ratios {10, 50, 100}. In long-tailed object detection and instance segmentation, LVIS , providing precise bounding box and mask annotations, is the widely-used benchmark. In multi-label image classification, the benchmarks are VOC-LT and COCO-LT , which are sampled from PASCAL VOC 2012 and COCO, respectively.\"\n\n\"Recently, a large-scale “untrimmed” video dataset, namely VideoLT , was released for long-tailed video recognition.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"markdown","source":"#First samples from each key","metadata":{}},{"cell_type":"markdown","source":"

Evaluation Metrics in the Deep Long-Tailed Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\"In long-tailed learning, the overall performance on all classes and the performance for head, middle and tail classes are usually reported. The used evaluation metrics differ in various tasks. For example, Top-1 Accuracy (or Error Rate) is the widely-used metric for long-tailed image classification, while mean Average Precision (mAP) is adopted for long-tailed object detection and instance segmentation. Moreover, mAP is also used in long-tailed multilabel image classification as a metric, while video recognition applies both Top-1 Accuracy and mAP for evaluation.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nprint(metadata[\"annotations\"][0])\nprint(metadata[\"images\"][0])\nprint(metadata[\"categories\"][0])\nprint(metadata[\"genera\"][0])\nprint(metadata[\"institutions\"][0])\nprint(metadata[\"distances\"][0])\nprint(metadata[\"license\"][0])","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:50:29.603365Z","iopub.execute_input":"2022-02-20T22:50:29.604023Z","iopub.status.idle":"2022-02-20T22:50:29.61429Z","shell.execute_reply.started":"2022-02-20T22:50:29.603972Z","shell.execute_reply":"2022-02-20T22:50:29.612948Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Calculate the total number of classes","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nlen(set([annotation[\"category_id\"] for annotation in metadata[\"annotations\"]]))","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:50:35.972263Z","iopub.execute_input":"2022-02-20T22:50:35.972742Z","iopub.status.idle":"2022-02-20T22:50:36.071929Z","shell.execute_reply.started":"2022-02-20T22:50:35.972706Z","shell.execute_reply":"2022-02-20T22:50:36.07107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Classic Methods in The Deep Long-Tailed Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\"The authors divided existing deep long-tailed learning methods into three main categories, including class re-balancing,\ninformation augmentation, and module improvement. More specifically, class re-balancing consists of three sub-categories: resampling, cost-sensitive learning (CSL), and logit adjustment (LA).\"\n\n\"Information augmentation comprises transfer learning (TL) and data augmentation (Aug). Module improvement includes representation learning (RL), classifier design (CD), decoupled training (DT) and ensemble learning (Ensemble). According to this taxonomy, the authors sorted out existing deep long-tailed learning methods and will review them in detail.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nids = []\ncategories = []\npaths = []\n\nfor annotation, image in zip(metadata[\"annotations\"], metadata[\"images\"]):\n #assert annotation[\"image_id\"] == image[\"id\"]\n ids.append(image[\"image_id\"])#Read above print metadata samples from each key\n categories.append(annotation[\"category_id\"])\n paths.append(image[\"file_name\"])\n \ndf_meta = pd.DataFrame({\"id\": ids, \"category\": categories, \"path\": paths})","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:50:45.964422Z","iopub.execute_input":"2022-02-20T22:50:45.964708Z","iopub.status.idle":"2022-02-20T22:50:47.084559Z","shell.execute_reply.started":"2022-02-20T22:50:45.964678Z","shell.execute_reply":"2022-02-20T22:50:47.083452Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Logit Adjustment in The Deep Long-Tailed Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\"Logit adjustment, post-hoc shifting the model logits based on label frequencies, is a classic idea to obtain a large relative margin between classes in class-imbalanced problems. Recently, one study comprehensively analyzed logit adjustment methods in long-tailed recognition, and theoretically showed that logit adjustment is Fisher consistent to minimize the average per-class error.\"\n\n\"Following this idea, RoBal applied a post-processing strategy to adjust the cosine classification boundary based on training label frequencies. Instead of using label frequencies of training data, LADE proposed to use the label frequencies of test data (if available) to post-adjust model outputs, so that the trained model can be calibrated for arbitrary test class distribution.\"\n\n\"UNO-IC proposed to use a hyper-parameter, tuned on a balanced meta validation set, to calibrate the model classifier for handling class imbalance, leading to better performance on the uniform test set. De-confound introduced a causal classifier that records the bias information by computing the exponential moving average of features during training, and then removes the bad causal effect by subtracting the bias information during inference. DisAlign applied an adaptive calibration function for logit adjustment, where the calibration function is learned by matching the calibrated prediction distribution to a relatively balanced class distribution.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ndf_meta","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:50:55.483229Z","iopub.execute_input":"2022-02-20T22:50:55.48356Z","iopub.status.idle":"2022-02-20T22:50:55.510438Z","shell.execute_reply.started":"2022-02-20T22:50:55.483521Z","shell.execute_reply":"2022-02-20T22:50:55.509537Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Information Augmentation in The Deep Long-Tailed Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\"Information augmentation based methods seek to introduce additional information into model training, so that the model\nperformance can be improved in long-tailed learning. There are two kinds of methods in this method type: transfer learning and data augmentation.\n\nTRASFER LEARNING\n\n\"Transfer learning seeks to transfer the knowledge from a source domain (e.g., datasets, tasks or classes)\nto enhance model training on a target domain. In deep long-tailed learning, there are four main transfer learning schemes, i.e., head-to tail knowledge transfer, model pre-training, knowledge distillation, and self-training.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"markdown","source":"#Classes distribution","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ndf_meta[\"category\"].value_counts()","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-02-20T22:51:02.494002Z","iopub.execute_input":"2022-02-20T22:51:02.49444Z","iopub.status.idle":"2022-02-20T22:51:02.525001Z","shell.execute_reply.started":"2022-02-20T22:51:02.494407Z","shell.execute_reply":"2022-02-20T22:51:02.524136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Data Augmentation in The Deep Long-Tailed Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\"Data Augmentation aims to pack a set of augmentation techniques to enhance the size and quality of datasets for model training.\"\n\n\"In long-tailed learning, there are two kinds of data augmentation methods having been explored, including transfer-based augmentation and conventional (non-transfer) augmentation.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf\n","metadata":{}},{"cell_type":"markdown","source":"#Change order (original code) to Genus","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nd_categories = {category[\"category_id\"]: category[\"scientificName\"] for category in metadata[\"categories\"]}\nd_families = {category[\"category_id\"]: category[\"family\"] for category in metadata[\"categories\"]}\nd_genus = {category[\"category_id\"]: category[\"genus\"] for category in metadata[\"categories\"]}\nd_species = {category[\"category_id\"]: category[\"species\"] for category in metadata[\"categories\"]}\n\ndf_meta[\"category_name\"] = df_meta[\"category\"].map(d_categories)\ndf_meta[\"family_name\"] = df_meta[\"category\"].map(d_families)\ndf_meta[\"genus_name\"] = df_meta[\"category\"].map(d_genus)\ndf_meta[\"species_name\"] = df_meta[\"category\"].map(d_species)\ndf_meta","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:52:03.192364Z","iopub.execute_input":"2022-02-20T22:52:03.192743Z","iopub.status.idle":"2022-02-20T22:52:03.387622Z","shell.execute_reply.started":"2022-02-20T22:52:03.192707Z","shell.execute_reply":"2022-02-20T22:52:03.386466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Module Improvement in The Deep Long-Tailed Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\"Besides class re-balancing and information augmentation, researchers also explored methods to improve network modules\nin long-tailed learning. These methods can be divided into four categories: (1) representation learning improves the feature extractor; (2) classifier design enhances the model classifier; (3) decoupled training boosts the learning of both the feature extractor and the classifier; (4) ensemble learning improves the whole architecture.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ndef visualize_train_batch(paths, categories, families, genus, species):\n plt.figure(figsize=(16, 16))\n \n for ind, info in enumerate(zip(paths, categories, families, genus, species)):\n path, category, family, genus, species = info\n \n plt.subplot(2, 3, ind + 1)\n \n image = cv2.imread(os.path.join(\"../input/herbarium-2022-fgvc9/train_images\", path))\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n plt.imshow(image)\n \n plt.title(\n f\"FAMILY: {family} GENUS: {genus}\\n{species}\", \n fontsize=10,\n )\n plt.axis(\"off\")\n \n plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:21:39.335395Z","iopub.execute_input":"2022-02-20T23:21:39.335744Z","iopub.status.idle":"2022-02-20T23:21:39.344305Z","shell.execute_reply.started":"2022-02-20T23:21:39.335707Z","shell.execute_reply":"2022-02-20T23:21:39.343348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Representation Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\n\"Existing representation learning methods for long-tailed learning are based on four main paradigms, i.e., metric learning, sequential training, prototype learning, and transfer learning.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ndef visualize_by_id(df, _id=None):\n tmp = df.sample(6)\n if _id is not None:\n tmp = df[df[\"category\"] == _id].sample(6)\n\n visualize_train_batch(\n tmp[\"path\"].tolist(), \n tmp[\"category_name\"].tolist(),\n tmp[\"family_name\"].tolist(),\n tmp[\"genus_name\"].tolist(),\n tmp[\"species_name\"].tolist(),\n )","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:21:47.345914Z","iopub.execute_input":"2022-02-20T23:21:47.346311Z","iopub.status.idle":"2022-02-20T23:21:47.353568Z","shell.execute_reply.started":"2022-02-20T23:21:47.346273Z","shell.execute_reply":"2022-02-20T23:21:47.352401Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Decoupled Training and Ensemble Learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\nDECOUPLED TRAINING\n\n\"Decoupled training decouples the learning procedure into representation learning and classifier training.\"\n\n\"Decoupling was the pioneering work to introduce the two-stage training scheme. It empirically evaluated different\nsampling strategies for representation learning in the first stage, and then evaluated different classifier training\nschemes by fixing the trained feature extractor in the second stage. In the classifier learning stage, there are also four methods, including classifier re-training with class-balanced sampling, the nearest class mean classifier, the τ -normalized classifier, and a learnable weight scaling scheme.\" \n\nENSEMBLE LEARNING\n\n\"Ensemble learning based methods strategically generate and combine multiple network modules (namely, multiple experts)\nto solve long-tailed visual learning problems.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nvisualize_by_id(df_meta, 0) #zero is taken from df_meta category column","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:24:59.670064Z","iopub.execute_input":"2022-02-20T23:24:59.670364Z","iopub.status.idle":"2022-02-20T23:25:01.171459Z","shell.execute_reply.started":"2022-02-20T23:24:59.670331Z","shell.execute_reply":"2022-02-20T23:25:01.170733Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Image Classification


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\n\"The most common application of long-tailed learning is multiclass classification. There are many artificially sampled long-tailed datasets from widely-used image classification datasets, i.e., ImageNet, CIFAR, and Places. Based on these datasets, various long-tailed learning methods have been proposed.\"\n\n\"Besides these artificial tasks, long-tailed learning is also applied to real-world image classification tasks, including species classification, face recognition, age classification, logo detection, rail surface defect detection and medical image diagnosis.\"\n\n\"In addition to multi-class classification, long-tailed learning is also applied to multi-label classification based on both artificial tasks (i.e., VOC-LT and COCO-LT) and real-world tasks, including web image classification, face attribute classification and cloth attribute classification.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nvisualize_by_id(df_meta, 4)","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:26:42.297148Z","iopub.execute_input":"2022-02-20T23:26:42.297872Z","iopub.status.idle":"2022-02-20T23:26:43.760078Z","shell.execute_reply.started":"2022-02-20T23:26:42.297835Z","shell.execute_reply":"2022-02-20T23:26:43.759127Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Image Detection and Segmentation


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\n\"Object detection and instance segmentation has attracted increasing attention in the long-tailed learning community, where most existing studies are conducted based on LVIS and COCO. In addition to these widely used benchmarks, many other applications have also been explored, including urban scene understanding, unmanned aerial vehicle detection, point cloud segmentation\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nvisualize_by_id(df_meta, 15504) #15504 is taken from df_meta category column","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:24:25.450094Z","iopub.execute_input":"2022-02-20T23:24:25.450381Z","iopub.status.idle":"2022-02-20T23:24:27.632496Z","shell.execute_reply.started":"2022-02-20T23:24:25.450352Z","shell.execute_reply":"2022-02-20T23:24:27.631461Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Class-incremental and Multi-domain long-tailed learning


\n\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\n\nCLASS INCREMENTAL LONG_TAILED LEARNING\n\n\"In real-world applications, long-tailed data may come in a continual and class incremental manner. To deal with this scenario, class incremental long-tailed learning aims to learn deep models from class-incremental long-tailed data, suffering two key challenges: (1)how to handle long-tailed class imbalance when different classes come sequentially, and the model has no information about the future input regarding classes as well as label frequencies; (2) how to overcome catastrophic forgetting of previous class knowledge when learning new classes.Such a task setting can also be named continual long-tailed learning.\"\n\nMULTI-DOMAIN LONG TAILED LEARNING\n\n\"Current long-tailed methods generally assume that all long-tailed samples come from the same data marginal distribution. However, in practice, long-tailed data may also get from different domains with distinct data distributions, e.g., the DomainNet dataset.\"\n\n\"Motivated by this, multi-domain long-tailed learning seeks to handle both class imbalance and domain distribution shift, simultaneously. One more challenging issue may be the inconsistency of class imbalance among different domains. In other words, various domains may have different class distributions, which further enlarges the domain shift in multi-domain long-tailed learning.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"markdown","source":"#Random samples","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nvisualize_by_id(df_meta)","metadata":{"_kg_hide-output":false,"execution":{"iopub.status.busy":"2022-02-20T23:21:52.166478Z","iopub.execute_input":"2022-02-20T23:21:52.167192Z","iopub.status.idle":"2022-02-20T23:21:53.680806Z","shell.execute_reply.started":"2022-02-20T23:21:52.16714Z","shell.execute_reply":"2022-02-20T23:21:53.679456Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Robust long-tailed learning


\n\nDeep Long-Tailed Learning: A Survey\n\nAuthors: Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng\n\nROBUST LONG_TAILED LEARNING\n\n\"Real-world long-tailed samples may also suffer image noise or label noise. Most long-tailed methods, however, assume all images and labels are clean, leading to poor model robustness in practical applications. This issue would be particularly severe for tail classes, as they have very limited training samples. Inspired by this, robust long-tailed learning seeks to handle class imbalance and improve model robustness, simultaneously.\"\n\n

Long-tailed Regression


\n\nLONG-TAILED REGRESSION\n\n\"Most existing studies of long-tailed visual learning focus on classification, detection and segmentation, which have discrete labels with class indices. However, many tasks involve continuous labels, where hard classification boundaries among classes do not exist. Motivated by this, long-tailed regression aims to deal with long-tailed learning with continuous label space. In such a task, how to simultaneously resolve long-tailed class imbalance and handle potential missing data for certain labels remains an open question.\"\n\n

Long-tailed Video Learning


\n\nLONG-TAILED VIDEO LEARNING.\n\n\"Most existing deep long-tailed learning studies focus on the image level, but ignore that the video domain also suffers the issue of long-tail class imbalance. Considering the additional temporal dimension in video data, long-tailed video learning should be more difficult than long-tailed image learning. Thanks to the recent release of a VideoLT dataset, long-tailed video learning can be explored in the near future.\"\n\nhttps://arxiv.org/pdf/2110.04596.pdf","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ndf_submission = pd.read_csv(\n \"../input/herbarium-2022-fgvc9/sample_submission.csv\",\n index_col=0,\n)","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:31:14.859813Z","iopub.execute_input":"2022-02-20T23:31:14.860816Z","iopub.status.idle":"2022-02-20T23:31:15.007614Z","shell.execute_reply.started":"2022-02-20T23:31:14.860767Z","shell.execute_reply":"2022-02-20T23:31:15.006667Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#One of the most frequently class from train data????? I don't know. No clue about that. Better ask the original author.","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ndf_submission[\"Predicted\"] = 25229","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:31:19.006007Z","iopub.execute_input":"2022-02-20T23:31:19.006727Z","iopub.status.idle":"2022-02-20T23:31:19.013342Z","shell.execute_reply.started":"2022-02-20T23:31:19.006679Z","shell.execute_reply":"2022-02-20T23:31:19.012094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Long-tailed Learning Papers


\n\nLEARNING FROM LONG-TAILED DATA WITH NOISY LABELS\nhttps://arxiv.org/pdf/2108.11096.pdf\n\nSelf-supervised Learning is More Robust to Dataset Imbalance\nhttps://arxiv.org/pdf/2110.05025.pdf\n\nAdaptive Logit Adjustment Loss for Long-Tailed Visual Recognition\nhttps://arxiv.org/pdf/2104.06094.pdf\n\nBalanced Knowledge Distillation for Long-tailed Learning\nhttps://arxiv.org/pdf/2104.10510.pdf\n\nAdversarial Robustness under Long-Tailed Distribution\nhttps://arxiv.org/pdf/2104.02703.pdf\n\nLearning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification\nhttps://arxiv.org/pdf/2001.01536.pdf\n\nDECOUPLING REPRESENTATION AND CLASSIFIER FOR LONG-TAILED RECOGNITION\nhttps://openreview.net/pdf?id=r1gRTCVFvB\n\nBBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition\nhttps://openaccess.thecvf.com/content_CVPR_2020/papers/Zhou_BBN_Bilateral-Branch_Network_With_Cumulative_Learning_for_Long-Tailed_Visual_Recognition_CVPR_2020_paper.pdf\n\n\niNaturalist 2018 Competition\nhttps://github.com/visipedia/inat_comp/tree/master/2018\n\nDeep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspective\nhttps://openaccess.thecvf.com/content_CVPR_2020/papers/Liu_Deep_Representation_Learning_on_Long-Tailed_Data_A_Learnable_Embedding_Augmentation_CVPR_2020_paper.pdf\n\n\nThe Devil is in the Tails: Fine-grained Classification in the Wild\nhttps://arxiv.org/pdf/1709.01450.pdf\n\nSource: https://github.com/Stomach-ache/awesome-long-tailed-learning","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ndf_submission.to_csv(\"submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:31:23.297997Z","iopub.execute_input":"2022-02-20T23:31:23.298302Z","iopub.status.idle":"2022-02-20T23:31:23.785374Z","shell.execute_reply.started":"2022-02-20T23:31:23.298269Z","shell.execute_reply":"2022-02-20T23:31:23.784286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\npd.read_csv(\"submission.csv\", index_col=0)","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:31:27.433098Z","iopub.execute_input":"2022-02-20T23:31:27.433415Z","iopub.status.idle":"2022-02-20T23:31:27.545322Z","shell.execute_reply.started":"2022-02-20T23:31:27.433372Z","shell.execute_reply":"2022-02-20T23:31:27.544163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nFULL_PIPELINE = False","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:31:32.276894Z","iopub.execute_input":"2022-02-20T23:31:32.27721Z","iopub.status.idle":"2022-02-20T23:31:32.281499Z","shell.execute_reply.started":"2022-02-20T23:31:32.277176Z","shell.execute_reply":"2022-02-20T23:31:32.280539Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import random\nfrom numpy import save, load\nimport albumentations as A\nfrom albumentations import pytorch as ATorch\nimport torch\nfrom torch.utils import data as torch_data\nfrom torch import nn as torch_nn\nfrom torch.nn import functional as torch_functional\nimport torchvision\nfrom tqdm import tqdm\nfrom sklearn.metrics.pairwise import euclidean_distances","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:31:47.611393Z","iopub.execute_input":"2022-02-20T23:31:47.611656Z","iopub.status.idle":"2022-02-20T23:31:50.234453Z","shell.execute_reply.started":"2022-02-20T23:31:47.611627Z","shell.execute_reply":"2022-02-20T23:31:50.233416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nclass MobileNetV2(torch.nn.Module):\n def __init__(self):\n super().__init__()\n tmp_net = torch.hub.load(\n \"pytorch/vision:v0.6.0\", \"mobilenet_v2\", pretrained=True\n )\n self.net = torch_nn.Sequential(*(list(tmp_net.children())[:-1]))\n\n def forward(self, x):\n return self.net(x)","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:31:52.651635Z","iopub.execute_input":"2022-02-20T23:31:52.651993Z","iopub.status.idle":"2022-02-20T23:31:52.658729Z","shell.execute_reply.started":"2022-02-20T23:31:52.651928Z","shell.execute_reply":"2022-02-20T23:31:52.657523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nclass DataRetriever(torch_data.Dataset):\n def __init__(\n self, \n paths, \n categories=None,\n transforms=None,\n base_path=PATH_TRAIN\n ):\n self.paths = paths\n self.categories = categories\n self.transforms = transforms\n self.base_path = base_path\n \n def __len__(self):\n return len(self.paths)\n \n def __getitem__(self, index):\n img = cv2.imread(os.path.join(self.base_path, self.paths[index]))\n img = cv2.resize(img, (224, 224))\n img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n \n if self.transforms:\n img = self.transforms(image=img)[\"image\"]\n \n if self.categories is None:\n return img\n \n y = self.categories[index] \n return img, y\n \n \ndef get_transforms():\n return A.Compose(\n [\n A.Normalize(\n mean=[0.485, 0.456, 0.406], \n std=[0.229, 0.224, 0.225], \n p=1.0\n ),\n ATorch.transforms.ToTensorV2(p=1.0),\n ], \n p=1.0\n )","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:31:57.508927Z","iopub.execute_input":"2022-02-20T23:31:57.509251Z","iopub.status.idle":"2022-02-20T23:31:57.522017Z","shell.execute_reply.started":"2022-02-20T23:31:57.509214Z","shell.execute_reply":"2022-02-20T23:31:57.521264Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Take for each category (target) all images from the train set and after processing average their vectors","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ndf_train = df_meta[[\"category\", \"path\"]].sort_values(by=\"category\")\n\ndf_train","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:32:03.990719Z","iopub.execute_input":"2022-02-20T23:32:03.991196Z","iopub.status.idle":"2022-02-20T23:32:04.160937Z","shell.execute_reply.started":"2022-02-20T23:32:03.991157Z","shell.execute_reply":"2022-02-20T23:32:04.160106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ntmp_path = df_train[\"path\"].tolist()\ntmp_category = df_train[\"category\"].tolist()\n# If FULL_PIPELINE is False we use small subset of data\nif not FULL_PIPELINE:\n tmp_path = tmp_path[:256 * 8]\n tmp_category = tmp_category[:256 * 8]\n\ntrain_data_retriever = DataRetriever(\n tmp_path,\n tmp_category,\n transforms=get_transforms(),\n)\n\ntrain_loader = torch_data.DataLoader(\n train_data_retriever,\n batch_size=256,\n shuffle=False,\n num_workers=8,\n)","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:32:08.951981Z","iopub.execute_input":"2022-02-20T23:32:08.952554Z","iopub.status.idle":"2022-02-20T23:32:09.037032Z","shell.execute_reply.started":"2022-02-20T23:32:08.952442Z","shell.execute_reply":"2022-02-20T23:32:09.035926Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Model","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\nmodel = MobileNetV2()\nmodel.to(device)\nmodel.eval();","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:32:13.771455Z","iopub.execute_input":"2022-02-20T23:32:13.771748Z","iopub.status.idle":"2022-02-20T23:32:15.525198Z","shell.execute_reply.started":"2022-02-20T23:32:13.771716Z","shell.execute_reply":"2022-02-20T23:32:15.524281Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\ncategory_counts = collections.Counter(df_train[\"category\"].tolist())","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:32:19.550381Z","iopub.execute_input":"2022-02-20T23:32:19.550656Z","iopub.status.idle":"2022-02-20T23:32:19.658332Z","shell.execute_reply.started":"2022-02-20T23:32:19.550627Z","shell.execute_reply":"2022-02-20T23:32:19.657249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nfinal_vectors = np.zeros((len(category_counts), 1280))\n\nwith torch.no_grad():\n for batch in tqdm(train_loader):\n X, y = batch\n vectors = model(X.to(device)).mean(axis=(2, 3))\n \n _y = y.numpy().tolist()\n for ind in range(len(_y)):\n final_vectors[_y[ind]] += vectors[ind].cpu().numpy().copy() / category_counts[_y[ind]]","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-02-20T23:32:26.27095Z","iopub.execute_input":"2022-02-20T23:32:26.271557Z","iopub.status.idle":"2022-02-20T23:32:26.787316Z","shell.execute_reply.started":"2022-02-20T23:32:26.27151Z","shell.execute_reply":"2022-02-20T23:32:26.783441Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#cv2.error: OpenCV(4.5.4) /tmp/pip-req-build-21t5esfk/opencv/modules/imgproc/src/resize.cpp:4051: error: (-215:Assertion failed) !ssize.empty() in function 'resize'","metadata":{}},{"cell_type":"markdown","source":"#Save and load category vectors. You can pretrain them. I obviously don't can.","metadata":{}},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nsave(\"average_vectors.npy\", final_vectors)","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:33:36.071071Z","iopub.execute_input":"2022-02-20T23:33:36.071606Z","iopub.status.idle":"2022-02-20T23:33:36.249746Z","shell.execute_reply.started":"2022-02-20T23:33:36.071562Z","shell.execute_reply":"2022-02-20T23:33:36.248422Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Yaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook\n\nfinal_vectors = load(\"average_vectors.npy\")","metadata":{"execution":{"iopub.status.busy":"2022-02-20T23:33:40.780693Z","iopub.execute_input":"2022-02-20T23:33:40.781058Z","iopub.status.idle":"2022-02-20T23:33:40.86507Z","shell.execute_reply.started":"2022-02-20T23:33:40.780998Z","shell.execute_reply":"2022-02-20T23:33:40.864063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#After another TypeError: list indices must be integers or slices, not str. I gave up","metadata":{}},{"cell_type":"markdown","source":"#Acknowledgement:\n\nYaroslav Isaienkov https://www.kaggle.com/ihelon/herbarium-2021-exploratory-data-analysis/notebook","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-02-20T22:25:28.696889Z","iopub.execute_input":"2022-02-20T22:25:28.697226Z","iopub.status.idle":"2022-02-20T22:25:28.746255Z","shell.execute_reply.started":"2022-02-20T22:25:28.697193Z","shell.execute_reply":"2022-02-20T22:25:28.745152Z"}}},{"cell_type":"markdown","source":"![](https://miro.medium.com/max/1400/1*oBjI7KW-BlVND9XbFns0tA.png)medium.com","metadata":{"execution":{"iopub.status.busy":"2022-02-20T22:22:54.135156Z","iopub.execute_input":"2022-02-20T22:22:54.135463Z","iopub.status.idle":"2022-02-20T22:22:54.276647Z","shell.execute_reply.started":"2022-02-20T22:22:54.135433Z","shell.execute_reply":"2022-02-20T22:22:54.27557Z"}}},{"cell_type":"markdown","source":"#Indeed, a fight for time and money. The Long-tailed is a Beast? I'm so screwed!","metadata":{}}]}