{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Taking the [public solution by ks2019](https://www.kaggle.com/code/ks2019/happywhale-arcface-baseline-tpu) from a private lb score of 0.470 to 0.804.\n\nA summary of all key changes taken from [my discussion thread](https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/319787).\n\n- Image size: 864x846.\n- Pretrained model: EfficientNet b5 with noisy student.\n- Pooling: concat pooling (from fastai)\n\n    ```\n    avg_pool = tf.keras.layers.GlobalAveragePooling2D()(x)\n    max_pool = tf.keras.layers.GlobalMaxPooling2D()(x)\n    pretrained_out = tf.keras.layers.Concatenate()([avg_pool, max_pool])\n    ```\n\n- Head:\n  - Dual-head model: Species classification output and metric.\n  - Add [Multi-Sample Dropout](https://arxiv.org/abs/1905.09788) (idea thanks to @dhakshiin) before metric.\n\n- Metric function: [Elastic Margin Loss](https://arxiv.org/abs/2109.09416) (w=0.3, std=0.025, s=30) (minor improvement over ArcFace).\n- Embed size: 1024.\n- Data: entire dataset with 12157 extra pseudo labels. Pseudo label algorithm by @dhakshiin: Use multiple models, find top-1 examples that the majority agree on, ensuring 0.2 separation between confidence of 1st class and 2nd class.\n- Crops: Select between 8x crops randomly selected at training time: full body crops, detic crops, original yolov5 crops, 3x TokenCut crops and full-sized images ([See my TFRecords notebook here](https://www.kaggle.com/lextoumbourou/happywhale-generate-tfrecords-with-pseudo))\n- Augmentations: `random_flip_left_right`, `random_hue`, `random_saturation`, `random_contrast`, `random_brightness` and `random_rgb_to_gray` (parameters tuned slightly from original kernel)\n- Optimiser: Adam (unchanged from original)\n- Learning Rate: exponential decay with 4 epochs warm up (unchanged from original)\n- Epochs: 30\n\n- Inference:\n  - Generated embeddings for each crop with a horizontal flip.\n  - Take a weighted mean of all crop embeddings to create final embeddings.\n  - Inference: standard KNN inference (use KNN=150 when added to final ensemble)","metadata":{"execution":{"iopub.execute_input":"2022-02-09T10:49:44.080928Z","iopub.status.busy":"2022-02-09T10:49:44.080598Z","iopub.status.idle":"2022-02-09T10:49:56.003075Z","shell.execute_reply":"2022-02-09T10:49:56.002155Z","shell.execute_reply.started":"2022-02-09T10:49:44.080867Z"},"papermill":{"duration":0.054007,"end_time":"2022-04-11T13:35:31.010608","exception":false,"start_time":"2022-04-11T13:35:30.956601","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# Install Drivers\n\nThis setup code was shared in [this discussion thread](https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/315363) by [Andrij](https://www.kaggle.com/aikhmelnytskyy).","metadata":{}},{"cell_type":"code","source":"import os\nis_kaggle = os.environ.get('KAGGLE_KERNEL_RUN_TYPE', '')\nif is_kaggle:\n    print('Running in Kaggle Kernels')\n    from kaggle_secrets import UserSecretsClient\n    user_secrets = UserSecretsClient()\n    wandb_creds = user_secrets.get_secret(\"wandb\")\n    \n    !pip3 install -Uq tensorflow==2.7\n    print(\"update TPU server tensorflow version…\")\n\n    !pip install -q cloud-tpu-client\n    import tensorflow as tf\n    from cloud_tpu_client import Client\n    print(tf.version)\n    Client().configure_tpu_version(2.7, restart_type='ifNeeded')\n\n    !pip install -Uq tensorflow-gcs-config==2.7\n\n    !pip install wandb > /dev/null\n    !pip install -q efficientnet > /dev/null\n    !pip install tensorflow_addons > /dev/null\n    \n    \n    !wandb login {wandb_creds}","metadata":{"papermill":{"duration":128.714774,"end_time":"2022-04-11T13:37:39.892901","exception":false,"start_time":"2022-04-11T13:35:31.178127","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:25:12.371478Z","iopub.execute_input":"2022-04-19T03:25:12.371923Z","iopub.status.idle":"2022-04-19T03:27:33.029833Z","shell.execute_reply.started":"2022-04-19T03:25:12.371853Z","shell.execute_reply":"2022-04-19T03:27:33.028135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"import re\nimport math\nimport random\nimport pickle\nimport json\nfrom datetime import datetime\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nimport tensorflow as tf\nimport efficientnet.tfkeras as efn\nfrom sklearn import metrics\nfrom sklearn.model_selection import KFold, train_test_split\nfrom tensorflow.keras import backend as K\nimport tensorflow_addons as tfa\nfrom tqdm.auto import tqdm\nimport matplotlib.pyplot as plt\nimport wandb","metadata":{"papermill":{"duration":1.141798,"end_time":"2022-04-11T13:37:41.089454","exception":false,"start_time":"2022-04-11T13:37:39.947656","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:26.196141Z","iopub.execute_input":"2022-04-19T03:28:26.196526Z","iopub.status.idle":"2022-04-19T03:28:27.823811Z","shell.execute_reply.started":"2022-04-19T03:28:26.196478Z","shell.execute_reply":"2022-04-19T03:28:27.82237Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Setup TPU","metadata":{}},{"cell_type":"code","source":"def get_strategy(tpu_arg):\n    try:\n        tpu = tf.distribute.cluster_resolver.TPUClusterResolver(tpu=tpu_arg)\n        print('Running on TPU ', tpu.master())\n    except ValueError as e:\n        print(f'No TPU: {e}')\n        tpu = None\n\n    if tpu:\n        tf.config.experimental_connect_to_cluster(tpu)\n        tf.tpu.experimental.initialize_tpu_system(tpu)\n        strategy = tf.distribute.TPUStrategy(tpu)\n    else:\n        # Default distribution strategy in Tensorflow. Works on CPU and single GPU.\n        strategy = tf.distribute.get_strategy()\n        \n    return strategy","metadata":{"papermill":{"duration":0.065462,"end_time":"2022-04-11T13:37:41.210745","exception":false,"start_time":"2022-04-11T13:37:41.145283","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:27.827489Z","iopub.execute_input":"2022-04-19T03:28:27.827837Z","iopub.status.idle":"2022-04-19T03:28:27.837281Z","shell.execute_reply.started":"2022-04-19T03:28:27.8278Z","shell.execute_reply":"2022-04-19T03:28:27.835864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tpu_arg = None\nif not is_kaggle:\n    tpu_arg = 'local'\n    \nstrategy = get_strategy(tpu_arg)\nAUTO = tf.data.experimental.AUTOTUNE\nprint(\"REPLICAS: \", strategy.num_replicas_in_sync)","metadata":{"papermill":{"duration":8.509785,"end_time":"2022-04-11T13:37:49.774649","exception":false,"start_time":"2022-04-11T13:37:41.264864","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:27.838887Z","iopub.execute_input":"2022-04-19T03:28:27.839283Z","iopub.status.idle":"2022-04-19T03:28:36.25638Z","shell.execute_reply.started":"2022-04-19T03:28:27.839215Z","shell.execute_reply":"2022-04-19T03:28:36.254865Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Config and WanDB\n\nI used [Weights & Biases](https://wandb.ai/site) for tracking experiment runs.","metadata":{"papermill":{"duration":0.05554,"end_time":"2022-04-11T13:37:49.886989","exception":false,"start_time":"2022-04-11T13:37:49.831449","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class config:\n    \n    SEED = 420\n    # Trained on the full dataset.\n    FOLD_TO_RUN = None\n    VAL_FOLD_TO_RUN = 0\n    FOLDS = 5\n    DEBUG = False\n    EVALUATE = True\n    RESUME = False\n    RESUME_EPOCH = None\n\n    BATCH_SIZE = 16 * strategy.num_replicas_in_sync\n    IMAGE_SIZE = 864\n    N_CLASSES = 15587\n\n    model_type = 'effnetv1'  \n    EFF_NET = 5\n\n    head = 'arcface'\n\n    EPOCHS = 30\n    EPOCHS_STAGE_2 = 0\n\n    LR = 0.001\n\n    RGB_TO_GRAY_PROB = 0.15\n\n    WEIGHT_METRIC = 0.5\n    WEIGHT_SPECIES = 0.5\n    \n    save_dir = './output'\n    \n    KNN = 100\n    \n    MIN_JPEG_QUAL_AUG = 75\n    MAX_JPEG_QUAL_AUG = 100\n    \n    FULLBODY_CROP_PROB = 0.2\n\n    DETIC_CROP_PROB = 0.14\n    YOLOV5_CROP_PROB = 0.14\n    \n    VIT_BASE_CROP_PROB = 0.14\n    VIT_SMALL_CROP_PROB = 0.14\n    VIT_MOCO_CROP_PROB = 0.14\n    \n    RANDOM_HUE_MAX_DELTA = 0.1\n\n    RANDOM_SATURATION_LOWER = 0.75\n    RANDOM_SATURATION_UPPER = 1.25\n    \n    RANDOM_CONTRAST_LOWER = 0.75\n    RANDOM_CONTRAST_UPPER = 1.25\n    \n    RANDOM_BRIGHTNESS_MAX_DELTA = 0.1\n\n    NONE_CROP_PROB = 0.1\n    \n    ARC_FACE_M = 0.3\n    \n    EMB_DIM = 1024\n\n    LOAD_WEIGHTS_STAGE_2 = None\n    LOAD_WEIGHTS = None\n    INF_LOAD_WEIGHTS = None\n    \n    GCS_DS_PATH = 'happywhale-tfrecords-private2'\n    GCS_PSEUDO_DS_PATH = 'happywhale-tfrecords-pseudo-dhak'\n    \n    INPUT_DATA_PATH = Path('../input/happy-whale-and-dolphin/')\n    LABEL_INFO_PATH = Path('../input/happywhale-generate-tfrecords-with-pseudo/')\n    \n    GCS_UPLOAD_BUCKET = None\n    \n    CROP_WEIGHTS = {\n        'fullbody': 10,\n        'detic': 1,\n        'yolov5': 1,\n        'vit_base': 1,\n        'vit_small': 1,\n        'vit_moco': 1,\n        'no_crop': 2,\n    }\n\n    \ndef count_data_items(filenames):\n    n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) \n         for filename in filenames]\n    return np.sum(n)\n\ndef seed_everything(seed):\n    random.seed(seed)\n    np.random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    tf.random.set_seed(seed)\n    \ndef is_interactive():\n    return 'runtime'    in get_ipython().config.IPKernelApp.connection_file\nIS_INTERACTIVE = is_interactive()\nprint(IS_INTERACTIVE)\n\nMODEL_NAME = None\nif config.model_type == 'effnetv1':\n    MODEL_NAME = f'effnetv1_b{config.EFF_NET}'\nelif config.model_type == 'effnetv2':\n    MODEL_NAME = f'effnetv2_{config.EFF_NETV2}'\n\nconfig.MODEL_NAME = MODEL_NAME\nprint(MODEL_NAME)\n\nif is_kaggle:\n    print('Loading GCS path from KaggleDatasets')\n    from kaggle_secrets import UserSecretsClient\n    from kaggle_datasets import KaggleDatasets\n\n    user_secrets = UserSecretsClient()\n    user_credential = user_secrets.get_gcloud_credential()\n    user_secrets.set_tensorflow_credential(user_credential)\n\n    config.GCS_DS_PATH = KaggleDatasets().get_gcs_path(config.GCS_DS_PATH)\n    if config.GCS_PSEUDO_DS_PATH:\n        config.GCS_PSEUDO_DS_PATH = KaggleDatasets().get_gcs_path(config.GCS_PSEUDO_DS_PATH)","metadata":{"papermill":{"duration":1.810965,"end_time":"2022-04-11T13:37:51.753393","exception":false,"start_time":"2022-04-11T13:37:49.942428","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:36.258554Z","iopub.execute_input":"2022-04-19T03:28:36.259128Z","iopub.status.idle":"2022-04-19T03:28:37.255129Z","shell.execute_reply.started":"2022-04-19T03:28:36.25909Z","shell.execute_reply":"2022-04-19T03:28:37.254126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'Doing fold {config.FOLD_TO_RUN} (total fold: {config.FOLDS})')","metadata":{"papermill":{"duration":0.064858,"end_time":"2022-04-11T13:37:51.874402","exception":false,"start_time":"2022-04-11T13:37:51.809544","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:37.256704Z","iopub.execute_input":"2022-04-19T03:28:37.256996Z","iopub.status.idle":"2022-04-19T03:28:37.263102Z","shell.execute_reply.started":"2022-04-19T03:28:37.256962Z","shell.execute_reply":"2022-04-19T03:28:37.262358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"OUTPUT_DIR = Path(config.save_dir) / datetime.now().strftime('%Y%m%d-%H%M%S')","metadata":{"papermill":{"duration":0.063082,"end_time":"2022-04-11T13:37:51.993281","exception":false,"start_time":"2022-04-11T13:37:51.930199","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:37.26422Z","iopub.execute_input":"2022-04-19T03:28:37.264754Z","iopub.status.idle":"2022-04-19T03:28:37.275324Z","shell.execute_reply.started":"2022-04-19T03:28:37.26472Z","shell.execute_reply":"2022-04-19T03:28:37.274367Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(OUTPUT_DIR)","metadata":{"papermill":{"duration":0.06464,"end_time":"2022-04-11T13:37:52.113144","exception":false,"start_time":"2022-04-11T13:37:52.048504","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:37.2768Z","iopub.execute_input":"2022-04-19T03:28:37.277378Z","iopub.status.idle":"2022-04-19T03:28:37.289485Z","shell.execute_reply.started":"2022-04-19T03:28:37.277337Z","shell.execute_reply":"2022-04-19T03:28:37.288606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"OUTPUT_DIR.mkdir(exist_ok=True, parents=True)","metadata":{"papermill":{"duration":0.069037,"end_time":"2022-04-11T13:37:52.238628","exception":false,"start_time":"2022-04-11T13:37:52.169591","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:37.29095Z","iopub.execute_input":"2022-04-19T03:28:37.291911Z","iopub.status.idle":"2022-04-19T03:28:37.302867Z","shell.execute_reply.started":"2022-04-19T03:28:37.291814Z","shell.execute_reply":"2022-04-19T03:28:37.301786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if is_kaggle == 'Interactive':\n    print('Wandb in offline mode.')\n    os.environ['WANDB_MODE'] = 'offline'","metadata":{"papermill":{"duration":0.063478,"end_time":"2022-04-11T13:37:52.470427","exception":false,"start_time":"2022-04-11T13:37:52.406949","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:37.304362Z","iopub.execute_input":"2022-04-19T03:28:37.305183Z","iopub.status.idle":"2022-04-19T03:28:37.318168Z","shell.execute_reply.started":"2022-04-19T03:28:37.305137Z","shell.execute_reply":"2022-04-19T03:28:37.317165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wandb.config = {\n    'seed': config.SEED,\n    'image_size': config.IMAGE_SIZE,\n    'model_type': config.model_type,\n    'lr': config.LR,\n    'epochs': config.EPOCHS,\n    'knn': 100,\n    'metric_out': config.WEIGHT_METRIC,\n    'specie_out': config.WEIGHT_SPECIES,\n    'rgb_to_gray_prob': config.RGB_TO_GRAY_PROB,\n    'min_jpeg_quality': config.MIN_JPEG_QUAL_AUG,\n    'max_jpeg_quality': config.MAX_JPEG_QUAL_AUG,\n    'fullbody_crop_prob': config.FULLBODY_CROP_PROB,\n    'detic_crop_prob': config.DETIC_CROP_PROB,\n    'yolov5_crop_prob': config.YOLOV5_CROP_PROB,\n    'vit_base_crop_prob': config.VIT_BASE_CROP_PROB,\n    'vit_small_crop_prob': config.VIT_SMALL_CROP_PROB,\n    'vit_moco_crop_prob': config.VIT_MOCO_CROP_PROB,\n    'non_crop_prob': config.NONE_CROP_PROB,\n    'arcface_m': config.ARC_FACE_M,\n    'emb_dim': config.EMB_DIM,\n    'gcs_ds_path': config.GCS_DS_PATH\n}","metadata":{"papermill":{"duration":0.066136,"end_time":"2022-04-11T13:37:52.592606","exception":false,"start_time":"2022-04-11T13:37:52.52647","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:37.319695Z","iopub.execute_input":"2022-04-19T03:28:37.320195Z","iopub.status.idle":"2022-04-19T03:28:37.328958Z","shell.execute_reply.started":"2022-04-19T03:28:37.32016Z","shell.execute_reply":"2022-04-19T03:28:37.327847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wandb.init(\n    project=\"happy-whale-and-dolphin\",\n    entity=\"lexkaggle\",\n    name=f'effnet-b{config.EFF_NET}-img-{config.IMAGE_SIZE}-fullbody-f-{config.FOLD_TO_RUN}')","metadata":{"papermill":{"duration":7.5037,"end_time":"2022-04-11T13:38:00.152888","exception":false,"start_time":"2022-04-11T13:37:52.649188","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:37.332936Z","iopub.execute_input":"2022-04-19T03:28:37.333464Z","iopub.status.idle":"2022-04-19T03:28:43.846008Z","shell.execute_reply.started":"2022-04-19T03:28:37.333422Z","shell.execute_reply":"2022-04-19T03:28:43.845139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"config.GCS_DS_PATH","metadata":{"papermill":{"duration":0.067954,"end_time":"2022-04-11T13:38:00.279474","exception":false,"start_time":"2022-04-11T13:38:00.21152","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:43.84722Z","iopub.execute_input":"2022-04-19T03:28:43.84814Z","iopub.status.idle":"2022-04-19T03:28:43.854994Z","shell.execute_reply.started":"2022-04-19T03:28:43.848103Z","shell.execute_reply":"2022-04-19T03:28:43.85432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_files = np.sort(np.array(tf.io.gfile.glob(config.GCS_DS_PATH + '/happywhale-2022-train*.tfrec')))\ntest_files = np.sort(np.array(tf.io.gfile.glob(config.GCS_DS_PATH + '/happywhale-2022-test*.tfrec')))\nprint(config.GCS_DS_PATH)\nprint(len(train_files),len(test_files),count_data_items(train_files),count_data_items(test_files))","metadata":{"papermill":{"duration":1.29462,"end_time":"2022-04-11T13:38:01.632556","exception":false,"start_time":"2022-04-11T13:38:00.337936","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:43.856047Z","iopub.execute_input":"2022-04-19T03:28:43.856927Z","iopub.status.idle":"2022-04-19T03:28:45.18131Z","shell.execute_reply.started":"2022-04-19T03:28:43.856888Z","shell.execute_reply":"2022-04-19T03:28:45.180378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data","metadata":{"papermill":{"duration":0.060352,"end_time":"2022-04-11T13:38:01.770656","exception":false,"start_time":"2022-04-11T13:38:01.710304","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def random_rgb_to_gray(image, probability, seed=config.SEED):\n    with tf.name_scope('RandomRGBtoGray'):\n        do_gray_random = tf.random.uniform([], seed=seed)\n\n        image = tf.cond(\n            tf.greater(do_gray_random, probability), lambda: image,\n            lambda: tf.image.grayscale_to_rgb(tf.image.rgb_to_grayscale(image)))\n\n    return image","metadata":{"papermill":{"duration":0.071418,"end_time":"2022-04-11T13:38:01.901896","exception":false,"start_time":"2022-04-11T13:38:01.830478","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:45.182725Z","iopub.execute_input":"2022-04-19T03:28:45.183785Z","iopub.status.idle":"2022-04-19T03:28:45.191153Z","shell.execute_reply.started":"2022-04-19T03:28:45.183748Z","shell.execute_reply":"2022-04-19T03:28:45.190247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"crops = tf.convert_to_tensor(['fullbody', 'detic', 'yolov5', 'vit_base', 'vit_small', 'vit_moco', 'none'])\n\ndef arcface_format(posting_id, image, label_group, species):\n    return posting_id, {'inp1': image, 'inp2': label_group, 'inp3': species}, label_group, species\n\ndef arcface_inference_format(posting_id, image, label_group, species):\n    return image,posting_id\n\ndef arcface_eval_format(posting_id, image, label_group, species):\n    return image,label_group\n\ndef data_augment(posting_id, image, label_group, species):\n    image = tf.image.random_flip_left_right(image)\n    image = tf.image.random_hue(image, config.RANDOM_HUE_MAX_DELTA)\n    image = tf.image.random_saturation(image, config.RANDOM_SATURATION_LOWER, config.RANDOM_SATURATION_UPPER)\n    image = tf.image.random_contrast(image, config.RANDOM_CONTRAST_LOWER, config.RANDOM_CONTRAST_UPPER)\n    image = tf.image.random_brightness(image, config.RANDOM_BRIGHTNESS_MAX_DELTA)\n    image = random_rgb_to_gray(image, config.RGB_TO_GRAY_PROB)\n    return posting_id, image, label_group, species\n\ndef decode_image(image_data, bbs):\n    if bbs is not None and bbs[0] != -1:\n        left, top, right, bottom = bbs[0], bbs[1], bbs[2], bbs[3]\n        bbs = tf.convert_to_tensor([top, left, bottom - top, right - left])\n        image = tf.io.decode_and_crop_jpeg(image_data, bbs, channels=3)\n    else:\n        image = tf.image.decode_jpeg(image_data, channels = 3)\n\n    image = tf.image.resize(image, [config.IMAGE_SIZE,config.IMAGE_SIZE])\n    image = tf.clip_by_value(image, 0, 255)\n    image = tf.cast(image, tf.float32) / 255.0\n    return image\n\ndef read_labeled_tfrecord(example, crop_method, flip=False):\n    LABELED_TFREC_FORMAT = {\n        \"image_name\": tf.io.FixedLenFeature([], tf.string),\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"target\": tf.io.FixedLenFeature([], tf.int64),\n        \"species\": tf.io.FixedLenFeature([], tf.int64),\n        'detic_box': tf.io.FixedLenFeature([4], tf.int64),\n        'yolov5_box': tf.io.FixedLenFeature([4], tf.int64),\n        'tc_vitbase': tf.io.FixedLenFeature([4], tf.int64),\n        'tc_vitsmall': tf.io.FixedLenFeature([4], tf.int64),\n        'tc_mocovit': tf.io.FixedLenFeature([4], tf.int64),\n        'fullbody': tf.io.FixedLenFeature([4], tf.int64)\n    }\n\n    example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n    posting_id = example['image_name']\n    \n    if crop_method == 'random':\n        rand = tf.random.categorical(tf.math.log([[\n            config.FULLBODY_CROP_PROB,\n            config.DETIC_CROP_PROB,\n            config.YOLOV5_CROP_PROB,\n            config.VIT_BASE_CROP_PROB,\n            config.VIT_SMALL_CROP_PROB,\n            config.VIT_MOCO_CROP_PROB,\n            config.NONE_CROP_PROB\n        ]]), 1, seed=config.SEED)\n        \n        crop_method = crops[rand[0][0]]\n    \n    if crop_method == 'detic':\n        bbs = tf.cast(example['detic_box'], tf.int32)\n    elif crop_method == 'yolov5':\n        bbs = tf.cast(example['yolov5_box'], tf.int32)\n    elif crop_method == 'vit_base':\n        bbs = tf.cast(example['tc_vitbase'], tf.int32)\n    elif crop_method == 'vit_small':\n        bbs = tf.cast(example['tc_vitsmall'], tf.int32)\n    elif crop_method == 'vit_moco':\n        bbs = tf.cast(example['tc_mocovit'], tf.int32)\n    elif crop_method == 'fullbody':\n        bbs = tf.cast(example['fullbody'], tf.int32)\n    else:\n        bbs = tf.convert_to_tensor([-1, -1, -1, -1])\n\n    image = decode_image(example['image'], bbs)\n    \n    if flip:\n        image = tf.image.flip_left_right(image)\n    \n    label_group = tf.cast(example['target'], tf.int32)\n    species = tf.cast(example['species'], tf.int32)\n    \n    return posting_id, image, label_group, species\n\ndef load_dataset(filenames, ordered = False, crop_method='random', flip=False):\n    ignore_order = tf.data.Options()\n    if not ordered:\n        ignore_order.experimental_deterministic = False \n        \n    dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads = AUTO)\n    dataset = dataset.with_options(ignore_order)\n    dataset = dataset.map(lambda x: read_labeled_tfrecord(x, crop_method, flip), num_parallel_calls = AUTO) \n    return dataset\n\ndef get_training_dataset(filenames, pseudo=False):\n    if pseudo:\n        print('Loading pseudo labels')\n        filenames = list(filenames)\n        filenames.append(config.GCS_PSEUDO_DS_PATH + '/happywhale-2022-pseudo--12157.tfrec')\n        filenames = np.array(filenames)\n    dataset = load_dataset(filenames, ordered = False)\n    dataset = dataset.map(data_augment, num_parallel_calls = AUTO)\n    dataset = dataset.map(arcface_format, num_parallel_calls = AUTO)\n    dataset = dataset.map(lambda posting_id, image, label_group, species: (image, {'metric_out': label_group, 'specie_out': species}))\n    dataset = dataset.repeat()\n    dataset = dataset.shuffle(1024)\n    dataset = dataset.batch(config.BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO)\n    return dataset\n\ndef get_val_dataset(filenames):\n    dataset = load_dataset(filenames, ordered = True, crop_method='detic')\n    # dataset = dataset.map(data_augment, num_parallel_calls = AUTO)\n    dataset = dataset.map(arcface_format, num_parallel_calls = AUTO)\n    dataset = dataset.map(lambda posting_id, image, label_group, species: (image, {'metric_out': label_group, 'specie_out': species}))\n    dataset = dataset.batch(config.BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO)\n    return dataset\n\ndef get_eval_dataset(filenames, get_targets = True):\n    dataset = load_dataset(filenames, ordered = True, crop_method='detic')\n    # dataset = dataset.map(data_augment, num_parallel_calls = AUTO)\n    dataset = dataset.map(arcface_eval_format, num_parallel_calls = AUTO)\n    if not get_targets:\n        dataset = dataset.map(lambda image, target: image)\n    dataset = dataset.batch(config.BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO)\n    return dataset\n\ndef get_test_dataset(filenames, get_names = True, crop_method='random', flip=False):\n    dataset = load_dataset(filenames, ordered = True, crop_method=crop_method, flip=flip)\n    # dataset = dataset.map(data_augment, num_parallel_calls = AUTO)\n    dataset = dataset.map(arcface_inference_format, num_parallel_calls = AUTO)\n    if not get_names:\n        dataset = dataset.map(lambda image, posting_id: image)\n    dataset = dataset.batch(config.BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO)\n    return dataset","metadata":{"papermill":{"duration":0.105646,"end_time":"2022-04-11T13:38:02.067716","exception":false,"start_time":"2022-04-11T13:38:01.96207","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:45.193023Z","iopub.execute_input":"2022-04-19T03:28:45.193532Z","iopub.status.idle":"2022-04-19T03:28:45.237514Z","shell.execute_reply.started":"2022-04-19T03:28:45.193479Z","shell.execute_reply":"2022-04-19T03:28:45.236521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"species = json.load(open(config.LABEL_INFO_PATH / 'species.json'))\nindividual_ids = json.load(open(config.LABEL_INFO_PATH / 'individual_ids.json'))\n\nid2species = {s: i for i, s in species.items()}\nid2individual_ids = {s: i for i, s in individual_ids.items()}","metadata":{"papermill":{"duration":0.113959,"end_time":"2022-04-11T13:38:02.247694","exception":false,"start_time":"2022-04-11T13:38:02.133735","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:45.238823Z","iopub.execute_input":"2022-04-19T03:28:45.239076Z","iopub.status.idle":"2022-04-19T03:28:45.293851Z","shell.execute_reply.started":"2022-04-19T03:28:45.239046Z","shell.execute_reply":"2022-04-19T03:28:45.292812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"id2species","metadata":{"papermill":{"duration":0.071412,"end_time":"2022-04-11T13:38:02.378954","exception":false,"start_time":"2022-04-11T13:38:02.307542","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:45.295366Z","iopub.execute_input":"2022-04-19T03:28:45.29582Z","iopub.status.idle":"2022-04-19T03:28:45.304467Z","shell.execute_reply.started":"2022-04-19T03:28:45.295787Z","shell.execute_reply":"2022-04-19T03:28:45.30351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"NUM_SPECIES = len(id2species)","metadata":{"papermill":{"duration":0.067694,"end_time":"2022-04-11T13:38:02.506366","exception":false,"start_time":"2022-04-11T13:38:02.438672","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:45.307192Z","iopub.execute_input":"2022-04-19T03:28:45.30796Z","iopub.status.idle":"2022-04-19T03:28:45.315969Z","shell.execute_reply.started":"2022-04-19T03:28:45.307918Z","shell.execute_reply":"2022-04-19T03:28:45.315255Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"row = 5; col = 4;\nrow = min(row,config.BATCH_SIZE//col)\nN_TRAIN = count_data_items(train_files)\nprint(N_TRAIN)\nds = get_training_dataset(train_files, pseudo=True)\n\nfor (sample,label) in ds:\n    img = sample['inp1']\n    plt.figure(figsize=(25,int(25*row/col)))\n    for j in range(row*col):\n        plt.subplot(row,col,j+1)\n        plt.title(f\"{id2individual_ids[label['metric_out'][j].numpy()]} - {id2species[label['specie_out'][j].numpy()]}\")\n        plt.axis('off')\n        plt.imshow(img[j,])\n    plt.show()\n    break","metadata":{"papermill":{"duration":11.75942,"end_time":"2022-04-11T13:38:14.32543","exception":false,"start_time":"2022-04-11T13:38:02.56601","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:28:45.317175Z","iopub.execute_input":"2022-04-19T03:28:45.318027Z","iopub.status.idle":"2022-04-19T03:29:00.315574Z","shell.execute_reply.started":"2022-04-19T03:28:45.317992Z","shell.execute_reply":"2022-04-19T03:29:00.314826Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"row = 3; col = 4;\nrow = min(row,config.BATCH_SIZE//col)\nN_TEST = count_data_items(test_files)\nprint(N_TEST)\nds = get_test_dataset(test_files)\n\nfor (img,label) in ds:\n    plt.figure(figsize=(25,int(25*row/col)))\n    for j in range(row*col):\n        plt.subplot(row,col,j+1)\n        plt.title(label[j].numpy())\n        plt.axis('off')\n        plt.imshow(img[j,])\n    plt.show()\n    break","metadata":{"papermill":{"duration":7.990621,"end_time":"2022-04-11T13:38:22.47968","exception":false,"start_time":"2022-04-11T13:38:14.489059","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:29:00.316776Z","iopub.execute_input":"2022-04-19T03:29:00.317583Z","iopub.status.idle":"2022-04-19T03:29:12.819242Z","shell.execute_reply.started":"2022-04-19T03:29:00.317543Z","shell.execute_reply":"2022-04-19T03:29:12.81821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"row = 3; col = 4;\nrow = min(row,config.BATCH_SIZE//col)\nN_TEST = count_data_items(test_files)\nprint(N_TEST)\nds = get_test_dataset(test_files, flip=True)\n\nfor (img,label) in ds:\n    plt.figure(figsize=(25,int(25*row/col)))\n    for j in range(row*col):\n        plt.subplot(row,col,j+1)\n        plt.title(label[j].numpy())\n        plt.axis('off')\n        plt.imshow(img[j,])\n    plt.show()\n    break","metadata":{"papermill":{"duration":4.044057,"end_time":"2022-04-11T13:38:26.749885","exception":false,"start_time":"2022-04-11T13:38:22.705828","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:29:12.8206Z","iopub.execute_input":"2022-04-19T03:29:12.820873Z","iopub.status.idle":"2022-04-19T03:29:17.497501Z","shell.execute_reply.started":"2022-04-19T03:29:12.820842Z","shell.execute_reply":"2022-04-19T03:29:17.496329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Architecture","metadata":{"papermill":{"duration":0.276915,"end_time":"2022-04-11T13:38:27.312691","exception":false,"start_time":"2022-04-11T13:38:27.035776","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Modified version of ArcFace from original kernel to create this. Based on paper ElasticFace: Elastic Margin Loss for Deep Face Recognition (https://arxiv.org/pdf/2109.09416.pdf).\n\nclass ElasticArcFace(tf.keras.layers.Layer):\n    def __init__(\n        self,\n        n_classes,\n        s=30,\n        mean=0.50,\n        std=0.025,\n        easy_margin=False,\n        ls_eps=0.0,\n        **kwargs\n    ):\n\n        super(ElasticArcFace, self).__init__(**kwargs)\n        \n        print(f'ElasticArcFace mean: {mean}, s: {std}')\n\n        self.n_classes = n_classes\n        self.s = s\n        self.mean = mean\n        self.std = std\n        self.ls_eps = ls_eps\n\n    def get_config(self):\n\n        config = super().get_config().copy()\n        config.update({\n            'n_classes': self.n_classes,\n            's': self.s,\n            'mean': self.mean,\n            'std': self.std,\n            'ls_eps': self.ls_eps\n        })\n        return config\n\n    def build(self, input_shape):\n        super(ElasticArcFace, self).build(input_shape[0])\n\n        self.W = self.add_weight(\n            name='W',\n            shape=(int(input_shape[0][-1]), self.n_classes),\n            initializer='glorot_uniform',\n            dtype='float32',\n            trainable=True,\n            regularizer=None)\n\n    def call(self, inputs):\n        X, y = inputs\n        y = tf.cast(y, dtype=tf.int32)\n        cosine = tf.matmul(\n            tf.math.l2_normalize(X, axis=1),\n            tf.math.l2_normalize(self.W, axis=0)\n        )\n        sine = tf.math.sqrt(1.0 - tf.math.pow(cosine, 2))\n\n        m = tf.random.normal((tf.shape(y)[0], 1), mean=self.mean, stddev=self.std, seed=config.SEED)\n\n        cos_m = tf.math.cos(m)\n        sin_m = tf.math.sin(m)\n        th = tf.math.cos(math.pi - m)\n        mm = tf.math.sin(math.pi - m) * m\n        \n        phi = cosine * cos_m - sine * sin_m\n\n        phi = tf.where(cosine > th, phi, cosine - mm)\n        one_hot = tf.cast(\n            tf.one_hot(y, depth=self.n_classes),\n            dtype=cosine.dtype\n            )\n        if self.ls_eps > 0:\n            one_hot = (1 - self.ls_eps) * one_hot + self.ls_eps / self.n_classes\n\n        output = (one_hot * phi) + ((1.0 - one_hot) * cosine)\n        output *= self.s\n        return output","metadata":{"papermill":{"duration":0.323978,"end_time":"2022-04-11T13:38:27.909372","exception":false,"start_time":"2022-04-11T13:38:27.585394","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:29:17.499005Z","iopub.execute_input":"2022-04-19T03:29:17.499329Z","iopub.status.idle":"2022-04-19T03:29:17.522546Z","shell.execute_reply.started":"2022-04-19T03:29:17.499292Z","shell.execute_reply":"2022-04-19T03:29:17.520998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"EFNS = [efn.EfficientNetB0, efn.EfficientNetB1, efn.EfficientNetB2, efn.EfficientNetB3, \n        efn.EfficientNetB4, efn.EfficientNetB5, efn.EfficientNetB6, efn.EfficientNetB7]\n\ndef freeze_BN(model):\n    # Unfreeze layers while leaving BatchNorm layers frozen\n    for layer in model.layers:\n        if not isinstance(layer, tf.keras.layers.BatchNormalization):\n            layer.trainable = True\n        else:\n            layer.trainable = False\n\ndef get_model():    \n    with strategy.scope():\n        margin = ElasticArcFace(\n            n_classes = config.N_CLASSES, \n            s = 30, \n            mean = config.ARC_FACE_M,\n            std=0.025,\n            name=f'head/{config.head}', \n            dtype='float32'\n        )\n\n        inp = tf.keras.layers.Input(shape = [config.IMAGE_SIZE, config.IMAGE_SIZE, 3], name = 'inp1')\n        label = tf.keras.layers.Input(shape = (), name = 'inp2')\n    \n        species = tf.keras.layers.Input(shape = (), name = 'inp3')\n        \n        if config.model_type == 'effnetv1':\n            x = EFNS[config.EFF_NET](weights = 'noisy-student', include_top = False)(inp)\n            # Concat pooling\n            avg_pool = tf.keras.layers.GlobalAveragePooling2D()(x)\n            max_pool = tf.keras.layers.GlobalMaxPooling2D()(x)\n            pretrained_out = tf.keras.layers.Concatenate()([avg_pool, max_pool])\n        elif config.model_type == 'effnetv2':\n            FEATURE_VECTOR = f'{EFFNETV2_ROOT}/tfhub_models/efficientnetv2-{config.EFF_NETV2}/feature_vector'\n            embed = tfhub.KerasLayer(FEATURE_VECTOR, trainable=True)(inp)\n        \n        # Do specie classification.\n        specie_in = tf.keras.layers.Dropout(0.2)(pretrained_out)\n        specie_out = tf.keras.layers.Dense(2048)(specie_in)\n        specie_out = tf.keras.layers.Dropout(0.2)(specie_out)\n        specie_out = tf.keras.layers.Dense(NUM_SPECIES, activation='softmax', name='specie_out')(specie_out)\n\n        # Do metric\n        print(f'Size of embed {config.EMB_DIM}')\n        pre_margin_dense_layer = tf.keras.layers.Dense(config.EMB_DIM)\n    \n        # Multiple-sample dropout https://arxiv.org/abs/1905.09788\n        dropout_base = 0.17\n        drop_ls = [tf.keras.layers.Dropout((dropout_base + 0.01), seed=420),\n                   tf.keras.layers.Dropout((dropout_base + 0.02), seed=4200),\n                   tf.keras.layers.Dropout((dropout_base + 0.03), seed=42000),\n                   tf.keras.layers.Dropout((dropout_base + 0.04), seed=420000),\n                   tf.keras.layers.Dropout((dropout_base + 0.05), seed=4200000)]\n\n        for ii, drop in enumerate(drop_ls):\n            if ii == 0:\n                embed = (pre_margin_dense_layer(drop(pretrained_out)) / 5.0)\n            else:\n                embed += (pre_margin_dense_layer(drop(pretrained_out)) / 5.0)\n                \n        embed = tf.keras.layers.BatchNormalization()(embed)\n        embed = tf.math.l2_normalize(embed, axis=1)\n        \n        x = margin([embed, label])\n        output = tf.keras.layers.Softmax(dtype='float32', name='metric_out')(x)\n        \n        model = tf.keras.models.Model(inputs = [inp, label, species], outputs = [output, specie_out])\n        embed_model = tf.keras.models.Model(inputs = inp, outputs = embed)  \n        \n        opt = tf.keras.optimizers.Adam(learning_rate = config.LR)\n\n        model.compile(\n            optimizer = opt,\n            loss = {\n                'metric_out': tf.keras.losses.SparseCategoricalCrossentropy(), \n                'specie_out': tf.keras.losses.SparseCategoricalCrossentropy()\n            },\n            loss_weights = {\n                'metric_out': config.WEIGHT_METRIC,\n                'specie_out': config.WEIGHT_SPECIES\n            },\n            metrics = {\n                'metric_out': [tf.keras.metrics.SparseCategoricalAccuracy(), tf.keras.metrics.SparseTopKCategoricalAccuracy(k=5)],\n                'specie_out': tf.keras.metrics.SparseCategoricalAccuracy()\n            }\n        ) \n        \n        return model,embed_model","metadata":{"papermill":{"duration":0.312318,"end_time":"2022-04-11T13:38:28.497026","exception":false,"start_time":"2022-04-11T13:38:28.184708","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:29:17.524422Z","iopub.execute_input":"2022-04-19T03:29:17.524977Z","iopub.status.idle":"2022-04-19T03:29:17.55324Z","shell.execute_reply.started":"2022-04-19T03:29:17.524936Z","shell.execute_reply":"2022-04-19T03:29:17.552288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training","metadata":{}},{"cell_type":"code","source":"def get_lr_callback(plot=False):\n    lr_start   = 0.000001\n    lr_max     = 0.000005 * config.BATCH_SIZE  \n    lr_min     = 0.000001\n    lr_ramp_ep = 4\n    lr_sus_ep  = 0\n    lr_decay   = 0.9\n   \n    def lrfn(epoch):\n        if config.RESUME:\n            epoch = epoch + config.RESUME_EPOCH\n        if epoch < lr_ramp_ep:\n            lr = (lr_max - lr_start) / lr_ramp_ep * epoch + lr_start\n            \n        elif epoch < lr_ramp_ep + lr_sus_ep:\n            lr = lr_max\n            \n        else:\n            lr = (lr_max - lr_min) * lr_decay**(epoch - lr_ramp_ep - lr_sus_ep) + lr_min\n            \n        return lr\n        \n    if plot:\n        epochs = list(range(config.EPOCHS))\n        learning_rates = [lrfn(x) for x in epochs]\n        print([f'{i:.20f}' for i in learning_rates])\n        plt.scatter(epochs,learning_rates)\n        plt.show()\n\n    lr_callback = tf.keras.callbacks.LearningRateScheduler(lrfn, verbose=False)\n    return lr_callback\n\nget_lr_callback(plot=True)","metadata":{"papermill":{"duration":0.50614,"end_time":"2022-04-11T13:38:29.323302","exception":false,"start_time":"2022-04-11T13:38:28.817162","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:29:17.554407Z","iopub.execute_input":"2022-04-19T03:29:17.55503Z","iopub.status.idle":"2022-04-19T03:29:17.817147Z","shell.execute_reply.started":"2022-04-19T03:29:17.554984Z","shell.execute_reply":"2022-04-19T03:29:17.815995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Snapshot(tf.keras.callbacks.Callback):\n    \n    def __init__(self,fold,snapshot_epochs=[]):\n        super(Snapshot, self).__init__()\n        self.snapshot_epochs = snapshot_epochs\n        self.fold = fold\n        \n        \n    def on_epoch_end(self, epoch, logs=None):\n        self.model.save_weights(config.save_dir+f\"/{config.MODEL_NAME}_last.h5\")","metadata":{"papermill":{"duration":0.2871,"end_time":"2022-04-11T13:38:29.889369","exception":false,"start_time":"2022-04-11T13:38:29.602269","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:29:17.818939Z","iopub.execute_input":"2022-04-19T03:29:17.819348Z","iopub.status.idle":"2022-04-19T03:29:17.83155Z","shell.execute_reply.started":"2022-04-19T03:29:17.819299Z","shell.execute_reply":"2022-04-19T03:29:17.828064Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{"papermill":{"duration":0.276168,"end_time":"2022-04-11T13:38:30.440833","exception":false,"start_time":"2022-04-11T13:38:30.164665","status":"completed"},"tags":[]}},{"cell_type":"code","source":"TRAINING_FILENAMES = [x for i,x in enumerate(train_files) if i%config.FOLDS!=config.FOLD_TO_RUN]\nVALIDATION_FILENAMES = [x for i,x in enumerate(train_files) if i%config.FOLDS==config.FOLD_TO_RUN]\nprint(len(TRAINING_FILENAMES),len(VALIDATION_FILENAMES),count_data_items(TRAINING_FILENAMES),count_data_items(VALIDATION_FILENAMES))","metadata":{"papermill":{"duration":0.322744,"end_time":"2022-04-11T13:38:31.03707","exception":false,"start_time":"2022-04-11T13:38:30.714326","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:29:17.833366Z","iopub.execute_input":"2022-04-19T03:29:17.836664Z","iopub.status.idle":"2022-04-19T03:29:17.848096Z","shell.execute_reply.started":"2022-04-19T03:29:17.836548Z","shell.execute_reply":"2022-04-19T03:29:17.847096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"seed_everything(config.SEED)\nVERBOSE = 1\ntrain_dataset = get_training_dataset(TRAINING_FILENAMES, pseudo=True)\nval_dataset = None\nif config.FOLD_TO_RUN is not None:\n    val_dataset = get_val_dataset(VALIDATION_FILENAMES)\nelse:\n    print('No val dataset')\n\nnum_pseudo = 12157\nSTEPS_PER_EPOCH = (count_data_items(TRAINING_FILENAMES) + num_pseudo) // config.BATCH_SIZE\n\n# SAVE BEST MODEL EACH FOLD        \nsv_loss = tf.keras.callbacks.ModelCheckpoint(\n    OUTPUT_DIR / f'{config.MODEL_NAME}_loss.h5',\n    monitor='val_loss',\n    verbose=0,\n    save_best_only=True,\n    save_weights_only=True,\n    mode='min',\n    save_freq='epoch')\n\n# BUILD MODEL\nK.clear_session()\nmodel,embed_model = get_model()\nsnap = Snapshot(fold=config.FOLD_TO_RUN)\nmodel.summary()\n\ncallbacks = [get_lr_callback(), snap]\nif config.FOLD_TO_RUN is not None:\n    callbacks.append(sv_loss)\n\nif config.LOAD_WEIGHTS:\n    print('Loading weights  ' + config.LOAD_WEIGHTS)\n    model.load_weights(config.LOAD_WEIGHTS)","metadata":{"papermill":{"duration":70.560706,"end_time":"2022-04-11T13:39:41.871103","exception":false,"start_time":"2022-04-11T13:38:31.310397","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:29:17.84981Z","iopub.execute_input":"2022-04-19T03:29:17.850151Z","iopub.status.idle":"2022-04-19T03:30:19.282467Z","shell.execute_reply.started":"2022-04-19T03:29:17.850105Z","shell.execute_reply":"2022-04-19T03:30:19.281342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Remove this line to train. It should train in Kaggle just fine.\nconfig.EPOCHS = 0","metadata":{"execution":{"iopub.status.busy":"2022-04-19T03:30:19.283942Z","iopub.execute_input":"2022-04-19T03:30:19.284198Z","iopub.status.idle":"2022-04-19T03:30:19.291086Z","shell.execute_reply.started":"2022-04-19T03:30:19.284168Z","shell.execute_reply":"2022-04-19T03:30:19.290076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if config.EPOCHS:\n    print('#### Image Size %i with EfficientNet B%i and batch_size %i'%\n          (config.IMAGE_SIZE,config.EFF_NET,config.BATCH_SIZE))\n\n    history = model.fit(\n        train_dataset,\n        validation_data = val_dataset,\n        steps_per_epoch = STEPS_PER_EPOCH,\n        epochs = config.EPOCHS,\n        callbacks = callbacks,\n        verbose = VERBOSE)\n\n    if config.FOLD_TO_RUN is not None:\n        model.load_weights(OUTPUT_DIR / f'{config.MODEL_NAME}_loss.h5')\n\n    try:\n        plt.plot(history.history['loss'])\n        plt.plot(history.history['val_loss'])\n        plt.title('model loss')\n        plt.ylabel('loss')\n        plt.xlabel('epoch')\n        plt.legend(['train', 'test'], loc='upper left')\n        plt.show()\n    except:\n        pass\nelse:\n    print('No training.')","metadata":{"papermill":{"duration":21357.575967,"end_time":"2022-04-11T19:35:41.582843","exception":false,"start_time":"2022-04-11T13:39:44.006876","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:30:19.292689Z","iopub.execute_input":"2022-04-19T03:30:19.292994Z","iopub.status.idle":"2022-04-19T03:30:19.308393Z","shell.execute_reply.started":"2022-04-19T03:30:19.292962Z","shell.execute_reply":"2022-04-19T03:30:19.307337Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Val Inference","metadata":{"papermill":{"duration":5.086841,"end_time":"2022-04-11T19:36:53.311162","exception":false,"start_time":"2022-04-11T19:36:48.224321","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def get_ids(filename):\n    ds = get_test_dataset([filename],get_names=True).map(lambda image, image_name: image_name).unbatch()\n    NUM_IMAGES = count_data_items([filename])\n    ids = next(iter(ds.batch(NUM_IMAGES))).numpy().astype('U')\n    return ids\n\ndef get_targets(filename):\n    ds = get_eval_dataset([filename],get_targets=True).map(lambda image, target: target).unbatch()\n    NUM_IMAGES = count_data_items([filename])\n    ids = next(iter(ds.batch(NUM_IMAGES))).numpy()\n    return ids\n\ndef get_embeddings(filename, crop_method, flip=False):\n    ds = get_test_dataset([filename],get_names=False, crop_method=crop_method, flip=flip)\n    embeddings = embed_model.predict(ds,verbose=0)\n    return embeddings\n\ndef get_predictions(test_df,threshold=0.2):\n    predictions = {}\n    for i,row in tqdm(test_df.iterrows()):\n        if row.image in predictions:\n            if len(predictions[row.image])==5:\n                continue\n            predictions[row.image].append(row.target)\n        elif row.confidence>threshold:\n            predictions[row.image] = [row.target,'new_individual']\n        else:\n            predictions[row.image] = ['new_individual',row.target]\n\n    for x in tqdm(predictions):\n        if len(predictions[x])<5:\n            remaining = [y for y in sample_list if y not in predictions]\n            predictions[x] = predictions[x]+remaining\n            predictions[x] = predictions[x][:5]\n        \n    return predictions\n\ndef map_per_image(label, predictions):\n    \"\"\"Computes the precision score of one image.\n\n    Parameters\n    ----------\n    label : string\n            The true label of the image\n    predictions : list\n            A list of predicted elements (order does matter, 5 predictions allowed per image)\n\n    Returns\n    -------\n    score : double\n    \"\"\"    \n    try:\n        return 1 / (predictions[:5].index(label) + 1)\n    except ValueError:\n        return 0.0\n    \nf = open (config.LABEL_INFO_PATH / 'individual_ids.json', \"r\")\ntarget_encodings = json.loads(f.read())\ntarget_encodings = {target_encodings[x]:x for x in target_encodings}\n\nsample_list = [\n    '938b7e931166', 'ca69a5d7c122', '18efa8d0b472', '91ed5caeb0d3', '7362d7a01d00',\n    'ae4343270756', '8e5253662392', '0b180ad0afa2', '6a6fa3ec3810', '6a3af6e0c55c'\n]","metadata":{"papermill":{"duration":5.194055,"end_time":"2022-04-11T19:37:03.520531","exception":false,"start_time":"2022-04-11T19:36:58.326476","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:30:19.310136Z","iopub.execute_input":"2022-04-19T03:30:19.310747Z","iopub.status.idle":"2022-04-19T03:30:19.346841Z","shell.execute_reply.started":"2022-04-19T03:30:19.310696Z","shell.execute_reply":"2022-04-19T03:30:19.345861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_embeddings_for_filenames(filenames, crop_method, flip=False):\n    output = []\n    for filename in tqdm(filenames):\n        embeddings = get_embeddings(filename, crop_method, flip=flip)\n        output.append(embeddings)\n        \n    return np.concatenate(output)","metadata":{"papermill":{"duration":5.073402,"end_time":"2022-04-11T19:37:44.281963","exception":false,"start_time":"2022-04-11T19:37:39.208561","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:30:19.348415Z","iopub.execute_input":"2022-04-19T03:30:19.348752Z","iopub.status.idle":"2022-04-19T03:30:19.356786Z","shell.execute_reply.started":"2022-04-19T03:30:19.348708Z","shell.execute_reply":"2022-04-19T03:30:19.355883Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prepare_all_embeddings(filenames, prefix):\n    # This will be lazily set by the first generated embedding, as only then will we know the dimensions.\n    all_embeddings = None\n\n    crop_weights = config.CROP_WEIGHTS\n    total_weight = sum(crop_weights.values())\n    crop_weights_adjusted = {k: (v / total_weight) for (k, v) in crop_weights.items()}\n    \n    i = 0\n    for crop_method, weight in crop_weights_adjusted.items():\n        print(f'Do {crop_method} no flip')\n        train_embeddings_no_flip = get_embeddings_for_filenames(filenames, crop_method)\n        \n        print(f'Do {crop_method} flip')\n        train_embeddings_flip = get_embeddings_for_filenames(filenames, crop_method, flip=True)\n        \n        emb_arr = np.mean(np.stack([train_embeddings_no_flip, train_embeddings_flip]), axis=0)\n        #fh = open(OUTPUT_DIR / f'{crop_method}_{prefix}_embeddings_fold{config.FOLD_TO_RUN or \"\"}.npz', 'wb')\n        #np.save(fh, emb_arr)\n        \n        if all_embeddings is None:\n            all_embeddings = np.zeros((len(crop_weights_adjusted), emb_arr.shape[0], emb_arr.shape[1]))\n            print(all_embeddings.shape)\n    \n        all_embeddings[i] = emb_arr * weight\n        i += 1\n    \n    # Try different style of merging embeddings in future.\n    mean = np.mean(all_embeddings, axis=0)\n    fh = open(OUTPUT_DIR / f'{prefix}_mean_embeddings_fold{config.FOLD_TO_RUN or \"\"}.npz', 'wb')\n    np.save(fh, mean)\n    \n    return mean","metadata":{"papermill":{"duration":5.441215,"end_time":"2022-04-11T19:38:05.202622","exception":false,"start_time":"2022-04-11T19:37:59.761407","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:30:19.357962Z","iopub.execute_input":"2022-04-19T03:30:19.358306Z","iopub.status.idle":"2022-04-19T03:30:19.415525Z","shell.execute_reply.started":"2022-04-19T03:30:19.358265Z","shell.execute_reply":"2022-04-19T03:30:19.391849Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def save_ids(filenames, prefix, include_targets=False):\n    train_targets = []\n    train_ids = []\n    for filename in tqdm(filenames):\n        if include_targets:\n            targets = get_targets(filename)\n            train_targets.append(targets)\n    \n        ids = get_ids(filename)\n        train_ids.append(ids)\n\n    if include_targets:\n        train_targets = np.concatenate(train_targets)\n        targets_fh = open(OUTPUT_DIR / f'{prefix}_targets_fold{config.FOLD_TO_RUN or \"\"}.npz', 'wb')\n        np.save(targets_fh, train_targets)\n\n    train_ids = np.concatenate(train_ids)\n    ids_fh = open(OUTPUT_DIR / f'{prefix}_ids_fold{config.FOLD_TO_RUN or \"\"}.npz', 'wb')\n    np.save(ids_fh, train_ids)\n    \n    return train_ids, train_targets if include_targets else None\n\ndef load_all_embeddings(path, embed_name):\n    return np.load(open(path / f'{embed_name}_mean_embeddings_fold.npz', 'rb'))\n\ndef get_ids(path, prefix):\n    return np.load(open(path / f'{prefix}_ids_fold.npz', 'rb'))\n\ndef get_targets(path, prefix):\n    return np.load(open(path / f'{prefix}_targets_fold.npz', 'rb'))","metadata":{"papermill":{"duration":5.361701,"end_time":"2022-04-11T19:38:15.902936","exception":false,"start_time":"2022-04-11T19:38:10.541235","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:30:47.625675Z","iopub.execute_input":"2022-04-19T03:30:47.626046Z","iopub.status.idle":"2022-04-19T03:30:47.638689Z","shell.execute_reply.started":"2022-04-19T03:30:47.626003Z","shell.execute_reply":"2022-04-19T03:30:47.637972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"OUTPUT_DIR.mkdir(exist_ok=True, parents=True)","metadata":{"papermill":{"duration":5.186669,"end_time":"2022-04-11T19:38:46.885542","exception":false,"start_time":"2022-04-11T19:38:41.698873","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:30:52.033347Z","iopub.execute_input":"2022-04-19T03:30:52.034066Z","iopub.status.idle":"2022-04-19T03:30:52.040114Z","shell.execute_reply.started":"2022-04-19T03:30:52.034025Z","shell.execute_reply":"2022-04-19T03:30:52.039439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Remove the if False if training in Kaggle.\nis_training = False\nif is_training:\n    TRAINING_FILENAMES = [x for i,x in enumerate(train_files) if i%config.FOLDS!=config.VAL_FOLD_TO_RUN]\n    VALIDATION_FILENAMES = [x for i,x in enumerate(train_files) if i%config.FOLDS==config.VAL_FOLD_TO_RUN]\n\n    print(TRAINING_FILENAMES, VALIDATION_FILENAMES)\n\n    train_embeddings = prepare_all_embeddings(TRAINING_FILENAMES, 'train')\n    val_embeddings = prepare_all_embeddings(VALIDATION_FILENAMES, 'val')\n    test_embeddings = prepare_all_embeddings(test_files, 'test')\n\n    train_ids, train_targets = save_ids(TRAINING_FILENAMES, 'train', include_targets=True)\n    print(train_targets.shape)\n\n    val_ids, val_targets = save_ids(VALIDATION_FILENAMES, 'val', include_targets=True)\n    print(val_targets.shape)\n\n    test_ids, _ = save_ids(test_files, 'test')\nelse:\n    embed_path = Path('../input/happywhale-download-b5-with-dhak-pseudo-multisam/b5-with-dhak-pseudo-multisampledropout')\n\n    train_embeddings = load_all_embeddings(embed_path, 'train')\n    val_embeddings = load_all_embeddings(embed_path, 'val')\n    test_embeddings = load_all_embeddings(embed_path, 'test')\n\n    train_ids = get_ids(embed_path, 'train')\n    train_targets = get_targets(embed_path, 'train')\n\n    val_ids = get_ids(embed_path, 'val')\n    val_targets = get_targets(embed_path, 'val')\n\n    test_ids = get_ids(embed_path, 'test')","metadata":{"papermill":{"duration":5652.317683,"end_time":"2022-04-11T21:13:04.200987","exception":false,"start_time":"2022-04-11T19:38:51.883304","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:31:55.945003Z","iopub.execute_input":"2022-04-19T03:31:55.94539Z","iopub.status.idle":"2022-04-19T03:31:56.384625Z","shell.execute_reply.started":"2022-04-19T03:31:55.945349Z","shell.execute_reply":"2022-04-19T03:31:56.383669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# KNN","metadata":{"papermill":{"duration":5.10221,"end_time":"2022-04-11T21:18:28.089328","exception":false,"start_time":"2022-04-11T21:18:22.987118","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from sklearn.neighbors import NearestNeighbors","metadata":{"papermill":{"duration":5.243062,"end_time":"2022-04-11T21:18:38.4","exception":false,"start_time":"2022-04-11T21:18:33.156938","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:31:57.151533Z","iopub.execute_input":"2022-04-19T03:31:57.151887Z","iopub.status.idle":"2022-04-19T03:31:57.159068Z","shell.execute_reply.started":"2022-04-19T03:31:57.151848Z","shell.execute_reply":"2022-04-19T03:31:57.158404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Note: validation is just to ensure there are no bugs. The model was trained on the whole dataset, so validation scores can't be trusted.","metadata":{}},{"cell_type":"code","source":"best_threshold_adjusted = 0.6\n\nneigh = NearestNeighbors(n_neighbors=config.KNN,metric='cosine')\nneigh.fit(train_embeddings)\nval_nn_distances, val_nn_idxs = neigh.kneighbors(val_embeddings, config.KNN, return_distance=True)\nallowed_targets = set([target_encodings[x] for x in np.unique(train_targets)])\nval_targets_df = pd.DataFrame(np.stack([val_ids,val_targets],axis=1),columns=['image','target'])\nval_targets_df['target'] = val_targets_df['target'].astype(int).map(target_encodings)\nval_targets_df.loc[~val_targets_df.target.isin(allowed_targets),'target'] = 'new_individual'\nval_targets_df.target.value_counts()\nval_df = []\nfor i in tqdm(range(len(val_ids))):\n    id_ = val_ids[i]\n    targets = train_targets[val_nn_idxs[i]]\n    distances = val_nn_distances[i]\n    subset_preds = pd.DataFrame(np.stack([targets,distances],axis=1),columns=['target','distances'])\n    subset_preds['image'] = id_\n    val_df.append(subset_preds)\nval_df = pd.concat(val_df).reset_index(drop=True)\nval_df['confidence'] = 1-val_df['distances']\nval_df = val_df.groupby(['image','target']).confidence.max().reset_index()\nval_df = val_df.sort_values('confidence',ascending=False).reset_index(drop=True)\nval_df['target'] = val_df['target'].map(target_encodings)\nval_df.to_csv('val_neighbors.csv')\nval_df.image.value_counts().value_counts()\n\n## Compute CV\nth = 0.6\ncv = 0\n\nall_preds = get_predictions(val_df,threshold=th)\nfor i,row in val_targets_df.iterrows():\n    target = row.target\n    preds = all_preds[row.image]\n    val_targets_df.loc[i,th] = map_per_image(target,preds)\ncv = val_targets_df[th].mean()\nprint(f\"CV at threshold {th}: {cv}\")\n\nval_targets_df.describe()\n\nif config.FOLD_TO_RUN == config.VAL_FOLD_TO_RUN:\n    wandb.log({\"best_cv\": best_cv, \"best_threshold\": best_th})\n\n## Adjustment: Since Public lb has nearly 10% 'new_individual' (Be Careful for private LB)\nval_targets_df['is_new_individual'] = val_targets_df.target=='new_individual'\nprint(val_targets_df.is_new_individual.value_counts().to_dict())\nval_scores = val_targets_df.groupby('is_new_individual').mean().T\nval_scores['adjusted_cv'] = val_scores[True]*0.1+val_scores[False]*0.9\nbest_threshold_adjusted = val_scores['adjusted_cv'].idxmax()\nprint(\"best_threshold\",best_threshold_adjusted)\nval_scores\n\ntrain_embeddings = np.concatenate([train_embeddings,val_embeddings])\ntrain_targets = np.concatenate([train_targets,val_targets])\nprint(train_embeddings.shape,train_targets.shape)","metadata":{"papermill":{"duration":402.192515,"end_time":"2022-04-11T21:25:25.854501","exception":false,"start_time":"2022-04-11T21:18:43.661986","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:34:10.111364Z","iopub.execute_input":"2022-04-19T03:34:10.111681Z","iopub.status.idle":"2022-04-19T03:35:20.656379Z","shell.execute_reply.started":"2022-04-19T03:34:10.11165Z","shell.execute_reply":"2022-04-19T03:35:20.655416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Test Submission","metadata":{"papermill":{"duration":5.076378,"end_time":"2022-04-11T21:25:36.015075","exception":false,"start_time":"2022-04-11T21:25:30.938697","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from sklearn.neighbors import NearestNeighbors\nneigh = NearestNeighbors(n_neighbors=config.KNN,metric='cosine')\nneigh.fit(train_embeddings)","metadata":{"papermill":{"duration":5.085945,"end_time":"2022-04-11T21:25:46.519763","exception":false,"start_time":"2022-04-11T21:25:41.433818","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:35:20.658347Z","iopub.execute_input":"2022-04-19T03:35:20.658596Z","iopub.status.idle":"2022-04-19T03:35:20.724292Z","shell.execute_reply.started":"2022-04-19T03:35:20.658566Z","shell.execute_reply":"2022-04-19T03:35:20.723325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_nn_distances, test_nn_idxs = neigh.kneighbors(test_embeddings, config.KNN, return_distance=True)","metadata":{"papermill":{"duration":80.661211,"end_time":"2022-04-11T21:27:12.407496","exception":false,"start_time":"2022-04-11T21:25:51.746285","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:35:20.725589Z","iopub.execute_input":"2022-04-19T03:35:20.725832Z","iopub.status.idle":"2022-04-19T03:36:43.815268Z","shell.execute_reply.started":"2022-04-19T03:35:20.725805Z","shell.execute_reply":"2022-04-19T03:36:43.814175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_submission = pd.read_csv(config.INPUT_DATA_PATH / 'sample_submission.csv',index_col='image')\nprint(len(test_ids),len(sample_submission))\ntest_df = []\nfor i in tqdm(range(len(test_ids))):\n    id_ = test_ids[i]\n    targets = train_targets[test_nn_idxs[i]]\n    distances = test_nn_distances[i]\n    subset_preds = pd.DataFrame(np.stack([targets,distances],axis=1),columns=['target','distances'])\n    subset_preds['image'] = id_\n    test_df.append(subset_preds)\ntest_df = pd.concat(test_df).reset_index(drop=True)\ntest_df['confidence'] = 1-test_df['distances']\ntest_df = test_df.groupby(['image','target']).confidence.max().reset_index()\ntest_df = test_df.sort_values('confidence',ascending=False).reset_index(drop=True)\ntest_df['target'] = test_df['target'].map(target_encodings)\ntest_df.to_csv('test_neighbors.csv')\ntest_df.image.value_counts().value_counts()","metadata":{"papermill":{"duration":43.520913,"end_time":"2022-04-11T21:28:01.00284","exception":false,"start_time":"2022-04-11T21:27:17.481927","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:36:43.817833Z","iopub.execute_input":"2022-04-19T03:36:43.818242Z","iopub.status.idle":"2022-04-19T03:37:17.497898Z","shell.execute_reply.started":"2022-04-19T03:36:43.818175Z","shell.execute_reply":"2022-04-19T03:37:17.496786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_list = ['938b7e931166', '5bf17305f073', '7593d2aee842', '7362d7a01d00','956562ff2888']","metadata":{"papermill":{"duration":5.054206,"end_time":"2022-04-11T21:28:11.10911","exception":false,"start_time":"2022-04-11T21:28:06.054904","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:37:17.499689Z","iopub.execute_input":"2022-04-19T03:37:17.500817Z","iopub.status.idle":"2022-04-19T03:37:17.508248Z","shell.execute_reply.started":"2022-04-19T03:37:17.500754Z","shell.execute_reply":"2022-04-19T03:37:17.507166Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predictions = {}\nfor i,row in tqdm(test_df.iterrows()):\n    if row.image in predictions:\n        if len(predictions[row.image])==5:\n            continue\n        predictions[row.image].append(row.target)\n    elif row.confidence>best_threshold_adjusted:\n        predictions[row.image] = [row.target,'new_individual']\n    else:\n        predictions[row.image] = ['new_individual',row.target]\n        \nfor x in tqdm(predictions):\n    if len(predictions[x])<5:\n        remaining = [y for y in sample_list if y not in predictions]\n        predictions[x] = predictions[x]+remaining\n        predictions[x] = predictions[x][:5]\n    predictions[x] = ' '.join(predictions[x])\n    \npredictions = pd.Series(predictions).reset_index()\npredictions.columns = ['image','predictions']\npredictions.to_csv('submission.csv',index=False)\npredictions.head()","metadata":{"papermill":{"duration":84.328233,"end_time":"2022-04-11T21:29:40.640032","exception":false,"start_time":"2022-04-11T21:28:16.311799","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-04-19T03:37:17.509691Z","iopub.execute_input":"2022-04-19T03:37:17.510057Z","iopub.status.idle":"2022-04-19T03:38:40.823768Z","shell.execute_reply.started":"2022-04-19T03:37:17.510008Z","shell.execute_reply":"2022-04-19T03:38:40.82253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}