{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# What is in this notebook 🤔 ?\n\nThe goal of this notebook is to create an enhanced version of the dataset for the last [happywhale competition](https://www.kaggle.com/c/happy-whale-and-dolphin).<br>\n\nThis notebook implements a few tricks that came out of discussions and public notebooks while sticking to the same file structure as the official competition's data, and remains fully configurable.<br>\nIt has been made such that you can fork it and change the settings to your convenience.<br>\n\n## To fork this notebook\n1. Fork this notebook by clicking th \"Copy & edit notebook\" button in the three dots menu on the top right corner of the page\n2. Upload your kaggle api creds in a **PRIVATE** dataset and add them as a dataset to your notebook\n3. Change the settings as you like\n4. Commit !\n\n## Use the datasets created by this notebook !\n* [light version (224x224)](https://www.kaggle.com/wolfy73/happywhale-enhanced-dataset-light)\n* [normal version (380x380)](https://www.kaggle.com/wolfy73/happywhale-enhanced-dataset-normal)\n\n\nIf you want this notebooks to integrate other tricks, please tell me in the comment section !<br>\n\nIf you found this notebook useful, please upvote it 👍 ","metadata":{}},{"cell_type":"markdown","source":"# What tricks are implemented in this notebook 💡 ?\n\n|Trick|Source|Description|\n|---:|:---|:---|\n|Bitmap format|[W&D - (224x224) Fast dataset](https://www.kaggle.com/wolfy73/w-d-224x224-fast-dataset)|The images are converted to the BMP format which is a format that, unlike the jpg, does not need computation when an image is readed. As such, image reading becomes faster|\n|Cropped images|[Happywhale: Cropped Dataset [YOLOv5] ✂️](https://www.kaggle.com/awsaf49/happywhale-cropped-dataset-yolov5)|Images are cropped using a YOLOv5 model that is train to label images with bounding boxes. This allows to crop out unwanted informations such as land, people and boats, and \"zoom\" on the part we are interested in|\n|Fixed species|[Fix all known species column problems](https://www.kaggle.com/c/happy-whale-and-dolphin/discussion/305574)|Merge similar species in the training set, reduce the number of classes to 26|\n|Stratified KFold|[Stratified KFold v. Group KFold (aka. I'm a dummy)](https://www.kaggle.com/c/happy-whale-and-dolphin/discussion/306521)|Attribute a fold to each training sample, thus you can track which sample was used to train which model. This also allows to perform cross-validation|\n|OOD Detection|[🐳&🐬 - Filter YOLOv5 failure cases](https://www.kaggle.com/wolfy73/filter-yolov5-failure-cases)|As the YOLOv5 model isn't perfect, we use OOD sample detection to filter out wrong bounding boxes|\n|Kaggle dataset|[HappyWhale TFRecords](https://www.kaggle.com/ks2019/happywhale-tfrecords)|The kaggle outputs sizes are limited to 19.6GB. You can utilize kaggle datasets to remove this constraint|\n|Maximize contrast| [🐳&🐬 - Extract more infromation from images 🖼️](https://www.kaggle.com/wolfy73/extract-more-infromation-from-images) | Try to maximize information in the image by remapping the colors |\n|Detic cropping|[cropped&resized(512x512) dataset using detic](https://www.kaggle.com/c/happy-whale-and-dolphin/discussion/305503)| Use a CLIP based cropping method to crop the dataset|\n|YOLOv5 cropping #2|[🐳&🐬 - Train YOLOv5 w/ crowd-sourced dataset 📊](https://www.kaggle.com/wolfy73/train-yolov5-w-crowd-sourced-dataset)| Images are cropped using a YOLOv5 model that is train to label images with bounding boxes. The difference with the previous one is that this time it was train on a [specific crowed sourced dataset](https://www.kaggle.com/wolfy73/wandd-crowed-sourced-bounging-boxes)|","metadata":{}},{"cell_type":"markdown","source":"# Changelog 📝\n|Version|Description|\n|:---|:---|\n|v1| All tricks turned on ✅ |\n|v2| Padding turned off ✅ |\n|v3| Image size increased to 380 + flag quantile increased to 5% ❌ (Out of disk space) |\n|v4| Outputing in a kaggle dataset + changelog + test mode ✅ |\n|v5| OOD_DETECTION_FLAG_QUANTILE=0.1, IMAGE_SIZE=224 + Maximize contrast trick ✅ |\n|v6| Disabling maximized contrast for comparison ✅ |\n|v7| Image size re-increased to 380 Cropping based on detic + OOD_DETECTION_FLAG_QUANTILE=0.01 ✅ |\n|v8| Image size increased to 512 + OOD_DETECTION_FLAG_QUANTILE=0.02 ❌ (Out of disk space) |\n|v9| Image size back to 380 + crowed sourced dataset YOLOv5 ✅  |\n|v10| Image size to 512 + save disk space ✅  |\n|v11| Disabled bitmap ✅ |","metadata":{}},{"cell_type":"markdown","source":"# Configuration ⚙️","metadata":{}},{"cell_type":"markdown","source":"Feel free to tweak theses settings to your convenience !","metadata":{}},{"cell_type":"code","source":"# Size of final images\nMAX_IMAGE_SIZE = 512\n\n### Turn on and off the mentionned tricks\n\n## Conversion of images into the bitmap format to speed up image reading\nUSE_BITMAP_FORMAT = False\n\n## Crop images using Yolov5 to cut out parts of the image that may not be informative\nUSE_CROPPED_IMAGES = True\nCROPPED_IMAGES_CONFIDENCE = 0.01 # Minimum confidence to keep the bounding box\nCROPPED_IMAGES_MARGIN = 0.1 # Proportion of the extra margin to add around the bounding box (avoid under-cropping)\n\n## Merge species that can be considered as the same to have less different classes\nUSE_FIXED_SPECIES = True\n\n# Assign a fold to each sample of the data to perform cross-validation\nUSE_STRATIFIED_K_FOLD = True\nSTRATIFIED_K_FOLD_K = 4 # Number of folds\n\n# Use OOD methods to filter out failure cases in the yolo bounding box dataset\nUSE_OOD_DETECTION = False\nOOD_DETECTION_FLAG_QUANTILE = 0.01 # Proportion of the data to flag by each method\nOOD_DETECTION_N_FLAGS = 2 # Minimum number of flag to filter out a sample\nOOD_DETECTION_USE_MAX_FLAG = True # Use the max confidence as a metric to detect OOD samples\nOOD_DETECTION_USE_DELTA_MAX_FLAG = True # Use the delta between the max confidence of original and cropped as a metric to detect OOD samples\nOOD_DETECTION_USE_ENTROPY_FLAG = True # Use the entropy of the prediction distribution as a metric to detect OOD samples\nOOD_DETECTION_USE_DELTA_ENTROPY_FLAG = True # Use the delta between the entropy of f original and cropped prediction distribution as a metric to detect OOD samples\n\n# Output the dataset as a kaggle dataset (higher memory limit)\nUSE_KAGGLE_DATASET = True\nKAGGLE_DATASET_NAME = 'happywhale-enhanced-dataset-large' # Name of the resulting dataset\n\n# Maximize the information in image by remaping the colors\nUSE_MAXIMIZE_CONTRAST = False\n\n# Pad with zeros to obtain squared images, the goal is to avoid squeezing images\nUSE_ZERO_PADDING = False\n\n\nTEST_MODE = False # Used for debugging","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:17.536643Z","iopub.execute_input":"2022-03-06T22:28:17.537348Z","iopub.status.idle":"2022-03-06T22:28:17.572316Z","shell.execute_reply.started":"2022-03-06T22:28:17.537212Z","shell.execute_reply":"2022-03-06T22:28:17.571639Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install -q bbox-utility # check https://github.com/awsaf49/bbox for source code","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:17.574500Z","iopub.execute_input":"2022-03-06T22:28:17.575143Z","iopub.status.idle":"2022-03-06T22:28:29.301563Z","shell.execute_reply.started":"2022-03-06T22:28:17.575094Z","shell.execute_reply":"2022-03-06T22:28:29.300419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport cv2\nimport matplotlib.pyplot as plt\nimport time\nimport glob\nimport shutil\nimport json\nimport datetime\nfrom tqdm.notebook import tqdm\nfrom bbox.utils import yolo2voc, draw_bboxes\nfrom sklearn.model_selection import StratifiedKFold","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-06T22:28:29.303148Z","iopub.execute_input":"2022-03-06T22:28:29.303410Z","iopub.status.idle":"2022-03-06T22:28:31.823914Z","shell.execute_reply.started":"2022-03-06T22:28:29.303380Z","shell.execute_reply":"2022-03-06T22:28:31.822730Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_images(batch, row=2, col=2, base_path=\"../input/happy-whale-and-dolphin/train_images/\"):\n    \"\"\"\n        Copied and adapted from https://www.kaggle.com/awsaf49/happywhale-data-distribution\n    \"\"\"\n    plt.figure(figsize=(col*3, row*3))\n    for i in range(row*col):\n        plt.subplot(row, col, i+1)\n        path = os.path.join(base_path,  batch[\"image\"].iloc[i])\n        img = cv2.imread(path)\n        if img is None:\n            continue\n        img = img[:, :, ::-1]\n        plt.imshow(img)\n        if \"species\" in batch:\n            plt.title(batch[\"species\"].iloc[i])\n        plt.axis('off')\n    plt.tight_layout()\n    plt.show()\n    \ndef get_size(start_path = '.'):\n    total_size = 0\n    for dirpath, dirnames, filenames in os.walk(start_path):\n        for f in filenames:\n            fp = os.path.join(dirpath, f)\n            # skip if it is symbolic link\n            if not os.path.islink(fp):\n                total_size += os.path.getsize(fp)\n\n    return total_size","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:31.826392Z","iopub.execute_input":"2022-03-06T22:28:31.826752Z","iopub.status.idle":"2022-03-06T22:28:31.838533Z","shell.execute_reply.started":"2022-03-06T22:28:31.826706Z","shell.execute_reply":"2022-03-06T22:28:31.837332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cp ../input/happy-whale-and-dolphin/sample_submission.csv sample_submission.csv\n!cp ../input/happy-whale-and-dolphin/train.csv train.csv","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:31.840065Z","iopub.execute_input":"2022-03-06T22:28:31.840398Z","iopub.status.idle":"2022-03-06T22:28:33.440989Z","shell.execute_reply.started":"2022-03-06T22:28:31.840352Z","shell.execute_reply":"2022-03-06T22:28:33.439768Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Authenticate (needs your kaggle.json file in secret named 'kaggle')\nfrom kaggle_secrets import UserSecretsClient\nuser_secrets = UserSecretsClient()\napitoken = user_secrets.get_secret(\"kaggle\")\n!mkdir -p ~/.kaggle\n%store apitoken >~/.kaggle/kaggle.json\n!mkdir -p dataset","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:33.442674Z","iopub.execute_input":"2022-03-06T22:28:33.442971Z","iopub.status.idle":"2022-03-06T22:28:35.108697Z","shell.execute_reply.started":"2022-03-06T22:28:33.442937Z","shell.execute_reply":"2022-03-06T22:28:35.107459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\"\"\"\n    Copy-paste from https://www.kaggle.com/ks2019/happywhale-tfrecords\n\"\"\"\n\nif USE_KAGGLE_DATASET:    \n    !rm -r /tmp/{KAGGLE_DATASET_NAME}\n\n    os.makedirs(f'/tmp/{KAGGLE_DATASET_NAME}', exist_ok=True)\n\n    with open('/root/.kaggle/kaggle.json') as f:\n        kaggle_creds = json.load(f)\n\n    os.environ['KAGGLE_USERNAME'] = kaggle_creds['username']\n    os.environ['KAGGLE_KEY'] = kaggle_creds['key']\n\n    !kaggle datasets init -p /tmp/{KAGGLE_DATASET_NAME}\n\n    with open(f'/tmp/{KAGGLE_DATASET_NAME}/dataset-metadata.json') as f:\n        dataset_meta = json.load(f)\n    dataset_meta['id'] = f'wolfy73/{KAGGLE_DATASET_NAME}'\n    dataset_meta['title'] = KAGGLE_DATASET_NAME\n    with open(f'/tmp/{KAGGLE_DATASET_NAME}/dataset-metadata.json', \"w\") as outfile:\n        json.dump(dataset_meta, outfile)\n    print(dataset_meta)\n\n    !cp /tmp/{KAGGLE_DATASET_NAME}/dataset-metadata.json /tmp/{KAGGLE_DATASET_NAME}/meta.json\n    !ls /tmp/{KAGGLE_DATASET_NAME}\n\n    !kaggle datasets create -u -p /tmp/{KAGGLE_DATASET_NAME} \n    \n    BASE_PATH=f\"/tmp/{KAGGLE_DATASET_NAME}\"\nelse:\n    BASE_PATH=\".\"\nprint(\"BASE_PATH =\", BASE_PATH)\n!mkdir {BASE_PATH}/test_images\n!mkdir {BASE_PATH}/train_images","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:35.110484Z","iopub.execute_input":"2022-03-06T22:28:35.110796Z","iopub.status.idle":"2022-03-06T22:28:41.956049Z","shell.execute_reply.started":"2022-03-06T22:28:35.110760Z","shell.execute_reply":"2022-03-06T22:28:41.954825Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# train.csv","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv(\"./train.csv\")\ntrain","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:41.958984Z","iopub.execute_input":"2022-03-06T22:28:41.959740Z","iopub.status.idle":"2022-03-06T22:28:42.060485Z","shell.execute_reply.started":"2022-03-06T22:28:41.959699Z","shell.execute_reply":"2022-03-06T22:28:42.059605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if USE_FIXED_SPECIES:\n    train[\"species\"][train[\"species\"] == \"bottlenose_dolpin\"] = \"bottlenose_dolphin\"\n    train[\"species\"][train[\"species\"] == \"kiler_whale\"] = \"killer_whale\"\n    train[\"species\"][train[\"species\"] == \"globis\"] = \"short_finned_pilot_whale\"\n    train[\"species\"][train[\"species\"] == \"pilot_whale\"] = \"short_finned_pilot_whale\"\n\nif USE_BITMAP_FORMAT:\n    train[\"image\"] = train[\"image\"].str[:-3] + \"bmp\"\n    \nif USE_STRATIFIED_K_FOLD:\n    \"\"\"\n        Copied and adapted from https://www.kaggle.com/debarshichanda/pytorch-arcface-gem-pooling-starter\n    \"\"\"\n    skf = StratifiedKFold(n_splits=STRATIFIED_K_FOLD_K)\n    folds = np.zeros(len(train), dtype=np.uint8)\n    for fold, ( _, val_) in enumerate(skf.split(X=train, y=train.individual_id)):\n        folds[val_] = fold\n    train[\"fold\"] = folds\n        \ntrain.to_csv(os.path.join(BASE_PATH, \"./train.csv\"), index=False)\ntrain","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:42.063054Z","iopub.execute_input":"2022-03-06T22:28:42.063320Z","iopub.status.idle":"2022-03-06T22:28:43.023784Z","shell.execute_reply.started":"2022-03-06T22:28:42.063287Z","shell.execute_reply":"2022-03-06T22:28:43.022735Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# sample_submission.csv","metadata":{}},{"cell_type":"code","source":"sample_submission = pd.read_csv(\"./sample_submission.csv\")\nsample_submission","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:43.025241Z","iopub.execute_input":"2022-03-06T22:28:43.025557Z","iopub.status.idle":"2022-03-06T22:28:43.073390Z","shell.execute_reply.started":"2022-03-06T22:28:43.025513Z","shell.execute_reply":"2022-03-06T22:28:43.072748Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if USE_BITMAP_FORMAT:\n    sample_submission[\"inference_image\"] = sample_submission[\"image\"].str[:-3] + \"bmp\"\nsample_submission.to_csv(os.path.join(BASE_PATH, \"./sample_submission.csv\"), index=False)\nsample_submission","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:43.074801Z","iopub.execute_input":"2022-03-06T22:28:43.075332Z","iopub.status.idle":"2022-03-06T22:28:43.225499Z","shell.execute_reply.started":"2022-03-06T22:28:43.075297Z","shell.execute_reply":"2022-03-06T22:28:43.224663Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"yolo_bp = '../input/train-yolov5-w-crowd-sourced-dataset'\nmetrics_bp = \"../input/filter-yolov5-failure-cases\"\n\nyolo_train = pd.read_csv(os.path.join(yolo_bp, 'train.csv'), index_col=\"image\").fillna(\"[]\")\nbboxes_train = yolo_train[\"bbox\"].map(eval)\nbboxes_train_confidence = yolo_train[\"conf\"].map(eval)\n\nmetrics_train = pd.read_csv(os.path.join(metrics_bp, 'train.csv'))\nmetrics_train.index = metrics_train[\"image\"].str[:-3] + \"jpg\"\nassert (bboxes_train.index == metrics_train.index).all()\nmetrics_train[\"bbox\"] = bboxes_train\nmetrics_train[\"bbox_conf\"] = bboxes_train_confidence\n\nyolo_test = pd.read_csv(os.path.join(yolo_bp, 'test.csv'), index_col=\"image\").fillna(\"[]\")\nbboxes_test = yolo_test[\"bbox\"].map(eval)\nbboxes_test_confidence = yolo_test[\"conf\"].map(eval)\n\nmetrics_test = pd.read_csv(os.path.join(metrics_bp, 'test.csv'))\nmetrics_test.index = metrics_test[\"image\"].str[:-3] + \"jpg\"\nassert (bboxes_test.index == metrics_test.index).all()\nmetrics_test[\"bbox\"] = bboxes_test\nmetrics_test[\"bbox_conf\"] = bboxes_test_confidence","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:43.227139Z","iopub.execute_input":"2022-03-06T22:28:43.227612Z","iopub.status.idle":"2022-03-06T22:28:47.696675Z","shell.execute_reply.started":"2022-03-06T22:28:43.227568Z","shell.execute_reply":"2022-03-06T22:28:47.695727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_flags = np.zeros(len(metrics_train))\ntest_flags = np.zeros(len(metrics_test))\n\n## MAX FLAG\nif OOD_DETECTION_USE_MAX_FLAG:\n    # Train\n    train_cropped_pred_max_conf = metrics_train[\"cropped_pred_max_conf\"].values\n    train_m = metrics_train[\"cropped_pred_max_conf\"] <= np.quantile(train_cropped_pred_max_conf, OOD_DETECTION_FLAG_QUANTILE)\n    print(f\"MAX_FLAG: {train_m.sum()} samples flagged in train\")\n    train_flags[train_m] += 1\n    # test\n    test_cropped_pred_max_conf = metrics_test[\"cropped_pred_max_conf\"].values\n    test_m = metrics_test[\"cropped_pred_max_conf\"] <= np.quantile(test_cropped_pred_max_conf, OOD_DETECTION_FLAG_QUANTILE)\n    print(f\"MAX_FLAG: {test_m.sum()} samples flagged in test\")\n    test_flags[test_m] += 1\n\n## DELTA MAX FLAG\nif OOD_DETECTION_USE_DELTA_MAX_FLAG:\n    # Train\n    train_pred_max_conf_delta = metrics_train[\"pred_max_conf_delta\"].values\n    train_m = metrics_train[\"pred_max_conf_delta\"] > np.quantile(train_pred_max_conf_delta, 1 - OOD_DETECTION_FLAG_QUANTILE)\n    print(f\"DELTA_MAX_FLAG: {train_m.sum()} samples flagged in train\")\n    train_flags[train_m] += 1\n    # test\n    test_pred_max_conf_delta = metrics_test[\"pred_max_conf_delta\"].values\n    test_m = metrics_test[\"pred_max_conf_delta\"] > np.quantile(test_pred_max_conf_delta, 1 - OOD_DETECTION_FLAG_QUANTILE)\n    print(f\"DELTA_MAX_FLAG: {test_m.sum()} samples flagged in test\")\n    test_flags[test_m] += 1\n\n## ENTROPY FLAG\nif OOD_DETECTION_USE_ENTROPY_FLAG:\n    # Train\n    train_cropped_pred_entropy = metrics_train[\"cropped_pred_entropy\"].values\n    train_m = metrics_train[\"cropped_pred_entropy\"] > np.quantile(train_cropped_pred_entropy, 1 - OOD_DETECTION_FLAG_QUANTILE)\n    print(f\"ENTROPY_FLAG: {train_m.sum()} samples flagged in train\")\n    train_flags[train_m] += 1\n    # test\n    test_cropped_pred_entropy = metrics_test[\"cropped_pred_entropy\"].values\n    test_m = metrics_test[\"cropped_pred_entropy\"] > np.quantile(test_cropped_pred_entropy, 1 - OOD_DETECTION_FLAG_QUANTILE)\n    print(f\"ENTROPY_FLAG: {test_m.sum()} samples flagged in test\")\n    test_flags[test_m] += 1\n\n## DELTA ENTROPY FLAG\nif OOD_DETECTION_USE_DELTA_ENTROPY_FLAG:\n    # Train\n    train_pred_entropy_delta = metrics_train[\"pred_entropy_delta\"].values\n    train_m = metrics_train[\"pred_entropy_delta\"] <= np.quantile(train_pred_entropy_delta, OOD_DETECTION_FLAG_QUANTILE)\n    print(f\"DELTA_ENTROPY_FLAG: {train_m.sum()} samples flagged in train\")\n    train_flags[train_m] += 1\n    # test\n    test_pred_entropy_delta = metrics_test[\"pred_entropy_delta\"].values\n    test_m = metrics_test[\"pred_entropy_delta\"] <= np.quantile(test_pred_entropy_delta, OOD_DETECTION_FLAG_QUANTILE)\n    print(f\"DELTA_ENTROPY_FLAG: {test_m.sum()} samples flagged in test\")\n    test_flags[test_m] += 1\n\nif USE_OOD_DETECTION:\n    train_m = train_flags >= OOD_DETECTION_N_FLAGS\n    metrics_train[\"bbox\"][train_m] = None\n    print(f\"Removed {train_m.sum()} bounding boxes in the train dataset\")\n    test_m = test_flags >= OOD_DETECTION_N_FLAGS\n    metrics_test[\"bbox\"][test_m] = None\n    print(f\"Removed {test_m.sum()} bounding boxes in the test dataset\")","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:47.698294Z","iopub.execute_input":"2022-03-06T22:28:47.699235Z","iopub.status.idle":"2022-03-06T22:28:47.730240Z","shell.execute_reply.started":"2022-03-06T22:28:47.699192Z","shell.execute_reply":"2022-03-06T22:28:47.729327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def crop_image(image, row):\n    if row['bbox'] is None or len(row['bbox']) == 0: # if there is no bbox\n        return image\n    bbox = row['bbox'][0]\n    conf = row['bbox_conf'][0]\n    if conf>=CROPPED_IMAGES_CONFIDENCE: # don't crop for poor confident bboxes\n        xmin, ymin, xmax, ymax = bbox\n        width = xmax - xmin\n        height = ymax - ymin\n        dw = int(round(CROPPED_IMAGES_MARGIN * width))\n        dh = int(round(CROPPED_IMAGES_MARGIN * height))\n        xmin, xmax, ymin, ymax = max(xmin-dw, 0), min(xmax+dw, image.shape[1]), max(ymin-dh, 0), min(ymax+dh, image.shape[0])\n        image = image[ymin:ymax, xmin:xmax] # crop image\n        \n    return image\n\ndef pad_and_resize(image):\n    r = MAX_IMAGE_SIZE / max(image.shape[0], image.shape[1])\n    dim = (int(image.shape[1] * r), int(image.shape[0] * r))\n    image = cv2.resize(image, dim, interpolation=cv2.INTER_CUBIC)\n    nimage = np.zeros((MAX_IMAGE_SIZE, MAX_IMAGE_SIZE, 3), dtype=np.uint8)\n    \n    middle = MAX_IMAGE_SIZE // 2\n    \n    vh1, hh1 = image.shape[0] // 2, image.shape[1] // 2\n    vh2, hh2 = image.shape[0] - vh1, image.shape[1] - hh1\n    \n    iy1, iy2 = middle - vh1, middle + vh2\n    ix1, ix2 = middle - hh1, middle + hh2\n    \n    nimage[iy1:iy2, ix1:ix2] = image\n    \n    return nimage\n\ndef remap_channel(image):\n    # Argsort of the image to keep spatial information\n    ids_sorted = np.argsort((image + np.random.random(image.shape) - 0.5).ravel())\n    # Shades of grey\n    values = np.floor(np.linspace(0.0, 256.0, num=len(ids_sorted), endpoint=False)).astype(np.uint8)\n    s = image.shape\n    image = image.ravel()\n    # Reorder the shades of greyto look like the original image\n    image[ids_sorted] = values\n    image = image.reshape(s)\n    return image\n\ndef remap_colors(image):\n    \"\"\"\n        The remapping is equivalent to create an image with n shades of grey and move the pixels in it such that it look like the original image\n    \"\"\"\n    if len(image.shape) == 2:\n        return remap_channel(image)\n    image[:, :, 0] = remap_channel(image[:, :, 0])\n    image[:, :, 1] = remap_channel(image[:, :, 1])\n    image[:, :, 2] = remap_channel(image[:, :, 2])\n    return image","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:47.733542Z","iopub.execute_input":"2022-03-06T22:28:47.734344Z","iopub.status.idle":"2022-03-06T22:28:47.753932Z","shell.execute_reply.started":"2022-03-06T22:28:47.734298Z","shell.execute_reply":"2022-03-06T22:28:47.752628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def copy_dir(collection, base_path=\"../input/happy-whale-and-dolphin/\"):\n    dirname = \"train_images\" if collection == \"train\" else \"test_images\"\n    print(\"Copying\", collection, \"images...\")\n    path = os.path.join(base_path, dirname)\n    images = list(os.listdir(path))\n    if TEST_MODE:\n        images = images[:50]\n    n = len(images)\n    for i, f in tqdm(enumerate(images), total=n):\n        if not bool(i % 1000):\n            print(\"{}/{}: {:.4f} Gb used so far\".format(i, n, get_size(f\"/tmp/{KAGGLE_DATASET_NAME}\") / 1e9))\n        \n        image_path = os.path.join(path, f)\n        image = cv2.imread(image_path)\n        \n        if USE_CROPPED_IMAGES:\n            df = metrics_train if collection == \"train\" else metrics_test\n            image = crop_image(image, df.loc[f])\n        \n        if USE_ZERO_PADDING and image.shape[0] != image.shape[1]:\n            image = pad_and_resize(image)\n        else:\n            if image.shape[0] > MAX_IMAGE_SIZE or image.shape[1] > MAX_IMAGE_SIZE:\n                image = cv2.resize(image, (MAX_IMAGE_SIZE, MAX_IMAGE_SIZE), interpolation=cv2.INTER_CUBIC)\n            \n        if USE_MAXIMIZE_CONTRAST:\n            image = remap_colors(image)\n            \n        if USE_BITMAP_FORMAT:\n            new_path = os.path.join(BASE_PATH, dirname, f.split('.')[0] + \".bmp\")\n        else:\n            new_path = os.path.join(BASE_PATH, dirname, f)\n        cv2.imwrite(new_path, image)","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:47.755841Z","iopub.execute_input":"2022-03-06T22:28:47.756269Z","iopub.status.idle":"2022-03-06T22:28:47.773379Z","shell.execute_reply.started":"2022-03-06T22:28:47.756135Z","shell.execute_reply":"2022-03-06T22:28:47.772621Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train images","metadata":{}},{"cell_type":"code","source":"copy_dir(\"train\")","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:47.774714Z","iopub.execute_input":"2022-03-06T22:28:47.775506Z","iopub.status.idle":"2022-03-06T22:28:52.223125Z","shell.execute_reply.started":"2022-03-06T22:28:47.775462Z","shell.execute_reply":"2022-03-06T22:28:52.221755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Test images","metadata":{}},{"cell_type":"code","source":"copy_dir(\"test\")","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:52.225039Z","iopub.execute_input":"2022-03-06T22:28:52.225461Z","iopub.status.idle":"2022-03-06T22:28:57.009771Z","shell.execute_reply.started":"2022-03-06T22:28:52.225416Z","shell.execute_reply":"2022-03-06T22:28:57.008922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if not TEST_MODE and USE_KAGGLE_DATASET:\n    !ls /tmp/{KAGGLE_DATASET_NAME}\n    verion_name = datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n    !kaggle datasets version -m {verion_name} -p /tmp/{KAGGLE_DATASET_NAME} -r zip -q","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:57.011212Z","iopub.execute_input":"2022-03-06T22:28:57.012059Z","iopub.status.idle":"2022-03-06T22:28:57.020037Z","shell.execute_reply.started":"2022-03-06T22:28:57.012009Z","shell.execute_reply":"2022-03-06T22:28:57.019358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"⬆️ You can access your dataset by clicking the link above ! ⬆️","metadata":{}},{"cell_type":"code","source":"if TEST_MODE:\n    train = pd.read_csv(os.path.join(BASE_PATH,  \"./train.csv\"))\n    path = os.path.join(BASE_PATH, \"train_images\")\n    plot_images(\n        train[train[\"image\"].isin(list(os.listdir(path)))],\n        row=5, col=5, \n        base_path=os.path.join(BASE_PATH, \"train_images\")\n    )","metadata":{"execution":{"iopub.status.busy":"2022-03-06T22:28:57.021462Z","iopub.execute_input":"2022-03-06T22:28:57.022006Z","iopub.status.idle":"2022-03-06T22:28:59.979131Z","shell.execute_reply.started":"2022-03-06T22:28:57.021958Z","shell.execute_reply":"2022-03-06T22:28:59.977845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/Thats_all_folks.svg/2560px-Thats_all_folks.svg.png)","metadata":{}}]}