{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Check marking of ISIC by [@shonenkov](https://www.kaggle.com/shonenkov)\n\n\nHi everyone!\n\nI have found really good reasons for unstable validation scheme - duplicates!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Main Idea\n\nLets use my best single model from [this kernel](https://www.kaggle.com/shonenkov/inference-single-model-melanoma-starter) for searching duplicated images and check correctness of marking! \n\nYou will see approach for searching duplicated images using clustering DBSCAN and `imagededup`! Also simple EDA with mistakes of marking!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Changelog\n\n- v5: initial, found ~490 duplicates\n- v7: add imagededup and tune epsilon for DBSCAN, found 1130 duplicates, calculated precision of clustering for `DBSCAN` and `IMAGEDEDUP`","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"from glob import glob\nimport json\nfrom tqdm import tqdm\nimport pandas as pd\nfrom sklearn.model_selection import GroupKFold\nfrom sklearn.cluster import DBSCAN\nfrom collections import defaultdict\nimport cv2\nfrom skimage import io\nimport albumentations as A\nimport scipy as sp\nimport torch\nimport os\nfrom datetime import datetime\nimport time\nimport random\nimport cv2\nimport pandas as pd\nimport numpy as np\nimport albumentations as A\nimport matplotlib.pyplot as plt\nfrom albumentations.pytorch.transforms import ToTensorV2\nfrom sklearn.model_selection import StratifiedKFold\nfrom torch.utils.data import Dataset,DataLoader\nfrom torch.utils.data.sampler import SequentialSampler, RandomSampler\nfrom torch.nn import functional as F\nfrom glob import glob\nimport sklearn\nfrom torch import nn\nimport warnings\n\nwarnings.filterwarnings(\"ignore\") \nwarnings.filterwarnings(\"ignore\", category=DeprecationWarning) \n\nSEED = 42\n\ndef seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = True\n\nseed_everything(SEED)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Clustering TRAIN","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"DATA_PATH = '../input/melanoma-merged-external-data-512x512-jpeg'\n\ndf_folds = pd.read_csv(f'{DATA_PATH}/folds_08062020.csv', index_col='image_id')\n\nTRAIN_IMAGE_IDS = df_folds.index.values\nTRAIN_CLUSTERS = []\nTRAIN_CLUSTERING_CACHE = set()\nBAD_CASES_CACHE = set()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Using [imagededup](https://github.com/idealo/imagededup)\n\nThanks a lot [@ebouteillon](https://www.kaggle.com/ebouteillon) for advise to use `imagededup`. Good tool, fast and good precision \"in box\"!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install --no-deps imagededup==0.2.2 > /dev/null","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\nfrom imagededup.methods import PHash\n\nphasher = PHash()\n\nencodings = phasher.encode_images(image_dir=f'{DATA_PATH}/512x512-dataset-melanoma/512x512-dataset-melanoma')\nduplicates = phasher.find_duplicates(encoding_map=encodings, max_distance_threshold=0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Not duplicates but found as duplicates using imagededup (manually checked):\nBAD_CASES = [\n    ['ISIC_0148783', 'ISIC_2016546'],\n    ['ISIC_2697895', 'ISIC_4591526'],\n    ['ISIC_6927689', 'ISIC_7106012'],\n    ['ISIC_6088194', 'ISIC_7559729'],\n    ['ISIC_3840133', 'ISIC_7168301'],\n    ['ISIC_0641480', 'ISIC_3273370'],\n    ['ISIC_4007215', 'ISIC_7219333'],\n    ['ISIC_2372032', 'ISIC_2484560'],\n    ['ISIC_3107526', 'ISIC_3900903'],\n    ['ISIC_3828455', 'ISIC_6577087'],\n    ['ISIC_1536014', 'ISIC_5709767'],\n    ['ISIC_0033743', 'ISIC_1356715'],\n    ['ISIC_5044766', 'ISIC_7912183'],\n    ['ISIC_4089379', 'ISIC_8432001'],\n    ['ISIC_0188415', 'ISIC_9626241'],\n    ['ISIC_4848047', 'ISIC_9117456'],\n    ['ISIC_0032336', 'ISIC_9125216'],\n    ['ISIC_0030513', 'ISIC_0688622'],\n    ['ISIC_3520750', 'ISIC_9639348'],\n    ['ISIC_2496831', 'ISIC_9540109'],\n    ['ISIC_1410153', 'ISIC_5950041'],\n    ['ISIC_3866081', 'ISIC_6754247'],\n    ['ISIC_2129226', 'ISIC_2647198'],\n    ['ISIC_0148783', 'ISIC_2016546', 'ISIC_3455285', 'ISIC_7460560'],\n    ['ISIC_2697895', 'ISIC_4591526', 'ISIC_6625344', 'ISIC_9367832'],\n    ['ISIC_0789732', 'ISIC_8303710', 'ISIC_9167141'],\n]\n\nfor case in BAD_CASES:\n    BAD_CASES_CACHE.add('.'.join(sorted(case)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for image_id, values in tqdm(duplicates.items(), total=len(duplicates)):\n    image_id = image_id.split('.')[0]\n    if len(values) < 1:\n        continue\n    if image_id not in TRAIN_IMAGE_IDS:\n        continue\n    sorted_cluster = [image_id]\n    for value in values:\n        value = value.split('.')[0]\n        if value in TRAIN_IMAGE_IDS:\n            sorted_cluster.append(value)\n\n    sorted_cluster = sorted(sorted_cluster)\n    if len(sorted_cluster) > 1:\n        cluster_name = '.'.join(sorted_cluster)\n        if cluster_name in BAD_CASES_CACHE:\n            continue\n        if cluster_name not in TRAIN_CLUSTERING_CACHE:\n            TRAIN_CLUSTERING_CACHE.add(cluster_name)\n            TRAIN_CLUSTERS.append(sorted_cluster)\n            \nTRAIN_CLUSTERS = sorted(TRAIN_CLUSTERS, key=lambda x: -len(x))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"margin = 0\ncount = 20\n\ndraw_clusters = TRAIN_CLUSTERS[margin:margin+count]\n\nsize = min([5, len(draw_clusters[0])])\n\nfig, ax = plt.subplots(count, size, figsize=(size*3, 4*count))\n\nfor j, image_ids in enumerate(draw_clusters):\n    for i, image_id in enumerate(image_ids[:size]):\n        image_id = image_id.split('.')[0]\n        image = cv2.imread(f'{DATA_PATH}/512x512-dataset-melanoma/512x512-dataset-melanoma/{image_id}.jpg', cv2.IMREAD_COLOR)\n        image = cv2.resize(image, (256, 256), cv2.INTER_AREA)\n        ax[j][i].imshow(image);\n        patient_id = df_folds.loc[image_id]['patient_id']\n        ax[j][i].set_title(f'{patient_id}\\n{image_id}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('-'*10 + '[imagededup]' + '-'*10)\nprint(f'[Clusters found]:', len(TRAIN_CLUSTERS))\nprint(f'[Precision]: ~{1 - round(len(BAD_CASES_CACHE) / (len(TRAIN_CLUSTERS) + len(BAD_CASES_CACHE)), 3)}')\nprint('-'*32)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# DBSCAN\n\n### Embeddings\n\nI would like to use [dataset](https://www.kaggle.com/shonenkov/melanoma-image-embeddings) for fast loading embeddings, if you need more information about getting this matrix you can see [version5](https://www.kaggle.com/shonenkov/dbscan-clustering-check-marking?scriptVersionId=36026391)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"TRAIN_DBSCAN_CLUSTERS = []\nTRAIN_DBSCAN_CLUSTERING_CACHE = set()\nBAD_CASES_DBSCAN_CACHE = set()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_embeddings = np.load('../input/melanoma-image-embeddings/train_embeddings.npy')\ntrain_image_names = json.load(open('../input/melanoma-image-embeddings/train_image_names.json', 'rb'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\nclusters = defaultdict(list)\nfor image_name, cluster_id in zip(train_image_names, DBSCAN(eps=3.0, min_samples=1, n_jobs=4).fit_predict(train_embeddings)):\n    clusters[cluster_id].append(image_name)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dbscan_clusters = sorted(clusters.items(), key=lambda x: -len(x[1]))\nsorted_dbscan_clusters = [\n    image_ids\n    for _, image_ids in dbscan_clusters if 1 < len(image_ids) <= 5\n]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BAD_CASES_DBSCAN = [\n    ['ISIC_0063674', 'ISIC_0073214'],\n    ['ISIC_0025789', 'ISIC_0031713'],\n    ['ISIC_0024371', 'ISIC_0064216'],\n    ['ISIC_2697083', 'ISIC_5258657'],\n    ['ISIC_1665944', 'ISIC_3475660'],\n    ['ISIC_0959735', 'ISIC_2028658'],\n    ['ISIC_0645454', 'ISIC_0851556'],\n    ['ISIC_0268080', 'ISIC_7364244'],\n    ['ISIC_0188432', 'ISIC_2459552'],\n    ['ISIC_0058863', 'ISIC_0062880', 'ISIC_0068056', 'ISIC_0068631'],\n    ['ISIC_1068686', 'ISIC_4214813', 'ISIC_5844037'],\n    ['ISIC_3593913', 'ISIC_4569978', 'ISIC_8509430'],\n]\n\nfor case in BAD_CASES_DBSCAN:\n    BAD_CASES_DBSCAN_CACHE.add('.'.join(sorted(case)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for image_ids in tqdm(sorted_dbscan_clusters, total=len(sorted_dbscan_clusters)):\n    sorted_cluster = []\n    for image_id in image_ids:\n        sorted_cluster.append(image_id)\n\n    sorted_cluster = sorted(sorted_cluster)\n    if len(sorted_cluster) > 1:\n        cluster_name = '.'.join(sorted_cluster)\n        if cluster_name in BAD_CASES_DBSCAN_CACHE:\n            continue\n        if cluster_name not in TRAIN_DBSCAN_CLUSTERING_CACHE:\n            TRAIN_DBSCAN_CLUSTERING_CACHE.add(cluster_name)\n            TRAIN_DBSCAN_CLUSTERS.append(sorted_cluster)\n\nTRAIN_DBSCAN_CLUSTERS = sorted(TRAIN_DBSCAN_CLUSTERS, key=lambda x: -len(x))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(TRAIN_DBSCAN_CLUSTERS)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"margin = 0\ncount = 20\n\ndraw_clusters = TRAIN_DBSCAN_CLUSTERS[margin:margin+count]\n\nsize = min([5, len(draw_clusters[0])])\n\nfig, ax = plt.subplots(count, size, figsize=(size*3, 4*count))\n\nfor j, image_ids in enumerate(draw_clusters):\n    for i, image_id in enumerate(image_ids[:size]):\n        image_id = image_id.split('.')[0]\n        image = cv2.imread(f'{DATA_PATH}/512x512-dataset-melanoma/512x512-dataset-melanoma/{image_id}.jpg', cv2.IMREAD_COLOR)\n        image = cv2.resize(image, (256, 256), cv2.INTER_AREA)\n        ax[j][i].imshow(image);\n        patient_id = df_folds.loc[image_id]['patient_id']\n        ax[j][i].set_title(f'{patient_id}\\n{image_id}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('-'*10 + '[DBSCAN]' + '-'*10)\nprint(f'[Clusters found]:', len(TRAIN_DBSCAN_CLUSTERS))\nprint(f'[Precision]: ~{1 - round(len(BAD_CASES_DBSCAN_CACHE) / (len(TRAIN_DBSCAN_CLUSTERS) + len(BAD_CASES_DBSCAN_CACHE)), 3)}')\nprint('-'*32)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Union Clusters","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"intersection = TRAIN_CLUSTERING_CACHE.intersection(TRAIN_DBSCAN_CLUSTERING_CACHE)\n\nprint(f'DBSCAN results contain {len(intersection)} cases from imagededup results ({len(TRAIN_CLUSTERING_CACHE)})')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Merge other cases","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def check_subcase(case):\n    case_image_ids = case.split('.')\n    for dbscan_case in TRAIN_DBSCAN_CLUSTERING_CACHE:\n        dbscan_case_image_ids = dbscan_case.split('.')\n        if len(set(dbscan_case_image_ids).intersection(case_image_ids)) == len(case_image_ids):\n            return True\n    return False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ORIGINALS_CLUSTERS = []\nfor case in list(TRAIN_CLUSTERING_CACHE.difference(intersection)):\n    if check_subcase(case):\n        continue\n    ORIGINALS_CLUSTERS.append(case.split('.'))\n\nlen(ORIGINALS_CLUSTERS)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"RESULT_CLUSTERS = sorted(ORIGINALS_CLUSTERS + TRAIN_DBSCAN_CLUSTERS, key=lambda x: -len(x))\nprint('-'*30)\nprint('CLUSTERS COUNT:', len(RESULT_CLUSTERS))\nprint('DUPLICATED IMAGES COUNT:', sum([len(image_ids) for image_ids in RESULT_CLUSTERS]) )\nprint('-'*30)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Simple EDA for Duplicates","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"TRAIN_ROOT_PATH = f'{DATA_PATH}/512x512-dataset-melanoma/512x512-dataset-melanoma'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data = []\nfor image_ids in RESULT_CLUSTERS:\n    sample = {}\n    sample['image_ids'] = '.'.join(image_ids)\n    sample.update(df_folds.loc[image_ids][['patient_id', 'target', 'source', 'sex', 'age_approx', 'anatom_site_general_challenge']].nunique())\n    data.append(sample)\n\ndata = pd.DataFrame(data)\nimage_ids = [image_id[0] for image_id in data['image_ids'].str.split('.')]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Target","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(df_folds.loc[image_ids]['target'].value_counts())\ndf_folds.loc[image_ids]['target'].hist();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Source","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(df_folds.loc[image_ids]['source'].value_counts())\ndf_folds.loc[image_ids]['source'].hist();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Sex","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(df_folds.loc[image_ids]['sex'].value_counts())\ndf_folds.loc[image_ids]['sex'].hist();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Age","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df_folds.loc[image_ids]['age_approx'].hist(bins=50);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Anatom","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df_folds.loc[image_ids]['anatom_site_general_challenge'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# MISTAKES in marking metadata:","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Diff Source","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"source_diff = data[data['source'] != 1]\ncount = source_diff.shape[0]\ncount","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(count, 2, figsize=(8, 2*2*count))\nfor j, (_, row) in enumerate(source_diff.iterrows()):\n    image_ids = row['image_ids'].split('.')\n    for i, image_id in enumerate(image_ids):\n        image = cv2.imread(f'{TRAIN_ROOT_PATH}/{image_id}.jpg', cv2.IMREAD_COLOR)\n        image = cv2.resize(image, (256, 256), cv2.INTER_AREA)\n        ax[j][i].imshow(image);\n        ax[j][i].set_title(f'{image_id}, source: {df_folds.loc[image_id].source}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Diff Target","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"target_diff = data[data['target'] != 1]\ncount = target_diff.shape[0]\ncount","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(count, 2, figsize=(8, 2*2*count))\nfor j, (_, row) in enumerate(target_diff.iterrows()):\n    image_ids = row['image_ids'].split('.')\n    for i, image_id in enumerate(image_ids):\n        image = cv2.imread(f'{TRAIN_ROOT_PATH}/{image_id}.jpg', cv2.IMREAD_COLOR)\n        image = cv2.resize(image, (256, 256), cv2.INTER_AREA)\n        ax[j][i].imshow(image);\n        ax[j][i].set_title(f'{image_id}, target: {df_folds.loc[image_id].target}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Diff Sex","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sex_diff = data[data['sex'] != 1]\ncount = sex_diff.shape[0]\ncount","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(count, 2, figsize=(8, 2*2*count))\nfor j, (_, row) in enumerate(sex_diff.iterrows()):\n    image_ids = row['image_ids'].split('.')\n    for i, image_id in enumerate(image_ids):\n        image = cv2.imread(f'{TRAIN_ROOT_PATH}/{image_id}.jpg', cv2.IMREAD_COLOR)\n        image = cv2.resize(image, (256, 256), cv2.INTER_AREA)\n        ax[j][i].imshow(image);\n        ax[j][i].set_title(f'{image_id}, sex: {df_folds.loc[image_id].sex}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Diff anatom_site_general_challenge","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"anatom_diff = data[data['anatom_site_general_challenge'] != 1]\ncount = anatom_diff.shape[0]\ncount","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(count, 2, figsize=(8, 2*2*count))\nfor j, (_, row) in enumerate(anatom_diff.iterrows()):\n    image_ids = row['image_ids'].split('.')\n    for i, image_id in enumerate(image_ids):\n        image = cv2.imread(f'{TRAIN_ROOT_PATH}/{image_id}.jpg', cv2.IMREAD_COLOR)\n        image = cv2.resize(image, (256, 256), cv2.INTER_AREA)\n        ax[j][i].imshow(image);\n        ax[j][i].set_title(f'{image_id}, {df_folds.loc[image_id].anatom_site_general_challenge}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Diff Patient_id","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"patient_diff = data[data['patient_id'] != 1]\ncount = patient_diff.shape[0]\ncount","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(count, 2, figsize=(8, 2*3*count))\nfor j, (_, row) in enumerate(patient_diff.iterrows()):\n    image_ids = row['image_ids'].split('.')[:2]\n    for i, image_id in enumerate(image_ids):\n        image = cv2.imread(f'{TRAIN_ROOT_PATH}/{image_id}.jpg', cv2.IMREAD_COLOR)\n        image = cv2.resize(image, (256, 256), cv2.INTER_AREA)\n        ax[j][i].imshow(image);\n        ax[j][i].set_title(f'{df_folds.loc[image_id].patient_id}\\n{image_id}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Diff Age approx","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"age_diff = data[data['age_approx'] != 1]\ncount = age_diff.shape[0]\ncount","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(count, 2, figsize=(8, 2*2*count))\nfor j, (_, row) in enumerate(age_diff.iterrows()):\n    image_ids = row['image_ids'].split('.')\n    for i, image_id in enumerate(image_ids):\n        image = cv2.imread(f'{TRAIN_ROOT_PATH}/{image_id}.jpg', cv2.IMREAD_COLOR)\n        image = cv2.resize(image, (256, 256), cv2.INTER_AREA)\n        ax[j][i].imshow(image);\n        ax[j][i].set_title(f'{image_id}, {df_folds.loc[image_id].age_approx}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Diff Common marking","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"marking_diff = data[\n    (data['target'] != 1) |\n    (data['sex'] != 1) |\n    (data['anatom_site_general_challenge'] != 1) |\n    (data['age_approx'] != 1)\n]\nmarking_diff.shape[0]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Save data with duplicates","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"data.to_csv('duplicates.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Clustering TEST\n\nWork in progress","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Thank you for reading my kernel! \n\nDon't forget to read my other kernel for this competition:\n\nTPU with PyTorch:\n\n- [[Torch XLA] Melanoma Crazy Fast](https://www.kaggle.com/shonenkov/torch-xla-melanoma-crazy-fast)\n- [[Inference] Melanoma Crazy Fast](https://www.kaggle.com/shonenkov/inference-melanoma-crazy-fast)\n\nGPU:\n\n- [[Training CV] Melanoma Starter](https://www.kaggle.com/shonenkov/training-cv-melanoma-starter)\n- [[Inference Single Model] Melanoma Starter](https://www.kaggle.com/shonenkov/inference-single-model-melanoma-starter)\n\nMerge external data:\n\n- [[Merge External Data]](https://www.kaggle.com/shonenkov/merge-external-data)","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}