{"cells":[{"metadata":{},"cell_type":"markdown","source":"# RAPIDS cuML kNN - Find Similar Images\nIn this notebook, we show how to use RAPIDS cuML kNN with CNN image embeddings to find similar images. This is a useful technique that can be used in many computer vision tasks. If you have one image and want to find similar images from a collection of images, you can use this technique. (For example, search internet images for similar images to a specific image, or search a database of images, or compete in Kaggle's Google Landmark Retrieval Competition [here][1])\n\nUsing this technique, we find 6 images in this competition's test dataset that are in last years 2019 train dataset. Therefore we know these 6 answers perfectly.\n\nExtracting CNN embeddings is a useful skill by itself. Once you have embeddings in a dataframe, you can train any ML model to classify Melanoma images using embeddings dataframe and ignoring the original images. And you can add more features like meta features to your embeddings for improved accuracy. I will demonstrate training a simple ML model using embeddings.\n\nLastly we will explore clusters of images in the embeddings space using RAPIDS cuML KMeans and RAPIDS cuML TSNE.\n\n[1]: https://www.kaggle.com/c/landmark-retrieval-2020\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Initialize Environment\nWe will use TensorFlow to extract CNN image embeddings and RAPIDS cuML kNN to compare them. We will restrict TensorFlow to only use 75% of GPU VRAM so that RAPIDS has 25% of GPU VRAM. Note that we only need to extract embeddings in the first version of this notebook and save them to a Kaggle dataset. Then in subsequent versions, we can load the embeddings from the Kaggle dataset. Do this if you wish to explore larger embeddings that take longer to extract.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"DIM = 256; EFFN = 0; BATCH_SIZE = 128\nLOAD_EMBEDDINGS = False\nif LOAD_EMBEDDINGS: print('We will read embeddings from Kaggle dataset')\nelse: print('We will extract embeddings from pretrained CNN')\nPATH_TO_EMBEDDINGS = '../input/embeddings-melanoma/'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# INSTALL RAPIDS\nimport sys\n!cp ../input/rapids/rapids.0.14.0 /opt/conda/envs/rapids.tar.gz\n!cd /opt/conda/envs/ && tar -xzvf rapids.tar.gz > /dev/null\nsys.path = [\"/opt/conda/envs/rapids/lib/python3.7/site-packages\"] + sys.path\nsys.path = [\"/opt/conda/envs/rapids/lib/python3.7\"] + sys.path\nsys.path = [\"/opt/conda/envs/rapids/lib\"] + sys.path \n!cp /opt/conda/envs/rapids/lib/libxgboost.so /opt/conda/lib/\n\n# INSTALL EFFICIENT NET\n!pip install -q efficientnet >> /dev/null","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# LOAD LIBRARIES\nimport pandas as pd, numpy as np\nimport tensorflow as tf, re, math\nimport tensorflow.keras.backend as K\nimport efficientnet.tfkeras as efn\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import roc_auc_score\nimport matplotlib.pyplot as plt, cv2\nimport cuml #, cupy\nprint('RAPIDS version',cuml.__version__)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# RESTRICT TENSORFLOW TO 12GB OF GPU RAM\n# SO THAT WE HAVE GPU RAM FOR RAPIDS CUML KNN\nLIMIT = 12\nif LOAD_EMBEDDINGS: LIMIT = 0\ngpus = tf.config.experimental.list_physical_devices('GPU')\nif gpus:\n  try:\n    tf.config.experimental.set_virtual_device_configuration(\n        gpus[0],\n        [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=1024*12)])\n    logical_gpus = tf.config.experimental.list_logical_devices('GPU')\n    print(len(gpus), \"Physical GPUs,\", len(logical_gpus), \"Logical GPUs\")\n  except RuntimeError as e:\n    print(e)\nprint('Restrict TensorFlow to max %iGB GPU RAM'%LIMIT)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# TFRecords Helper Functions\nThe following are helper functions to load TFRecords. This are from AgentAuers' notebook [here][1]\n\n[1]: https://www.kaggle.com/agentauers/incredible-tpus-finetune-effnetb0-b6-at-once","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def read_labeled_tfrecord(example):\n    tfrec_format = {\n        'image'                        : tf.io.FixedLenFeature([], tf.string),\n        'image_name'                   : tf.io.FixedLenFeature([], tf.string),\n        'target'                       : tf.io.FixedLenFeature([], tf.int64)\n    }           \n    example = tf.io.parse_single_example(example, tfrec_format)\n    return example['image'], example['target']\n\n\ndef read_unlabeled_tfrecord(example, return_image_name):\n    tfrec_format = {\n        'image'                        : tf.io.FixedLenFeature([], tf.string),\n        'image_name'                   : tf.io.FixedLenFeature([], tf.string),\n    }\n    example = tf.io.parse_single_example(example, tfrec_format)\n    return example['image'], example['image_name'] if return_image_name else 0\n\n \ndef prepare_image(img, augment=True, dim=256):    \n    img = tf.image.decode_jpeg(img, channels=3)\n    # NORMALIZE IMAGES TO IMAGENET PRETRAIN\n    img = ((tf.cast(img, tf.float32) / 255.0) - 0.449) / 0.226                      \n    img = tf.reshape(img, [dim,dim, 3])\n    return img\n\ndef count_data_items(filenames):\n    n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) \n         for filename in filenames]\n    return np.sum(n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_dataset(files, augment = False, shuffle = False, repeat = False, \n                labeled=True, return_image_names=True, batch_size=32, dim=128):\n    \n    ds = tf.data.TFRecordDataset(files, num_parallel_reads=AUTO)\n    ds = ds.cache()\n    \n    if repeat:\n        ds = ds.repeat()\n    \n    if shuffle: \n        ds = ds.shuffle(1024*8)\n        opt = tf.data.Options()\n        opt.experimental_deterministic = False\n        ds = ds.with_options(opt)\n        \n    if labeled: \n        ds = ds.map(read_labeled_tfrecord, num_parallel_calls=AUTO)\n    else:\n        ds = ds.map(lambda example: read_unlabeled_tfrecord(example, return_image_names), \n                    num_parallel_calls=AUTO)      \n    \n    ds = ds.map(lambda img, imgname_or_label: (prepare_image(img, augment=augment, dim=dim), \n                                               imgname_or_label), \n                num_parallel_calls=AUTO)\n    \n    ds = ds.batch(batch_size * REPLICAS)\n    ds = ds.prefetch(AUTO)\n    return ds","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Extract Image Features\nWe will extract image embeddings from EfficientNet B0 using images of size `256x256`. If the variable `LOAD_EMBEDDINGS=True` then we will load them from a previous notebook version Kaggle dataset. If `LOAD_EMBEDDINGS=False`, we will extract them. Note that we are using `noisy-student` pretrained weights for better features. And in the TFRecord read function above, we normalize our inputs by subtracting ImageNet mean of `0.449` and dividing by ImageNet standard deviation of `0.226`. This is important for image feature extraction because we are not training the EfficientNet anymore. If we were training more, then the model would correct itself if we don't normalize.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# EXTRACT LAST LAYER OF EFFICIENT NET WITH GLOBAL AVERAGE POOLING\nEFNS = [efn.EfficientNetB0, efn.EfficientNetB1, efn.EfficientNetB2, efn.EfficientNetB3, \n        efn.EfficientNetB4, efn.EfficientNetB5, efn.EfficientNetB6]\n\ndef build_model(dim=256, ef=EFFN):\n    inp = tf.keras.layers.Input(shape=(dim,dim,3))\n    base = EFNS[ef](input_shape=(dim,dim,3),weights='imagenet',include_top=False)\n    x = base(inp)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    model = tf.keras.Model(inputs=inp,outputs=x)\n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# TFRECORDS PATH\nGCS_PATH = '../input/melanoma-%ix%i'%(DIM,DIM) # GPU does not need KaggleDatasets()\nfiles_train = np.sort(np.array(tf.io.gfile.glob(GCS_PATH + '/train*.tfrec')))\nfiles_test  = np.sort(np.array(tf.io.gfile.glob(GCS_PATH + '/test*.tfrec')))\nAUTO     = tf.data.experimental.AUTOTUNE; REPLICAS = 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Test Image Embeddings","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"if not LOAD_EMBEDDINGS:\n    model = build_model(dim=DIM,ef=EFFN)\n    ds_test = get_dataset(files_test,labeled=False,return_image_names=False,augment=False,\n            repeat=False,shuffle=False,dim=DIM,batch_size=BATCH_SIZE)\n    embed_test = model.predict(ds_test,verbose=1)\n    np.save('embed_test_%i_%i'%(DIM,EFFN),embed_test.astype('float32'))\nelse:\n    embed_test = np.load(PATH_TO_EMBEDDINGS+'embed_test_%i_%i.npy'%(DIM,EFFN))\nprint('test embeddings shape',embed_test.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Train Image Embeddings","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"if not LOAD_EMBEDDINGS:\n    model = build_model(dim=DIM,ef=EFFN)\n    ds_train = get_dataset(files_train,labeled=False,return_image_names=False,augment=False,\n            repeat=False,shuffle=False,dim=DIM,batch_size=BATCH_SIZE)\n    embed = model.predict(ds_train,verbose=1)\n    np.save('embed_train_%i_%i'%(DIM,EFFN),embed.astype('float32'))\nelse:\n    embed = np.load(PATH_TO_EMBEDDINGS+'embed_train_%i_%i.npy'%(DIM,EFFN))\nprint('train embeddings shape',embed.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## External Image Embeddings (2019 comp data)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"GCS_PATH2 = '../input/isic2019-%ix%i'%(DIM,DIM) # GPU does not need KaggleDatasets()\nfiles_ext = np.sort(np.array(tf.io.gfile.glob(GCS_PATH2 + '/train*.tfrec')))\n\nif not LOAD_EMBEDDINGS:\n    model = build_model(dim=DIM,ef=EFFN)\n    ds_ext = get_dataset(files_ext,labeled=False,return_image_names=False,augment=False,\n            repeat=False,shuffle=False,dim=DIM,batch_size=BATCH_SIZE)\n    embed_ext = model.predict(ds_ext,verbose=1)\n    np.save('embed_ext_%i_%i'%(DIM,EFFN),embed_ext.astype('float32'))\nelse:\n    embed_ext = np.load(PATH_TO_EMBEDDINGS+'embed_ext_%i_%i.npy'%(DIM,EFFN))\nprint('ext embeddings shape',embed_ext.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Train Test Image Names\nHere we extract the image names from TFRecords. We use our smallest TFRecord of size `128x128` for maximum speed.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# WE WILL READ IMAGE NAMES FROM 128X128 IMAGES for speed\nif not LOAD_EMBEDDINGS:\n    DIM = 128; BATCH_SIZE = 32\n    GCS_PATH = '../input/melanoma-%ix%i'%(DIM,DIM)\n    files_train = np.sort(np.array(tf.io.gfile.glob(GCS_PATH + '/train*.tfrec')))\n    files_test  = np.sort(np.array(tf.io.gfile.glob(GCS_PATH + '/test*.tfrec')))\n\n# READ TEST IMAGE NAMES\nif not LOAD_EMBEDDINGS:\n    ds_test = get_dataset(files_test,labeled=False,return_image_names=True,augment=False,\n            repeat=False,shuffle=False,dim=DIM,batch_size=BATCH_SIZE)\n    names_test = np.array([img_name.numpy().decode(\"utf-8\") for img, img_name in iter(ds_test.unbatch())])\n    np.save('names_test',names_test)\nelse:\n    names_test = np.load(PATH_TO_EMBEDDINGS+'names_test.npy')\nprint('test names',names_test.shape)\n\n# READ TRAIN IMAGE NAMES\nif not LOAD_EMBEDDINGS:\n    ds_train = get_dataset(files_train,labeled=False,return_image_names=True,augment=False,\n            repeat=False,shuffle=False,dim=DIM,batch_size=BATCH_SIZE)\n    names = np.array([img_name.numpy().decode(\"utf-8\") for img, img_name in iter(ds_train.unbatch())])\n    np.save('names_train',names)\nelse:\n    names = np.load(PATH_TO_EMBEDDINGS+'names_train.npy')\nprint('train names',names.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## External Image Names","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# READ EXT IMAGE NAMES\nif not LOAD_EMBEDDINGS:\n    GCS_PATH2 = '../input/isic2019-%ix%i'%(DIM,DIM) # GPU does not need KaggleDatasets()\n    files_ext = np.sort(np.array(tf.io.gfile.glob(GCS_PATH2 + '/train*.tfrec')))\n\nif not LOAD_EMBEDDINGS:\n    ds_ext = get_dataset(files_ext,labeled=False,return_image_names=True,augment=False,\n            repeat=False,shuffle=False,dim=DIM,batch_size=BATCH_SIZE)\n    names_ext = np.array([img_name.numpy().decode(\"utf-8\") for img, img_name in iter(ds_ext.unbatch())])\n    np.save('names_ext',names_ext)\nelse:\n    names_ext = np.load(PATH_TO_EMBEDDINGS+'names_ext.npy')\nprint('ext names',names_ext.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# CSV Files","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# LOAD TRAIN AND TEST CSV\ntest = pd.read_csv( '../input/siim-isic-melanoma-classification/test.csv' ).set_index('image_name',drop=True)\ntest = test.loc[names_test].reset_index()\nprint('Test csv shape',test.shape)\n\ntrain = pd.read_csv( '../input/melanoma-%ix%i/train.csv'%(DIM,DIM) ).set_index('image_name',drop=True)\ntrain = train.loc[names].reset_index()\ntrain.target = train.target.astype('float32')\nprint('Train csv shape',train.shape)\n\n# LOAD EXTERNAL DATA CSV\ntrain_ext = pd.read_csv( '../input/isic2019-%ix%i/train.csv'%(DIM,DIM) ).set_index('image_name',drop=True)\ntrain_ext = train_ext.loc[names_ext].reset_index()\ntrain_ext.target = train_ext.target.astype('float32')\nprint('Train_ext csv shape',train_ext.shape)\n\nprint('Displaying train.csv below...')\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# RAPIDS cuML kNN - Find Duplicates","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Find Duplicates","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"KNN = 3\nmodel = cuml.neighbors.NearestNeighbors(n_neighbors=KNN)\nmodel.fit(embed_ext)\ndistances, indices = model.kneighbors(embed_test)\n\nmm = np.min(distances,axis=1)\nplt.title('Shortest Distances of Test Images to any 2019 Image')\nplt.hist(mm)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"CUTOFF = 2\nidx = np.where( (mm<CUTOFF) )[0]\nprint('There are %i potential duplicate images that have distance < %i'%(len(idx),CUTOFF))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Display Duplicates","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"PATH_TEST = '../input/jpeg-melanoma-128x128/test/'\nPATH_EXT = '../input/jpeg-isic2019-128x128/train/'\n\na = []; b = []; c = []\n\nfor k in idx:\n    \n    a.append(names_test[k])\n    b.append(train_ext.target.iloc[int(indices[k,0])])\n    c.append(names_ext[int(indices[k,0])])\n    \n    plt.figure(figsize=(10,5))\n    \n    plt.subplot(1,2,1)\n    img = cv2.imread(PATH_TEST+names_test[k]+'.jpg')\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    plt.imshow(img)\n    plt.title('2020 Test Image - Target = ?\\n%s'%names_test[k])\n\n    plt.subplot(1,2,2)\n    img = cv2.imread(PATH_EXT+names_ext[int(indices[k,0])]+'.jpg')\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    plt.imshow(img)\n    t = train_ext.target.iloc[int(indices[k,0])]\n    plt.title('2019 Image - Target = %i\\n%s'%(t,names_ext[int(indices[k,0])]))\n    \n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# RAPIDS cuML kNN - Train and Predict Test\nLet's build a simple kNN model using the embeddings. We will use triple stratified KFold validation and inference.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"skf = KFold(n_splits=5,shuffle=True,random_state=42)\noof = np.zeros((train.shape[0]))\nfolds = np.zeros((train.shape[0]))\npreds = np.zeros((test.shape[0]))\n\nfor fold,(idxT,idxV) in enumerate(skf.split(np.arange(15))):\n    \n    # DISPLAY FOLD INFO\n    print('#'*25); print('#### FOLD',fold+1)\n    \n    # CREATE TRAIN AND VALIDATION SUBSETS\n    idxT2 = train.loc[train.tfrecord.isin(idxT)].index.values #2020 train\n    idxV2 = train.loc[train.tfrecord.isin(idxV)].index.values #2020 valid\n    \n    model = cuml.neighbors.KNeighborsClassifier(n_neighbors=299)\n    model.fit(embed[idxT2,],train.target.values[idxT2])\n        \n    oof[idxV2] = model.predict_proba(embed[idxV2,])[:,1]  \n    preds += model.predict_proba(embed_test)[:,1] / skf.n_splits\n    \n    # REPORT RESULTS\n    folds[idxV2] = fold\n    auc = roc_auc_score(train.target.values[idxV2],oof[idxV2])\n    print('#### OOF AUC = %.3f'%auc)\n    print('#'*25); print()    \n    \nauc = roc_auc_score(train.target.values,oof)\nprint('Overall OOF AUC = %.3f'%auc)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Submit to Kaggle","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# WRITE OOF TO DISK\ntrain['pred'] = oof; train['fold'] = folds\ntrain[['image_name','target','pred','fold']].to_csv('oof.csv',index=False)\ntrain[['image_name','target','pred','fold']].head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# WRITE SUBMISSION TO DISK\nsubmission = pd.DataFrame(dict(image_name=names_test, target=preds))\nsubmission = submission.sort_values('image_name') \nsubmission.to_csv('submission.csv', index=False)\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# PLOT TEST PREDICTION HISTOGRAM\nplt.hist(submission.target,bins=50)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# APPENDIX\nIn this appendix, we will explore our embeddings","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# RAPIDS cuML KMeans\nEmbeddings from EfficientNetB0 has dimension 1280. So each image is now represented by a point in 1280 dimension space. Let's cluster these points into 20 clusters and display what images look like in the different clusters.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"CLUSTERS = 20\nmodel = cuml.KMeans(n_clusters=CLUSTERS)\nmodel.fit(embed)\ntrain['cluster'] = model.labels_\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"PATH_TRAIN = '../input/jpeg-melanoma-128x128/train/'\n\nfor k in range(CLUSTERS):\n    print('#'*25);\n    print('#### Cluster %i of similar train images'%k)\n    print('#'*25)\n    df = train.loc[train.cluster==k]\n    plt.figure(figsize=(20,10))\n    for j in range(8):\n        plt.subplot(2,4,j+1)\n        img = cv2.imread(PATH_TRAIN+names[df.index[j]]+'.jpg')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        plt.axis('off')\n        plt.title('%s, Target = %i'%(names[df.index[j]],df.loc[df.index[j],'target']))\n        plt.imshow(img)  \n    plt.show()\n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# RAPIDS cuML TSNE\nEmbeddings from EfficientNetB0 has dimension 1280. So each image is now represented by a point in 1280 dimension space. Let's map this 1280 dimensional space to 2 dimensions. Below we plot each image's new 2 dimension representation. We color benign orange and malignant blue. Next we will find a random square in this 2 dimensional space and plot some of the images inside the square.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"model = cuml.TSNE()\nembed2D = model.fit_transform(embed)\ntrain['x'] = embed2D[:,0]\ntrain['y'] = embed2D[:,1]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"X_DIV = 10; Y_DIV = 10\nx_min = train.x.min()\nx_max = train.x.max()\ny_min = train.y.min()\ny_max = train.y.max()\nx_step = (x_max - x_min)/X_DIV\ny_step = (y_max - y_min)/Y_DIV\nmx = 0; xa_mx = 0; xb_mx=0; ya_mx = 0; yb_mx = 0\nfor k in range(X_DIV+1):\n    for j in range(Y_DIV+1):\n        xa = k*x_step + x_min\n        xb = (k+1)*x_step + x_min\n        ya = j*y_step + y_min\n        yb = (j+1)*y_step + y_min\n        df = train.loc[(train.x>xa)&(train.x<xb)&(train.y>ya)&(train.y<yb)]\n        t = df.target.mean()\n        if (t>mx)&(len(df)>=16):\n            mx = t\n            xa_mx = xa\n            xb_mx = xb\n            ya_mx = ya\n            yb_mx = yb\n        #print(k,j,t)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(10,10))\ndf1 = train.loc[train.target==0]\nplt.scatter(df1.x,df1.y,color='orange',s=10,label='Benign')\ndf2 = train.loc[train.target==1]\nplt.scatter(df2.x,df2.y,color='blue',s=10,label='Malignant')\nplt.plot([xa_mx,xa_mx],[ya_mx,yb_mx],color='black')\nplt.plot([xa_mx,xb_mx],[ya_mx,ya_mx],color='black')\nplt.plot([xb_mx,xb_mx],[ya_mx,yb_mx],color='black')\nplt.plot([xa_mx,xb_mx],[yb_mx,yb_mx],color='black')\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"df = train.loc[(train.x>xa_mx)&(train.x<xb_mx)&(train.y>ya_mx)&(train.y<yb_mx)]\nprint('This region has %.2f %% malignant'%(100*df.target.mean())) \n\nROW = 4\nCOL = 4\nplt.figure(figsize=(20,20*ROW/COL))\nfor k in range(ROW):\n    for j in range(COL):\n        plt.subplot(ROW,COL,k*COL+j+1)\n        img = cv2.imread(PATH_TRAIN+names[df.index[k*COL+j]]+'.jpg')\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        plt.axis('off')\n        plt.title('%s, Target = %i'%(names[df.index[k*COL+j]],df.loc[df.index[k*COL+j],'target']))\n        plt.imshow(img)  \nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"for it in range(5):\n    i = 0\n    while i<16:\n        k = np.random.randint(0,X_DIV)\n        j = np.random.randint(0,Y_DIV)\n        xa_mx = k*x_step + x_min\n        xb_mx = (k+1)*x_step + x_min\n        ya_mx = j*y_step + y_min\n        yb_mx = (j+1)*y_step + y_min\n        df = train.loc[(train.x>xa_mx)&(train.x<xb_mx)&(train.y>ya_mx)&(train.y<yb_mx)]\n        i = len(df)\n\n    plt.figure(figsize=(10,10))\n    df1 = train.loc[train.target==0]\n    plt.scatter(df1.x,df1.y,color='orange',s=10,label='Benign')\n    df2 = train.loc[train.target==1]\n    plt.scatter(df2.x,df2.y,color='blue',s=10,label='Malignant')\n    plt.plot([xa_mx,xa_mx],[ya_mx,yb_mx],color='black')\n    plt.plot([xa_mx,xb_mx],[ya_mx,ya_mx],color='black')\n    plt.plot([xb_mx,xb_mx],[ya_mx,yb_mx],color='black')\n    plt.plot([xa_mx,xb_mx],[yb_mx,yb_mx],color='black')\n    plt.legend()\n    plt.show()\n\n    print('This region has %.2f %% malignant'%(100*df.target.mean()))    \n    \n    ROW = 4\n    COL = 4\n    plt.figure(figsize=(20,20*ROW/COL))\n    for k in range(ROW):\n        for j in range(COL):\n            plt.subplot(ROW,COL,k*COL+j+1)\n            img = cv2.imread(PATH_TRAIN+names[df.index[k*COL+j]]+'.jpg')\n            img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n            plt.axis('off')\n            plt.title('%s, Target = %i'%(names[df.index[k*COL+j]],df.loc[df.index[k*COL+j],'target']))\n            plt.imshow(img)  \n    plt.show()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}