{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Hello, this seems like a fun task so I thought it would good to write up a simple transfer learning model to see the results. I tend to write my notebooks as a tutorial format for those who are unfamiliar with tensorflow v2 and visual recognition. Feel free to skip all of the comments if it is annoying!\n\nThe following works were referenced when creating this tutorial:\n* https://www.kaggle.com/drcapa/human-trafficking-2021-starter \n\n* https://www.kaggle.com/shanmukh05/combat-human-trafficking-2k21-tpu-training \n\n* https://www.kaggle.com/ateplyuk/human-trafficking-2021-baseline \n\n\n\nAll mistakes are self-inflicted; please notify me if any mistakes are spotted.\n\n","metadata":{}},{"cell_type":"markdown","source":"**Loading in Libraries**","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nfrom kaggle_datasets import KaggleDatasets\nimport numpy as np\nimport pandas as pd\nimport os\nfrom sklearn import preprocessing\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"First things first, lets initalize the kaggle's accelerators units for faster computation. This is done by changing the notebook settings in the top right. You will find the TPU and GPU setting under Hardware accelerator. Unfortunately, for this task, you will have to first train the model using TPU, and then submit the best model that you found.","metadata":{}},{"cell_type":"code","source":"\ndef intialize_accel(hardware):\n    \"\"\"\n    input:\n    str: GPU or TPU for hardware accelerator\n    \n    output:\n    strategy -- used later for model definition and fitting\n    \"\"\"\n    if hardware =='TPU':\n        try:\n            tpu = tf.distribute.cluster_resolver.TPUClusterResolver().connect()  \n            strategy = tf.distribute.experimental.TPUStrategy(tpu)\n\n            print('TPU Initialized')\n            print(\"TPU Units:\", strategy.num_replicas_in_sync)\n            return strategy\n        except:\n            print('TPU Initialization Failed')\n    \n    \n    elif hardware == 'GPU':\n        print(\"Num GPUs Available: \", len(tf.config.list_physical_devices('GPU')))\n        strategy = tf.distribute.MirroredStrategy()\n        return strategy\n        \nstrategy = intialize_accel('TPU')\nAUTO = tf.data.experimental.AUTOTUNE\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Okay, now that the accelerator units are initalized: lets start exploring our data","metadata":{}},{"cell_type":"code","source":"#Setting up Pathways\n#DIR = '../input/hotel-id-2021-fgvc8/'\nDIR = KaggleDatasets().get_gcs_path()\n\nTrain_PATH = DIR + \"/train_images/\"\nTest_PATH = DIR + \"/test_images/\"\n\n##Loading in Training csv\ntrain_df = pd.read_csv(\"../input/hotel-id-2021-fgvc8/train.csv\")\n#dropping duplicates\ntrain_df = train_df.drop_duplicates(subset=['image'])\n\n#checking data properties\nprint(\"Number of unique hotel chains: \",train_df.chain.nunique())\nprint(\"Number of unique hotels: \",train_df.hotel_id.nunique())\nprint(\"Number of Training Samples: \", train_df.shape[0])\n\nprint(train_df.head())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Great! The hotel id is what we want, so our final model should output ~7000 classes. Now let us define our parameters. There are two types; data-specific parameters and training parameters. Feel free to play around with the training parameters.\n\n* Class: Number of classes\n* Channels: Number of color channesl\n* Batch_size: Size of the batches\n* size: Size of the final processed image\n* Split: Size of training vs validation split","metadata":{}},{"cell_type":"code","source":"#Specific to data\nClasses =  train_df.hotel_id.nunique()\nChannels = 3\n\n##Specific to the model\nBatch_size = 8 * strategy.num_replicas_in_sync\nsize = (200,200)\nSplit = int(0.9*train_df.shape[0])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now lets start processing our data. First, let us change the encoding of our hotel ids so that they are lined with each other. This can simply be done using the sklearn preprocessing label encoder. ","metadata":{}},{"cell_type":"code","source":"le = preprocessing.LabelEncoder()\ntrain_df['label'] = le.fit_transform(train_df['hotel_id'])\nprint(train_df[['hotel_id','label']])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We are then going to process our input data. To do this in keras, we are going to formulate the data into datasets and pass the datasets directly to our model. It is also here where we define our data augmentation.","metadata":{}},{"cell_type":"code","source":"def image_proces(Path, labels):\n    \"\"\"\n    inputs: Tensorflow Dataset that contains the following two properties\n        Path: Paths to images\n        labels: image labels\n        \n    output: Tensorflow Dataset that contains\n        data: processed images\n        labels: image labels\n    \n    Function:\n    Takes a tensorflow dataset of image paths and decodes the images into numpy arrays. The images are then resized.\n    \"\"\"\n    data = tf.io.read_file(Path)\n    data = tf.image.decode_jpeg(data, channels=3)\n    data = tf.image.resize(data, size)\n    #data = tf.image.per_image_standardization(data)\n    return data,labels\n\ndef import_image(Paths,labels):\n    \"\"\"\n    inputs:\n    Paths: a list of paths to the images\n    Y: a list of labels (hotel ids)\n    \n    outputs:\n    dataset: a tensorflow datset that contains images which are decoded and resized\n    \n    Function: \n    Takes a list of paths and returns a formatted tensorflow dataset\n    \"\"\"\n    dataset = tf.data.Dataset.from_tensor_slices((Paths,labels))\n    dataset = dataset.map(image_proces,num_parallel_calls=AUTO)\n    return dataset\n\ndef data_augment(image, labels):\n    \"\"\"\n    inputs: Tensorflow Dataset that contains the following two properties\n        Images: Images arrays\n        labels: image labels\n        \n    output: Tensorflow Dataset that contains\n        Image: processed images\n        labels: image labels\n    \n    Function:\n    Takes a tensorflow dataset of image arrays and augments the brightness and contrast.\n    \"\"\"\n    #image = tf.image.random_flip_left_right(image)\n    #image = tf.image.random_flip_up_down(image)\n    image = tf.image.random_brightness(image, max_delta=0.1)\n    image = tf.image.random_contrast(image,lower=0.8,upper=1.2)\n    return image, labels\n\n\nPaths = Train_PATH + train_df['chain'].astype(str) + '/' + train_df['image']\nPaths_train  = Paths[:Split]\nPaths_valid  = Paths[Split:]\nPaths_Test = tf.io.gfile.glob(Test_PATH + '*.jpg')\n\n\ntrain_dataset= import_image(Paths_train,train_df['label'][:Split])\n\n\nval_dataset = import_image(Paths_valid,train_df['label'][Split:])\ndataset_Test = import_image(Paths_Test,np.arange(len(Paths_Test)))\n\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A quick sanity check: 90% training, 10% validation samples. The input shape matches our intended value.","metadata":{}},{"cell_type":"code","source":"print(\"Number of Validation Samples:\",val_dataset.cardinality().numpy())\nprint(\"Number of Train Samples:\",train_dataset.cardinality().numpy())\nprint(\"Number of Test Samples:\",dataset_Test.cardinality().numpy())\n\ninspect= list(train_dataset.take(1).as_numpy_iterator())\nprint('Input Shape:',inspect[0][0].shape, 'Example Label:',inspect[0][1])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here we are going to define our model. We will be using a pretrained image recognition network (ResNet50) to recognize important features and then train a neural network on top to recognize the hotel. Rather than training the whole thing, we will be fixing the resnet (for now) and training the classification network on top. Since we have $\\sim$ 7000 classes, we would need to first perform feature extraction before classifcation, or else our model would be far too large ($\\sim$9 million parameters) to train properly. ","metadata":{}},{"cell_type":"code","source":"input_shape=[200,200,Channels]\nBase = tf.keras.applications.ResNet50(include_top=False,input_shape=input_shape)\nBase.trainable = False\n#Base.summary()\n\n\ndef create_model(Base,input_shape):\n    inputs = tf.keras.Input(shape=(input_shape))\n    norm =  tf.keras.layers.experimental.preprocessing.Normalization()\n    x = norm(inputs)\n    x = Base(x,training=False)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    x = tf.keras.layers.Dense(50,activation=\"relu\", dtype='float32')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    outputs = tf.keras.layers.Dense(Classes,activation=\"softmax\", dtype='float32')(x)\n        \n    model = tf.keras.Model(inputs, outputs)\n    return model\n\n\nmodel = create_model(Base,input_shape)\n\nmodel.summary()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we define our compiler with ADAM optimizer. Since we didn't encode our labels into one hot representations, we will be using the sparse categorical cross entropy. Additionally, since I am using the TPU accelerator, we can process multiple step in parallel. This is controlled by the steps_per_execution hyperparameter","metadata":{}},{"cell_type":"code","source":"import tensorflow_addons as tfa\n\ndef compile_model(model, lr):\n    \n    optimizer = tf.keras.optimizers.Adam(lr=lr)\n    \n    loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\n    metrics = [tf.keras.metrics.SparseCategoricalAccuracy(name='accuracy')]\n\n    model.compile(optimizer=optimizer, loss=loss, metrics=metrics,steps_per_execution=8)\n    \n\n    return model","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Final steps of data processing: all we are doing here is preparing our training and validation datasets for training. \n\n* Cache: Stores the dataset after first pass (This might result in memory issues!)\n* batch: batches the data up\n* shuffle: shuffles the training data\n* prefetch: prefetches the next batch during training\n","metadata":{}},{"cell_type":"code","source":"EPOCHS= 1\nVERBOSE =1\nTrain_steps  = Split//Batch_size\n\ntrain_dataset = train_dataset.cache()\ntrain_dataset = train_dataset.map(data_augment,num_parallel_calls=AUTO)\ntrain_dataset = train_dataset.shuffle(2048).batch(Batch_size)\ntrain_dataset = train_dataset.prefetch(AUTO)\n\nval_dataset = val_dataset.cache()\nval_dataset = val_dataset.batch(Batch_size)\nval_dataset = val_dataset.prefetch(AUTO)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Setting up callbacks which will runned at the end of every epoch of training.","metadata":{}},{"cell_type":"code","source":"reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_accuracy', factor=0.1,\n                              patience=3, mode='max', min_delta=0.0001,verbose=1)\n\ncheckpoint_filepath = './best_model.h5'\nmodel_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(\n                            filepath=checkpoint_filepath,\n                            save_weights_only=True,\n                            monitor='val_accuracy',\n                            mode='max',\n                            save_best_only=True,verbose=1)\n\n\ncallback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=10, mode='max', min_delta=0.0001,verbose=1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now let's start training!","metadata":{}},{"cell_type":"code","source":"\nwith strategy.scope():\n    Base = tf.keras.applications.ResNet50(include_top=False,input_shape=input_shape)\n    Base.trainable = False\n    model = create_model(Base,input_shape)\n    model = compile_model(model, lr=0.001)\n   \n\nprint('Fitting') \n\nHistory = model.fit(train_dataset, \n                epochs=EPOCHS,\n                callbacks=[reduce_lr,model_checkpoint_callback,callback],\n                validation_data = val_dataset,  \n                verbose=VERBOSE\n               )","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from matplotlib import pyplot as plt\nplt.figure(1)\nplt.plot(History.history['accuracy'][1:],label='Train')\nplt.plot(History.history['val_accuracy'][1:],label='Valid')\nplt.title('Accuracy')\nplt.xlabel('Epoch')\nplt.legend()\n\nplt.figure(2)\nplt.plot(History.history['loss'][1:],label='Train')\nplt.plot(History.history['val_loss'][1:],label='Valid')\nplt.xlabel('Epoch')\nplt.title('Loss')\nplt.legend()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now let's load up the best model and see how it performs on the test set.","metadata":{}},{"cell_type":"code","source":"best_model = create_model(Base,input_shape)\nbest_model.load_weights(checkpoint_filepath)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dataset_Test = dataset_Test.batch(3)\npredictions = best_model.predict(dataset_Test,verbose=1)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"images_names= [i.split('/')[-1] for i in Paths_Test]\nhotels = le.inverse_transform(np.argmax(predictions,axis=1))\nsubmission = pd.DataFrame(list(zip(images_names,hotels)), columns=['image','hotel_id'])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv(\"submission.csv\", index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}