{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Neural Network Basic Using Tensorflow\n","metadata":{}},{"cell_type":"markdown","source":"#  Importing the libraries","metadata":{}},{"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport tensorflow as tf","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:20.626840Z","iopub.execute_input":"2022-07-07T05:02:20.628068Z","iopub.status.idle":"2022-07-07T05:02:20.633578Z","shell.execute_reply.started":"2022-07-07T05:02:20.627993Z","shell.execute_reply":"2022-07-07T05:02:20.632749Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Read the Dataset","metadata":{}},{"cell_type":"code","source":"data= pd.read_csv('../input/titanic/train.csv')","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:20.670092Z","iopub.execute_input":"2022-07-07T05:02:20.670788Z","iopub.status.idle":"2022-07-07T05:02:20.683304Z","shell.execute_reply.started":"2022-07-07T05:02:20.670741Z","shell.execute_reply":"2022-07-07T05:02:20.682087Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(data.shape)\ndata.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:20.724987Z","iopub.execute_input":"2022-07-07T05:02:20.727673Z","iopub.status.idle":"2022-07-07T05:02:20.746930Z","shell.execute_reply.started":"2022-07-07T05:02:20.727634Z","shell.execute_reply":"2022-07-07T05:02:20.745710Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Select the required data frames and by intuition class, fare, age and sex are the most important predictors. \nThis is known because higher class people were given priority over the lower class. Sex is important because women were given preference over men. Age becomes a predictor because children were also given preference. \nSo as per my intuition lower class men were most likely to die. \n\nBut we can’t rely on intuition only so let's go for the machine learning algorithm. \n","metadata":{}},{"cell_type":"code","source":"data = data[['Survived', 'Pclass', 'Sex', 'Age', 'Fare']]","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:20.768158Z","iopub.execute_input":"2022-07-07T05:02:20.768554Z","iopub.status.idle":"2022-07-07T05:02:20.774344Z","shell.execute_reply.started":"2022-07-07T05:02:20.768520Z","shell.execute_reply":"2022-07-07T05:02:20.773139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = data.dropna()\nprint(data.shape)","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:20.813083Z","iopub.execute_input":"2022-07-07T05:02:20.813791Z","iopub.status.idle":"2022-07-07T05:02:20.823071Z","shell.execute_reply.started":"2022-07-07T05:02:20.813741Z","shell.execute_reply":"2022-07-07T05:02:20.821965Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"target = data.pop('Survived')","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:20.855975Z","iopub.execute_input":"2022-07-07T05:02:20.856660Z","iopub.status.idle":"2022-07-07T05:02:20.861676Z","shell.execute_reply.started":"2022-07-07T05:02:20.856622Z","shell.execute_reply":"2022-07-07T05:02:20.860805Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As per the datasets out of 800 people around 500 people died and the rest survived. \n\nNow that the NULL values have been dropped and the target is separated from the data time to build the machine learning pipeline. \n\nOne hot encoding on the categorical dataset and normalisation of the numeric dataset. \n Normalisation is to make sure that the dataset fits between 0 and 1. \nOne hot encoding creates a new column for each category. All are filled with 0s and 1s. 1s refer to the existence of that category for the row. \n\n\n# Split the dataset into parts for the same\n","metadata":{}},{"cell_type":"code","source":"categorical_feature_names = ['Pclass','Sex']\nnumeric_feature_names = ['Fare', 'Age']\npredicted_feature_name = ['Survived']","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:20.897441Z","iopub.execute_input":"2022-07-07T05:02:20.898274Z","iopub.status.idle":"2022-07-07T05:02:20.903146Z","shell.execute_reply.started":"2022-07-07T05:02:20.898230Z","shell.execute_reply":"2022-07-07T05:02:20.902135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### To feed the dataset to Tensorflow it must be pre-processed in a certain way. The first task is to create the tensor dictionary \n","metadata":{}},{"cell_type":"code","source":"def create_tensor_dict(data, categorical_feature_names):\n    inputs = {}\n    for name, column in data.items():\n      if type(column[0]) == str:\n        dtype = tf.string\n      elif (name in categorical_feature_names):\n        dtype = tf.int64\n      else:\n        dtype = tf.float32\n\n      inputs[name] = tf.keras.Input(shape=(), name=name, dtype=dtype)\n    return inputs\n\ninputs = create_tensor_dict(data, categorical_feature_names)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:20.949665Z","iopub.execute_input":"2022-07-07T05:02:20.950423Z","iopub.status.idle":"2022-07-07T05:02:20.967326Z","shell.execute_reply.started":"2022-07-07T05:02:20.950386Z","shell.execute_reply":"2022-07-07T05:02:20.966039Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here each column is assigned a particular TensorFlow datatype based on its current datatype and a dictionary is created to uniquely identify each column and its data type. \n\nNext up is normalising the dataset\nBefore normalizing the features a helper function is needed to convert pandas dataframe to tenroflow floats and converts it into one big tensor. \n","metadata":{}},{"cell_type":"code","source":"def stack_dict(inputs, fun=tf.stack):\n    values = []\n    for key in sorted(inputs.keys()):\n      values.append(tf.cast(inputs[key], tf.float32))\n\n    return fun(values, axis=-1)","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:20.979883Z","iopub.execute_input":"2022-07-07T05:02:20.981640Z","iopub.status.idle":"2022-07-07T05:02:20.990198Z","shell.execute_reply.started":"2022-07-07T05:02:20.981582Z","shell.execute_reply":"2022-07-07T05:02:20.989272Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Next its time to normalise using Keras’s inbuilt normalizer. \n","metadata":{}},{"cell_type":"code","source":"def create_normalizer(numeric_feature_names, data):\n    numeric_features = data[numeric_feature_names]\n    \n    normalizer = tf.keras.layers.Normalization(axis=-1)\n    normalizer.adapt(stack_dict(dict(numeric_features)))  \n    return normalizer","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.018412Z","iopub.execute_input":"2022-07-07T05:02:21.019274Z","iopub.status.idle":"2022-07-07T05:02:21.028718Z","shell.execute_reply.started":"2022-07-07T05:02:21.019188Z","shell.execute_reply":"2022-07-07T05:02:21.027278Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Using the stack_dict and create_normalizer function time to create a dictionary in a way the normalizer can process it. ","metadata":{}},{"cell_type":"code","source":"def normalize_numeric_input(numeric_feature_names, inputs, normalizer):\n    numeric_inputs = {}\n    for name in numeric_feature_names:\n      numeric_inputs[name]=inputs[name]\n\n    numeric_inputs = stack_dict(numeric_inputs)\n    numeric_normalized = normalizer(numeric_inputs) \n    return numeric_normalized","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.056959Z","iopub.execute_input":"2022-07-07T05:02:21.057656Z","iopub.status.idle":"2022-07-07T05:02:21.063228Z","shell.execute_reply.started":"2022-07-07T05:02:21.057619Z","shell.execute_reply":"2022-07-07T05:02:21.062218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"normalizer = create_normalizer(numeric_feature_names, data)\nnumeric_normalized = normalize_numeric_input(numeric_feature_names, inputs, normalizer)\nprint(numeric_normalized)","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.102593Z","iopub.execute_input":"2022-07-07T05:02:21.103544Z","iopub.status.idle":"2022-07-07T05:02:21.344421Z","shell.execute_reply.started":"2022-07-07T05:02:21.103499Z","shell.execute_reply":"2022-07-07T05:02:21.343257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Creating a way to store all the preprocessed dataset. ","metadata":{}},{"cell_type":"code","source":"preprocessed = []\npreprocessed.append(numeric_normalized)","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.346336Z","iopub.execute_input":"2022-07-07T05:02:21.346840Z","iopub.status.idle":"2022-07-07T05:02:21.351663Z","shell.execute_reply.started":"2022-07-07T05:02:21.346791Z","shell.execute_reply":"2022-07-07T05:02:21.350597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# One Hot Encoding\nNow the numeric part of the dataset has been normalised it is time to do one hot encoding to the categorical features. \n\nHere we iterate through the columns and find the string and integer type of columns and convert into the one-hot encoded columns for strings. Then we do the same for integer values. To do this placeholders from the input dictionary created above are taken. \nAt the end one hot encodings are returned from the function. \n","metadata":{}},{"cell_type":"code","source":"def one_hot_encode_categorical_features(categorical_feature_names, data, inputs):\n    one_hot = []\n    for name in categorical_feature_names:\n      value = sorted(set(data[name]))\n\n      if type(value[0]) is str:\n        lookup = tf.keras.layers.StringLookup(vocabulary=value, output_mode='one_hot')\n      else:\n        lookup = tf.keras.layers.IntegerLookup(vocabulary=value, output_mode='one_hot')\n\n      x = inputs[name][:, tf.newaxis]\n      x = lookup(x)\n      one_hot.append(x)\n    return one_hot","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.352874Z","iopub.execute_input":"2022-07-07T05:02:21.353221Z","iopub.status.idle":"2022-07-07T05:02:21.367199Z","shell.execute_reply.started":"2022-07-07T05:02:21.353193Z","shell.execute_reply":"2022-07-07T05:02:21.365948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Adding one hot encoded data to the preprocessed one. ","metadata":{}},{"cell_type":"code","source":"one_hot = one_hot_encode_categorical_features(categorical_feature_names, data, inputs)\npreprocessed = preprocessed + one_hot\n","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.370626Z","iopub.execute_input":"2022-07-07T05:02:21.371167Z","iopub.status.idle":"2022-07-07T05:02:21.421667Z","shell.execute_reply.started":"2022-07-07T05:02:21.371122Z","shell.execute_reply":"2022-07-07T05:02:21.420624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preprocesssed_result = tf.concat(preprocessed, axis=-1)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.423469Z","iopub.execute_input":"2022-07-07T05:02:21.424229Z","iopub.status.idle":"2022-07-07T05:02:21.436384Z","shell.execute_reply.started":"2022-07-07T05:02:21.424182Z","shell.execute_reply":"2022-07-07T05:02:21.435252Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Keras preprocessing before the model is constructed","metadata":{}},{"cell_type":"code","source":"preprocessor = tf.keras.Model(inputs, preprocesssed_result)","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.439252Z","iopub.execute_input":"2022-07-07T05:02:21.440597Z","iopub.status.idle":"2022-07-07T05:02:21.450417Z","shell.execute_reply.started":"2022-07-07T05:02:21.440548Z","shell.execute_reply":"2022-07-07T05:02:21.449393Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preprocessor(dict(data.iloc[:1]))","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.452454Z","iopub.execute_input":"2022-07-07T05:02:21.452883Z","iopub.status.idle":"2022-07-07T05:02:21.469947Z","shell.execute_reply.started":"2022-07-07T05:02:21.452844Z","shell.execute_reply":"2022-07-07T05:02:21.468554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Build the Neural Network\n\nUsing Keras Sequential its time to define the neural network. We will be using 2 dense hidden layers with 10 neurons each and apply the ReLU activation function. \n","metadata":{}},{"cell_type":"code","source":"network = tf.keras.Sequential([\n  tf.keras.layers.Dense(10, activation='relu'),\n  tf.keras.layers.Dense(10, activation='relu'),\n  tf.keras.layers.Dense(1)\n])","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.473447Z","iopub.execute_input":"2022-07-07T05:02:21.473878Z","iopub.status.idle":"2022-07-07T05:02:21.485164Z","shell.execute_reply.started":"2022-07-07T05:02:21.473844Z","shell.execute_reply":"2022-07-07T05:02:21.483987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Next preprocessor and network are tied together","metadata":{}},{"cell_type":"code","source":"x = preprocessor(inputs)\nresult = network(x)\nmodel = tf.keras.Model(inputs, result)","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.486362Z","iopub.execute_input":"2022-07-07T05:02:21.486699Z","iopub.status.idle":"2022-07-07T05:02:21.561616Z","shell.execute_reply.started":"2022-07-07T05:02:21.486665Z","shell.execute_reply":"2022-07-07T05:02:21.560812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Compiling the Model\nFinally the entire model is compiled using the Adam optimizer ( Adam is generally used as default) along with binary cross as loss function and accuracy as the evaluation function. ","metadata":{}},{"cell_type":"code","source":"model.compile(optimizer='adam',\n                loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n                metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.562781Z","iopub.execute_input":"2022-07-07T05:02:21.563310Z","iopub.status.idle":"2022-07-07T05:02:21.574388Z","shell.execute_reply.started":"2022-07-07T05:02:21.563279Z","shell.execute_reply":"2022-07-07T05:02:21.573132Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Split the dataset into test and training datasets","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\ntrain_data, val_data, train_target, val_target = train_test_split(data,target, train_size=0.8)\nhistory = model.fit(dict(train_data), train_target, validation_data=(dict(val_data), val_target), epochs=20, batch_size=8)","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:21.575978Z","iopub.execute_input":"2022-07-07T05:02:21.576993Z","iopub.status.idle":"2022-07-07T05:02:26.788395Z","shell.execute_reply.started":"2022-07-07T05:02:21.576950Z","shell.execute_reply":"2022-07-07T05:02:26.787306Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Accuracy","metadata":{}},{"cell_type":"code","source":"results = model.evaluate(dict(train_data), train_target, batch_size=128)\nprint(\"test accuracy:\", results[1]*100 , \"%\")","metadata":{"execution":{"iopub.status.busy":"2022-07-07T05:02:26.790815Z","iopub.execute_input":"2022-07-07T05:02:26.791260Z","iopub.status.idle":"2022-07-07T05:02:26.945142Z","shell.execute_reply.started":"2022-07-07T05:02:26.791215Z","shell.execute_reply":"2022-07-07T05:02:26.944030Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}