{"cells":[{"metadata":{},"cell_type":"markdown","source":"Good day kagglers,this is going to be my first ever kaggle kernel and i am pretty much excited to share some neural network basics with you,in this notebook i will show you how a very simple neural network can learn x to y variable mapping(this notebook is for deep learning or neural network beginners)."},{"metadata":{},"cell_type":"markdown","source":"**importing the necessary libraries**"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom tensorflow import keras\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\n#import os\n#print(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"let's define x and y variable for our neural network to learn,the cell below is containing 1-d array x with some numbers and corresponding y value is stored in 1-d array y which denoting our dependent variable"},{"metadata":{"trusted":true},"cell_type":"code","source":"x = np.array([-1.0,0.0,1.0,2.0,3.0,4.0], dtype = float)\ny = np.array([-3.0,-1.0,1.0,3.0,5.0,7.0], dtype = float)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"now you might have already observed the pattern between x to y relaion which is : if x is 1 then y is also 1 but if x is 0 then y is 0-1 = -1,when x is -1.0,y is -1.0-2.0 = -3.0,again if we go in the positive direction we can see that for x==1 y is also equal 1,for x = 2,y =2+1 =3,for x = 3,y = 3+2 = 5 and for x = 4,y = 4+3 = 7, now if x = 5 then what should be the y value for this equation? 5+4 = 9 right? similarly if x is equal to -2 then y should be -2-3 = -5 right?now lets visualize this simple mathematical function only for our defined x and y variable"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.plot(x,y)\nplt.title('Visualizing relation between x and y')\nplt.ylabel('Dependent variable (y)')\nplt.xlabel('Independent variable (x)')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**lets define our simple neural network with single unit (because we have a simple 1-d array x),we will use sgd optimizer and mean_squared_error function for computing the loss function**\n\n1.to understand keras optimizer [click here](https://keras.io/optimizers/)\n\n2.[keras losses](https://keras.io/losses/)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = keras.Sequential([keras.layers.Dense(units =1, input_shape=[1])])\nmodel.compile(optimizer = 'sgd', loss = 'mean_squared_error')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"telling our neural network(defined above) to learn x to y mapping for 100 epochs.An epoch is a single step in training a neural network. in other words when a neural network is trained on every training samples only in one pass we say that one epoch is finished.One Epoch is when an ENTIRE dataset is passed forward and backward through the neural network only ONCE.As the number of epochs increases, more number of times the weight are changed in the neural network and the curve goes from underfitting to optimal to overfitting curve.to learn more [watch this](https://towardsdatascience.com/epoch-vs-iterations-vs-batch-size-4dfb9c7ce9c9)"},{"metadata":{"trusted":true},"cell_type":"code","source":"model.fit(x,y, epochs = 100)\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**asking our network to predict what should be the corresponding y value of given x input as 10.0 which is a new x input to our learnt neural network **"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"predicted Answer : \" ,  model.predict([10.0]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**the network predicts that for x=10.0 ,y should be 18.098911**\n\nnow lets see what the correct answer is,\n\nfor x = 4,y was 4+3 = 7,\n\nso,\n\nfor x = 5,y should be 5+4 = 9\n\nfor x = 6, y should be 6+5 = 11\n\nfor x = 7,y should be 7+6 = 13\n\nfor x = 8, y should be 8+7 = 15\n\nfor x = 9, y should be 9+8 = 17\n\nfor x = 10,y should be 10+9 = 19(where our neural network predicting 18.098911 which is pretty close to correct answer 19 right?)\n"},{"metadata":{},"cell_type":"markdown","source":"***Note that Our Network only learnt for 6 samples so if you ask it to predict what should be the answer of very large input like 10000000 then the network will not do well because the value you want to predict is too far away from our learnt samples,to fix this issue you will need to feed this network much more samples to learn***"},{"metadata":{},"cell_type":"markdown","source":"**let's try again**\n\nremember the old school lesson for converting Celsius to Fahrenheit?\n\nusing tensorflow we will do the same now\n\nour network will learn **Approximate Formula which is : f = c * 1.8 + 32** to predict Fahrenheit for given Celsius as input"},{"metadata":{},"cell_type":"markdown","source":"**Setting up training data**"},{"metadata":{"trusted":true},"cell_type":"code","source":"celsius_q = np.array([-40,-10,0,8,15,22,38], dtype = float) #x variables\n\nfahrenheit_a = np.array([-40,14,32,46,59,72,100], dtype = float) #y variables\n\n#lets print these\n\nfor f,c in enumerate (celsius_q):\n  print(\"{} degree cesius = {} degree fahrenheit\".format(c,fahrenheit_a[f]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Creating the model**"},{"metadata":{"trusted":true},"cell_type":"code","source":"ten = tf.keras.layers.Dense(units = 1, input_shape = [1])\n\nmodel = tf.keras.Sequential([ten])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.compile(loss = 'mean_squared_error',\n             \n              optimizer = tf.keras.optimizers.Adam(0.1) #0.1 here is the learning rate\n             )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Training the Model**"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = model.fit(celsius_q, fahrenheit_a, epochs = 100, verbose = False )\n\nprint(\"training has been finished\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Displaying Training Statistics**"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.xlabel('Epoch Number')\n\nplt.ylabel('Loss Magnitude')\n\nplt.plot(train.history['loss'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**MAKING PREDICTION**"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(model.predict([100.0]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Looking at the layer weights**"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"These are the layer variables: {}\".format(ten.get_weights()))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"*i hope you enjoyed it,thanks for your time,in the upcoming kernel i will be working on CNN's in shaa allah*"},{"metadata":{},"cell_type":"markdown","source":"**References : **\n\n1. https://www.coursera.org/learn/introduction-tensorflow#utm_source=email&utm_medium=dl.aiGeneralListCTA&utm_campaign=TFSC1Announcement\n\n2. https://classroom.udacity.com/courses/ud187"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}