{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## A gentle start to Pytorch\n\n![](https://miro.medium.com/max/2400/1*aqNgmfyBIStLrf9k7d9cng.jpeg)","metadata":{}},{"cell_type":"markdown","source":"\nThis notebook aims to give a nice and easy introduction with easy and few steps. It doesn't aim high score! I tried to keep it simple without adding new features. The only and only aim is to a give a very gentle intro to Pytorch world\n\n\nBefore that let's give some introduction to Pytorch:\n\nPyTorch is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing. It is primarily developed by Facebook's AI Research lab. It is free and open-source software released under the Modified BSD license. Below is taken from the Pytorch Documentation (you can reach it from https://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html\n\nYou can check more advanced tutorials: \n\n- https://pytorch.org/tutorials/\n\nOr its github page:\n\n- https://github.com/pytorch/pytorch\n\nWHAT IS PYTORCH?\n\nIt’s a Python-based scientific computing package targeted at two sets of audiences:\n\n- A replacement for NumPy to use the power of GPUs\n- A deep learning research platform that provides maximum flexibility and speed","metadata":{}},{"cell_type":"markdown","source":"![](https://www.sciencealert.com/images/articles/processed/titanic-1_600.jpg)\n\nFor this exercise we will be using the famous **Titanic** Dataset:\n\nThis is a very famous dataset but if you need an introduction on this, please check this useful links:\n\nhttps://web.stanford.edu/class/archive/cs/cs109/cs109.1166/problem12.html\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"Several Cells ara taken from my other notebooks, where I don't use pythorch please check it here:\n\n1. https://www.kaggle.com/frtgnn/titanic-survival-classifier\n\n2. https://www.kaggle.com/frtgnn/beginner-s-stop-pipeline-introduction","metadata":{}},{"cell_type":"markdown","source":"# DataSet & Library Loading","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom collections import Counter\nfrom sklearn.utils import shuffle\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\n\ndf_train = pd.read_csv('../input/titanic/train.csv')\ndf_test  = pd.read_csv('../input/titanic/test.csv')\ndf_sub   = pd.read_csv('../input/titanic/gender_submission.csv')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Making the dataset ready for the model\n\n- let's drop the unnecessary columns\n- encode the categorical (no details)\n- impute the necessary columns (again no details)\n- scale both the train and test data for linear models\n- split the data for the model","metadata":{}},{"cell_type":"code","source":"df_train.drop(['Name','Ticket','Cabin'],axis=1,inplace=True)\ndf_test.drop( ['Name','Ticket','Cabin'],axis=1,inplace=True)\n\nsex      = pd.get_dummies(df_train['Sex'],drop_first=True)\nembark   = pd.get_dummies(df_train['Embarked'],drop_first=True)\ndf_train = pd.concat([df_train,sex,embark],axis=1)\n\ndf_train.drop(['Sex','Embarked'],axis=1,inplace=True)\n\nsex     = pd.get_dummies(df_test['Sex'],drop_first=True)\nembark  = pd.get_dummies(df_test['Embarked'],drop_first=True)\ndf_test = pd.concat([df_test,sex,embark],axis=1)\n\ndf_test.drop(['Sex','Embarked'],axis=1,inplace=True)\n\ndf_train.fillna(df_train.mean(),inplace=True)\ndf_test.fillna(df_test.mean(),inplace=True)\n\nScaler1 = StandardScaler()\nScaler2 = StandardScaler()\n\ntrain_columns = df_train.columns\ntest_columns  = df_test.columns\n\ndf_train = pd.DataFrame(Scaler1.fit_transform(df_train))\ndf_test  = pd.DataFrame(Scaler2.fit_transform(df_test))\n\ndf_train.columns = train_columns\ndf_test.columns  = test_columns\n\nfeatures = df_train.iloc[:,2:].columns.tolist()\ntarget   = df_train.loc[:, 'Survived'].name\n\nX_train = df_train.iloc[:,2:].values\ny_train = df_train.loc[:, 'Survived'].values","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Pytorch","metadata":{}},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nfrom torch.nn import functional as F\nfrom torch.autograd import Variable","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Pytorch Logistic Regression Model","metadata":{}},{"cell_type":"code","source":"#thank you very much https://www.kaggle.com/mburakergenc/ttianic-minimal-pytorch-mlp\nclass Net(nn.Module):\n    def __init__(self):\n        super(Net, self).__init__()\n        self.fc1 = nn.Linear(8, 512)\n        self.fc2 = nn.Linear(512, 512)\n        self.fc3 = nn.Linear(512, 2)\n        self.dropout = nn.Dropout(0.2)\n        \n    def forward(self, x):\n        x = F.relu(self.fc1(x))\n        x = self.dropout(x)\n        x = F.relu(self.fc2(x))\n        x = self.dropout(x)\n        x = self.fc3(x)\n        return x\nmodel = Net()\nprint(model)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Pytorch Loss Function (Cross Entropy CE)","metadata":{}},{"cell_type":"code","source":"criterion = nn.CrossEntropyLoss()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Pytorch Optimizer (Stochastic Gradient Descent SGD)","metadata":{}},{"cell_type":"code","source":"optimizer = torch.optim.SGD(model.parameters(), lr=0.01)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Pytorch Training","metadata":{}},{"cell_type":"code","source":"#thank you very much https://www.kaggle.com/mburakergenc/ttianic-minimal-pytorch-mlp\n\nbatch_size = 64\nn_epochs = 500\nbatch_no = len(X_train) // batch_size\n\ntrain_loss = 0\ntrain_loss_min = np.Inf\nfor epoch in range(n_epochs):\n    for i in range(batch_no):\n        start = i * batch_size\n        end   = start + batch_size\n        x_var = Variable(torch.FloatTensor(X_train[start:end]))\n        y_var = Variable(torch.LongTensor(y_train[start:end])) \n        \n        optimizer.zero_grad()\n        output = model(x_var)\n        loss   = criterion(output,y_var)\n        loss.backward()\n        optimizer.step()\n        \n        values, labels = torch.max(output, 1)\n        num_right   = np.sum(labels.data.numpy() == y_train[start:end])\n        train_loss += loss.item()*batch_size\n    \n    train_loss = train_loss / len(X_train)\n    if train_loss <= train_loss_min:\n        print(\"Validation loss decreased ({:6f} ===> {:6f}). Saving the model...\".format(train_loss_min,train_loss))\n        torch.save(model.state_dict(), \"model.pt\")\n        train_loss_min = train_loss\n    \n    if epoch % 200 == 0:\n        print('')\n        print(\"Epoch: {} \\tTrain Loss: {} \\tTrain Accuracy: {}\".format(epoch+1, train_loss,num_right / len(y_train[start:end]) ))\nprint('Training Ended! ')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# predictions","metadata":{}},{"cell_type":"code","source":"X_test     = df_test.iloc[:,1:].values\nX_test_var = Variable(torch.FloatTensor(X_test), requires_grad=False) \nwith torch.no_grad():\n    test_result = model(X_test_var)\nvalues, labels = torch.max(test_result, 1)\nsurvived = labels.data.numpy()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# submission","metadata":{}},{"cell_type":"code","source":"submission = pd.DataFrame({'PassengerId': df_sub['PassengerId'], 'Survived': survived})\nsubmission.to_csv('submission.csv', index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I have found 3 nice courses online, checked them out and they are really nice!\n\n- https://www.udemy.com/course/pytorch-for-deep-learning-with-python-bootcamp/?ranMID=39197&ranEAID=vedj0cWlu2Y&ranSiteID=vedj0cWlu2Y-FiBKCfRWMo8DOXuV0uYFLg&LSNPUBID=vedj0cWlu2Y\n\n- https://www.coursera.org/learn/deep-neural-networks-with-pytorch?ranMID=40328&ranEAID=vedj0cWlu2Y&ranSiteID=vedj0cWlu2Y-TFDEo8s4j9f2CxC59L3_8w&siteID=vedj0cWlu2Y-TFDEo8s4j9f2CxC59L3_8w&utm_content=10&utm_medium=partners&utm_source=linkshare&utm_campaign=vedj0cWlu2Y\n\n- https://www.udacity.com/course/deep-learning-pytorch--ud188?cjevent=becd1b75759d11ea83f301a10a24060d\n(this one's free)","metadata":{}},{"cell_type":"markdown","source":"## thank you!\n## I'll try to add more info and make it better asap","metadata":{}}]}