{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Titanic Exploratory Data Analysis & Basic Decision Tree\nData clean-up insipred by: [Naveen's Titanic-Survival-Prediction Notebook](https://www.kaggle.com/code/naveensenthilkumar/titanic-survival-prediction)","metadata":{}},{"cell_type":"code","source":"#Import python libraries\nimport numpy as np\nimport pandas as pd \nimport matplotlib.pyplot as plt","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-08-10T22:01:16.736148Z","iopub.execute_input":"2022-08-10T22:01:16.736994Z","iopub.status.idle":"2022-08-10T22:01:16.763934Z","shell.execute_reply.started":"2022-08-10T22:01:16.736894Z","shell.execute_reply":"2022-08-10T22:01:16.762775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load Data & Inspect","metadata":{}},{"cell_type":"code","source":"train= pd.read_csv(\"../input/titanic/train.csv\")\npred=pd.read_csv(\"../input/titanic/test.csv\", header=0)","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:01:16.765920Z","iopub.execute_input":"2022-08-10T22:01:16.766716Z","iopub.status.idle":"2022-08-10T22:01:16.798876Z","shell.execute_reply.started":"2022-08-10T22:01:16.766678Z","shell.execute_reply":"2022-08-10T22:01:16.797809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:01:16.800055Z","iopub.execute_input":"2022-08-10T22:01:16.800583Z","iopub.status.idle":"2022-08-10T22:01:16.824961Z","shell.execute_reply.started":"2022-08-10T22:01:16.800552Z","shell.execute_reply":"2022-08-10T22:01:16.824193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:01:16.828704Z","iopub.execute_input":"2022-08-10T22:01:16.829481Z","iopub.status.idle":"2022-08-10T22:01:16.847336Z","shell.execute_reply.started":"2022-08-10T22:01:16.829446Z","shell.execute_reply":"2022-08-10T22:01:16.846104Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Training Data: ',train.shape)\nprint('Test Data: ',pred.shape)","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:01:16.848742Z","iopub.execute_input":"2022-08-10T22:01:16.849497Z","iopub.status.idle":"2022-08-10T22:01:16.855451Z","shell.execute_reply.started":"2022-08-10T22:01:16.849459Z","shell.execute_reply":"2022-08-10T22:01:16.854347Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Look for NaN data","metadata":{}},{"cell_type":"code","source":"train.info()\npred.info()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:01:16.857537Z","iopub.execute_input":"2022-08-10T22:01:16.858180Z","iopub.status.idle":"2022-08-10T22:01:16.895213Z","shell.execute_reply.started":"2022-08-10T22:01:16.858129Z","shell.execute_reply":"2022-08-10T22:01:16.894000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Cleanup\nFrom the above info, we see that several of the columns have null values, so we need to do something with them to clean up the data.\n\n1. Remove Cabin column since it is mostly nan\n2. Replace Nan in Age with the average\n3. Replace Nan in Embarked with the mode since it is a string\n4. Replace Nan in Fare with the average","metadata":{}},{"cell_type":"code","source":"#Remove Cabin\ntrain.drop('Cabin', axis=1, inplace=True)\npred.drop('Cabin', axis=1, inplace=True)\n\n#Replace Nan in age with the mean value\ntrain[\"Age\"].replace(np.nan, train[\"Age\"].mean(), inplace=True)\ntrain[\"Embarked\"].replace(np.nan, train.Embarked.mode()[0], inplace=True)\npred[\"Embarked\"].replace(np.nan, pred.Embarked.mode()[0], inplace=True)\npred[\"Age\"].replace(np.nan, pred[\"Age\"].mean(), inplace=True)\npred[\"Fare\"].replace(np.nan, pred[\"Fare\"].mean(),inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:01:16.896390Z","iopub.execute_input":"2022-08-10T22:01:16.897277Z","iopub.status.idle":"2022-08-10T22:01:16.912690Z","shell.execute_reply.started":"2022-08-10T22:01:16.897244Z","shell.execute_reply":"2022-08-10T22:01:16.911550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Check to make sure everything is non-null\ntrain.info()\npred.info()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:01:16.914166Z","iopub.execute_input":"2022-08-10T22:01:16.914777Z","iopub.status.idle":"2022-08-10T22:01:16.936141Z","shell.execute_reply.started":"2022-08-10T22:01:16.914743Z","shell.execute_reply":"2022-08-10T22:01:16.935240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Exploration\nLet's first try to get a feel of what variables have a correlation with the binary output. I think a good way to visualize the categrical data will be with a [grouped bar chart](https://www.geeksforgeeks.org/plotting-multiple-bar-charts-using-matplotlib-in-python/) and the numerical data with histograms [plt.hist](https://matplotlib.org/stable/gallery/statistics/hist.html)/[np.histogram](https://numpy.org/doc/stable/reference/generated/numpy.histogram.html).","metadata":{}},{"cell_type":"code","source":"survBool = train.Survived.astype('bool')\ndieBool = survBool == False","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:13:15.143919Z","iopub.execute_input":"2022-08-10T22:13:15.144371Z","iopub.status.idle":"2022-08-10T22:13:15.150356Z","shell.execute_reply.started":"2022-08-10T22:13:15.144333Z","shell.execute_reply":"2022-08-10T22:13:15.149131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Male vs Female\nX = ['Male', 'Female']\nMsurv = np.count_nonzero(np.logical_and(train.Sex=='male',survBool))\nMdie = np.count_nonzero(np.logical_and(train.Sex=='male',dieBool))\nFsurv = np.count_nonzero(np.logical_and(train.Sex=='female',survBool))\nFdie = np.count_nonzero(np.logical_and(train.Sex=='female',dieBool))\nBsurv = [Msurv, Fsurv]\nBdie = [Mdie, Fdie]\n  \nX_axis = np.arange(len(X))\n  \nplt.bar(X_axis - 0.2, Bsurv, 0.4, label = 'Survived')\nplt.bar(X_axis + 0.2, Bdie, 0.4, label = 'Died')\n  \nplt.xticks(X_axis, X)\nplt.ylabel(\"Number of Passengers\")\nplt.title(\"Male vs. Female Survival Rate\")\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:37:37.828641Z","iopub.execute_input":"2022-08-10T22:37:37.829778Z","iopub.status.idle":"2022-08-10T22:37:37.968102Z","shell.execute_reply.started":"2022-08-10T22:37:37.829735Z","shell.execute_reply":"2022-08-10T22:37:37.966962Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Male vs Female Survival Rate\nX = ['Male', 'Female']\nMsurv = np.count_nonzero(np.logical_and(train.Sex=='male',survBool))\nMtot = np.count_nonzero(train.Sex=='male')\nFsurv = np.count_nonzero(np.logical_and(train.Sex=='female',survBool))\nFtot = np.count_nonzero(train.Sex=='female')\nBrate = [Msurv/Mtot, Fsurv/Ftot]\n  \nX_axis = np.arange(len(X))\n  \nplt.bar(X_axis, Brate, 0.5, label = 'Survival Rate')\n  \nplt.xticks(X_axis, X)\nplt.ylabel(\"Number of Passengers\")\nplt.title(\"Male vs. Female Survival Rate\")\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:39:37.899131Z","iopub.execute_input":"2022-08-10T22:39:37.899533Z","iopub.status.idle":"2022-08-10T22:39:38.395172Z","shell.execute_reply.started":"2022-08-10T22:39:37.899498Z","shell.execute_reply":"2022-08-10T22:39:38.394245Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Clearly, it is much more favorable to be female.","metadata":{}},{"cell_type":"code","source":"#Pclass\nX = ['1', '2', '3']\nP1surv = np.count_nonzero(np.logical_and(train.Pclass==1,survBool))\nP1tot = np.count_nonzero(train.Pclass==1)\nP2surv = np.count_nonzero(np.logical_and(train.Pclass==2,survBool))\nP2tot = np.count_nonzero(train.Pclass==2)\nP3surv = np.count_nonzero(np.logical_and(train.Pclass==3,survBool))\nP3tot = np.count_nonzero(train.Pclass==3)\nBrate = [P1surv/P1tot,P2surv/P2tot,P3surv/P3tot]\n  \nX_axis = np.arange(len(X))\n  \nplt.bar(X_axis, Brate, 0.5, label = 'Survival Rate')\n  \nplt.xticks(X_axis, X)\nplt.xlabel('Passenger Class')\nplt.ylabel(\"Number of Passengers\")\nplt.title(\"Passenger Class Survival Rate\")\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:47:24.606444Z","iopub.execute_input":"2022-08-10T22:47:24.607328Z","iopub.status.idle":"2022-08-10T22:47:24.808399Z","shell.execute_reply.started":"2022-08-10T22:47:24.607286Z","shell.execute_reply":"2022-08-10T22:47:24.807279Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- The passenger class also appears to be a good predictor of survival, \nwhere first class (Pclass==1) passengers are more likely to survive","metadata":{}},{"cell_type":"code","source":"#Fare Price: Normalized Histograms\nfareHist,binEdge1 = np.histogram(train.Fare)\nfareHistSurv,binEdge2 = np.histogram(train.Fare[survBool])\nfareHist[fareHist==0] = 1\nfareSurvRat = fareHistSurv/fareHist\nplt.figure()\nplt.bar(binEdge1[0:-1],fareSurvRat,(binEdge1[1:]-binEdge1[0:-1]),\n        align='edge',edgecolor='w')\nplt.xlabel(\"Fare Price\")\nplt.ylabel(\"Fraction Survived\")\nplt.title(\"Survival Rate per Ticket Fare\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:32:27.163474Z","iopub.execute_input":"2022-08-10T22:32:27.163916Z","iopub.status.idle":"2022-08-10T22:32:27.365375Z","shell.execute_reply.started":"2022-08-10T22:32:27.163883Z","shell.execute_reply":"2022-08-10T22:32:27.364266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Similar to the passenger class, the more expensive fares also appear to survive more frequently","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:35:09.035736Z","iopub.execute_input":"2022-08-10T22:35:09.037131Z","iopub.status.idle":"2022-08-10T22:35:09.048478Z","shell.execute_reply.started":"2022-08-10T22:35:09.037074Z","shell.execute_reply":"2022-08-10T22:35:09.047274Z"}}},{"cell_type":"code","source":"#Age: Normalized Histograms\nageHist,binEdge1 = np.histogram(train.Age)\nageHistSurv,binEdge2 = np.histogram(train.Age[survBool])\nageHist[ageHist==0] = 1\nageSurvRat = ageHistSurv/ageHist\nplt.figure()\nplt.bar(binEdge1[0:-1],ageSurvRat,(binEdge1[1:]-binEdge1[0:-1]),\n        align='edge',edgecolor='w')\nplt.xlabel(\"age Price\")\nplt.ylabel(\"Fraction Survived\")\nplt.title(\"Survival Rate By Age\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T22:52:21.394610Z","iopub.execute_input":"2022-08-10T22:52:21.395048Z","iopub.status.idle":"2022-08-10T22:52:21.625310Z","shell.execute_reply.started":"2022-08-10T22:52:21.395010Z","shell.execute_reply":"2022-08-10T22:52:21.624340Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Age appears to slightly predict survivability, with the lowest age group (< 10 yrs old) surviving more, but all other age groups are rather similar","metadata":{}},{"cell_type":"markdown","source":"# Basic Decision Tree\nFrom the EDA, it looks like we should be able to make a good prediction just using a [decision tree](https://scikit-learn.org/stable/modules/tree.html). Since we have categorical data, we need to implement [one-hot-encoding](https://stackoverflow.com/a/51624006) to be able to use scikit-learn.","metadata":{}},{"cell_type":"code","source":"from sklearn import tree\n\nclf = tree.DecisionTreeClassifier()\n\none_hot_data = pd.get_dummies(train[['Age','SibSp','Fare','Sex','Pclass','Embarked']],drop_first=True)\nclf = clf.fit(one_hot_data, train['Survived'])","metadata":{"execution":{"iopub.status.busy":"2022-08-10T23:23:56.101929Z","iopub.execute_input":"2022-08-10T23:23:56.102370Z","iopub.status.idle":"2022-08-10T23:23:56.124694Z","shell.execute_reply.started":"2022-08-10T23:23:56.102333Z","shell.execute_reply":"2022-08-10T23:23:56.123941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Check the accuracy of the decision tree on the training data:","metadata":{}},{"cell_type":"code","source":"clf.score(one_hot_data, train['Survived'])","metadata":{"execution":{"iopub.status.busy":"2022-08-10T23:26:18.903855Z","iopub.execute_input":"2022-08-10T23:26:18.904264Z","iopub.status.idle":"2022-08-10T23:26:18.914399Z","shell.execute_reply.started":"2022-08-10T23:26:18.904231Z","shell.execute_reply":"2022-08-10T23:26:18.913601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Visualize the decision tree","metadata":{}},{"cell_type":"code","source":"import graphviz \ndot_data = tree.export_graphviz(clf, out_file=None, \n                      feature_names=one_hot_data.keys(),  \n                      filled=True, rounded=True,  \n                      special_characters=True)  \ngraph = graphviz.Source(dot_data)  \ngraph","metadata":{"execution":{"iopub.status.busy":"2022-08-10T23:21:36.104742Z","iopub.execute_input":"2022-08-10T23:21:36.105131Z","iopub.status.idle":"2022-08-10T23:21:36.289949Z","shell.execute_reply.started":"2022-08-10T23:21:36.105099Z","shell.execute_reply":"2022-08-10T23:21:36.288744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is a horribly complicated tree, but it is working well on the training data. Let's make a submission and see how it does","metadata":{}},{"cell_type":"markdown","source":"# Submission","metadata":{}},{"cell_type":"code","source":"pred_one_hot_data = pd.get_dummies(pred[['Age','SibSp','Fare','Sex','Pclass','Embarked']],drop_first=True)\n\nfinal_pred = pd.DataFrame({\"PassengerId\": pred['PassengerId'], \"Survived\": clf.predict(pred_one_hot_data)})\nfinal_pred.reset_index(drop=True, inplace = True)\nfinal_pred.to_csv('submission.csv', index = False)","metadata":{"execution":{"iopub.status.busy":"2022-08-10T23:35:45.551510Z","iopub.execute_input":"2022-08-10T23:35:45.552486Z","iopub.status.idle":"2022-08-10T23:35:45.571927Z","shell.execute_reply.started":"2022-08-10T23:35:45.552447Z","shell.execute_reply":"2022-08-10T23:35:45.571066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!head submission.csv","metadata":{"execution":{"iopub.status.busy":"2022-08-10T23:36:34.674522Z","iopub.execute_input":"2022-08-10T23:36:34.674955Z","iopub.status.idle":"2022-08-10T23:36:35.815687Z","shell.execute_reply.started":"2022-08-10T23:36:34.674919Z","shell.execute_reply":"2022-08-10T23:36:35.814566Z"},"trusted":true},"execution_count":null,"outputs":[]}]}