{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introduction\n\nThe sinking of Titanic is one of the most notorious shipwrecks in the history. In 1912, during her voyage , the Titanic sank after colliding with an iceberg. Killing 1502 out of 2224 passangers and crews.\n\n<font color='Blue' >\nContent :\n\n1. [Load and Check Data](#1)\n1. [Veriable Description](#2)\n    * [Univariate Variable Analysis](#3)\n        *  [Categorical Variable Analysis](#4)\n        *  [Numberical Variable Analysis](#5)\n1. [Basic Data Analysis](#6)\n1. [OutLier Detection](#7)\n1. [Missing Value](#8)\n    * [Find Missing Value](#9)\n    * [Fill Missing Value](#10)\n1. [Visualization](#11)\n    * [Corelation Between Sibsp -- Parch -- Age -- Fare -- Surived](#12)\n    * [SibSp -- Survived](#13)\n    * [Parch -- Survived](#14)\n    * [Pclass -- Survived](#15)\n    * [Age -- Survived](#16)\n    * [Pclass -- Age -- Survived](#17)\n    * [Embarked -- Sex -- Pclass -- Survived](#18)\n    * [Embarked -- Sex -- Fare -- Survived](#19)\n    * [Fill Missing: Age Value](#20)\n1. Machine Learning\n    * [Feature Engineering](#21) \n        * [Name -- Title](#22)\n        * [Family Size](#23)\n        * [Embarked](#24)\n        * [Ticket](#25)\n        * [Pclass](#26)\n        * [Sex](#27)\n        * [Drop Passenger Id and Cabin](#28)\n1.[Modeling](#29)\n    * [Train Test Split](#30)\n    * [Simple Logistic Regretion](#31)\n    * [Hyperparameter Tuning -- Grid Search -- Cross Validation](#32)\n    * [Ensemble Modeling](#33)\n    * [Prediction and Submission](#34)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-05-11T15:36:20.608394Z","iopub.execute_input":"2022-05-11T15:36:20.609085Z","iopub.status.idle":"2022-05-11T15:36:21.739831Z","shell.execute_reply.started":"2022-05-11T15:36:20.60895Z","shell.execute_reply":"2022-05-11T15:36:21.738863Z"}}},{"cell_type":"code","source":"#This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport matplotlib.pyplot as plt\nplt.style.use(\"seaborn-whitegrid\")\n#plt.style.available -> you can choose other styles\n\nimport seaborn as sns\n\nfrom collections import Counter\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:04.038986Z","iopub.execute_input":"2022-08-12T18:54:04.039263Z","iopub.status.idle":"2022-08-12T18:54:04.047335Z","shell.execute_reply.started":"2022-08-12T18:54:04.039232Z","shell.execute_reply":"2022-08-12T18:54:04.046454Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='1'></a><br>\n## Load and Check Data\n\n","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(\"/kaggle/input/titanic/train.csv\")\ntest_df = pd.read_csv(\"/kaggle/input/titanic/test.csv\")\ntest_PassengerId = test_df[\"PassengerId\"]","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:04.240280Z","iopub.execute_input":"2022-08-12T18:54:04.241054Z","iopub.status.idle":"2022-08-12T18:54:04.256592Z","shell.execute_reply.started":"2022-08-12T18:54:04.241004Z","shell.execute_reply":"2022-08-12T18:54:04.256019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.columns","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:04.440442Z","iopub.execute_input":"2022-08-12T18:54:04.440950Z","iopub.status.idle":"2022-08-12T18:54:04.447485Z","shell.execute_reply.started":"2022-08-12T18:54:04.440903Z","shell.execute_reply":"2022-08-12T18:54:04.446664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head(5)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:04.677123Z","iopub.execute_input":"2022-08-12T18:54:04.677557Z","iopub.status.idle":"2022-08-12T18:54:04.692135Z","shell.execute_reply.started":"2022-08-12T18:54:04.677526Z","shell.execute_reply":"2022-08-12T18:54:04.691503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.describe()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:04.938482Z","iopub.execute_input":"2022-08-12T18:54:04.938764Z","iopub.status.idle":"2022-08-12T18:54:04.973224Z","shell.execute_reply.started":"2022-08-12T18:54:04.938722Z","shell.execute_reply":"2022-08-12T18:54:04.972283Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='2'></a><br>\n## Veriable Description\n\n1. PassangerId -> unique id number to each passenger\n1. Survived -> passenger survive(1) or died(0)\n1. Pclass ->  passenger class\n1. Name ->  name\n1. Sex ->  male/female\n1. Age ->  age of passenger\n1. SibSp ->  number of sibligns/spouse\n1. Parch ->  number of parents/children\n1. Ticket ->  Ticket number\n1. Fare ->  Price of ticket\n1. Cabin -> Room number of passenger \n1. Embarked -> Port where passenger embarked (C= Cherbourg, Q= Queenstown, S=Southampton)\n","metadata":{}},{"cell_type":"markdown","source":"* float64(2) -> Age and Fare\n* int64(5) -> PassengerId, Survived, Pclass, SibSp, Parch\n* object(5)-> Name, Sex, Ticket, Cabin, Embarked","metadata":{}},{"cell_type":"markdown","source":"<a id='3'></a><br>\n# Univariate Variable Analysis\n* Categorical Variable: Survived, Sex, Pclass, Embarked, Cabin, Name, Ticket, Sibsp and Parch\n* Numberical Variable: Age, Fare and PassengerId","metadata":{}},{"cell_type":"markdown","source":"<a id='4'></a><br>\n## Categorical Variable Analysis\n","metadata":{}},{"cell_type":"code","source":"def bar_plot(variable):\n    \"\"\"\n        input: variable ex: \"Sex\"\n        output: bar plat & value count\n    \"\"\"\n    # get feature\n    var = train_df[variable]\n    \n    # count number of categorical variable(value/sample)\n    varValue = var.value_counts()\n    \n    # visualize\n    plt.figure(figsize=(9,3))\n    plt.bar(varValue.index,varValue)\n    plt.xticks(varValue.index,varValue.index.values)\n    plt.ylabel(\"Frequency\")\n    plt.title(variable)\n    plt.show()\n    print(\"{}: \\n {}\".format(variable,varValue))","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:05.214845Z","iopub.execute_input":"2022-08-12T18:54:05.215151Z","iopub.status.idle":"2022-08-12T18:54:05.222131Z","shell.execute_reply.started":"2022-08-12T18:54:05.215117Z","shell.execute_reply":"2022-08-12T18:54:05.221071Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"category1 = [\"Survived\",\"Sex\",\"Pclass\",\"Embarked\",\"SibSp\",\"Parch\"]\n\nfor c in category1:\n    bar_plot(c)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:05.419599Z","iopub.execute_input":"2022-08-12T18:54:05.420361Z","iopub.status.idle":"2022-08-12T18:54:06.451614Z","shell.execute_reply.started":"2022-08-12T18:54:05.420321Z","shell.execute_reply":"2022-08-12T18:54:06.451038Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"category2 = [\"Cabin\",\"Name\",\"Ticket\"]\nfor c in category2:\n    print(\"{} \\n\".format(train_df[c].value_counts()))","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:06.453053Z","iopub.execute_input":"2022-08-12T18:54:06.453963Z","iopub.status.idle":"2022-08-12T18:54:06.466697Z","shell.execute_reply.started":"2022-08-12T18:54:06.453915Z","shell.execute_reply":"2022-08-12T18:54:06.465898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='5'></a><br>\n## Numberical Variable Analysis","metadata":{}},{"cell_type":"code","source":"def plot_hist(variable):\n    \n    plt.figure(figsize=(9,3))\n    plt.hist(train_df[variable])\n    plt.xlabel(variable)\n    plt.ylabel(\"Frequency\")\n    plt.title(\"{} distribution with histogram\".format(variable))\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:06.468020Z","iopub.execute_input":"2022-08-12T18:54:06.468251Z","iopub.status.idle":"2022-08-12T18:54:06.477265Z","shell.execute_reply.started":"2022-08-12T18:54:06.468222Z","shell.execute_reply":"2022-08-12T18:54:06.476451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"numericVar = [\"Fare\",\"Age\"] # \"PassengerId\" is not a good parameter\n\nfor n in numericVar:\n    plot_hist(n)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:06.479177Z","iopub.execute_input":"2022-08-12T18:54:06.479598Z","iopub.status.idle":"2022-08-12T18:54:06.859743Z","shell.execute_reply.started":"2022-08-12T18:54:06.479564Z","shell.execute_reply":"2022-08-12T18:54:06.858821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='6'></a><br>\n# Basic Data Analysis\n\n* Pclass - Survived\n* Sex - Survived\n* Sibsp - Survived\n* Parch - Survived","metadata":{}},{"cell_type":"code","source":"# Pclass vs Survived\n\ntrain_df[[\"Pclass\",\"Survived\"]].groupby([\"Pclass\"],as_index= False).mean().sort_values(by=\"Survived\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:06.861369Z","iopub.execute_input":"2022-08-12T18:54:06.861818Z","iopub.status.idle":"2022-08-12T18:54:06.879991Z","shell.execute_reply.started":"2022-08-12T18:54:06.861762Z","shell.execute_reply":"2022-08-12T18:54:06.878807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Sex vs Survived\n\ntrain_df[[\"Sex\",\"Survived\"]].groupby([\"Sex\"],as_index=False).mean().sort_values(by=\"Survived\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:06.881435Z","iopub.execute_input":"2022-08-12T18:54:06.882314Z","iopub.status.idle":"2022-08-12T18:54:06.897198Z","shell.execute_reply.started":"2022-08-12T18:54:06.882270Z","shell.execute_reply":"2022-08-12T18:54:06.896109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Sibsp vs Survived\ntrain_df[[\"SibSp\",\"Survived\"]].groupby([\"SibSp\"],as_index=False).mean().sort_values(by=\"Survived\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:06.899272Z","iopub.execute_input":"2022-08-12T18:54:06.899597Z","iopub.status.idle":"2022-08-12T18:54:06.912190Z","shell.execute_reply.started":"2022-08-12T18:54:06.899568Z","shell.execute_reply":"2022-08-12T18:54:06.911598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Parch vs Survived\n\ntrain_df[[\"Parch\",\"Survived\"]].groupby([\"Parch\"],as_index=False).mean().sort_values(by=\"Survived\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:06.913321Z","iopub.execute_input":"2022-08-12T18:54:06.913690Z","iopub.status.idle":"2022-08-12T18:54:06.933636Z","shell.execute_reply.started":"2022-08-12T18:54:06.913659Z","shell.execute_reply":"2022-08-12T18:54:06.932945Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='7'></a><br>\n# OutLier Detection","metadata":{}},{"cell_type":"code","source":"def detect_outliers(df,features):\n    outlier_indices =[]\n    \n    for d in features:\n        # 1st quartile\n        Q1= np.percentile(df[d],25)\n        \n        # 3rd quartile\n        Q3= np.percentile(df[d],75)\n        \n        # IQR\n        IQR = Q3-Q1\n        \n        # Outlier step\n        outlier_step= IQR * 1.5\n        \n        # detect outlier and their indices\n        outlier_list_col=df[(df[d]<Q1 - outlier_step) | (df[d]> Q3 + outlier_step)].index\n        \n        # store indeces \n        outlier_indices.extend(outlier_list_col)\n        \n    outlier_indices = Counter(outlier_indices)\n    multiple_outliers = list(i for i,v in outlier_indices.items() if v>2 )\n    \n    return multiple_outliers","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:07.070747Z","iopub.execute_input":"2022-08-12T18:54:07.071143Z","iopub.status.idle":"2022-08-12T18:54:07.077362Z","shell.execute_reply.started":"2022-08-12T18:54:07.071112Z","shell.execute_reply":"2022-08-12T18:54:07.076552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.loc[detect_outliers(train_df,[\"Age\",\"SibSp\",\"Parch\",\"Fare\"])]","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:07.253292Z","iopub.execute_input":"2022-08-12T18:54:07.253833Z","iopub.status.idle":"2022-08-12T18:54:07.278800Z","shell.execute_reply.started":"2022-08-12T18:54:07.253782Z","shell.execute_reply":"2022-08-12T18:54:07.277837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Drop putliers\n\ntrain_df = train_df.drop(detect_outliers(train_df,[\"Age\",\"SibSp\",\"Parch\",\"Fare\"]),axis=0).reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:07.448327Z","iopub.execute_input":"2022-08-12T18:54:07.449242Z","iopub.status.idle":"2022-08-12T18:54:07.461047Z","shell.execute_reply.started":"2022-08-12T18:54:07.449185Z","shell.execute_reply":"2022-08-12T18:54:07.460257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='8'></a><br>\n# Missing Value\n* Find Missing Value\n* Fill Missing Value","metadata":{}},{"cell_type":"code","source":"train_df_len=len(train_df)\ntrain_df= pd.concat([train_df,test_df],axis=0).reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:07.660064Z","iopub.execute_input":"2022-08-12T18:54:07.660864Z","iopub.status.idle":"2022-08-12T18:54:07.669807Z","shell.execute_reply.started":"2022-08-12T18:54:07.660818Z","shell.execute_reply":"2022-08-12T18:54:07.668779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:07.871201Z","iopub.execute_input":"2022-08-12T18:54:07.872232Z","iopub.status.idle":"2022-08-12T18:54:07.895697Z","shell.execute_reply.started":"2022-08-12T18:54:07.872166Z","shell.execute_reply":"2022-08-12T18:54:07.892275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='9'></a><br>\n## Find Missing Value","metadata":{}},{"cell_type":"code","source":"train_df.columns[train_df.isnull().any()]","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:08.069337Z","iopub.execute_input":"2022-08-12T18:54:08.070231Z","iopub.status.idle":"2022-08-12T18:54:08.078946Z","shell.execute_reply.started":"2022-08-12T18:54:08.070188Z","shell.execute_reply":"2022-08-12T18:54:08.077978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:08.272891Z","iopub.execute_input":"2022-08-12T18:54:08.273171Z","iopub.status.idle":"2022-08-12T18:54:08.283159Z","shell.execute_reply.started":"2022-08-12T18:54:08.273142Z","shell.execute_reply":"2022-08-12T18:54:08.282269Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='10'></a><br>\n## Fill missing Value\n\n* Embarked has 2 missing value\n* Fare has only 1","metadata":{}},{"cell_type":"code","source":"train_df[train_df[\"Embarked\"].isnull()]","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:08.493759Z","iopub.execute_input":"2022-08-12T18:54:08.494057Z","iopub.status.idle":"2022-08-12T18:54:08.509928Z","shell.execute_reply.started":"2022-08-12T18:54:08.494025Z","shell.execute_reply":"2022-08-12T18:54:08.509234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.boxplot(column=\"Fare\",by= \"Embarked\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:08.690858Z","iopub.execute_input":"2022-08-12T18:54:08.691670Z","iopub.status.idle":"2022-08-12T18:54:08.832700Z","shell.execute_reply.started":"2022-08-12T18:54:08.691619Z","shell.execute_reply":"2022-08-12T18:54:08.831824Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Embarked\"]= train_df[\"Embarked\"].fillna(\"C\")","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:08.877957Z","iopub.execute_input":"2022-08-12T18:54:08.878981Z","iopub.status.idle":"2022-08-12T18:54:08.884409Z","shell.execute_reply.started":"2022-08-12T18:54:08.878936Z","shell.execute_reply":"2022-08-12T18:54:08.883240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Fare\ntrain_df[train_df[\"Fare\"].isnull()]","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:09.076703Z","iopub.execute_input":"2022-08-12T18:54:09.077287Z","iopub.status.idle":"2022-08-12T18:54:09.092687Z","shell.execute_reply.started":"2022-08-12T18:54:09.077252Z","shell.execute_reply":"2022-08-12T18:54:09.091800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Fare\"]= train_df[\"Fare\"].fillna(np.mean(train_df[train_df[\"Pclass\"]==3][\"Fare\"]))","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:09.269298Z","iopub.execute_input":"2022-08-12T18:54:09.269566Z","iopub.status.idle":"2022-08-12T18:54:09.276478Z","shell.execute_reply.started":"2022-08-12T18:54:09.269536Z","shell.execute_reply":"2022-08-12T18:54:09.275579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='11'></a><br>\n# Visualization","metadata":{}},{"cell_type":"markdown","source":"<a id='12'></a><br>\n## Corelation Between Sibsp -- Parch -- Age -- Fare -- Surived","metadata":{}},{"cell_type":"code","source":"list1= [\"SibSp\",\"Parch\",\"Age\",\"Fare\",\"Survived\"]\nsns.heatmap(train_df[list1].corr(),annot=True, fmt=\".2f\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:09.459301Z","iopub.execute_input":"2022-08-12T18:54:09.459597Z","iopub.status.idle":"2022-08-12T18:54:09.750816Z","shell.execute_reply.started":"2022-08-12T18:54:09.459562Z","shell.execute_reply":"2022-08-12T18:54:09.749937Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Fare feature seems to have correlation with survived feature(0.26).\n","metadata":{}},{"cell_type":"markdown","source":"<a id='13'></a><br>\n## SipSp -- Survived","metadata":{}},{"cell_type":"code","source":"g =sns.factorplot(x=\"SibSp\",y= \"Survived\",data=train_df, kind=\"bar\",size= 6)\ng.set_ylabels(\"Survived Probability\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:09.752398Z","iopub.execute_input":"2022-08-12T18:54:09.752637Z","iopub.status.idle":"2022-08-12T18:54:10.189409Z","shell.execute_reply.started":"2022-08-12T18:54:09.752606Z","shell.execute_reply":"2022-08-12T18:54:10.188488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Having a lot of SibSp have less chance to survive\n* if sibsp == 0 or 1 or 2,passenger has more chance to survive\n* we can consider a new feature describing  these categories","metadata":{}},{"cell_type":"markdown","source":"<a id='14'></a><br>\n## Parch -- Survived","metadata":{}},{"cell_type":"code","source":"g = sns.factorplot(x = \"Parch\", y =\"Survived\", data=train_df, kind=\"bar\",size=6)\ng.set_ylabels(\"Survived Probability\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:10.191576Z","iopub.execute_input":"2022-08-12T18:54:10.191941Z","iopub.status.idle":"2022-08-12T18:54:10.628667Z","shell.execute_reply.started":"2022-08-12T18:54:10.191896Z","shell.execute_reply":"2022-08-12T18:54:10.627635Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Sibsp and Parch can be used for new feature extraction with th =3\n* small families have more chance to survive.\n* there is a std in survival of passenger with parch =3 .","metadata":{}},{"cell_type":"markdown","source":"<a id='15'></a><br>\n## Pclass -- Survived","metadata":{}},{"cell_type":"code","source":"g = sns.factorplot(x = \"Pclass\", y =\"Survived\", data=train_df, kind=\"bar\",size=6)\ng.set_ylabels(\"Survived Probability\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:10.630516Z","iopub.execute_input":"2022-08-12T18:54:10.630852Z","iopub.status.idle":"2022-08-12T18:54:10.993766Z","shell.execute_reply.started":"2022-08-12T18:54:10.630804Z","shell.execute_reply":"2022-08-12T18:54:10.992730Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* if the passenger was in the first class, the passenger will be alive.\n* third class's passengers has less chance.\n","metadata":{}},{"cell_type":"markdown","source":"<a id='16'></a><br>\n## Age -- Survived","metadata":{}},{"cell_type":"code","source":"g =sns.FacetGrid(train_df,col=\"Survived\")\ng.map(sns.distplot, \"Age\",bins=25)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:10.996026Z","iopub.execute_input":"2022-08-12T18:54:10.998754Z","iopub.status.idle":"2022-08-12T18:54:11.480634Z","shell.execute_reply.started":"2022-08-12T18:54:10.998692Z","shell.execute_reply":"2022-08-12T18:54:11.479722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Age <= 10 has more survival rate,\n* oldest passengers(=>75) survived,\n* large number of 20 years old did not survive\n* most passengers are in 15-35 age range,\n* use age distribution for missing value of age","metadata":{}},{"cell_type":"markdown","source":"<a id='17'></a><br>\n## Pclass -- Age -- Survived","metadata":{}},{"cell_type":"code","source":"g = sns.FacetGrid(train_df,col= \"Survived\",row=\"Pclass\",size =2)\ng.map(plt.hist,\"Age\",bins=25)\ng.add_legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:11.482011Z","iopub.execute_input":"2022-08-12T18:54:11.482313Z","iopub.status.idle":"2022-08-12T18:54:12.791809Z","shell.execute_reply.started":"2022-08-12T18:54:11.482274Z","shell.execute_reply":"2022-08-12T18:54:12.791132Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Pclass is important feature for model training\n","metadata":{}},{"cell_type":"markdown","source":"<a id='18'></a><br>\n## Embarked -- Sex -- Pclass -- Survived","metadata":{}},{"cell_type":"code","source":"g= sns.FacetGrid(train_df,row=\"Embarked\",size=2)\ng.map(sns.pointplot, \"Pclass\",\"Survived\",\"Sex\")\ng.add_legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:12.792772Z","iopub.execute_input":"2022-08-12T18:54:12.793493Z","iopub.status.idle":"2022-08-12T18:54:13.848385Z","shell.execute_reply.started":"2022-08-12T18:54:12.793445Z","shell.execute_reply":"2022-08-12T18:54:13.847429Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Famale passengers have much better survival rate than males.\n* Males have better survival rate in pclass 1-2 in Embarked = C.\n* Embarked and sex will be used in training.","metadata":{}},{"cell_type":"markdown","source":"<a id='19'></a><br>\n## Embarked -- Sex -- Fare -- Survived","metadata":{}},{"cell_type":"code","source":"g= sns.FacetGrid(train_df,row=\"Embarked\",col =\"Survived\",size=3)\ng.map(sns.barplot, \"Sex\",\"Fare\")\ng.add_legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:13.849645Z","iopub.execute_input":"2022-08-12T18:54:13.849930Z","iopub.status.idle":"2022-08-12T18:54:15.790024Z","shell.execute_reply.started":"2022-08-12T18:54:13.849897Z","shell.execute_reply":"2022-08-12T18:54:15.789088Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Passengers who pay higher fare have better survival rate. Fare can be used as categorical for training.\n","metadata":{}},{"cell_type":"markdown","source":"<a id='20'></a><br>\n## Fill Missing: Age Value\n","metadata":{}},{"cell_type":"code","source":"train_df[train_df[\"Age\"].isnull()]","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:15.792246Z","iopub.execute_input":"2022-08-12T18:54:15.792498Z","iopub.status.idle":"2022-08-12T18:54:15.818179Z","shell.execute_reply.started":"2022-08-12T18:54:15.792468Z","shell.execute_reply":"2022-08-12T18:54:15.817303Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.factorplot(x= \"Sex\",y =\"Age\",data=train_df,kind=\"box\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:15.819311Z","iopub.execute_input":"2022-08-12T18:54:15.819526Z","iopub.status.idle":"2022-08-12T18:54:16.091132Z","shell.execute_reply.started":"2022-08-12T18:54:15.819499Z","shell.execute_reply":"2022-08-12T18:54:16.090475Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Sex is not inforamtive for age prediction, age distribution seems to be same.","metadata":{}},{"cell_type":"code","source":"sns.factorplot(x=\"Sex\",y=\"Age\",hue=\"Pclass\",data=train_df,kind=\"box\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:16.092109Z","iopub.execute_input":"2022-08-12T18:54:16.092939Z","iopub.status.idle":"2022-08-12T18:54:16.503270Z","shell.execute_reply.started":"2022-08-12T18:54:16.092901Z","shell.execute_reply":"2022-08-12T18:54:16.502352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"First class passengers are older than second class, and second class is older than third class.","metadata":{}},{"cell_type":"code","source":"sns.factorplot(x=\"Parch\",y=\"Age\",data=train_df,kind=\"box\")\nsns.factorplot(x=\"SibSp\",y=\"Age\",data=train_df,kind=\"box\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:16.504478Z","iopub.execute_input":"2022-08-12T18:54:16.504746Z","iopub.status.idle":"2022-08-12T18:54:17.205993Z","shell.execute_reply.started":"2022-08-12T18:54:16.504701Z","shell.execute_reply":"2022-08-12T18:54:17.205245Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Sex\"]= [1 if i ==\"male\" else 0 for i in train_df[\"Sex\"]]","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:17.207094Z","iopub.execute_input":"2022-08-12T18:54:17.207517Z","iopub.status.idle":"2022-08-12T18:54:17.214108Z","shell.execute_reply.started":"2022-08-12T18:54:17.207479Z","shell.execute_reply":"2022-08-12T18:54:17.213138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.heatmap(train_df[[\"Age\",\"Sex\",\"SibSp\",\"Parch\",\"Pclass\"]].corr(),annot=True)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:17.215599Z","iopub.execute_input":"2022-08-12T18:54:17.215945Z","iopub.status.idle":"2022-08-12T18:54:17.513074Z","shell.execute_reply.started":"2022-08-12T18:54:17.215903Z","shell.execute_reply":"2022-08-12T18:54:17.512507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Age is not correlated with sex but it is correlated with parch,sibsp and pclass","metadata":{}},{"cell_type":"code","source":"index_nan_age= list(train_df[\"Age\"][train_df[\"Age\"].isnull()].index)\nfor i in index_nan_age:\n    age_pred = train_df[\"Age\"][((train_df[\"SibSp\"]== train_df.iloc[i][\"SibSp\"])&(train_df[\"Parch\"]== train_df.iloc[i][\"Parch\"]) & (train_df[\"Pclass\"]== train_df.iloc[i][\"Pclass\"]))].median()\n    age_med = train_df[\"Age\"].median()\n    \n    if not np.isnan(age_pred):\n        train_df[\"Age\"].iloc[i]= age_pred\n    else:\n        train_df[\"Age\"].iloc[i]= age_med","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:17.514191Z","iopub.execute_input":"2022-08-12T18:54:17.514529Z","iopub.status.idle":"2022-08-12T18:54:18.061888Z","shell.execute_reply.started":"2022-08-12T18:54:17.514500Z","shell.execute_reply":"2022-08-12T18:54:18.060638Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[train_df[\"Age\"].isnull()]\n# every age has a number.","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.063416Z","iopub.execute_input":"2022-08-12T18:54:18.063687Z","iopub.status.idle":"2022-08-12T18:54:18.075229Z","shell.execute_reply.started":"2022-08-12T18:54:18.063651Z","shell.execute_reply":"2022-08-12T18:54:18.074156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='21'></a><br>\n# Feature Engineering\n    ","metadata":{}},{"cell_type":"markdown","source":"<a id  ='22'></a><br>\n## Name -- Title","metadata":{}},{"cell_type":"code","source":"train_df[\"Name\"].head(10)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.076671Z","iopub.execute_input":"2022-08-12T18:54:18.077006Z","iopub.status.idle":"2022-08-12T18:54:18.085537Z","shell.execute_reply.started":"2022-08-12T18:54:18.076971Z","shell.execute_reply":"2022-08-12T18:54:18.084386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"name= train_df[\"Name\"]\ntrain_df[\"Title\"] = [i.split(\".\")[0].split(\",\")[-1].strip()for i in name]\n#i.split(\".\")[0] yaptığımızda -> \"Braund, Mr\",\"Owen Harris\" ikiye ayırdık ve ilk değeri aldık.\n#split(\",\")[-1].strip() dediğimizde ->\"Braund\",\" Mr\"diye ayırdık ve sonunucu değeri aldık. Önündeki boşluğu silmek içinde strip kullandık.\n","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.087298Z","iopub.execute_input":"2022-08-12T18:54:18.088352Z","iopub.status.idle":"2022-08-12T18:54:18.098886Z","shell.execute_reply.started":"2022-08-12T18:54:18.088308Z","shell.execute_reply":"2022-08-12T18:54:18.098133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Title\"].head(10)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.100319Z","iopub.execute_input":"2022-08-12T18:54:18.101515Z","iopub.status.idle":"2022-08-12T18:54:18.111089Z","shell.execute_reply.started":"2022-08-12T18:54:18.101470Z","shell.execute_reply":"2022-08-12T18:54:18.110230Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.countplot(x=\"Title\",data=train_df)\nplt.xticks(rotation = 60)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.115109Z","iopub.execute_input":"2022-08-12T18:54:18.115525Z","iopub.status.idle":"2022-08-12T18:54:18.389573Z","shell.execute_reply.started":"2022-08-12T18:54:18.115470Z","shell.execute_reply":"2022-08-12T18:54:18.388678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#convert to categorical\ntrain_df[\"Title\"]= train_df[\"Title\"].replace([\"Lady\",\"the Countess\",\"Capt\",\"Col\",\"Don\",\"Dr\",\"Major\",\"Rev\",\"Sir\",\"Jonkheer\",\"Dona\"],\"other\")\n","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.390727Z","iopub.execute_input":"2022-08-12T18:54:18.391003Z","iopub.status.idle":"2022-08-12T18:54:18.398517Z","shell.execute_reply.started":"2022-08-12T18:54:18.390971Z","shell.execute_reply":"2022-08-12T18:54:18.397560Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Title\"]= [0 if i == \"Master\" else 1 if i ==\"Miss\" or i ==\"Ms\" or i ==\"Mlle\" or i ==\"Mrs\" else 2 if i ==\"Mr\" else 3 for i in train_df[\"Title\"]]\ntrain_df[\"Title\"].head(10)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.399696Z","iopub.execute_input":"2022-08-12T18:54:18.400414Z","iopub.status.idle":"2022-08-12T18:54:18.413104Z","shell.execute_reply.started":"2022-08-12T18:54:18.400374Z","shell.execute_reply":"2022-08-12T18:54:18.412420Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.countplot(x=\"Title\",data=train_df)\nplt.xticks(rotation = 60)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.414257Z","iopub.execute_input":"2022-08-12T18:54:18.414505Z","iopub.status.idle":"2022-08-12T18:54:18.540793Z","shell.execute_reply.started":"2022-08-12T18:54:18.414475Z","shell.execute_reply":"2022-08-12T18:54:18.539674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"g = sns.factorplot(x='Title',y = \"Survived\",data=train_df,kind=\"bar\")\ng.set_xticklabels([\"Master\",\"Mrs\",\"Mr\",\"Others\"])\ng.set_ylabels(\"Survival Probability\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.542170Z","iopub.execute_input":"2022-08-12T18:54:18.542387Z","iopub.status.idle":"2022-08-12T18:54:18.828239Z","shell.execute_reply.started":"2022-08-12T18:54:18.542359Z","shell.execute_reply":"2022-08-12T18:54:18.827174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.drop(labels =[\"Name\"],axis=1,inplace = True)\n","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.829490Z","iopub.execute_input":"2022-08-12T18:54:18.829762Z","iopub.status.idle":"2022-08-12T18:54:18.835070Z","shell.execute_reply.started":"2022-08-12T18:54:18.829710Z","shell.execute_reply":"2022-08-12T18:54:18.834399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.835973Z","iopub.execute_input":"2022-08-12T18:54:18.836630Z","iopub.status.idle":"2022-08-12T18:54:18.867413Z","shell.execute_reply.started":"2022-08-12T18:54:18.836597Z","shell.execute_reply":"2022-08-12T18:54:18.866830Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df=pd.get_dummies(train_df,columns=[\"Title\"])\ntrain_df.head(10)\n#Titleları dört tabloya ayırdı","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.870643Z","iopub.execute_input":"2022-08-12T18:54:18.871469Z","iopub.status.idle":"2022-08-12T18:54:18.902316Z","shell.execute_reply.started":"2022-08-12T18:54:18.871417Z","shell.execute_reply":"2022-08-12T18:54:18.900383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='23'></a><br>\n# Family Size","metadata":{}},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.904717Z","iopub.execute_input":"2022-08-12T18:54:18.905374Z","iopub.status.idle":"2022-08-12T18:54:18.925411Z","shell.execute_reply.started":"2022-08-12T18:54:18.905326Z","shell.execute_reply":"2022-08-12T18:54:18.924562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Fsize\"]= train_df[\"SibSp\"]+ train_df[\"Parch\"]+ 1","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.927172Z","iopub.execute_input":"2022-08-12T18:54:18.927657Z","iopub.status.idle":"2022-08-12T18:54:18.941804Z","shell.execute_reply.started":"2022-08-12T18:54:18.927614Z","shell.execute_reply":"2022-08-12T18:54:18.940819Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"g = sns.factorplot(x=\"Fsize\",y= \"Survived\",data=train_df,kind= \"bar\")\ng.set_ylabels(\"Survival\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:18.943277Z","iopub.execute_input":"2022-08-12T18:54:18.946067Z","iopub.status.idle":"2022-08-12T18:54:19.345213Z","shell.execute_reply.started":"2022-08-12T18:54:18.946017Z","shell.execute_reply":"2022-08-12T18:54:19.344284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"family-size\"]=[1 if i < 5 else 0 for i in train_df[\"Fsize\"]]","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.346774Z","iopub.execute_input":"2022-08-12T18:54:19.347176Z","iopub.status.idle":"2022-08-12T18:54:19.354175Z","shell.execute_reply.started":"2022-08-12T18:54:19.347133Z","shell.execute_reply":"2022-08-12T18:54:19.353345Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.355849Z","iopub.execute_input":"2022-08-12T18:54:19.356150Z","iopub.status.idle":"2022-08-12T18:54:19.388965Z","shell.execute_reply.started":"2022-08-12T18:54:19.356102Z","shell.execute_reply":"2022-08-12T18:54:19.387937Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.countplot(x=\"family-size\",data =train_df)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.390281Z","iopub.execute_input":"2022-08-12T18:54:19.390991Z","iopub.status.idle":"2022-08-12T18:54:19.503610Z","shell.execute_reply.started":"2022-08-12T18:54:19.390953Z","shell.execute_reply":"2022-08-12T18:54:19.502716Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"g = sns.factorplot(x=\"family-size\",y= \"Survived\",data=train_df,kind= \"bar\")\ng.set_ylabels(\"Survival\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.507113Z","iopub.execute_input":"2022-08-12T18:54:19.507331Z","iopub.status.idle":"2022-08-12T18:54:19.714506Z","shell.execute_reply.started":"2022-08-12T18:54:19.507303Z","shell.execute_reply":"2022-08-12T18:54:19.713917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Small families have more change to survive  than large families.\n","metadata":{}},{"cell_type":"code","source":"train_df = pd.get_dummies(train_df,columns=[\"family-size\"])\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.715807Z","iopub.execute_input":"2022-08-12T18:54:19.716630Z","iopub.status.idle":"2022-08-12T18:54:19.741819Z","shell.execute_reply.started":"2022-08-12T18:54:19.716585Z","shell.execute_reply":"2022-08-12T18:54:19.741190Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='24'></a><br>\n# Embarked","metadata":{}},{"cell_type":"code","source":"train_df[\"Embarked\"].head()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.742981Z","iopub.execute_input":"2022-08-12T18:54:19.743516Z","iopub.status.idle":"2022-08-12T18:54:19.750521Z","shell.execute_reply.started":"2022-08-12T18:54:19.743482Z","shell.execute_reply":"2022-08-12T18:54:19.749523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.countplot(x=\"Embarked\",data=train_df)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.751647Z","iopub.execute_input":"2022-08-12T18:54:19.751921Z","iopub.status.idle":"2022-08-12T18:54:19.861109Z","shell.execute_reply.started":"2022-08-12T18:54:19.751892Z","shell.execute_reply":"2022-08-12T18:54:19.859994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = pd.get_dummies(train_df,columns=[\"Embarked\"])\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.863962Z","iopub.execute_input":"2022-08-12T18:54:19.864436Z","iopub.status.idle":"2022-08-12T18:54:19.889400Z","shell.execute_reply.started":"2022-08-12T18:54:19.864392Z","shell.execute_reply":"2022-08-12T18:54:19.888573Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='25'></a><br>\n# Ticket\n        \n    ","metadata":{}},{"cell_type":"code","source":"train_df[\"Ticket\"].head(20)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.891946Z","iopub.execute_input":"2022-08-12T18:54:19.892450Z","iopub.status.idle":"2022-08-12T18:54:19.900409Z","shell.execute_reply.started":"2022-08-12T18:54:19.892402Z","shell.execute_reply":"2022-08-12T18:54:19.899763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tickets =[]\nfor i in list(train_df.Ticket):\n    if not i.isdigit():\n        tickets.append(i.replace(\".\",\" \").replace(\"/\",\"\").strip().split(\" \")[0])\n    else :\n        tickets.append(\"x\")\ntrain_df[\"Ticket\"] = tickets","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.901991Z","iopub.execute_input":"2022-08-12T18:54:19.902617Z","iopub.status.idle":"2022-08-12T18:54:19.912035Z","shell.execute_reply.started":"2022-08-12T18:54:19.902569Z","shell.execute_reply":"2022-08-12T18:54:19.911031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head(20)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.915520Z","iopub.execute_input":"2022-08-12T18:54:19.915814Z","iopub.status.idle":"2022-08-12T18:54:19.945539Z","shell.execute_reply.started":"2022-08-12T18:54:19.915770Z","shell.execute_reply":"2022-08-12T18:54:19.944918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = pd.get_dummies(train_df,columns=[\"Ticket\"],prefix=\"T\")#Ticket_... yerine T_ yaptık. Kısaltma\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.946518Z","iopub.execute_input":"2022-08-12T18:54:19.947216Z","iopub.status.idle":"2022-08-12T18:54:19.978788Z","shell.execute_reply.started":"2022-08-12T18:54:19.947182Z","shell.execute_reply":"2022-08-12T18:54:19.978232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='26'></a><br>\n# Pclass","metadata":{}},{"cell_type":"code","source":"sns.countplot(x=\"Pclass\",data=train_df)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:19.979682Z","iopub.execute_input":"2022-08-12T18:54:19.980322Z","iopub.status.idle":"2022-08-12T18:54:20.096575Z","shell.execute_reply.started":"2022-08-12T18:54:19.980291Z","shell.execute_reply":"2022-08-12T18:54:20.095667Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Pclass\"] = train_df[\"Pclass\"].astype(\"category\")\ntrain_df = pd.get_dummies(train_df,columns=[\"Pclass\"])\ntrain_df.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.098524Z","iopub.execute_input":"2022-08-12T18:54:20.099143Z","iopub.status.idle":"2022-08-12T18:54:20.130517Z","shell.execute_reply.started":"2022-08-12T18:54:20.099095Z","shell.execute_reply":"2022-08-12T18:54:20.129685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='27'></a><br>\n# Sex\n","metadata":{}},{"cell_type":"code","source":"train_df[\"Sex\"]=train_df[\"Sex\"].astype(\"category\")\ntrain_Df = pd.get_dummies(train_df,columns=[\"Sex\"])\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.132304Z","iopub.execute_input":"2022-08-12T18:54:20.132901Z","iopub.status.idle":"2022-08-12T18:54:20.161355Z","shell.execute_reply.started":"2022-08-12T18:54:20.132854Z","shell.execute_reply":"2022-08-12T18:54:20.160511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='28'></a><br>\n# Drop Passenger Id and Cabin","metadata":{}},{"cell_type":"code","source":"train_df.drop(labels =[\"PassengerId\",\"Cabin\"],axis=1,inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.162931Z","iopub.execute_input":"2022-08-12T18:54:20.163410Z","iopub.status.idle":"2022-08-12T18:54:20.169500Z","shell.execute_reply.started":"2022-08-12T18:54:20.163366Z","shell.execute_reply":"2022-08-12T18:54:20.168848Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.columns","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.170941Z","iopub.execute_input":"2022-08-12T18:54:20.171422Z","iopub.status.idle":"2022-08-12T18:54:20.182438Z","shell.execute_reply.started":"2022-08-12T18:54:20.171376Z","shell.execute_reply":"2022-08-12T18:54:20.181779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='29'></a><br>\n# Modeling","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split, StratifiedKFold, GridSearchCV\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.svm import SVC\nfrom sklearn.ensemble import RandomForestClassifier, VotingClassifier\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.metrics import accuracy_score","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.187927Z","iopub.execute_input":"2022-08-12T18:54:20.188337Z","iopub.status.idle":"2022-08-12T18:54:20.194383Z","shell.execute_reply.started":"2022-08-12T18:54:20.188285Z","shell.execute_reply":"2022-08-12T18:54:20.193770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='30'></a><br>\n # Train - Test Split","metadata":{}},{"cell_type":"code","source":"test = train_df[train_df_len:]\ntest.drop(labels=[\"Survived\"],axis=1,inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.195914Z","iopub.execute_input":"2022-08-12T18:54:20.196215Z","iopub.status.idle":"2022-08-12T18:54:20.206918Z","shell.execute_reply.started":"2022-08-12T18:54:20.196176Z","shell.execute_reply":"2022-08-12T18:54:20.206068Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.208382Z","iopub.execute_input":"2022-08-12T18:54:20.208827Z","iopub.status.idle":"2022-08-12T18:54:20.235070Z","shell.execute_reply.started":"2022-08-12T18:54:20.208793Z","shell.execute_reply":"2022-08-12T18:54:20.234114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = train_df[:train_df_len]\nx_train = train.drop(labels= \"Survived\",axis=1)\ny_train = train[\"Survived\"]\nx_train,x_test,y_train,y_test = train_test_split(x_train,y_train,test_size=0.33,random_state=42)\nprint(\"x_train\",len(x_train))\nprint(\"x_test\",len(x_test))\nprint(\"y_train\",len(y_train))\nprint(\"y_test\",len(y_test))\nprint(\"test\",len(test))\n","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.236482Z","iopub.execute_input":"2022-08-12T18:54:20.237770Z","iopub.status.idle":"2022-08-12T18:54:20.251075Z","shell.execute_reply.started":"2022-08-12T18:54:20.236703Z","shell.execute_reply":"2022-08-12T18:54:20.250022Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='31'></a><br>\n# Simple Logistic Regretion","metadata":{}},{"cell_type":"code","source":"logreg = LogisticRegression()\nlogreg.fit(x_train,y_train)\nacc_log_train = round(logreg.score(x_train,y_train)*100,2)\nacc_log_test = round(logreg.score(x_test,y_test)*100,2)\n\nprint(\"Training accuracy: % {}\".format(acc_log_train))\nprint(\"Testing accuracy: % {}\".format(acc_log_test))","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.252668Z","iopub.execute_input":"2022-08-12T18:54:20.253071Z","iopub.status.idle":"2022-08-12T18:54:20.337852Z","shell.execute_reply.started":"2022-08-12T18:54:20.253030Z","shell.execute_reply":"2022-08-12T18:54:20.336714Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='32'></a><br>\n## Hyperparameter Tuning -- Grid Search -- Cross Validation\n\nWe will compare 5 ml classifier and evaluate mean accuracy of each of them by stratified cross validation.\n\n* Decision Tree\n* SVM\n* Random Forest\n* KNN\n* Logistic Regression\n","metadata":{}},{"cell_type":"code","source":"random_state = 42\nclassifier = [DecisionTreeClassifier(random_state = random_state),\n             SVC(random_state = random_state),\n             RandomForestClassifier(random_state = random_state),\n             LogisticRegression(random_state = random_state),\n             KNeighborsClassifier()]\n\ndt_param_grid = {\"min_samples_split\" : range(10,500,20),\n                \"max_depth\": range(1,20,2)}\n\nsvc_param_grid = {\"kernel\" : [\"rbf\"],\n                 \"gamma\": [0.001, 0.01, 0.1, 1],\n                 \"C\": [1,10,50,100,200,300,1000]}\n\nrf_param_grid = {\"max_features\": [1,3,10],\n                \"min_samples_split\":[2,3,10],\n                \"min_samples_leaf\":[1,3,10],\n                \"bootstrap\":[False],\n                \"n_estimators\":[100,300],\n                \"criterion\":[\"gini\"]}\n\nlogreg_param_grid = {\"C\":np.logspace(-3,3,7),\n                    \"penalty\": [\"l1\",\"l2\"]}\n\nknn_param_grid = {\"n_neighbors\": np.linspace(1,19,10, dtype = int).tolist(),\n                 \"weights\": [\"uniform\",\"distance\"],\n                 \"metric\":[\"euclidean\",\"manhattan\"]}\nclassifier_param = [dt_param_grid,\n                   svc_param_grid,\n                   rf_param_grid,\n                   logreg_param_grid,\n                   knn_param_grid]","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.339577Z","iopub.execute_input":"2022-08-12T18:54:20.340543Z","iopub.status.idle":"2022-08-12T18:54:20.357821Z","shell.execute_reply.started":"2022-08-12T18:54:20.340499Z","shell.execute_reply":"2022-08-12T18:54:20.356723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cv_result = []\nbest_estimators = []\nfor i in range(len(classifier)):\n    clf = GridSearchCV(classifier[i], param_grid=classifier_param[i], cv = StratifiedKFold(n_splits = 10), scoring = \"accuracy\", n_jobs = -1,verbose = 1)\n    clf.fit(x_train,y_train)\n    cv_result.append(clf.best_score_)\n    best_estimators.append(clf.best_estimator_)\n    print(cv_result[i])","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:54:20.359448Z","iopub.execute_input":"2022-08-12T18:54:20.360043Z","iopub.status.idle":"2022-08-12T18:55:37.971019Z","shell.execute_reply.started":"2022-08-12T18:54:20.359998Z","shell.execute_reply":"2022-08-12T18:55:37.969933Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cv_results = pd.DataFrame({\"Cross Validation Means\": cv_result, \"ML Models\":[\"DecisionTreeClassifier\",\n                          \"SVM\",\n                          \"RandomForestClassifier\",\n                          \"LogisticRegression\",\n                          \"KNeighborsClassifier\"]})\n\ng = sns.barplot(\"Cross Validation Means\", \"ML Models\", data = cv_results)\ng.set_xlabel(\"Mean Accuracy\")\ng.set_title(\"Cross Validation Scores\")","metadata":{"execution":{"iopub.status.busy":"2022-08-12T18:55:37.973096Z","iopub.execute_input":"2022-08-12T18:55:37.973641Z","iopub.status.idle":"2022-08-12T18:55:38.219661Z","shell.execute_reply.started":"2022-08-12T18:55:37.973583Z","shell.execute_reply":"2022-08-12T18:55:38.218786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='33'></a><br>\n# Ensemble Modeling","metadata":{}},{"cell_type":"code","source":"votingC = VotingClassifier(estimators=[(\"dt\",best_estimators[0]),\n                                       (\"rfc\",best_estimators[2]),\n                                       (\"lr\",best_estimators[3])],\n                                       voting= \"soft\",n_jobs=-1)\n\nvotingC = votingC.fit(x_train,y_train)\nprint(accuracy_score(votingC.predict(x_test),y_test))","metadata":{"execution":{"iopub.status.busy":"2022-08-12T19:13:57.585578Z","iopub.execute_input":"2022-08-12T19:13:57.585930Z","iopub.status.idle":"2022-08-12T19:13:59.890697Z","shell.execute_reply.started":"2022-08-12T19:13:57.585888Z","shell.execute_reply":"2022-08-12T19:13:59.889500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id  ='34'></a><br>\n# Ensemble Modeling","metadata":{}},{"cell_type":"code","source":"test_survived = pd.Series(votingC.predict(test),name=\"Survived\").astype(int)\nresults = pd.concat([test_PassengerId,test_survived],axis=1)\nresults.to_csv(\"titanic.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-12T19:17:01.255033Z","iopub.execute_input":"2022-08-12T19:17:01.255866Z","iopub.status.idle":"2022-08-12T19:17:01.360222Z","shell.execute_reply.started":"2022-08-12T19:17:01.255810Z","shell.execute_reply":"2022-08-12T19:17:01.358841Z"},"trusted":true},"execution_count":null,"outputs":[]}]}