{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# ***Introduction***\nThe sinking of Titanic is one of the most notorious shipwrecks in the history. In 1912 during her voyage, the Titanic sank after coliding with an iceberg, killing 1502 out of 2224 passengers and crew.\n\n\n<font color='blue'>\n    Content:\n   \n 1. [Load and Check Data](#1)\n 2. [Variable Description](#2)\n      *  [Univariate Variable Anlysis](#3)\n        *  [Categorical Variable Anlysis](#4)\n        *  [Numerical Variable Anlysis](#5)\n 3. [Basic Data Analysis](#6)\n 4. [Outliear Detection](#7)\n 4. [Missing Value](#8)\n      * [Find Missing Value](#9)\n      * [Fill Missing Value](#10)\n 4. [Visualization](#11)\n      * [Correlation Between SibSp -- Parch -- Age -- Fare -- Survived](#12)\n      * [SibSp -- Survived](#13)\n      * [Parch -- Survived](#14)\n      * [Pclass -- Survived](#15)\n      * [Age -- Survived](#16)\n      * [Pclass -- Survived -- Age](#17)\n      * [Embarked -- Sex -- Pclass -- Survived](#18)\n      * [Embarked -- Sex -- Fare -- Survived](#19)\n      * [Fill Missing: Age Feature](#20)\n 4. [Feature Engineering](#21)\n      * [Name--Title](#22)\n      * [Family Size](#23)\n      * [Embarked](#24)\n      * [Ticket](#25)\n      * [Pclass](#26)\n      * [Sex](#27)\n      * [Drop PassengerID and Cabin](#28)\n 4. [Modeling](#29)\n      * [Train-Test-Split](#30)\n      * [Simple Logistic Regression](#31)\n      * [Hyperparameter Tuning -- Grid Search -- Cross Validation](#32)\n      * [Ensemble Modeling](#33)\n      * [Prediction and Submission](#34)","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport matplotlib.pyplot as plt\nplt.style.use(\"seaborn-deep\")\n\nimport seaborn as sns\n\nfrom collections import Counter\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-07-13T17:12:12.179815Z","iopub.execute_input":"2022-07-13T17:12:12.180610Z","iopub.status.idle":"2022-07-13T17:12:13.393429Z","shell.execute_reply.started":"2022-07-13T17:12:12.180513Z","shell.execute_reply":"2022-07-13T17:12:13.391921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"1\"></a><br>\n# Load and Check Data","metadata":{}},{"cell_type":"code","source":"train_df=pd.read_csv(\"/kaggle/input/titanic/train.csv\")\ntest_df=pd.read_csv(\"/kaggle/input/titanic/test.csv\")\ntest_PassengerId=test_df[\"PassengerId\"]\ngender_df=pd.read_csv(\"/kaggle/input/titanic/gender_submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:14:30.471725Z","iopub.execute_input":"2022-07-13T17:14:30.471945Z","iopub.status.idle":"2022-07-13T17:14:30.494228Z","shell.execute_reply.started":"2022-07-13T17:14:30.471921Z","shell.execute_reply":"2022-07-13T17:14:30.493045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.columns","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:24.888868Z","iopub.execute_input":"2022-07-12T13:36:24.889514Z","iopub.status.idle":"2022-07-12T13:36:24.896043Z","shell.execute_reply.started":"2022-07-12T13:36:24.889465Z","shell.execute_reply":"2022-07-12T13:36:24.895426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:24.897019Z","iopub.execute_input":"2022-07-12T13:36:24.897690Z","iopub.status.idle":"2022-07-12T13:36:24.916396Z","shell.execute_reply.started":"2022-07-12T13:36:24.897606Z","shell.execute_reply":"2022-07-12T13:36:24.915511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.describe()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:24.917926Z","iopub.execute_input":"2022-07-12T13:36:24.918173Z","iopub.status.idle":"2022-07-12T13:36:24.955716Z","shell.execute_reply.started":"2022-07-12T13:36:24.918142Z","shell.execute_reply":"2022-07-12T13:36:24.954844Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#gender_df.columns just PassengerId and Survived\n#gender_df.head()\ngender_df[\"Survived\"].describe()","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:17:04.540281Z","iopub.execute_input":"2022-07-13T17:17:04.540573Z","iopub.status.idle":"2022-07-13T17:17:04.555519Z","shell.execute_reply.started":"2022-07-13T17:17:04.540543Z","shell.execute_reply":"2022-07-13T17:17:04.554778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"2\"></a><br>\n# Variable Description\n1. PassengerId: unique id number to each passenger\n1. Survived: passenger survive(1) or died(0)\n1. Pclass: passenger class\n1. Name: name\n1. Sex: gender of passenger\n1. Age: age of passenger\n1. SibSp: number of siblings/spouses\n1. Parch: number of parents/childrens\n1. Ticket: ticket number\n1. Fare: amount of money spent on ticket\n1. Cabin: cabin category\n1. Embarked: port where passenger embarked(C=Cherbourg, Q=Queenstown, S=Southampton)","metadata":{}},{"cell_type":"code","source":"train_df.info()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:24.957098Z","iopub.execute_input":"2022-07-12T13:36:24.957335Z","iopub.status.idle":"2022-07-12T13:36:24.970963Z","shell.execute_reply.started":"2022-07-12T13:36:24.957305Z","shell.execute_reply":"2022-07-12T13:36:24.970141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* float64(2): Fare and Age\n* int64(5): Pclass, Sibsp, Parch, PassengerId and Survived\n* object(5): Cabin, Embarked, Ticket, Name and Sex","metadata":{}},{"cell_type":"markdown","source":"<a id=\"3\"></a><br>\n# Univariate Variable Anlysis\n* Categorical Variable: Survived, Sec, Pclass, Embarked, Cabin Name, Ticket, SibSp and Parch\n* Numerical Variable: Age, PassengerId and Fare","metadata":{}},{"cell_type":"markdown","source":"<a id=\"4\"></a><br>\n# Categorical Variable # ","metadata":{}},{"cell_type":"code","source":"def bar_plot(variable):\n    \"\"\"\n       ınput: variable ex: sex\n       output: bar plot & value count\n    \n    \"\"\"\n    #get feature\n    var = train_df[variable]\n    #count number of categorical variable\n    varvalue = var.value_counts()\n    \n    #visualize\n    plt.figure(figsize=(9,3))\n    plt.bar(varvalue.index, varvalue)\n    plt.xticks(varvalue.index, varvalue.index.values)\n    plt.ylabel(\"Frequency\")\n    plt.title(variable)\n    plt.show()\n    print(\"{}: \\n {}\".format(variable,varvalue))","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:21:18.601135Z","iopub.execute_input":"2022-07-13T17:21:18.601427Z","iopub.status.idle":"2022-07-13T17:21:18.609846Z","shell.execute_reply.started":"2022-07-13T17:21:18.601398Z","shell.execute_reply":"2022-07-13T17:21:18.608782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"category1 = {\"Survived\",\"Sex\",\"Pclass\",\"Embarked\",\"SibSp\",\"Parch\"}\nfor c in category1:\n    bar_plot(c)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:21:30.193358Z","iopub.execute_input":"2022-07-13T17:21:30.193696Z","iopub.status.idle":"2022-07-13T17:21:31.046173Z","shell.execute_reply.started":"2022-07-13T17:21:30.193658Z","shell.execute_reply":"2022-07-13T17:21:31.045551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"category2 = {\"Cabin\",\"Name\",\"Ticket\"}\nfor c in category2:\n    print(\"{} \\n\".format(train_df[c].value_counts()))","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:22:11.681044Z","iopub.execute_input":"2022-07-13T17:22:11.681312Z","iopub.status.idle":"2022-07-13T17:22:11.695606Z","shell.execute_reply.started":"2022-07-13T17:22:11.681283Z","shell.execute_reply":"2022-07-13T17:22:11.693961Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"5\"></a><br>\n# Numerical Variable # ","metadata":{}},{"cell_type":"code","source":"def plot_hist(variable):\n    plt.figure(figsize=(9,3))\n    plt.hist(train_df[variable], bins=50)\n    plt.xlabel(variable)\n    plt.ylabel(\"Frequency\")\n    plt.title(\"{} distribution with hist\".format(variable))\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:23:01.379957Z","iopub.execute_input":"2022-07-13T17:23:01.380229Z","iopub.status.idle":"2022-07-13T17:23:01.385772Z","shell.execute_reply.started":"2022-07-13T17:23:01.380205Z","shell.execute_reply":"2022-07-13T17:23:01.384670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"numericVar= [\"Fare\", \"Age\", \"PassengerId\"]\nfor n in numericVar:\n    plot_hist(n)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:23:04.131011Z","iopub.execute_input":"2022-07-13T17:23:04.131461Z","iopub.status.idle":"2022-07-13T17:23:04.747618Z","shell.execute_reply.started":"2022-07-13T17:23:04.131432Z","shell.execute_reply":"2022-07-13T17:23:04.746859Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"6\"></a><br>\n# Basic Data Analysis\n* Pclass - Survived\n* Sex - Survived\n* SibSp - Survived\n* Parch - Survived","metadata":{}},{"cell_type":"code","source":"# Pclass vs Survived\ntrain_df[[\"Pclass\",\"Survived\"]].groupby([\"Pclass\"],as_index=False).mean().sort_values(by=\"Survived\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:23:14.790315Z","iopub.execute_input":"2022-07-13T17:23:14.790551Z","iopub.status.idle":"2022-07-13T17:23:14.808478Z","shell.execute_reply.started":"2022-07-13T17:23:14.790528Z","shell.execute_reply":"2022-07-13T17:23:14.807947Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Sex vs Survived\ntrain_df[[\"Sex\",\"Survived\"]].groupby([\"Sex\"],as_index=False).mean().sort_values(by=\"Survived\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:23:29.329133Z","iopub.execute_input":"2022-07-13T17:23:29.329378Z","iopub.status.idle":"2022-07-13T17:23:29.346203Z","shell.execute_reply.started":"2022-07-13T17:23:29.329355Z","shell.execute_reply":"2022-07-13T17:23:29.344972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# SibSp vs Survived\ntrain_df[[\"SibSp\",\"Survived\"]].groupby([\"SibSp\"],as_index=False).mean().sort_values(by=\"Survived\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:23:38.187801Z","iopub.execute_input":"2022-07-13T17:23:38.188136Z","iopub.status.idle":"2022-07-13T17:23:38.203664Z","shell.execute_reply.started":"2022-07-13T17:23:38.188106Z","shell.execute_reply":"2022-07-13T17:23:38.202954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Parch vs Survived\ntrain_df[[\"Parch\",\"Survived\"]].groupby([\"Parch\"],as_index=False).mean().sort_values(by=\"Survived\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:23:53.079571Z","iopub.execute_input":"2022-07-13T17:23:53.079858Z","iopub.status.idle":"2022-07-13T17:23:53.093402Z","shell.execute_reply.started":"2022-07-13T17:23:53.079832Z","shell.execute_reply":"2022-07-13T17:23:53.092660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Age vs Embarked\ntrain_df[[\"Age\",\"Embarked\",\"Survived\"]].groupby([\"Embarked\"],as_index=False).mean().sort_values(by=\"Survived\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:30:59.614727Z","iopub.execute_input":"2022-07-13T17:30:59.615071Z","iopub.status.idle":"2022-07-13T17:30:59.633798Z","shell.execute_reply.started":"2022-07-13T17:30:59.615039Z","shell.execute_reply":"2022-07-13T17:30:59.632915Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Parch vs Cabin\ntrain_df[[\"Cabin\",\"Parch\",\"Survived\"]].groupby([\"Cabin\"],as_index=False).mean().sort_values(by=\"Parch\",ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:32:09.091484Z","iopub.execute_input":"2022-07-13T17:32:09.091732Z","iopub.status.idle":"2022-07-13T17:32:09.112810Z","shell.execute_reply.started":"2022-07-13T17:32:09.091708Z","shell.execute_reply":"2022-07-13T17:32:09.112229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"7\"></a><br>\n# Outlier Detection","metadata":{}},{"cell_type":"code","source":"def detect_outliers(df,features):\n    outlier_indices=[]\n    \n    for c in features:\n        # 1st quartile\n        Q1=np.percentile(df[c],25)\n        # 3rd quartile\n        Q3=np.percentile(df[c],75)\n        # IQR\n        IQR=Q3-Q1\n        # Outlier step\n        outlier_step=IQR-1.5\n        # detect outlier and their indeces\n        outlier_list_col=df[(df[c]<Q1-outlier_step)|(df[c]>Q3+outlier_step)].index\n        # store indeces\n        outlier_indices.extend(outlier_list_col)\n        \n    outlier_indices=Counter(outlier_indices)\n    multiple_outliers=list(i for i,v in outlier_indices.items() if v>2)\n    return multiple_outliers","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:39:01.974311Z","iopub.execute_input":"2022-07-13T17:39:01.974588Z","iopub.status.idle":"2022-07-13T17:39:01.981885Z","shell.execute_reply.started":"2022-07-13T17:39:01.974563Z","shell.execute_reply":"2022-07-13T17:39:01.981043Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.loc[detect_outliers(train_df,[\"Age\",\"SibSp\",\"Parch\",\"Fare\"])]","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:39:27.199116Z","iopub.execute_input":"2022-07-13T17:39:27.199353Z","iopub.status.idle":"2022-07-13T17:39:27.235640Z","shell.execute_reply.started":"2022-07-13T17:39:27.199330Z","shell.execute_reply":"2022-07-13T17:39:27.234621Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#drop outliers\ntrain_df=train_df.drop(detect_outliers(train_df,[\"Age\",\"SibSp\",\"Parch\",\"Fare\"]),axis=0).reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:40:03.965171Z","iopub.execute_input":"2022-07-13T17:40:03.965703Z","iopub.status.idle":"2022-07-13T17:40:03.976739Z","shell.execute_reply.started":"2022-07-13T17:40:03.965668Z","shell.execute_reply":"2022-07-13T17:40:03.976206Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"8\"></a><br>\n# Missing Value\n* Find Miising Value\n* Fill Missing Value","metadata":{}},{"cell_type":"code","source":"train_df_len=len(train_df)\ntrain_df=pd.concat([train_df,test_df],axis=0).reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:40:08.040922Z","iopub.execute_input":"2022-07-13T17:40:08.041212Z","iopub.status.idle":"2022-07-13T17:40:08.050334Z","shell.execute_reply.started":"2022-07-13T17:40:08.041184Z","shell.execute_reply":"2022-07-13T17:40:08.049122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:40:11.710085Z","iopub.execute_input":"2022-07-13T17:40:11.710340Z","iopub.status.idle":"2022-07-13T17:40:11.726373Z","shell.execute_reply.started":"2022-07-13T17:40:11.710316Z","shell.execute_reply":"2022-07-13T17:40:11.724918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"9\"></a><br>\n## Find Misiing Value","metadata":{}},{"cell_type":"code","source":"train_df.columns[train_df.isnull().any()]","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:44:59.745060Z","iopub.execute_input":"2022-07-13T17:44:59.745303Z","iopub.status.idle":"2022-07-13T17:44:59.751562Z","shell.execute_reply.started":"2022-07-13T17:44:59.745279Z","shell.execute_reply":"2022-07-13T17:44:59.751167Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:45:02.168928Z","iopub.execute_input":"2022-07-13T17:45:02.169409Z","iopub.status.idle":"2022-07-13T17:45:02.185901Z","shell.execute_reply.started":"2022-07-13T17:45:02.169378Z","shell.execute_reply":"2022-07-13T17:45:02.181597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"10\"></a><br>\n## Fill Misiing Value\n* Fare has only 1 missing value","metadata":{}},{"cell_type":"code","source":"train_df[train_df[\"Fare\"].isnull()]","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:45:17.529723Z","iopub.execute_input":"2022-07-13T17:45:17.530134Z","iopub.status.idle":"2022-07-13T17:45:17.549488Z","shell.execute_reply.started":"2022-07-13T17:45:17.530088Z","shell.execute_reply":"2022-07-13T17:45:17.548464Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.mean(train_df[train_df[\"Pclass\"]==3][\"Fare\"])","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:45:24.768887Z","iopub.execute_input":"2022-07-13T17:45:24.769205Z","iopub.status.idle":"2022-07-13T17:45:24.778061Z","shell.execute_reply.started":"2022-07-13T17:45:24.769176Z","shell.execute_reply":"2022-07-13T17:45:24.777088Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Fare\"]=train_df[\"Fare\"].fillna(np.mean(train_df[train_df[\"Pclass\"]==3][\"Fare\"]))\ntrain_df[train_df[\"Fare\"].isnull()]","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:45:28.791785Z","iopub.execute_input":"2022-07-13T17:45:28.792058Z","iopub.status.idle":"2022-07-13T17:45:28.808054Z","shell.execute_reply.started":"2022-07-13T17:45:28.792031Z","shell.execute_reply":"2022-07-13T17:45:28.806820Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"11\"></a><br>\n# Visualization # ","metadata":{}},{"cell_type":"markdown","source":"<a id=\"12\"></a><br>\n# Correlation Between SibSp -- Parch -- Age -- Fare -- Survived # ","metadata":{}},{"cell_type":"code","source":"list1=[\"SibSp\",\"Parch\",\"Age\",\"Fare\",\"Survived\"]\nsns.heatmap(train_df[list1].corr(),annot=True,fmt=\".3f\") #annot matris üzerindeki sayıların gösterimiyle alakalı fmt ise virgülden sonra kaç basamak\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:45:47.392590Z","iopub.execute_input":"2022-07-13T17:45:47.392885Z","iopub.status.idle":"2022-07-13T17:45:47.643113Z","shell.execute_reply.started":"2022-07-13T17:45:47.392853Z","shell.execute_reply":"2022-07-13T17:45:47.642650Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Fare feature seems to have correlation with parch feature (0.229).","metadata":{}},{"cell_type":"markdown","source":"<a id=\"13\"></a><br>\n# SibSp -- Survived # ","metadata":{}},{"cell_type":"code","source":"g=sns.factorplot(x=\"SibSp\",y=\"Survived\",data=train_df,kind=\"bar\",height=6)\ng.set_ylabels(\"Survived Probability\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:47:17.672256Z","iopub.execute_input":"2022-07-13T17:47:17.672492Z","iopub.status.idle":"2022-07-13T17:47:17.994088Z","shell.execute_reply.started":"2022-07-13T17:47:17.672469Z","shell.execute_reply":"2022-07-13T17:47:17.993322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Let's first review the links between SibSp and Survived\n* Since the SibSp rate starts decrease after 2, we can say that:\n     Those with more than 2 siblings have a lower survival rate","metadata":{}},{"cell_type":"markdown","source":"<a id=\"14\"></a><br>\n# Parch -- Survived # ","metadata":{}},{"cell_type":"code","source":"g=sns.factorplot(x=\"Parch\",y=\"Survived\",data=train_df,kind=\"bar\",height=6)\ng.set_ylabels(\"Survived Probability\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:47:43.822777Z","iopub.execute_input":"2022-07-13T17:47:43.823204Z","iopub.status.idle":"2022-07-13T17:47:44.195485Z","shell.execute_reply.started":"2022-07-13T17:47:43.823178Z","shell.execute_reply":"2022-07-13T17:47:44.194350Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* SibSp and Parch can be used for new feature extraction with th = 3.\n* Small Families have more chance to survival\n* There is a std in survival of passenger with parch = 3","metadata":{}},{"cell_type":"markdown","source":"<a id=\"15\"></a><br>\n# Pclass -- Survived # ","metadata":{}},{"cell_type":"code","source":"g=sns.factorplot(x=\"Pclass\",y=\"Survived\",data=train_df,kind=\"bar\",height=6)\ng.set_ylabels(\"Survived Probability\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-13T17:48:17.696163Z","iopub.execute_input":"2022-07-13T17:48:17.696392Z","iopub.status.idle":"2022-07-13T17:48:17.947166Z","shell.execute_reply.started":"2022-07-13T17:48:17.696370Z","shell.execute_reply":"2022-07-13T17:48:17.945736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* See that high survival rate in first class and second class passengers","metadata":{}},{"cell_type":"markdown","source":"<a id=\"16\"></a><br>\n# Age -- Survived # ","metadata":{}},{"cell_type":"code","source":"g=sns.FacetGrid(train_df,col=\"Survived\")\ng.map(sns.distplot,\"Age\",bins=25)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:28.780595Z","iopub.execute_input":"2022-07-12T13:36:28.780930Z","iopub.status.idle":"2022-07-12T13:36:29.268631Z","shell.execute_reply.started":"2022-07-12T13:36:28.780885Z","shell.execute_reply":"2022-07-12T13:36:29.267593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Age<=10 has a high survival rate\n* Oldest Passengers (80 survived)\n* large number of 20 years old did not survive\n* most passengers are in 15-35 age range\n* use age feature in training\n* use age distribution for missing value of age","metadata":{}},{"cell_type":"markdown","source":"<a id=\"17\"></a><br>\n# Pclass -- Survived -- Age # ","metadata":{}},{"cell_type":"code","source":"g=sns.FacetGrid(train_df,col=\"Survived\",row=\"Pclass\",size=3)\ng.map(plt.hist,\"Age\",bins=25)\ng.add_legend()\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:29.270007Z","iopub.execute_input":"2022-07-12T13:36:29.270288Z","iopub.status.idle":"2022-07-12T13:36:30.856553Z","shell.execute_reply.started":"2022-07-12T13:36:29.270246Z","shell.execute_reply":"2022-07-12T13:36:30.855903Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Pclass important feature is train model","metadata":{}},{"cell_type":"markdown","source":"<a id=\"18\"></a><br>\n# Embarked -- Sex -- Pclass -- Survived # ","metadata":{}},{"cell_type":"code","source":"g=sns.FacetGrid(train_df,row=\"Embarked\",size=3)\ng.map(sns.pointplot,\"Pclass\",\"Survived\",\"Sex\")\ng.add_legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:30.858077Z","iopub.execute_input":"2022-07-12T13:36:30.858586Z","iopub.status.idle":"2022-07-12T13:36:32.110325Z","shell.execute_reply.started":"2022-07-12T13:36:30.858525Z","shell.execute_reply":"2022-07-12T13:36:32.109483Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Male passenger have much better survival rate than females\n* Females Have better survival rate in pclass 2 in C\n* Embarked and sex will be used in training.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"19\"></a><br>\n# Embarked -- Sex -- Fare -- Survived # ","metadata":{}},{"cell_type":"code","source":"g=sns.FacetGrid(train_df,col=\"Survived\",row=\"Embarked\",size=3)\ng.map(sns.barplot,\"Sex\",\"Fare\")\ng.add_legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:32.111697Z","iopub.execute_input":"2022-07-12T13:36:32.112346Z","iopub.status.idle":"2022-07-12T13:36:33.484783Z","shell.execute_reply.started":"2022-07-12T13:36:32.112296Z","shell.execute_reply":"2022-07-12T13:36:33.483965Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Passengers to pay higher fare have better survival.\n* Fare can be use categorical for training.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"20\"></a><br>\n# Fill Missing: Age Value # ","metadata":{}},{"cell_type":"code","source":"train_df[train_df[\"Age\"].isnull()]","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:33.486037Z","iopub.execute_input":"2022-07-12T13:36:33.486817Z","iopub.status.idle":"2022-07-12T13:36:33.516042Z","shell.execute_reply.started":"2022-07-12T13:36:33.486777Z","shell.execute_reply":"2022-07-12T13:36:33.515226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.factorplot(x=\"Sex\",y=\"Age\",data=train_df,kind=\"box\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:33.517158Z","iopub.execute_input":"2022-07-12T13:36:33.517386Z","iopub.status.idle":"2022-07-12T13:36:33.782151Z","shell.execute_reply.started":"2022-07-12T13:36:33.517354Z","shell.execute_reply":"2022-07-12T13:36:33.781302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Sex is not informative feature for Age prediction, because age distribution semmes to be same","metadata":{}},{"cell_type":"code","source":"sns.factorplot(x=\"Sex\",y=\"Age\",hue=\"Pclass\",row=\"Embarked\",data=train_df,kind=\"box\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:33.783619Z","iopub.execute_input":"2022-07-12T13:36:33.784041Z","iopub.status.idle":"2022-07-12T13:36:35.493170Z","shell.execute_reply.started":"2022-07-12T13:36:33.784004Z","shell.execute_reply":"2022-07-12T13:36:35.492404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"First class passengers are older than 2nd, and 2nd older than ","metadata":{}},{"cell_type":"code","source":"sns.factorplot(x=\"Parch\",y=\"Age\",hue=\"Pclass\",row=\"Embarked\",data=train_df,kind=\"box\")\nsns.factorplot(x=\"SibSp\",y=\"Age\",hue=\"Pclass\",row=\"Embarked\",data=train_df,kind=\"box\")\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:35.494279Z","iopub.execute_input":"2022-07-12T13:36:35.494519Z","iopub.status.idle":"2022-07-12T13:36:37.953846Z","shell.execute_reply.started":"2022-07-12T13:36:35.494490Z","shell.execute_reply":"2022-07-12T13:36:37.953052Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.heatmap(train_df[[\"Age\",\"Sex\",\"SibSp\",\"Parch\",\"Pclass\"]].corr(),annot=True)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:37.955024Z","iopub.execute_input":"2022-07-12T13:36:37.955252Z","iopub.status.idle":"2022-07-12T13:36:38.238239Z","shell.execute_reply.started":"2022-07-12T13:36:37.955224Z","shell.execute_reply":"2022-07-12T13:36:38.237291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Sex\"]=[1 if i==\"male\" else 0 for i in train_df[\"Sex\"]]","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:38.240092Z","iopub.execute_input":"2022-07-12T13:36:38.240487Z","iopub.status.idle":"2022-07-12T13:36:38.247688Z","shell.execute_reply.started":"2022-07-12T13:36:38.240435Z","shell.execute_reply":"2022-07-12T13:36:38.246576Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.heatmap(train_df[[\"Age\",\"Sex\",\"SibSp\",\"Parch\",\"Pclass\"]].corr(),annot=True)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:38.249576Z","iopub.execute_input":"2022-07-12T13:36:38.249913Z","iopub.status.idle":"2022-07-12T13:36:38.555830Z","shell.execute_reply.started":"2022-07-12T13:36:38.249868Z","shell.execute_reply":"2022-07-12T13:36:38.554987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Age-Pclass\n* Age-SibSp\n* Age-Parch are connected.\n\nSex is not connected with age\n","metadata":{}},{"cell_type":"code","source":"index_nan_age=list(train_df[\"Age\"][train_df[\"Age\"].isnull()].index)\nfor i in index_nan_age:\n    age_pred=train_df[\"Age\"][((train_df[\"SibSp\"]==train_df.iloc[i][\"SibSp\"])&(train_df[\"Pclass\"]==train_df.iloc[i][\"Pclass\"])&(train_df[\"Parch\"]==train_df.iloc[i][\"Parch\"]))].median()\n    age_med=train_df[\"Age\"].median()\n    \n    if not np.isnan(age_pred):\n        train_df[\"Age\"].iloc[i]=age_pred\n    else:\n        train_df[\"Age\"].iloc[i]=age_med\n    ","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:38.557062Z","iopub.execute_input":"2022-07-12T13:36:38.557299Z","iopub.status.idle":"2022-07-12T13:36:39.102450Z","shell.execute_reply.started":"2022-07-12T13:36:38.557269Z","shell.execute_reply":"2022-07-12T13:36:39.101684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[train_df[\"Age\"].isnull()]","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:39.104359Z","iopub.execute_input":"2022-07-12T13:36:39.104579Z","iopub.status.idle":"2022-07-12T13:36:39.120078Z","shell.execute_reply.started":"2022-07-12T13:36:39.104552Z","shell.execute_reply":"2022-07-12T13:36:39.119043Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"21\"></a><br>\n# Feature Engineering #","metadata":{}},{"cell_type":"markdown","source":"<a id=\"22\"></a><br>\n# Name--Title #","metadata":{}},{"cell_type":"code","source":"train_df[\"Name\"].head(10)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:39.135595Z","iopub.execute_input":"2022-07-12T13:36:39.136144Z","iopub.status.idle":"2022-07-12T13:36:39.144487Z","shell.execute_reply.started":"2022-07-12T13:36:39.136103Z","shell.execute_reply":"2022-07-12T13:36:39.143627Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"name=train_df[\"Name\"]\ntrain_df[\"Title\"]=[i.split(\".\")[0].split(\",\")[-1].strip() for i in name]","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:39.145760Z","iopub.execute_input":"2022-07-12T13:36:39.146098Z","iopub.status.idle":"2022-07-12T13:36:39.156060Z","shell.execute_reply.started":"2022-07-12T13:36:39.146055Z","shell.execute_reply":"2022-07-12T13:36:39.155094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Title\"].head(10)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:39.157324Z","iopub.execute_input":"2022-07-12T13:36:39.157660Z","iopub.status.idle":"2022-07-12T13:36:39.169831Z","shell.execute_reply.started":"2022-07-12T13:36:39.157616Z","shell.execute_reply":"2022-07-12T13:36:39.169206Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.countplot(x=\"Title\",data=train_df)\nplt.xticks(rotation=60)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:39.170688Z","iopub.execute_input":"2022-07-12T13:36:39.170964Z","iopub.status.idle":"2022-07-12T13:36:39.421299Z","shell.execute_reply.started":"2022-07-12T13:36:39.170934Z","shell.execute_reply":"2022-07-12T13:36:39.419378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#convert to categorical\ntrain_df[\"Title\"]=train_df[\"Title\"].replace([\"Lady\",\"the Countess\",\"Capt\",\"Col\",\"Don\",\"Dr\",\"Major\",\"Rev\",\"Sir\",\"Jonkheer\",\"Dona\"],\"other\")\ntrain_df[\"Title\"]=[0 if i==\"Master\" else 1 if i==\"Miss\" or i==\"Ms\" or i==\"Mrs\" or i==\"Mlle\" else 2 if i==\"Mr\" else 3 for i in train_df[\"Title\"]]\ntrain_df[\"Title\"].head(20)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:39.422702Z","iopub.execute_input":"2022-07-12T13:36:39.422968Z","iopub.status.idle":"2022-07-12T13:36:39.437667Z","shell.execute_reply.started":"2022-07-12T13:36:39.422939Z","shell.execute_reply":"2022-07-12T13:36:39.436864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.countplot(x=\"Title\",data=train_df)\nplt.xticks(rotation=60)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:39.438905Z","iopub.execute_input":"2022-07-12T13:36:39.439221Z","iopub.status.idle":"2022-07-12T13:36:39.611936Z","shell.execute_reply.started":"2022-07-12T13:36:39.439192Z","shell.execute_reply":"2022-07-12T13:36:39.611316Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"g=sns.factorplot(x=\"Title\", y=\"Survived\",data=train_df,kind=\"bar\")\ng.set_xticklabels([\"Master\",\"Mrs\",\"Mr\",\"Other\"])\ng.set_ylabels(\"Survived Probability\")\ng.add_legend()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:39.613034Z","iopub.execute_input":"2022-07-12T13:36:39.613352Z","iopub.status.idle":"2022-07-12T13:36:40.043902Z","shell.execute_reply.started":"2022-07-12T13:36:39.613323Z","shell.execute_reply":"2022-07-12T13:36:40.042922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.drop(labels=\"Name\",axis=1,inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.045488Z","iopub.execute_input":"2022-07-12T13:36:40.045836Z","iopub.status.idle":"2022-07-12T13:36:40.051920Z","shell.execute_reply.started":"2022-07-12T13:36:40.045799Z","shell.execute_reply":"2022-07-12T13:36:40.051229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.052983Z","iopub.execute_input":"2022-07-12T13:36:40.053576Z","iopub.status.idle":"2022-07-12T13:36:40.077787Z","shell.execute_reply.started":"2022-07-12T13:36:40.053504Z","shell.execute_reply":"2022-07-12T13:36:40.076856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df=pd.get_dummies(train_df,columns=[\"Title\"])\ntrain_df.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.079518Z","iopub.execute_input":"2022-07-12T13:36:40.080120Z","iopub.status.idle":"2022-07-12T13:36:40.108921Z","shell.execute_reply.started":"2022-07-12T13:36:40.080073Z","shell.execute_reply":"2022-07-12T13:36:40.108077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"23\"></a><br>\n# Family Size #","metadata":{}},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.110167Z","iopub.execute_input":"2022-07-12T13:36:40.111049Z","iopub.status.idle":"2022-07-12T13:36:40.129223Z","shell.execute_reply.started":"2022-07-12T13:36:40.111004Z","shell.execute_reply":"2022-07-12T13:36:40.128614Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Fsize\"]=train_df[\"SibSp\"]+train_df[\"Parch\"]+1","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.130189Z","iopub.execute_input":"2022-07-12T13:36:40.130977Z","iopub.status.idle":"2022-07-12T13:36:40.136589Z","shell.execute_reply.started":"2022-07-12T13:36:40.130900Z","shell.execute_reply":"2022-07-12T13:36:40.135971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.137453Z","iopub.execute_input":"2022-07-12T13:36:40.137994Z","iopub.status.idle":"2022-07-12T13:36:40.164778Z","shell.execute_reply.started":"2022-07-12T13:36:40.137962Z","shell.execute_reply":"2022-07-12T13:36:40.163931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"g=sns.factorplot(x=\"Fsize\",y=\"Survived\",data=train_df,kind=\"bar\")\ng.set_ylabels(\"Survival Rating\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.166108Z","iopub.execute_input":"2022-07-12T13:36:40.166343Z","iopub.status.idle":"2022-07-12T13:36:40.689138Z","shell.execute_reply.started":"2022-07-12T13:36:40.166314Z","shell.execute_reply":"2022-07-12T13:36:40.688309Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"family_size\"]=[1 if i<5 else 0 for i in train_df[\"Fsize\"]]\ntrain_df.head(20)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.690303Z","iopub.execute_input":"2022-07-12T13:36:40.690528Z","iopub.status.idle":"2022-07-12T13:36:40.719842Z","shell.execute_reply.started":"2022-07-12T13:36:40.690499Z","shell.execute_reply":"2022-07-12T13:36:40.718981Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"g=sns.countplot(x=\"family_size\",data=train_df)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.720998Z","iopub.execute_input":"2022-07-12T13:36:40.721594Z","iopub.status.idle":"2022-07-12T13:36:40.842168Z","shell.execute_reply.started":"2022-07-12T13:36:40.721522Z","shell.execute_reply":"2022-07-12T13:36:40.841479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"g=sns.barplot(x=\"family_size\",y=\"Survived\",data=train_df)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:40.843682Z","iopub.execute_input":"2022-07-12T13:36:40.844181Z","iopub.status.idle":"2022-07-12T13:36:41.028248Z","shell.execute_reply.started":"2022-07-12T13:36:40.844133Z","shell.execute_reply":"2022-07-12T13:36:41.027567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Small families have more chance to survive than large families","metadata":{}},{"cell_type":"code","source":"train_df=pd.get_dummies(train_df,columns=[\"family_size\"])\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:41.029718Z","iopub.execute_input":"2022-07-12T13:36:41.030224Z","iopub.status.idle":"2022-07-12T13:36:41.056220Z","shell.execute_reply.started":"2022-07-12T13:36:41.030177Z","shell.execute_reply":"2022-07-12T13:36:41.055137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"24\"></a><br>\n# Embarked #","metadata":{}},{"cell_type":"code","source":"train_df[\"Embarked\"].head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:41.058886Z","iopub.execute_input":"2022-07-12T13:36:41.059214Z","iopub.status.idle":"2022-07-12T13:36:41.067609Z","shell.execute_reply.started":"2022-07-12T13:36:41.059170Z","shell.execute_reply":"2022-07-12T13:36:41.066688Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.countplot(x=\"Embarked\",data=train_df)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:41.069029Z","iopub.execute_input":"2022-07-12T13:36:41.069837Z","iopub.status.idle":"2022-07-12T13:36:41.220515Z","shell.execute_reply.started":"2022-07-12T13:36:41.069783Z","shell.execute_reply":"2022-07-12T13:36:41.219373Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df=pd.get_dummies(train_df,columns=[\"Embarked\"])\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:41.222294Z","iopub.execute_input":"2022-07-12T13:36:41.222691Z","iopub.status.idle":"2022-07-12T13:36:41.249162Z","shell.execute_reply.started":"2022-07-12T13:36:41.222643Z","shell.execute_reply":"2022-07-12T13:36:41.248110Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"25\"></a><br>\n# Ticket #","metadata":{}},{"cell_type":"code","source":"train_df[\"Ticket\"].head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:41.250667Z","iopub.execute_input":"2022-07-12T13:36:41.251007Z","iopub.status.idle":"2022-07-12T13:36:41.259083Z","shell.execute_reply.started":"2022-07-12T13:36:41.250961Z","shell.execute_reply":"2022-07-12T13:36:41.258267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tickets=[]\nfor i in list(train_df.Ticket):\n    if not i.isdigit():\n        tickets.append(i.replace(\".\",\"\").replace(\"/\",\"\").strip().split(\" \")[0])\n    else:\n        tickets.append(\"x\")\n        \ntrain_df[\"Ticket\"]=tickets\n","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:41.260602Z","iopub.execute_input":"2022-07-12T13:36:41.260843Z","iopub.status.idle":"2022-07-12T13:36:41.271747Z","shell.execute_reply.started":"2022-07-12T13:36:41.260814Z","shell.execute_reply":"2022-07-12T13:36:41.270770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.Ticket.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:41.273420Z","iopub.execute_input":"2022-07-12T13:36:41.274019Z","iopub.status.idle":"2022-07-12T13:36:41.285059Z","shell.execute_reply.started":"2022-07-12T13:36:41.273985Z","shell.execute_reply":"2022-07-12T13:36:41.284220Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df=pd.get_dummies(train_df,columns=[\"Ticket\"],prefix=\"T\")","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:41.286472Z","iopub.execute_input":"2022-07-12T13:36:41.287043Z","iopub.status.idle":"2022-07-12T13:36:41.298811Z","shell.execute_reply.started":"2022-07-12T13:36:41.287004Z","shell.execute_reply":"2022-07-12T13:36:41.297823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:36:41.300214Z","iopub.execute_input":"2022-07-12T13:36:41.301149Z","iopub.status.idle":"2022-07-12T13:36:41.327305Z","shell.execute_reply.started":"2022-07-12T13:36:41.301099Z","shell.execute_reply":"2022-07-12T13:36:41.326527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"26\"></a><br>\n# Pclass #","metadata":{}},{"cell_type":"code","source":"sns.countplot(x=\"Pclass\",data=train_df)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:37:52.857802Z","iopub.execute_input":"2022-07-12T13:37:52.858308Z","iopub.status.idle":"2022-07-12T13:37:53.034584Z","shell.execute_reply.started":"2022-07-12T13:37:52.858253Z","shell.execute_reply":"2022-07-12T13:37:53.033710Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[\"Pclass\"]=train_df[\"Pclass\"].astype(\"category\")\ntrain_df=pd.get_dummies(train_df,columns=[\"Pclass\"])\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:40:10.136952Z","iopub.execute_input":"2022-07-12T13:40:10.137435Z","iopub.status.idle":"2022-07-12T13:40:10.165558Z","shell.execute_reply.started":"2022-07-12T13:40:10.137384Z","shell.execute_reply":"2022-07-12T13:40:10.164938Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"27\"></a><br>\n# Sex #","metadata":{}},{"cell_type":"code","source":"train_df[\"Sex\"]=train_df[\"Sex\"].astype(\"category\")\ntrain_df=pd.get_dummies(train_df,columns=[\"Sex\"])\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:46:45.172518Z","iopub.execute_input":"2022-07-12T13:46:45.173010Z","iopub.status.idle":"2022-07-12T13:46:45.202604Z","shell.execute_reply.started":"2022-07-12T13:46:45.172976Z","shell.execute_reply":"2022-07-12T13:46:45.201800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"28\"></a><br>\n# Drop PassengerID and Cabin #","metadata":{}},{"cell_type":"code","source":"train_df.drop(labels=[\"PassengerId\",\"Cabin\"],axis=1,inplace=True)\ntrain_df.columns","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:49:02.947037Z","iopub.execute_input":"2022-07-12T13:49:02.947619Z","iopub.status.idle":"2022-07-12T13:49:02.958865Z","shell.execute_reply.started":"2022-07-12T13:49:02.947580Z","shell.execute_reply":"2022-07-12T13:49:02.958109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"29\"></a><br>\n# Modelling #","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split,GridSearchCV,StratifiedKFold\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.svm import SVC\nfrom sklearn.ensemble import RandomForestClassifier,VotingClassifier\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.metrics import accuracy_score","metadata":{"execution":{"iopub.status.busy":"2022-07-12T13:54:27.445902Z","iopub.execute_input":"2022-07-12T13:54:27.446202Z","iopub.status.idle":"2022-07-12T13:54:27.452145Z","shell.execute_reply.started":"2022-07-12T13:54:27.446168Z","shell.execute_reply":"2022-07-12T13:54:27.451223Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"30\"></a><br>\n# Train-Test-Split #","metadata":{}},{"cell_type":"code","source":"train_df_len","metadata":{"execution":{"iopub.status.busy":"2022-07-12T14:03:09.211500Z","iopub.execute_input":"2022-07-12T14:03:09.211952Z","iopub.status.idle":"2022-07-12T14:03:09.217188Z","shell.execute_reply.started":"2022-07-12T14:03:09.211918Z","shell.execute_reply":"2022-07-12T14:03:09.216441Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test=train_df[train_df_len:]\ntest.drop(labels=[\"Survived\"],axis=1,inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T14:05:18.043647Z","iopub.execute_input":"2022-07-12T14:05:18.043947Z","iopub.status.idle":"2022-07-12T14:05:18.050786Z","shell.execute_reply.started":"2022-07-12T14:05:18.043917Z","shell.execute_reply":"2022-07-12T14:05:18.049874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-12T14:05:25.228092Z","iopub.execute_input":"2022-07-12T14:05:25.228565Z","iopub.status.idle":"2022-07-12T14:05:25.252046Z","shell.execute_reply.started":"2022-07-12T14:05:25.228501Z","shell.execute_reply":"2022-07-12T14:05:25.251454Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train=train_df[:train_df_len]\nx_train=train.drop(labels=[\"Survived\"],axis=1)\ny_train=train[\"Survived\"]\nx_train,x_test,y_train,y_test=train_test_split(x_train,y_train,test_size=0.33,random_state=42)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-12T14:11:08.514341Z","iopub.execute_input":"2022-07-12T14:11:08.514655Z","iopub.status.idle":"2022-07-12T14:11:08.559185Z","shell.execute_reply.started":"2022-07-12T14:11:08.514621Z","shell.execute_reply":"2022-07-12T14:11:08.558244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"x_train: \",len(x_train))\nprint(\"x_test: \",len(x_test))\nprint(\"y_train: \",len(y_train))\nprint(\"y_test: \",len(y_test))\nprint(\"test: \",len(test))","metadata":{"execution":{"iopub.status.busy":"2022-07-12T14:12:44.139471Z","iopub.execute_input":"2022-07-12T14:12:44.139823Z","iopub.status.idle":"2022-07-12T14:12:44.147990Z","shell.execute_reply.started":"2022-07-12T14:12:44.139782Z","shell.execute_reply":"2022-07-12T14:12:44.147110Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"31\"></a><br>\n# Simple Logistic Regression #","metadata":{}},{"cell_type":"code","source":"logreg=LogisticRegression()\nlogreg.fit(x_train,y_train)\nacc_log_train=round(logreg.score(x_train,y_train)*100,2)\nacc_log_test=round(logreg.score(x_test,y_test)*100,2)\nprint(\"Training Accuracy: %\",acc_log_train)\nprint(\"Testing Accuracy: %\",acc_log_test)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T14:37:58.395873Z","iopub.execute_input":"2022-07-12T14:37:58.396287Z","iopub.status.idle":"2022-07-12T14:37:58.472485Z","shell.execute_reply.started":"2022-07-12T14:37:58.396254Z","shell.execute_reply":"2022-07-12T14:37:58.471560Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"32\"></a><br>\n# Hyperparameter Tuning -- Grid Search -- Cross Validation #\n\nWe will compare 5 ml classifier and evaluate mean accuracy of each of them by stratified cross validation.\n\n* Decision Tree\n* SVM\n* Random Forest\n* KNN\n* Logistic Regression\n","metadata":{}},{"cell_type":"code","source":"random_state=42\nclassifier=[DecisionTreeClassifier(random_state=random_state),\n           SVC(random_state=random_state),\n           RandomForestClassifier(random_state=random_state),\n           LogisticRegression(random_state=random_state),\n           KNeighborsClassifier()]\n\ndt_param_grid={\"min_samples_split\": range(10,500,20),\n              \"max_depth\": range(1,20,2)}\n\nsvc_param_grid={\"kernel\":[\"rbf\"],\n               \"gamma\":[0.001,0.01,0.1,1],\n               \"C\": [1,10,50,100,200,300,1000]}\n\nrf_param_grid={\"max_features\": [1,3,10],\n                \"min_samples_split\":[2,3,10],\n                \"min_samples_leaf\":[1,3,10],\n                \"bootstrap\":[False],\n                \"n_estimators\":[100,300],\n                \"criterion\":[\"gini\"]}\n\nlogreg_param_grid={\"C\":np.logspace(-3,3,7),\n                  \"penalty\":[\"l1\",\"l2\"]}\n\nknn_param_grid={\"n_neighbors\":np.linspace(1,19,10,dtype=int).tolist(),\n               \"weights\":[\"uniform\",\"distance\"],\n               \"metric\":[\"euclidean\",\"manhattan\"]}\n\nclassifier_param=[dt_param_grid,\n                 svc_param_grid,\n                 rf_param_grid,\n                 logreg_param_grid,\n                 knn_param_grid]","metadata":{"execution":{"iopub.status.busy":"2022-07-12T15:43:00.230904Z","iopub.execute_input":"2022-07-12T15:43:00.231314Z","iopub.status.idle":"2022-07-12T15:43:00.247447Z","shell.execute_reply.started":"2022-07-12T15:43:00.231275Z","shell.execute_reply":"2022-07-12T15:43:00.246297Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cv_result=[]\nbest_estimators=[]\n\nfor i in range(len(classifier)):\n    clf=GridSearchCV(classifier[i],param_grid=classifier_param[i],cv=StratifiedKFold(n_splits=10),scoring=\"accuracy\",n_jobs=-1,verbose=1)\n    clf.fit(x_train,y_train)\n    cv_result.append(clf.best_score_)\n    best_estimators.append(clf.best_estimator_)\n    print(cv_result[i])","metadata":{"execution":{"iopub.status.busy":"2022-07-12T15:43:10.655410Z","iopub.execute_input":"2022-07-12T15:43:10.656061Z","iopub.status.idle":"2022-07-12T15:44:39.543685Z","shell.execute_reply.started":"2022-07-12T15:43:10.656021Z","shell.execute_reply":"2022-07-12T15:44:39.542751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cv_results = pd.DataFrame({\"Cross Validation Means\":cv_result, \"ML Models\":[\"DecisionTreeClassifier\", \"SVM\",\"RandomForestClassifier\",\n             \"LogisticRegression\",\n             \"KNeighborsClassifier\"]})\n\ng = sns.barplot(\"Cross Validation Means\", \"ML Models\", data = cv_results)\ng.set_xlabel(\"Mean Accuracy\")\ng.set_title(\"Cross Validation Scores\")","metadata":{"execution":{"iopub.status.busy":"2022-07-12T15:44:59.280755Z","iopub.execute_input":"2022-07-12T15:44:59.281737Z","iopub.status.idle":"2022-07-12T15:44:59.479753Z","shell.execute_reply.started":"2022-07-12T15:44:59.281682Z","shell.execute_reply":"2022-07-12T15:44:59.478602Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"33\"></a><br>\n# Ensemble Modeling #","metadata":{}},{"cell_type":"code","source":"votingC=VotingClassifier(estimators=[(\"dt\",best_estimators[0]),\n                                    (\"svc\",best_estimators[0]),\n                                    (\"rfc\",best_estimators[0]),\n                                    (\"lr\",best_estimators[0])],\n                                    voting=\"soft\",n_jobs=-1)\nvotingC=votingC.fit(x_train,y_train)\nprint(accuracy_score(votingC.predict(x_test),y_test))","metadata":{"execution":{"iopub.status.busy":"2022-07-12T15:55:52.198828Z","iopub.execute_input":"2022-07-12T15:55:52.199504Z","iopub.status.idle":"2022-07-12T15:55:52.253337Z","shell.execute_reply.started":"2022-07-12T15:55:52.199461Z","shell.execute_reply":"2022-07-12T15:55:52.252176Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"34\"></a><br>\n# Prediction and Submission #","metadata":{}},{"cell_type":"code","source":"test_survived=pd.Series(votingC.predict(test),name=\"Survived\").astype(int)\nresults=pd.concat([test_PassengerId,test_survived],axis=1)\nresults.to_csv(\"titanic.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-12T16:04:58.451684Z","iopub.execute_input":"2022-07-12T16:04:58.451991Z","iopub.status.idle":"2022-07-12T16:04:58.474002Z","shell.execute_reply.started":"2022-07-12T16:04:58.451960Z","shell.execute_reply":"2022-07-12T16:04:58.473330Z"},"trusted":true},"execution_count":null,"outputs":[]}]}