{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

Titanic - Complete Data Science Solution

","metadata":{}},{"cell_type":"markdown","source":"![](https://cdn.britannica.com/68/185468-050-267B9304/Titanic-iceberg-British-15-1912.jpg)\n\nReference: [Encyclopedia Britannica](https://www.britannica.com/topic/Titanic)","metadata":{}},{"cell_type":"markdown","source":"
\n

Problem Statement

\n
\nOn April 15, 1912, during her maiden voyage, the \"unsinkable\" Titanic sank after colliding with an iceberg. This unfortunate incident resulted in the demise of 1502 out of 2224 passengers and crew.\n\nCreate a model that can determine, given a labelled training set of samples listing passengers who either did or did not survive the disaster, which of the passengers on the test dataset would have survived.\n\n*Dataset description*\n\nsurvival: Survival (0 = No, 1 = Yes)\n\npclass: Ticket class (1 = 1st, 2 = 2nd, 3 = 3rd)\n\nsex: Sex\n\nAge: Age in years\n\nsibsp: # of siblings / spouses aboard the Titanic\n\nparch: # of parents / children aboard the Titanic\n\nticket: Ticket number\n\nfare: Passenger fare\n\ncabin: Cabin number\n\nembarked: Port of Embarkation (C = Cherbourg, Q = Queenstown, S = Southampton)","metadata":{}},{"cell_type":"markdown","source":"
\n

Importing relevant components

\n
","metadata":{}},{"cell_type":"markdown","source":"

Libraries

","metadata":{}},{"cell_type":"code","source":"# For data manipulation and visualization\nimport numpy as np\nimport pandas as pd\n# import pandas_profiling\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# For predictive data analysis\nfrom sklearn.preprocessing import OneHotEncoder, LabelEncoder\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.naive_bayes import GaussianNB\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.svm import SVC\nfrom sklearn.metrics import confusion_matrix, accuracy_score, precision_recall_curve ","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:20.096083Z","iopub.execute_input":"2022-07-06T06:06:20.096580Z","iopub.status.idle":"2022-07-06T06:06:21.772101Z","shell.execute_reply.started":"2022-07-06T06:06:20.096476Z","shell.execute_reply":"2022-07-06T06:06:21.771047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note:** A common Pandas function is describe() which provides a descriptive statistical summary of all the features of the dataset. pandas_profiling is an improvement on this function and offers web format report generation for the dataset with lots of features and customizations for the report generated. However, it is not compatible with the default Python version on Kaggle, as yet, and has been commented until the issue is solved.","metadata":{}},{"cell_type":"markdown","source":"

Dataset

\nWhile importing the datasets, the \"PassengerId\" field is made the index column.","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv(\"../input/titanic/train.csv\", index_col = \"PassengerId\")\ntest = pd.read_csv(\"../input/titanic/test.csv\", index_col = \"PassengerId\")","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.774227Z","iopub.execute_input":"2022-07-06T06:06:21.774758Z","iopub.status.idle":"2022-07-06T06:06:21.813049Z","shell.execute_reply.started":"2022-07-06T06:06:21.774690Z","shell.execute_reply":"2022-07-06T06:06:21.812053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.814290Z","iopub.execute_input":"2022-07-06T06:06:21.814712Z","iopub.status.idle":"2022-07-06T06:06:21.838867Z","shell.execute_reply.started":"2022-07-06T06:06:21.814677Z","shell.execute_reply":"2022-07-06T06:06:21.837798Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.840757Z","iopub.execute_input":"2022-07-06T06:06:21.841124Z","iopub.status.idle":"2022-07-06T06:06:21.858414Z","shell.execute_reply.started":"2022-07-06T06:06:21.841090Z","shell.execute_reply":"2022-07-06T06:06:21.857357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

Data wrangling

\n
","metadata":{}},{"cell_type":"markdown","source":"Data wrangling is the process of cleaning and unifying messy datasets for easy access and analysis. It is essentially the act of transforming data from a generally raw form to a more appropriate and valuable form, therbey making it suitable for a variety of downstream purposes such as analytics.\n\nUsing Pandas, we can describe the dataset and attain an in-depth understanding of the nature of our data. Using this understanding, we may then proceed to clean the data. As the popular aphorism goes... \"Garbage in, garbage out.\" The measure of a created model will be highly dependant on the data used to create it. Thus, data wrangling is a pivotal step in the predictive data analysis pipeline. ","metadata":{}},{"cell_type":"markdown","source":"**How many rows and columns do we have?**","metadata":{}},{"cell_type":"code","source":"train.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.860328Z","iopub.execute_input":"2022-07-06T06:06:21.861199Z","iopub.status.idle":"2022-07-06T06:06:21.870470Z","shell.execute_reply.started":"2022-07-06T06:06:21.861149Z","shell.execute_reply":"2022-07-06T06:06:21.869423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.872189Z","iopub.execute_input":"2022-07-06T06:06:21.872650Z","iopub.status.idle":"2022-07-06T06:06:21.882221Z","shell.execute_reply.started":"2022-07-06T06:06:21.872607Z","shell.execute_reply":"2022-07-06T06:06:21.881208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see that the train dataset has 891 records (rows) and 11 attributes/fields (columns). The test dataset has 418 records and 10 attributes. This is expected as the test dataset will not have the label (survived/died). ","metadata":{}},{"cell_type":"markdown","source":"**What features are available in the dataset?**\n\nFeatures and attributes are often used interchangeably. However, to be accurate from a terminology perspective, keep in mind that attributes refer to all available fields while features refer to those attributes used to create the model.","metadata":{}},{"cell_type":"code","source":"train.columns","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.883374Z","iopub.execute_input":"2022-07-06T06:06:21.883803Z","iopub.status.idle":"2022-07-06T06:06:21.892745Z","shell.execute_reply.started":"2022-07-06T06:06:21.883757Z","shell.execute_reply":"2022-07-06T06:06:21.891895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.columns","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.893923Z","iopub.execute_input":"2022-07-06T06:06:21.894273Z","iopub.status.idle":"2022-07-06T06:06:21.907427Z","shell.execute_reply.started":"2022-07-06T06:06:21.894242Z","shell.execute_reply":"2022-07-06T06:06:21.906331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Categorical fields: These fields have a certain fixed number of valid inputs. May be nominal, ordinal, ratio-based, or interval-based.\n* Survived\n* Pclass\n* Sex\n* Embarked\n\nContinuous fields: These fields have any number of valid inputs within theoretical minimum and maximum values. \n* Age\n* Fare\n* SibSp\n* Parch\n\nUnderstanding the data type of the various fields aids in selecting appropriate preprocessing and visualization techniques. ","metadata":{}},{"cell_type":"markdown","source":"**Which features contain empty/null values?**\n\nModel creation and employment fail when records with empty fields are passed. Thus, it becomes imperative to identify and address such records as early as possible. ","metadata":{}},{"cell_type":"code","source":"train.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.909154Z","iopub.execute_input":"2022-07-06T06:06:21.909668Z","iopub.status.idle":"2022-07-06T06:06:21.922326Z","shell.execute_reply.started":"2022-07-06T06:06:21.909638Z","shell.execute_reply":"2022-07-06T06:06:21.921567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.925034Z","iopub.execute_input":"2022-07-06T06:06:21.925768Z","iopub.status.idle":"2022-07-06T06:06:21.934197Z","shell.execute_reply.started":"2022-07-06T06:06:21.925711Z","shell.execute_reply":"2022-07-06T06:06:21.933470Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It can be seen that we indeed have a considerable number of null values. In the train dataset, the \"Cabin\", \"Age\", and \"Embarked\" fields have 687, 177, and 2 null values respectively. In the test dataset, the \"Cabin\", \"Age\", and \"Fare\" fields have 327, 86, and 1 null values respectively.","metadata":{}},{"cell_type":"markdown","source":"**Number of unique values in each field**","metadata":{}},{"cell_type":"code","source":"train.nunique()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.935111Z","iopub.execute_input":"2022-07-06T06:06:21.935656Z","iopub.status.idle":"2022-07-06T06:06:21.949716Z","shell.execute_reply.started":"2022-07-06T06:06:21.935622Z","shell.execute_reply":"2022-07-06T06:06:21.948800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.nunique()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.951780Z","iopub.execute_input":"2022-07-06T06:06:21.952409Z","iopub.status.idle":"2022-07-06T06:06:21.960900Z","shell.execute_reply.started":"2022-07-06T06:06:21.952374Z","shell.execute_reply":"2022-07-06T06:06:21.959988Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**What is the data type for each field?**","metadata":{}},{"cell_type":"code","source":"train.info()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.961825Z","iopub.execute_input":"2022-07-06T06:06:21.962409Z","iopub.status.idle":"2022-07-06T06:06:21.987120Z","shell.execute_reply.started":"2022-07-06T06:06:21.962380Z","shell.execute_reply":"2022-07-06T06:06:21.985924Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.info()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:21.988785Z","iopub.execute_input":"2022-07-06T06:06:21.989581Z","iopub.status.idle":"2022-07-06T06:06:22.004436Z","shell.execute_reply.started":"2022-07-06T06:06:21.989544Z","shell.execute_reply":"2022-07-06T06:06:22.003773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Looking at the dtype row gives an overview of the data types in the dataset.\n* The train dataset has 6 floating or integer fields and 5 object (string or another) fields\n* The test dataset has 5 floating or integer fields and 5 object (string or another) fields","metadata":{}},{"cell_type":"markdown","source":"**What is the statistical distribution of data in each feature?**\n\nBy default, the following function only considers numerical features. To generate descriptive statistics for categorical features as well, include the optional argument 'include = [O]' in the function call.","metadata":{}},{"cell_type":"code","source":"train.describe()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.005447Z","iopub.execute_input":"2022-07-06T06:06:22.006081Z","iopub.status.idle":"2022-07-06T06:06:22.039024Z","shell.execute_reply.started":"2022-07-06T06:06:22.006050Z","shell.execute_reply":"2022-07-06T06:06:22.038243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.describe(include = [\"O\"])","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.040290Z","iopub.execute_input":"2022-07-06T06:06:22.040594Z","iopub.status.idle":"2022-07-06T06:06:22.062105Z","shell.execute_reply.started":"2022-07-06T06:06:22.040567Z","shell.execute_reply":"2022-07-06T06:06:22.061255Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For numerical features: \n* Sample size is 40% of the population (891 records out of 2,224)\n* Around 38% samples survived (mean of binary class \"Survived\")\n* Most passengers (> 75%) did not travel with parents or children.\n* Nearly 30% of the passengers had siblings and/or spouse aboard.\n\nFor categorical features:\n* No duplicate values in \"Name\" (count = unique)\n* \"Sex\" is a binary field with 65% being male (freq. of top value (male) is 577 out of 891)\n* Embarked takes three possible values and port S port was most frequently used","metadata":{}},{"cell_type":"markdown","source":"**Modifying the Cabin coloumn**\n\nThe Cabin coloumn has an unjustifiably large number of unique values. This is based on the presumption that the number allocation in the coloumn following the cabin letter has a negligible impact on a passengers probability of survival. Thus, this coloumn will be modified so as to mitigate the numbers. For records that do not have an entry in this coloumn, \"na\" will be assigned. ","metadata":{}},{"cell_type":"code","source":"train[\"Cabin\"].unique()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.063321Z","iopub.execute_input":"2022-07-06T06:06:22.063618Z","iopub.status.idle":"2022-07-06T06:06:22.070089Z","shell.execute_reply.started":"2022-07-06T06:06:22.063591Z","shell.execute_reply":"2022-07-06T06:06:22.069155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train[\"Cabin\"] = train[\"Cabin\"].apply(lambda x : x[0] if pd.notna(x) else \"na\")\ntrain[\"Cabin\"].unique()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.071528Z","iopub.execute_input":"2022-07-06T06:06:22.072493Z","iopub.status.idle":"2022-07-06T06:06:22.084841Z","shell.execute_reply.started":"2022-07-06T06:06:22.072447Z","shell.execute_reply":"2022-07-06T06:06:22.083837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test[\"Cabin\"] = test[\"Cabin\"].apply(lambda x : x[0] if pd.notna(x) else \"na\")\ntest[\"Cabin\"].unique()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.085936Z","iopub.execute_input":"2022-07-06T06:06:22.086226Z","iopub.status.idle":"2022-07-06T06:06:22.095141Z","shell.execute_reply.started":"2022-07-06T06:06:22.086200Z","shell.execute_reply":"2022-07-06T06:06:22.094374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Handling missing values**\n\nIt is not possible to create a model using a dataset with missing values. Thus, it is important to address such records. The most common approaches to handling missing values are:\n* Dropping records/coloumns in their entirety\n* Filling in missing values using appropriate method\n\nIt can be seen above that Age, Cabin, Fare, and Embarked have missing values. However, Cabin was already addressed in the step above. Thus, only the Age, Fare, and Embarked coloumns will be filled.\n\nAdditionally, here is when unnecessary fields/attributes may be dropped. ","metadata":{}},{"cell_type":"code","source":"train.reset_index(inplace = True)\ntest.reset_index(inplace = True)\n\ntrain.drop([\"PassengerId\", \"Name\", \"Ticket\"], axis = 1, inplace = True)\ntest_passenger_ids = test[\"PassengerId\"] # Saved seperately for submission file\ntest.drop([\"PassengerId\", \"Name\", \"Ticket\"], axis = 1, inplace = True)\n\ntrain[\"Age\"].fillna(train[\"Age\"].mean(skipna = True), inplace = True)\ntest[\"Age\"].fillna(test[\"Age\"].mean(skipna = True), inplace = True)\n\ntrain[\"Embarked\"].fillna(\"S\", inplace = True)\ntest[\"Embarked\"].fillna(\"S\", inplace = True)\n\ntest[\"Fare\"].fillna(test[\"Fare\"].mean(skipna = True), inplace = True)","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.096269Z","iopub.execute_input":"2022-07-06T06:06:22.097133Z","iopub.status.idle":"2022-07-06T06:06:22.114364Z","shell.execute_reply.started":"2022-07-06T06:06:22.097101Z","shell.execute_reply":"2022-07-06T06:06:22.113455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.115566Z","iopub.execute_input":"2022-07-06T06:06:22.116275Z","iopub.status.idle":"2022-07-06T06:06:22.127522Z","shell.execute_reply.started":"2022-07-06T06:06:22.116245Z","shell.execute_reply":"2022-07-06T06:06:22.126500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.128841Z","iopub.execute_input":"2022-07-06T06:06:22.129633Z","iopub.status.idle":"2022-07-06T06:06:22.138021Z","shell.execute_reply.started":"2022-07-06T06:06:22.129601Z","shell.execute_reply":"2022-07-06T06:06:22.137069Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Label encoding**\n\nSince the system can not comprehend string data, all such entries must be converted into numbers using logical techniques.","metadata":{}},{"cell_type":"code","source":"sex = {'male': 0, 'female': 1}\ntrain[\"Sex\"] = [sex[i] for i in train[\"Sex\"]] \ntest[\"Sex\"] = [sex[i] for i in test[\"Sex\"]] \n\nembarked = {'S': 0, 'C': 1, 'Q':2}\ntrain[\"Embarked\"] = [embarked[i] for i in train[\"Embarked\"]] \ntest[\"Embarked\"] = [embarked[i] for i in test[\"Embarked\"]] \n# train[\"Embarked\"] = train[\"Embarked\"].map(embarked).astype(int)\n# test[\"Embarked\"] = test[\"Embarked\"].map(embarked).astype(int)\n\ncabin_plot = train[[\"Cabin\", \"Survived\"]]\ntrain[\"Cabin\"] = LabelEncoder().fit_transform(train[\"Cabin\"])\ntest[\"Cabin\"] = LabelEncoder().fit_transform(test[\"Cabin\"])","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.139472Z","iopub.execute_input":"2022-07-06T06:06:22.139996Z","iopub.status.idle":"2022-07-06T06:06:22.157685Z","shell.execute_reply.started":"2022-07-06T06:06:22.139929Z","shell.execute_reply":"2022-07-06T06:06:22.156372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

Exploratory data analysis (EDA)

\n
","metadata":{}},{"cell_type":"markdown","source":"A picture is worth a thousand words. We will now understand the data using visualization techniques, with the help of Seaborn. ","metadata":{}},{"cell_type":"markdown","source":"
\n
","metadata":{}},{"cell_type":"markdown","source":"**Distribution of age**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (20,10))\nsns.histplot(x = \"Age\", data = train)\nplt.title(\"Histogram (Age)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.159257Z","iopub.execute_input":"2022-07-06T06:06:22.159851Z","iopub.status.idle":"2022-07-06T06:06:22.489189Z","shell.execute_reply.started":"2022-07-06T06:06:22.159817Z","shell.execute_reply":"2022-07-06T06:06:22.488273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20,10))\nsns.kdeplot(x = \"Age\", data = train, fill = True)\nplt.title(\"KDE (Age)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.490866Z","iopub.execute_input":"2022-07-06T06:06:22.491324Z","iopub.status.idle":"2022-07-06T06:06:22.752676Z","shell.execute_reply.started":"2022-07-06T06:06:22.491271Z","shell.execute_reply":"2022-07-06T06:06:22.751460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20,2))\nsns.boxplot(x = \"Age\", data = train)\nplt.title(\"Boxplot (Age)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.754317Z","iopub.execute_input":"2022-07-06T06:06:22.754794Z","iopub.status.idle":"2022-07-06T06:06:22.928482Z","shell.execute_reply.started":"2022-07-06T06:06:22.754749Z","shell.execute_reply":"2022-07-06T06:06:22.927453Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20,2))\nsns.violinplot(x = \"Age\", data = train)\nplt.title(\"Violin plot (Age)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:22.930024Z","iopub.execute_input":"2022-07-06T06:06:22.930459Z","iopub.status.idle":"2022-07-06T06:06:23.101692Z","shell.execute_reply.started":"2022-07-06T06:06:22.930426Z","shell.execute_reply":"2022-07-06T06:06:23.100904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
","metadata":{}},{"cell_type":"markdown","source":"**Distribution of fare**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (20,10))\nsns.histplot(x = \"Fare\", data = train)\nplt.title(\"Histogram (Fare)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:23.106742Z","iopub.execute_input":"2022-07-06T06:06:23.107461Z","iopub.status.idle":"2022-07-06T06:06:23.519689Z","shell.execute_reply.started":"2022-07-06T06:06:23.107427Z","shell.execute_reply":"2022-07-06T06:06:23.518512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20,10))\nsns.kdeplot(x = \"Fare\", data = train, fill = True)\nplt.title(\"KDE (Fare)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:23.521487Z","iopub.execute_input":"2022-07-06T06:06:23.522335Z","iopub.status.idle":"2022-07-06T06:06:23.779061Z","shell.execute_reply.started":"2022-07-06T06:06:23.522284Z","shell.execute_reply":"2022-07-06T06:06:23.778285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20,2))\nsns.boxplot(x = \"Fare\", data = train)\nplt.title(\"Boxplot (Fare)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:23.780071Z","iopub.execute_input":"2022-07-06T06:06:23.780922Z","iopub.status.idle":"2022-07-06T06:06:23.947581Z","shell.execute_reply.started":"2022-07-06T06:06:23.780889Z","shell.execute_reply":"2022-07-06T06:06:23.946472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20,2))\nsns.violinplot(x = \"Fare\", data = train)\nplt.title(\"Violin plot (Fare)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:23.950689Z","iopub.execute_input":"2022-07-06T06:06:23.951068Z","iopub.status.idle":"2022-07-06T06:06:24.124993Z","shell.execute_reply.started":"2022-07-06T06:06:23.951036Z","shell.execute_reply":"2022-07-06T06:06:24.123794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
","metadata":{}},{"cell_type":"markdown","source":"**Distribution of gender**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (10, 5))\nsns.countplot(x = \"Sex\", data = train)\nplt.title(\"Countplot (Sex)\")\nplt.xticks([0, 1], [\"Male\", \"Female\"])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:24.126511Z","iopub.execute_input":"2022-07-06T06:06:24.126987Z","iopub.status.idle":"2022-07-06T06:06:24.302857Z","shell.execute_reply.started":"2022-07-06T06:06:24.126942Z","shell.execute_reply":"2022-07-06T06:06:24.301740Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
","metadata":{}},{"cell_type":"markdown","source":"**Relation of various features with Survived**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (20, 10))\nsns.countplot(x = \"Pclass\", hue = \"Survived\", data = train)\nplt.title(\"Survived VS Pclass\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:24.304511Z","iopub.execute_input":"2022-07-06T06:06:24.305340Z","iopub.status.idle":"2022-07-06T06:06:24.546482Z","shell.execute_reply.started":"2022-07-06T06:06:24.305275Z","shell.execute_reply":"2022-07-06T06:06:24.545788Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (10, 5))\ntrain.loc[train[\"Sex\"] == 0, \"Survived\"].value_counts().plot.pie(autopct=\"%1.1f%%\")\nplt.title(\"Percentage survived (Male)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:24.547575Z","iopub.execute_input":"2022-07-06T06:06:24.548017Z","iopub.status.idle":"2022-07-06T06:06:24.679254Z","shell.execute_reply.started":"2022-07-06T06:06:24.547985Z","shell.execute_reply":"2022-07-06T06:06:24.678176Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (10, 5))\ntrain.loc[train[\"Sex\"] == 1, \"Survived\"].value_counts().plot.pie(autopct=\"%1.1f%%\")\nplt.title(\"Percentage survived (Female)\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:24.680958Z","iopub.execute_input":"2022-07-06T06:06:24.681642Z","iopub.status.idle":"2022-07-06T06:06:24.815017Z","shell.execute_reply.started":"2022-07-06T06:06:24.681598Z","shell.execute_reply":"2022-07-06T06:06:24.813984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20, 10))\nsns.countplot(x = \"SibSp\", hue = \"Survived\", data = train)\nplt.title(\"Survived VS SibSp\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:24.816666Z","iopub.execute_input":"2022-07-06T06:06:24.817433Z","iopub.status.idle":"2022-07-06T06:06:25.312831Z","shell.execute_reply.started":"2022-07-06T06:06:24.817388Z","shell.execute_reply":"2022-07-06T06:06:25.311818Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20, 10))\nsns.countplot(x = \"Cabin\", hue = \"Survived\", data = cabin_plot)\nplt.title(\"Survived VS Cabin\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:25.314827Z","iopub.execute_input":"2022-07-06T06:06:25.315369Z","iopub.status.idle":"2022-07-06T06:06:25.679172Z","shell.execute_reply.started":"2022-07-06T06:06:25.315321Z","shell.execute_reply":"2022-07-06T06:06:25.678447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20, 10))\nsns.kdeplot(x = \"Age\", hue = \"Survived\", data = train, shade = True)\nplt.title(\"Survived VS Age\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:25.680428Z","iopub.execute_input":"2022-07-06T06:06:25.681036Z","iopub.status.idle":"2022-07-06T06:06:25.977600Z","shell.execute_reply.started":"2022-07-06T06:06:25.680992Z","shell.execute_reply":"2022-07-06T06:06:25.976589Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (20, 10))\nsns.kdeplot(x = \"Fare\", hue = \"Survived\", data = train, shade = True)\nplt.title(\"Survived VS Fare\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:25.978975Z","iopub.execute_input":"2022-07-06T06:06:25.979695Z","iopub.status.idle":"2022-07-06T06:06:26.302642Z","shell.execute_reply.started":"2022-07-06T06:06:25.979650Z","shell.execute_reply":"2022-07-06T06:06:26.301643Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
","metadata":{}},{"cell_type":"markdown","source":"**Relation of various features with Age and Survived**","metadata":{}},{"cell_type":"code","source":"fig, axes = plt.subplots(3, 2, figsize = (20, 20))\n\nsns.violinplot(x = \"Pclass\", y = \"Age\", hue = \"Survived\", split = True, data = train, ax = axes[0, 0])\nsns.violinplot(x = \"Sex\", y = \"Age\", hue = \"Survived\", split = True, data = train, ax = axes[0, 1])\nsns.violinplot(x = \"SibSp\", y = \"Age\", hue = \"Survived\", split = True, data = train, ax = axes[1, 0])\nsns.violinplot(x = \"Parch\", y = \"Age\", hue = \"Survived\", split = True, data = train, ax = axes[1, 1])\nsns.violinplot(x = \"Embarked\", y = \"Age\", hue = \"Survived\", split = True, data = train, ax = axes[2, 0])\nsns.violinplot(x = \"Cabin\", y = \"Age\", hue = \"Survived\", split = True, data = train, ax = axes[2, 1])\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:26.304072Z","iopub.execute_input":"2022-07-06T06:06:26.304977Z","iopub.status.idle":"2022-07-06T06:06:27.825459Z","shell.execute_reply.started":"2022-07-06T06:06:26.304929Z","shell.execute_reply":"2022-07-06T06:06:27.824291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
","metadata":{}},{"cell_type":"markdown","source":"
\n

Model creation and evaluation

\n
","metadata":{}},{"cell_type":"markdown","source":"There is a wide array of predictive modeling algorithms at our disposal. The algorithm(s) chosen is contingent on the problem itself and solution requirement. The problem posed here is a classification problem, in that the model must use the features provided to classify an unknown passenger as a survivior (1) or victim (0). Additionally, the category of machine learning seen here is supervised learning, since we have a labelled training dataset from which the model is created. Taking these points into consideration, we may decide on few algorithms to implement. The ones implemented here are:\n\n1. Decision tree\n2. Random forest\n3. Naive Bayes\n4. K-nearest neighbours\n5. Support vector machine","metadata":{}},{"cell_type":"code","source":"y = train[\"Survived\"]\nX = train.drop(\"Survived\", axis = 1)\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 1)\nX_train.shape, X_test.shape, y_train.shape, y_test.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:27.826860Z","iopub.execute_input":"2022-07-06T06:06:27.827747Z","iopub.status.idle":"2022-07-06T06:06:27.841507Z","shell.execute_reply.started":"2022-07-06T06:06:27.827692Z","shell.execute_reply":"2022-07-06T06:06:27.840473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Decision tree classifier**\n\nIt is a tree-structured classifier, where internal nodes represent the features of a dataset, branches represent the decision rules and each leaf node represents the outcome. In a Decision tree, there are two nodes, which are the Decision Node and Leaf Node. There are generally two types of decision trees. Models where the target variable can take a finite set of values are called classification trees. Trees where the target variable can take continuous values (typically real numbers) are called regression trees.\n\n![](https://cdn-images-1.medium.com/max/824/0*J2l5dvJ2jqRwGDfG.png)","metadata":{}},{"cell_type":"code","source":"decision_tree_model = DecisionTreeClassifier(random_state = 2)\ndecision_tree_model.fit(X_train, y_train)\ny_pred = decision_tree_model.predict(X_test)\ndecision_tree_model_acc = accuracy_score(y_pred, y_test) * 100\nprint(\"Accuracy:\", decision_tree_model_acc)","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:27.843073Z","iopub.execute_input":"2022-07-06T06:06:27.843659Z","iopub.status.idle":"2022-07-06T06:06:27.859256Z","shell.execute_reply.started":"2022-07-06T06:06:27.843625Z","shell.execute_reply":"2022-07-06T06:06:27.858288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Random forest classifier**\n\nThe random forest classifier is an improvement over decision tree classifiers. Based on ensemble learning, a random forest classifier contains a number of decision trees on various subsets of the given dataset and takes the average to improve the predictive accuracy of that dataset. In general, a greater number of trees in the forest leads to higher accuracy and prevents the problem of overfitting.\n\n![](https://static.javatpoint.com/tutorial/machine-learning/images/random-forest-algorithm.png)","metadata":{}},{"cell_type":"code","source":"random_forest_model = RandomForestClassifier(criterion='gini',\n n_estimators=1750,\n max_depth=7,\n min_samples_split=6,\n min_samples_leaf=6,\n max_features='auto',\n verbose=1,\n random_state = 3)\nrandom_forest_model.fit(X_train, y_train)\ny_pred = random_forest_model.predict(X_test)\nrandom_forest_model_acc = accuracy_score(y_pred, y_test) * 100\nprint(\"Accuracy:\", random_forest_model_acc)","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:27.860639Z","iopub.execute_input":"2022-07-06T06:06:27.861256Z","iopub.status.idle":"2022-07-06T06:06:31.525798Z","shell.execute_reply.started":"2022-07-06T06:06:27.861206Z","shell.execute_reply":"2022-07-06T06:06:31.524884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Naive Bayes classifier**\n\nThe Naïve Bayes classifier is a probabilistic classifier that uses Bayes theorem. It assumes complete independance between the features. \n![](https://s3.ap-south-1.amazonaws.com/techleer/204.png)\n","metadata":{}},{"cell_type":"code","source":"NB_model = GaussianNB()\nNB_model.fit(X_train, y_train)\ny_pred = NB_model.predict(X_test)\nNB_model_acc = accuracy_score(y_pred, y_test) * 100\nprint(\"Accuracy:\", NB_model_acc)","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:31.526904Z","iopub.execute_input":"2022-07-06T06:06:31.527202Z","iopub.status.idle":"2022-07-06T06:06:31.538033Z","shell.execute_reply.started":"2022-07-06T06:06:31.527175Z","shell.execute_reply":"2022-07-06T06:06:31.537221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**K-nearest neighbours classifier**\n\nThe K-nearest neighbours (KNN) classifier uses proximity to make classifications or predictions about independent data points. This technique may be used for both classification and regression scenarios and the output will vary. In classification instances, a decision is made based on majority vote, i.e., the class assigned to the new data point is taken to be the one that is most frequently seen in the vicinity of the point. KNN is also known as a lazy learner technique since a model is not learned. Instead, the raw data is stored and used everytime a prediction must be made.\n![](http://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1531424125/KNN_final1_ibdm8a.png)","metadata":{}},{"cell_type":"code","source":"knn_model = KNeighborsClassifier(n_neighbors = 5)\nknn_model.fit(X_train, y_train)\ny_pred = knn_model.predict(X_test)\nknn_model_acc = accuracy_score(y_pred, y_test) * 100\nprint(\"Accuracy:\", knn_model_acc)","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:31.539788Z","iopub.execute_input":"2022-07-06T06:06:31.540557Z","iopub.status.idle":"2022-07-06T06:06:31.559790Z","shell.execute_reply.started":"2022-07-06T06:06:31.540513Z","shell.execute_reply":"2022-07-06T06:06:31.558842Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Support vector machines**\n\nSupport vector machines (SVM) are a class of machine learning algorithms that map data to a high dimensionality feature space in such a manner that the data points can be categorized, even when they are not otherwise linearly seperable. A seperator between the categories is found and a hyperplane is drawn accordingly. Support vectors are those data points that are close to the hyperplane and influence the position and orientation of the hyperplane. The function used to map data to a high dimensionality feature space is called a kernel function.\n![](http://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1526288453/index3_souoaz.png)","metadata":{}},{"cell_type":"code","source":"svm_model = SVC()\nsvm_model.fit(X_train, y_train)\ny_pred = svm_model.predict(X_test)\nsvm_model_acc = accuracy_score(y_pred, y_test) * 100\nprint(\"Accuracy:\", svm_model_acc)","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:31.561142Z","iopub.execute_input":"2022-07-06T06:06:31.561581Z","iopub.status.idle":"2022-07-06T06:06:31.599193Z","shell.execute_reply.started":"2022-07-06T06:06:31.561536Z","shell.execute_reply":"2022-07-06T06:06:31.598422Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n

Submission file

\n
","metadata":{}},{"cell_type":"code","source":"all_models = pd.DataFrame({\n \"Model\" : [\"Decision tree classifier\", \"Random forest classifier\", \"Naive Bayes classifier\", \"KNN\", \"SVM\"],\n \"Accuracy score\" : [decision_tree_model_acc, random_forest_model_acc, NB_model_acc, knn_model_acc, svm_model_acc]\n})\nall_models.sort_values(by = \"Accuracy score\")","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:31.600287Z","iopub.execute_input":"2022-07-06T06:06:31.600733Z","iopub.status.idle":"2022-07-06T06:06:31.612401Z","shell.execute_reply.started":"2022-07-06T06:06:31.600691Z","shell.execute_reply":"2022-07-06T06:06:31.611701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"After comparing the models above, it appears that the Random Forest classifier had the best performance with an accuracy score of approximately 79.33. Ergo, this is the model that will be used to create the final submission file. ","metadata":{}},{"cell_type":"code","source":"random_forest_model = RandomForestClassifier(criterion='gini',\n n_estimators=1750,\n max_depth=7,\n min_samples_split=6,\n min_samples_leaf=6,\n max_features='auto',\n verbose=1,\n random_state = 3)\nrandom_forest_model.fit(X, y)\ny_pred = random_forest_model.predict(test)","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:31.613359Z","iopub.execute_input":"2022-07-06T06:06:31.614039Z","iopub.status.idle":"2022-07-06T06:06:35.345952Z","shell.execute_reply.started":"2022-07-06T06:06:31.613995Z","shell.execute_reply":"2022-07-06T06:06:35.345066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission = pd.DataFrame({\n \"PassengerId\" : test_passenger_ids,\n \"Survived\" : y_pred\n})\nsubmission.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:35.347040Z","iopub.execute_input":"2022-07-06T06:06:35.347357Z","iopub.status.idle":"2022-07-06T06:06:35.357712Z","shell.execute_reply.started":"2022-07-06T06:06:35.347329Z","shell.execute_reply":"2022-07-06T06:06:35.356798Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv('submission.csv', index = False)","metadata":{"execution":{"iopub.status.busy":"2022-07-06T06:06:35.358922Z","iopub.execute_input":"2022-07-06T06:06:35.359250Z","iopub.status.idle":"2022-07-06T06:06:35.372035Z","shell.execute_reply.started":"2022-07-06T06:06:35.359221Z","shell.execute_reply":"2022-07-06T06:06:35.370893Z"},"trusted":true},"execution_count":null,"outputs":[]}]}