{"cells":[{"metadata":{"_uuid":"be23f8ec4860be855f99d4173ed8942339c5cf49","colab_type":"text","id":"0XTPNdQQdY46"},"cell_type":"markdown","source":"# 1. Objective\nThis book is to analyze the Titanic dataset on Kaggle(https://www.kaggle.com/c/titanic), and predict survival of test dataset. Based on the analysis, obervations on people who are more likely to survive are proposed."},{"metadata":{"_uuid":"dd04b2eabaff7c1b45f8e1a5d74c5ca6c299a327","colab_type":"text","id":"mQcAkkXxeViy"},"cell_type":"markdown","source":"# 2. Problem Description\n\n>\"The sinking of the RMS Titanic is one of the most infamous shipwrecks in history.  On April 15, 1912, during her maiden voyage, the Titanic sank after colliding with an iceberg, killing 1502 out of 2224 passengers and crew. This sensational tragedy shocked the international community and led to better safety regulations for ships.\n\n>One of the reasons that the shipwreck led to such loss of life was that there were not enough lifeboats for the passengers and crew. Although there was some element of luck involved in surviving the sinking, some groups of people were more likely to survive than others, such as women, children, and the upper-class.\"\n"},{"metadata":{"_uuid":"198f7686b782625c222319fed09081ae940d6a5b","colab_type":"text","id":"5pPhvzH8iexw"},"cell_type":"markdown","source":"# 3. Exploratory Data Analysis"},{"metadata":{"_uuid":"7c3e244acfab63f865eafcce57f4e1e2a198ca34","colab_type":"text","id":"RV9liyiJiooj"},"cell_type":"markdown","source":"## 3.1 Import Libraries"},{"metadata":{"_uuid":"ec2b8fc0fd7577fed12498908cd809c9a55f54f6","colab":{},"colab_type":"code","id":"M-d2WgV4iaKk","trusted":false},"cell_type":"code","source":"# data analysis and wrangling\nimport pandas as pd\nimport numpy as np\nimport random as rnd\nimport re\n\n# visualization\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\n# machine learning\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.svm import SVC, LinearSVC\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.naive_bayes import GaussianNB\nfrom sklearn.linear_model import Perceptron\nfrom sklearn.linear_model import SGDClassifier\nfrom sklearn.tree import DecisionTreeClassifier","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7ef7936b72d58af0c343e187384c3cb43db746d1","colab_type":"text","id":"yzlKNEj8ACov"},"cell_type":"markdown","source":"## 3.2 Acquire Data"},{"metadata":{"_uuid":"e4a666767109a834916ff2d77effd61abb3e301c","colab":{},"colab_type":"code","id":"84eu06cNLZ6W","trusted":false},"cell_type":"code","source":"# Genenrate training set and testing set\ntrain = pd.read_csv('../input/train.csv')\ntest = pd.read_csv('../input/test.csv')\n#train = pd.read_csv('/Users/machen/Downloads/titanic data/train.csv')\n#test = pd.read_csv('/Users/machen/Downloads/titanic data/test.csv')\nfull = train.append( test , ignore_index = True )","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"72c1ea966bc8e5a426a904d1561eb6af825534a4","colab_type":"text","id":"sTK-p218IQCL"},"cell_type":"markdown","source":"View data."},{"metadata":{"_uuid":"bbe43584fc4d8d133d6fbab148cdd930942dd743","colab":{"base_uri":"https://localhost:8080/","height":204},"colab_type":"code","id":"LvHzXIBqIMfT","outputId":"9c038d8c-ad71-44e9-c1b1-f72491cd1373","trusted":false},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"04a16f92a3439cf01b249b006d488549dc93f937","colab_type":"text","id":"h3F76RUZiQy8"},"cell_type":"markdown","source":"View distribution of numerical data."},{"metadata":{"_uuid":"88f947711f768eb7cfefce219548f2c47da008a2","colab":{"base_uri":"https://localhost:8080/","height":297},"colab_type":"code","id":"M6QuK55oI0W0","outputId":"c5eb63a5-806e-455d-9a78-5eacc1062000","trusted":false},"cell_type":"code","source":"full.describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6dcf93f0b9003676be97f50914362932c06d4220","colab_type":"text","id":"ZA5jK4emL6ye"},"cell_type":"markdown","source":"## 3.3 Pivoting Categorical Data"},{"metadata":{"_uuid":"76d22d9bce16e608415abf71042fde81620ca5f6","colab_type":"text","id":"8l4n_BkVMcpK"},"cell_type":"markdown","source":"### Obervations\n1. People at higher class have higher survival rate, hence Pclass should be included in the model;\n2. Female suvived at higher chance than male;\n3. The correlation between SibSp/Parch and survival is random for certain values, these two variables can be combined into a new feature;\n4. Passengers got on board on port C have higher survival, but this was due to the constitution of the passengers: higher percentage of class 1, infants and elderly people; So we can conclude that Embarked variable is not related to survival, and drop it.\n5. Drop Name, Ticket and Cabin;\n"},{"metadata":{"_uuid":"173a980418652428b81a98296e14b068bbbacf7e","colab":{"base_uri":"https://localhost:8080/","height":142},"colab_type":"code","id":"SM66vztSMRv1","outputId":"a7fd3f34-b59f-4180-b85c-1fcd02c5a5d5","trusted":false},"cell_type":"code","source":"# Pivot pclass and survival rate\ntrain[['Pclass', 'Survived']].groupby(['Pclass'], as_index=False).mean().sort_values(by='Survived', ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ed4062c1323dba345ced1fbbd279e10ed8081e20","colab":{"base_uri":"https://localhost:8080/","height":111},"colab_type":"code","id":"wq00U4vcNC7v","outputId":"41ed7a8e-6ea4-4b4f-f60c-0de6813bdd87","trusted":false},"cell_type":"code","source":"# Pivot sex and survival rate\ntrain[['Sex', 'Survived']].groupby(['Sex'], as_index=False).mean().sort_values(by='Survived', ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b28c58f76a2ca71c78d6f4571a01ca3a3d373f43","colab":{"base_uri":"https://localhost:8080/","height":266},"colab_type":"code","id":"sSw8HyCOPpjm","outputId":"5ced2fe2-f47a-4628-e2f4-aaefd7136662","trusted":false},"cell_type":"code","source":"# Pivot sibling number and survival rate\nsibsp_sur = train[[\"SibSp\", \"Survived\"]].groupby(['SibSp'], as_index=False).mean().sort_values(by='SibSp', ascending=False)\nsibsp_sur.plot(x='SibSp', y='Survived')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8520ec5c7d08c82a051c9147fa01249d05faf826","colab":{"base_uri":"https://localhost:8080/","height":266},"colab_type":"code","id":"oqRupDNPQDUZ","outputId":"adb1eab2-58b4-4b9c-80a8-33e75317a1c8","trusted":false},"cell_type":"code","source":"# Pivot parch number and survival rate\nparch_sur = train[[\"Parch\", \"Survived\"]].groupby(['Parch'], as_index=False).mean().sort_values(by='Parch', ascending=False)\nparch_sur.plot(x='Parch', y = 'Survived')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9a56d37e7cd5b72fa464f4ad6e575a8553fa61d0","colab_type":"text","id":"-_gcGXBzifR2"},"cell_type":"markdown","source":"### Embarking\nPeople **embarking at port C** have higher survival rate, which raises our attention why this is happening.\n\nSo we slice the data with factors we find related to survival: Sex, Pclass, Age, and find that the reason that port C has higher survival rate is that higher percentage of passengers are at class 1."},{"metadata":{"_uuid":"38a63d31f381910746716e2e5d1ba74bc26bb0b0","colab":{"base_uri":"https://localhost:8080/","height":142},"colab_type":"code","id":"20tFDvgWVuYD","outputId":"2d773ef9-4624-4cda-8e92-d26f2e8486ff","trusted":false},"cell_type":"code","source":"## Pivot embark port and survival rate\ntrain[[\"Embarked\", \"Survived\"]].groupby(['Embarked'], as_index=False).mean().sort_values(by='Embarked', ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a62a3ab362339b292ea630e8bc485a587bcd1cee","colab":{"base_uri":"https://localhost:8080/","height":235},"colab_type":"code","id":"_BxcSUaCXXVz","outputId":"08b03e67-a16f-4006-ebc3-4945935e1dfe","trusted":false},"cell_type":"code","source":"# slicing data by embark and sex, people boarding from port C have higher survival, regardless of the sex. \ntrain[[\"Embarked\", \"Survived\",'Sex']].groupby(['Embarked','Sex'], as_index=False).mean().sort_values(by='Embarked', ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2485f2b2cb14585f58d35eff60041c9735fe40a0","colab":{"base_uri":"https://localhost:8080/","height":328},"colab_type":"code","id":"Z94p3d-bh-Op","outputId":"33b75a35-7ab5-4c98-e08a-acf3cceb1c6d","trusted":false},"cell_type":"code","source":"# slicing data by embark and class. Higher percentage of passengers are at class 1.\ntrain[[\"Embarked\", \"Survived\",'Pclass']].groupby(['Embarked','Pclass'], as_index=False).count().sort_values(by='Embarked', ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7326635768d9c000d3a32e8d035033ba3649be04","colab":{"base_uri":"https://localhost:8080/","height":328},"colab_type":"code","id":"JQABLgFWjqDZ","outputId":"e6b8a888-ff9f-44f0-c870-73abb14c9ecc","trusted":false},"cell_type":"code","source":"# slicing data by embark and sex, people boarding from port C have higher survival, regardless of the sex. \ntrain[[\"Embarked\", \"Survived\",'Pclass']].groupby(['Embarked','Pclass'], as_index=False).mean().sort_values(by='Embarked', ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"13261ac5a7563a3b2d9a8bab918b52504dca166b","colab":{"base_uri":"https://localhost:8080/","height":297},"colab_type":"code","id":"5itaBVCLdiF7","outputId":"1fb37ba1-f941-4a21-8d28-d43cd833b6ae","trusted":false},"cell_type":"code","source":"## plot histgram of age for each segment of embark data\n## passengers embarking at port C have higher percentage of infants and elderly people, resulting in higher survival rate.\nfig, (axis1,axis2, axis3) = plt.subplots(1,3,figsize=(15,4))\naxis1.set_title('Age hist - embark S')\naxis2.set_title('Age hist - embark Q')\naxis3.set_title('Age hist - embark C')\n\ntrain[train['Embarked']=='S']['Age'].hist(bins=70, ax=axis1)\ntrain[train['Embarked']=='Q']['Age'].hist(bins=70, ax=axis2)\ntrain[train['Embarked']=='C']['Age'].hist(bins=70, ax=axis3)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"edeffa2c78cd2fd1760d094663ab50e910fe647c"},"cell_type":"markdown","source":"### Name\nWe use regular expression to extract title from names."},{"metadata":{"trusted":false,"_uuid":"717e754b80681ebb982ac0365b2d812e316e5ce7"},"cell_type":"code","source":"def get_title(name):\n    title_search = re.search(' ([A-Za-z]+)\\. ', name)\n    if title_search:\n        return title_search.group(1)\n    return \"\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"8b3f6da6e1df3f918fefcf3256d481b68da0393f"},"cell_type":"code","source":"full['title'] = full['Name'].apply(get_title)\nfull['title'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"7e37a1e17298827a6b141006a27b3b51fb917cd6"},"cell_type":"code","source":"full['title'] = full['title'].replace(['Lady', 'Countess','Capt', 'Col','Don', 'Dr', 'Major', 'Rev', 'Sir', 'Jonkheer', 'Dona'],'Rare')\nfull['title'] = full['title'].replace('Mlle','Miss')\nfull['title'] = full['title'].replace('Ms','Miss')\nfull['title'] = full['title'].replace('Mme','Mrs')\nfull['title'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1a139ebbfd65261ad1f9c4da81b39bc214b44967","colab_type":"text","id":"zqe1PA-KRESO"},"cell_type":"markdown","source":"## 3.4 Explore the relationship between numerical variables and survival"},{"metadata":{"_uuid":"ecd4063ab26f21525aac9f50c4c88df274db6249","colab_type":"text","id":"Lu-8yGTeRP-3"},"cell_type":"markdown","source":"From analysis above, among all categorical variables, Sex has the biggest influence on survival. Hence, we segment data by Sex when exploring the relationship between numerical variables and survival.\n"},{"metadata":{"_uuid":"26b39f938aeb9cc90929fab90bda796514a783fc","colab_type":"text","id":"9e8EbYtUaWmy"},"cell_type":"markdown","source":"### Obeservations\n1. Age \n\nFrom below segmented survival rate vs. age by Sex plots, we can see female gender does not should strong relationship between age and survival rate. While male group show high survival rate for children and elderly people. So we decide to cut age into three bins: infants/childrean(age<=12), adult(12<age<80), elderly(age>=60)"},{"metadata":{"_uuid":"c6ffdad794c38216418049c3849bcd05e9f355c2","colab":{"base_uri":"https://localhost:8080/","height":330},"colab_type":"code","id":"KIq8iFgZfn-A","outputId":"30072629-836f-4986-92e9-00da40f193dc","scrolled":true,"trusted":false},"cell_type":"code","source":"# average survived passengers by age, sliced by Sex\nfig, (axis1,axis2) = plt.subplots(2,1,figsize=(30,16))\naverage_age = train[['Sex',\"Age\", \"Survived\"]].groupby(['Sex','Age'],as_index=False).mean()\nsns.barplot(x='Age', y='Survived', data=average_age[average_age['Sex']=='female'],ax = axis1)\nsns.barplot(x='Age', y='Survived', data=average_age[average_age['Sex']=='male'],ax = axis2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"fec5ba904878d283a0c2d39e20dca6d038e140da"},"cell_type":"code","source":"full['Age'].hist(bins = 60)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5256c88fde2eaf831256b260e004c9ed38f3c738"},"cell_type":"markdown","source":"2. Fare\n\nThere is clearly increase in survival rate as fare goes up. Hence we decide to cut fare into three bins."},{"metadata":{"trusted":false,"_uuid":"6a5204529b1143a65a50bc8c50ab56aa6643813a"},"cell_type":"code","source":"#full['Fare'].hist(bins = 60)\nfull[full['Fare']<100]['Fare'].hist(bins = 20)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9ccbb375e33ef99704fed1a722611a9e596dd2cd","colab_type":"text","id":"IvO5C-dyRPQq"},"cell_type":"markdown","source":"# 4. Transforming Data"},{"metadata":{"_uuid":"f16b96202edc656ad1adc9208e6caaa5c9ede932","colab_type":"text","id":"Vs-mp3eElTn-"},"cell_type":"markdown","source":"## 4.1 Deal with Missing Values\n\nCheck how many null values each feature has."},{"metadata":{"trusted":false,"_uuid":"2c9b98e05a7e4197e932814d948bff16f2f4717d"},"cell_type":"code","source":"full.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8a375ffd3db66f12893a3f04334bfb4cbea1cbb8","colab":{"base_uri":"https://localhost:8080/","height":204},"colab_type":"code","id":"KE1ObpIGlW5M","outputId":"f019d895-41de-4b6b-ba29-667952814cd3","trusted":false},"cell_type":"code","source":"# Fill missing values of Age with the average of Age (median)\nfull[ 'Age' ] = full.Age.fillna( full.Age.median() )\n\n# Fill missing values of Fare with the average of Fare (median)\nfull[ 'Fare' ] = full.Fare.fillna( full.Fare.median() )","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"93b038c27701a32ada648f9c65113a5833e33d85"},"cell_type":"code","source":"full.set_value((full['Fare']<40) , 'fare_category','40-')\nfull.set_value((full['Fare']>40) & (full['Fare']<=100), 'fare_category','40-100')\nfull.set_value((full['Fare']>100) & (full['Fare']<=200), 'fare_category','200-300')\nfull.set_value((full['Fare']>200) , 'fare_category','300+')\nfull.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"78fb92786b33e67f83443f5e4ef6580424acfef3"},"cell_type":"markdown","source":"## 4.2 Band numerical data (Age, Fare)"},{"metadata":{"trusted":false,"_uuid":"e40f10fc1d621cbd72f44c4a332e478c4052623c"},"cell_type":"code","source":"full.set_value(full['Age']<=12, 'age_category','childrean')\nfull.set_value((full['Age']>12) & (full['Age']<60), 'age_category','adult')\nfull.set_value(full['Age']>=60, 'age_category','elderly')\nfull.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0224fe111de51383c96e470cade5ee3ce27bc2b0","colab_type":"text","id":"JmQDXLPubcEm"},"cell_type":"markdown","source":"## 4.3 Transforming Categorical Data into Numerical Data"},{"metadata":{"_uuid":"86335a5f02b53a4148007c34e7a34b5bc8ede580","colab":{"base_uri":"https://localhost:8080/","height":119},"colab_type":"code","id":"OseWgi7zbwKy","outputId":"e3f56eed-dc36-4955-9a3d-7ebe84a5bb9d","trusted":false},"cell_type":"code","source":"# Transform Sex into binary values 0 and 1\nsex = pd.Series(np.where( full.Sex == 'male' , 1 , 0 ) , name = 'Sex' )\nsex.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5d2cd255421ae6dee183404689fcb4cb061c9c39","colab":{"base_uri":"https://localhost:8080/","height":204},"colab_type":"code","id":"JGFND6cBcLkM","outputId":"a6867e90-b6dc-461a-e415-8b7349ecbbf1","trusted":false},"cell_type":"code","source":"# Create a new variable for every unique value of Pclass\npclass = pd.get_dummies( full.Pclass , prefix='Pclass' )\npclass.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3189a732ced24f5675c002a7b26fbb988938220c","colab":{"base_uri":"https://localhost:8080/","height":119},"colab_type":"code","id":"UTyuDq8RfdH2","outputId":"a20bb4fe-2054-49f4-e0a3-d300bf73a98a","trusted":false},"cell_type":"code","source":"# Create a new variable to represent whether one passenger has siblings or parches\nfull['family']= full[ 'Parch' ] + full[ 'SibSp' ] +1 #including the passenger self\nfamily = pd.get_dummies(full['family'], prefix='Family Size')\nfamily.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"da30967bf2a33b274bb8e065aa2b6e7f0fff4c49"},"cell_type":"code","source":"# Createa a new variable for every age category\nage = pd.get_dummies(full['age_category'], prefix = 'Age')\nage.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"df1195d81914ccda65b4790531fed6ed7678179c"},"cell_type":"code","source":"# Create a new variable for every fare category\nfare = pd.get_dummies(full['fare_category'],prefix = 'Fare')\nfare.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2b1167b8efd3df0b85dec943f469f0aa0d61f362","colab_type":"text","id":"YHdNzLT4c0uX"},"cell_type":"markdown","source":"## 4.3 Create training set and testing set"},{"metadata":{"_uuid":"20bf8b918dde46704933bcd9c641b8d61a266aba","colab":{"base_uri":"https://localhost:8080/","height":1235},"colab_type":"code","id":"1p1YymmZg3vd","outputId":"bd70d04e-ca68-4770-f444-4dee875772cd","trusted":false},"cell_type":"code","source":"# Select which features/variables to include in the dataset:\n# pclass , sex , family , fare, Age, \n\nfull_X = pd.concat( [ pclass , sex , family , age, fare ] , axis=1 )\nfull_X.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d774319d6026a57c0b575375dd339eb65251281e","colab":{"base_uri":"https://localhost:8080/","height":34},"colab_type":"code","id":"T2QVUlZPiPL5","outputId":"0b4fdc44-0ae3-4908-b327-ec8572c6ade1","trusted":false},"cell_type":"code","source":"# Create all datasets that are necessary to train, validate and test models\nX_train = full_X[ 0:891 ]\nY_train = train.Survived\nX_test = full_X[ 891: ]\nX_train.shape, Y_train.shape, X_test.shape","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"efdb32371e39e247faf03c348b776a3021b9b254","colab_type":"text","id":"WogoLGKol4XM"},"cell_type":"markdown","source":"# 5. Modeling"},{"metadata":{"_uuid":"1b74dfa5dcefd1d2a7d7b334d0d9e444298ca824","colab_type":"text","id":"IIMDrSP-l7ei"},"cell_type":"markdown","source":"## 5.1 Running Models"},{"metadata":{"_uuid":"6a525771dae8141100e0aec5d5816b326be3695b","colab":{"base_uri":"https://localhost:8080/","height":108},"colab_type":"code","id":"_T1bk_tTl__h","outputId":"aec0507a-9084-4ad8-e2ef-9281499abaca","trusted":false},"cell_type":"code","source":"# Logistic Regression\n\nlogreg = LogisticRegression()\nlogreg.fit(X_train, Y_train)\nY_pred = logreg.predict(X_test)\nacc_log = round(logreg.score(X_train, Y_train) * 100, 2)\nacc_log","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3dfb488bd11227d798c501d605b5d064bf64b287","colab":{"base_uri":"https://localhost:8080/","height":108},"colab_type":"code","id":"vIOMvl48n-0t","outputId":"a18e7334-5c12-4381-ab3a-0b79e25ac515","trusted":false},"cell_type":"code","source":"# Support Vector Machines\n\nsvc = SVC()\nsvc.fit(X_train, Y_train)\nY_pred = svc.predict(X_test)\nacc_svc = round(svc.score(X_train, Y_train) * 100, 2)\nacc_svc","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a4ca5cab9525192fe62a843e5625c9ed0bc74f25","colab":{"base_uri":"https://localhost:8080/","height":108},"colab_type":"code","id":"ZoAs1ZV0o1fd","outputId":"3703b3b2-0ff9-4147-8010-e9a90d504e65","trusted":false},"cell_type":"code","source":"# Stochastic Gradient Descent\n\nsgd = SGDClassifier()\nsgd.fit(X_train, Y_train)\nY_pred = sgd.predict(X_test)\nacc_sgd = round(sgd.score(X_train, Y_train) * 100, 2)\nacc_sgd","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4455bf4e642982c4a61533e0272e8d243fc7ab9b","colab":{"base_uri":"https://localhost:8080/","height":34},"colab_type":"code","id":"9HO8B9B0o_Hc","outputId":"647888fb-fcb5-4021-e79a-8f56debba0da","trusted":false},"cell_type":"code","source":"# Decision Tree\n\ndecision_tree = DecisionTreeClassifier()\ndecision_tree.fit(X_train, Y_train)\nY_pred = decision_tree.predict(X_test)\nacc_decision_tree = round(decision_tree.score(X_train, Y_train) * 100, 2)\nacc_decision_tree","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"32e05c0881268c2e457d8d09dab50319ef4d538a","colab":{"base_uri":"https://localhost:8080/","height":34},"colab_type":"code","id":"MlnGwedQpAmP","outputId":"8af351c8-1bc5-4ae4-e22a-20b78b5ce1e3","trusted":false},"cell_type":"code","source":"# Random Forest\n\nrandom_forest = RandomForestClassifier(n_estimators=100)\nrandom_forest.fit(X_train, Y_train)\nY_pred = random_forest.predict(X_test)\nrandom_forest.score(X_train, Y_train)\nacc_random_forest = round(random_forest.score(X_train, Y_train) * 100, 2)\nacc_random_forest","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e3d2d6ce5ed4c0447d0d10a85de6cf19663a6d25","colab_type":"text","id":"XjWpvSGGpKR2"},"cell_type":"markdown","source":"## 5.2 Model Selection"},{"metadata":{"_uuid":"bb6b5780b9e7624b32dda451eae0f0183cae4110","trusted":false},"cell_type":"code","source":"models = pd.DataFrame({\n    'Model': ['Logistic Regression', \n              'Support Vector Machines',\n              'Stochastic Gradient Decent',\n              'Decision Tree',\n              'Random Forest'],\n    'Score': [acc_log, acc_svc, acc_sgd, acc_decision_tree, acc_random_forest\n              ]})\nmodels.sort_values(by='Score', ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"336feded97ede6b00c00d4e93b8859a1388f8ff1","trusted":false},"cell_type":"code","source":"## feature importance in random forest model\ncolnames = X_train.columns\nimportance_dic = dict(zip(colnames,random_forest.feature_importances_))\nimportance_dic","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bdbb01cf2963c68f43db0d8c60b868a99e23ed73","colab":{},"colab_type":"code","id":"f20TlBIapNkb","trusted":false},"cell_type":"code","source":"submission = pd.DataFrame({\n        \"PassengerId\": test[\"PassengerId\"],\n        \"Survived\": Y_pred\n    })\nsubmission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"colab":{"collapsed_sections":[],"name":"Titanic Prediction.ipynb","provenance":[],"toc_visible":true,"version":"0.3.2"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}