{ "cells": [ { "cell_type": "markdown", "metadata": { "_cell_guid": "040aa878-3ade-e76c-0bda-13fe3503f8db" }, "source": [ "## This is my first time using a Kaggle notebook...\n", "...And I am not really sure what I am doing. My main goal here is to write a really sketchy submission for the Titanic competition.\n", "\n", "### My goals are as follows\n", "\n", "1. Write a \"really bad\" first submission to the competition that works. Gotta jump in the pool some time.\n", "2. Improve on #1 using what I already know to create a simple model that uses sklearn.\n", "3. Make sure I'm adding in all this cool dank Pandas (a library I'm still learning) knowledge. It definitely saves time.\n", "4. Look at some other kernels / the forum and make minor improvements.\n", "5. When I feel comfortable navigating the Kaggle environment, take a look at another competition." ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "6752497d-bc46-3d65-27bc-1fe8a12e837b" }, "source": [ "## Read in the data" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "bdf5031b-01bd-34bf-fa3c-5d590b082b30" }, "outputs": [], "source": [ "# The first order of business, as always, is to read in the mother fucking data\n", "\n", "import pandas as pd\n", "dfTrain = pd.read_csv('../input/train.csv')\n", "dfTest = pd.read_csv('../input/test.csv')" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "1f1de9bb-2429-68eb-1780-6fdf2d0da590" }, "source": [ "## Look at the format of the data\n", "\n", "We need to know the expected data types for each column to be able to effectively clean the data of NaN's. \n", "\n", "We can see the following:\n", "\n", "1. PassengerID - int\n", "2. Survived - int\n", "3. Pclass - int\n", "4. Name - string\n", "5. Sex - string\n", "6. Age - float\n", "7. SibSp - int\n", "8. Parch - int\n", "9. Ticket - string\n", "10. Fare - float\n", "11. Cabin - String\n", "12. Embarked - String" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "3e82082b-a48f-165b-bde7-8147c9e5c068" }, "outputs": [], "source": [ "dfTrain.head()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "c4683143-818d-0cd9-2a0b-9fa9d0c4d31e" }, "source": [ "## Clean the NaN's from the data" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "568ec920-2d5e-c10d-37f8-7fb1089d4b04" }, "outputs": [], "source": [ "# Assign default values for each data type\n", "defaultInt = -1\n", "defaultString = 'NaN'\n", "defaultFloat = -1.0\n", "\n", "# Create lists by data tpe\n", "intFeatures = ['PassengerId', 'Survived', 'Pclass', 'SibSp', 'Parch']\n", "stringFeatures = ['Name', 'Sex', 'Ticket', 'Cabin', 'Embarked']\n", "floatFeatures = ['Age', 'Fare']\n", "\n", "# Clean the NaN's\n", "for feature in list(dfTrain):\n", " if feature in intFeatures:\n", " dfTrain[feature] = dfTrain[feature].fillna(defaultInt)\n", " elif feature in stringFeatures:\n", " dfTrain[feature] = dfTrain[feature].fillna(defaultString)\n", " elif feature in floatFeatures:\n", " dfTrain[feature] = dfTrain[feature].fillna(defaultFloat)\n", " else:\n", " print('Error: Feature %s not recognized.' % feature)\n", " \n", "for feature in list(dfTest):\n", " if feature in intFeatures:\n", " dfTest[feature] = dfTest[feature].fillna(defaultInt)\n", " elif feature in stringFeatures:\n", " dfTest[feature] = dfTest[feature].fillna(defaultString)\n", " elif feature in floatFeatures:\n", " dfTest[feature] = dfTest[feature].fillna(defaultFloat)\n", " else:\n", " print('Error: Feature %s not recognized.' % feature)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "b1643e36-8a8c-06db-f0ba-16c610af9818" }, "source": [ "## Encode data\n", "\n", "Encode all features except PassengerId, as this needs to be untouched for the Kaggle grading script to run properly." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "8e5eae6e-6ee5-df0f-ad40-c500e248c776" }, "outputs": [], "source": [ "from sklearn.preprocessing import LabelEncoder\n", "dfCombined = pd.concat([dfTrain, dfTest])\n", "for feature in list(dfCombined):\n", " \n", " le = LabelEncoder()\n", " le.fit(dfCombined[feature])\n", " \n", " if feature in dfTrain:\n", " if feature != 'PassengerId':\n", " dfTrain[feature] = le.transform(dfTrain[feature])\n", " if feature in dfTest:\n", " if feature != 'PassengerId':\n", " dfTest[feature] = le.transform(dfTest[feature])" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "fedbd16e-1c78-bbfd-8511-eba83431a061" }, "source": [ "## Split into a training and test set\n", "\n", "Sklearn can take a dataframe as input to its functions?? Cool!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "979f5050-c04b-7d96-4a4e-bfaf3694b955" }, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "\n", "X = dfTrain.drop(['Survived'], axis=1)\n", "y = dfTrain['Survived']\n", "\n", "num_test = 0.20\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=num_test, random_state=23)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "7f9b89b8-0c5e-7040-1bc7-95797e850087" }, "source": [ "## Train ML classifier\n", "\n", "For this purpose, we will use Random Forests." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "2fa5f596-88c7-787a-d5c9-aebbf1de3503" }, "outputs": [], "source": [ "from sklearn.ensemble import RandomForestClassifier\n", "\n", "clf = RandomForestClassifier()\n", "clf.fit(X_train, y_train)\n", "predictions = clf.predict(X_test)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "8b731473-1f33-f98c-2f87-7f80d94c27c2" }, "source": [ "## Calculate accuracy" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "01094ccd-5cc7-19a0-cdc4-7204d0109fc9" }, "outputs": [], "source": [ "from sklearn.metrics import accuracy_score\n", "\n", "accuracy = accuracy_score(y_test, predictions)\n", "\n", "print(accuracy)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "83fe4f78-f806-f6c5-2225-84ff312f4ba9" }, "source": [ "## Form predictions on test set" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "3fe1dda7-6db3-2081-4816-35a44d5b92ed" }, "outputs": [], "source": [ "# Generate predictions\n", "clf = RandomForestClassifier()\n", "clf.fit(X, y)\n", "dfTestPredictions = clf.predict(dfTest)\n", "\n", "# Write predictions to csv file\n", "results = pd.DataFrame({'PassengerId': dfTest['PassengerId'], 'Survived': dfTestPredictions})\n", "results.to_csv('results.csv', index=False)\n", "results.head()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "24d4c50c-a697-079d-3158-ecb885ad4069" }, "source": [ "## Possible Improvements\n", "\n", "1. Instead of default values for NaN's, clean missing values with column averages.\n", "2. Remove columns that offer little or no information. Eg. PassengerId.\n", "3. Add features related to family size.\n", "4. Assess the accuracy by implementing k-fold cross validation.\n", "5. Implement Gird Search to tune hyper parameters\n", "6. Learn how to make an evaluation for how valuable each feature is. Remove unimportant features." ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "1ff0607f-1c83-93c4-0068-2ef72f635d64" }, "source": [ "# Things I Learned\n", "\n", "1. In Python 3 specifically, for LabelEncoder to work, encodings of NaN's must be to a data type homogeneous with the rest of the data in a column. \n", "- This is not a problem in Python 2.7\n", "2. You can pass a Pandas data frame into an Sklearn classifier. That's pretty fucking cool, and saves a lot of time." ] } ], "metadata": { "_change_revision": 0, "_is_fork": false, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.0" } }, "nbformat": 4, "nbformat_minor": 0 }