{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-08-09T23:54:38.244226Z","iopub.execute_input":"2022-08-09T23:54:38.245466Z","iopub.status.idle":"2022-08-09T23:54:38.277980Z","shell.execute_reply.started":"2022-08-09T23:54:38.245332Z","shell.execute_reply":"2022-08-09T23:54:38.276994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Who survived the Spaceship Titanic?","metadata":{}},{"cell_type":"markdown","source":"Before starting, let's state the problem, and make sure we are familiar with each of the variables in the dataset.","metadata":{}},{"cell_type":"markdown","source":"## My mission 🛸 if I choose to accept it:","metadata":{}},{"cell_type":"markdown","source":"I will be attempting to figure out who was transported to an alternate dimension (*not* a favorable outcome btw) during the accident.  Yes, I said the accident.  BOY, I am never naming one of my cars *Titanic*.","metadata":{}},{"cell_type":"markdown","source":"## What does each field tell us?","metadata":{}},{"cell_type":"markdown","source":"Luckily we had a set of personal records recovered from the ship's damaged computer system.\n- PassengerId - A unique Id for each passenger. Each Id takes the form gggg_pp where gggg indicates a group the passenger is travelling with and pp is their number within the group. People in a group are often family members, but not always.\n- HomePlanet - The planet the passenger departed from, typically their planet of permanent residence.\n- CryoSleep - Indicates whether the passenger elected to be put into suspended animation for the duration of the voyage. Passengers in cryosleep are confined to their cabins.\n- Cabin - The cabin number where the passenger is staying. Takes the form deck/num/side, where side can be either P for Port or S for Starboard.\n- Destination - The planet the passenger will be debarking to.\n- Age - The age of the passenger.\n- VIP - Whether the passenger has paid for special VIP service during the voyage.\n- RoomService, FoodCourt, ShoppingMall, Spa, VRDeck - Amount the passenger has billed at each of the Spaceship Titanic's many - luxury amenities.\n- Name - The first and last names of the passenger.\n- Transported - Whether the passenger was transported to another dimension. This is the target, the column you are trying to predict.","metadata":{}},{"cell_type":"markdown","source":"## Read in the data","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/spaceship-titanic/train.csv')\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:54:49.123720Z","iopub.execute_input":"2022-08-09T23:54:49.124361Z","iopub.status.idle":"2022-08-09T23:54:49.206823Z","shell.execute_reply.started":"2022-08-09T23:54:49.124317Z","shell.execute_reply":"2022-08-09T23:54:49.205603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test = pd.read_csv('/kaggle/input/spaceship-titanic/test.csv')\ntest.shape","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:54:50.055823Z","iopub.execute_input":"2022-08-09T23:54:50.056434Z","iopub.status.idle":"2022-08-09T23:54:50.090200Z","shell.execute_reply.started":"2022-08-09T23:54:50.056380Z","shell.execute_reply":"2022-08-09T23:54:50.088972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## EDA","metadata":{}},{"cell_type":"code","source":"# import packages for graphing\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport plotly.express as px","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:54:52.859655Z","iopub.execute_input":"2022-08-09T23:54:52.860183Z","iopub.status.idle":"2022-08-09T23:54:54.644035Z","shell.execute_reply.started":"2022-08-09T23:54:52.860143Z","shell.execute_reply":"2022-08-09T23:54:54.642680Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.describe()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:54:54.646071Z","iopub.execute_input":"2022-08-09T23:54:54.647152Z","iopub.status.idle":"2022-08-09T23:54:54.692146Z","shell.execute_reply.started":"2022-08-09T23:54:54.647102Z","shell.execute_reply":"2022-08-09T23:54:54.690778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.HomePlanet.value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:54:59.239541Z","iopub.execute_input":"2022-08-09T23:54:59.240036Z","iopub.status.idle":"2022-08-09T23:54:59.251441Z","shell.execute_reply.started":"2022-08-09T23:54:59.239993Z","shell.execute_reply":"2022-08-09T23:54:59.250203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.Destination.value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:01.223334Z","iopub.execute_input":"2022-08-09T23:55:01.224092Z","iopub.status.idle":"2022-08-09T23:55:01.234334Z","shell.execute_reply.started":"2022-08-09T23:55:01.224033Z","shell.execute_reply":"2022-08-09T23:55:01.233260Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Look at Home planet vs destination and percent that got transported\nsns.catplot(x='HomePlanet', y='Transported', hue='Destination', kind='bar', data=train, height=5, aspect=2);","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:04.171576Z","iopub.execute_input":"2022-08-09T23:55:04.172101Z","iopub.status.idle":"2022-08-09T23:55:04.926182Z","shell.execute_reply.started":"2022-08-09T23:55:04.172039Z","shell.execute_reply":"2022-08-09T23:55:04.924884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The Home Planet is definitely worth keeping - Europa has the highest transportation rate. <br>\nNow we'll switch the home and destination.","metadata":{}},{"cell_type":"code","source":"sns.catplot(x='Destination', y='Transported', hue='HomePlanet', kind='bar', data=train, height=5, aspect=2);","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:06.159416Z","iopub.execute_input":"2022-08-09T23:55:06.159893Z","iopub.status.idle":"2022-08-09T23:55:06.790179Z","shell.execute_reply.started":"2022-08-09T23:55:06.159858Z","shell.execute_reply":"2022-08-09T23:55:06.788942Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.catplot(x='Destination', y='Transported', kind='bar', data=train, height=5, aspect=2);","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:06.839394Z","iopub.execute_input":"2022-08-09T23:55:06.839914Z","iopub.status.idle":"2022-08-09T23:55:07.221694Z","shell.execute_reply.started":"2022-08-09T23:55:06.839865Z","shell.execute_reply":"2022-08-09T23:55:07.220545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Destination doesn't seem to be as much of a factor, but maybe a little.  We'll keep it too.","metadata":{}},{"cell_type":"code","source":"# Are there any highly correlated factors?\nsns.heatmap(train.corr(), annot=True);","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:08.496215Z","iopub.execute_input":"2022-08-09T23:55:08.497181Z","iopub.status.idle":"2022-08-09T23:55:08.944747Z","shell.execute_reply.started":"2022-08-09T23:55:08.497132Z","shell.execute_reply":"2022-08-09T23:55:08.943479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There is a little correlation between money spent at the food court and spent at the spa or VR deck, but not a lot.","metadata":{}},{"cell_type":"markdown","source":"Now check out CryoSleep","metadata":{}},{"cell_type":"code","source":"train.CryoSleep.value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:12.167689Z","iopub.execute_input":"2022-08-09T23:55:12.168195Z","iopub.status.idle":"2022-08-09T23:55:12.177914Z","shell.execute_reply.started":"2022-08-09T23:55:12.168150Z","shell.execute_reply":"2022-08-09T23:55:12.176921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cryo_transport = train[['CryoSleep', 'Transported']]\ndisplay(cryo_transport)","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:13.191785Z","iopub.execute_input":"2022-08-09T23:55:13.192777Z","iopub.status.idle":"2022-08-09T23:55:13.208710Z","shell.execute_reply.started":"2022-08-09T23:55:13.192731Z","shell.execute_reply":"2022-08-09T23:55:13.207423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cryo_transport = pd.DataFrame(train.groupby('CryoSleep')['Transported'].sum())\ncryo_transport","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:14.143128Z","iopub.execute_input":"2022-08-09T23:55:14.144377Z","iopub.status.idle":"2022-08-09T23:55:14.159787Z","shell.execute_reply.started":"2022-08-09T23:55:14.144330Z","shell.execute_reply":"2022-08-09T23:55:14.158609Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We will also keep the CryoSleep feature.","metadata":{}},{"cell_type":"markdown","source":"VIP?","metadata":{}},{"cell_type":"code","source":"vip_transport = pd.DataFrame(train.groupby('VIP')['Transported'].sum())\nvip_transport","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:17.435662Z","iopub.execute_input":"2022-08-09T23:55:17.436202Z","iopub.status.idle":"2022-08-09T23:55:17.449590Z","shell.execute_reply.started":"2022-08-09T23:55:17.436155Z","shell.execute_reply":"2022-08-09T23:55:17.448192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Oh FOR SURE!!","metadata":{}},{"cell_type":"markdown","source":"Finally, I suspect if someone was traveling in a group, they were less likely to be transported.  Let's see.","metadata":{}},{"cell_type":"markdown","source":"### Feature Engineering","metadata":{}},{"cell_type":"code","source":"# create two new columns from the PassengerId\ntrain[['PassengerNo','GroupMemNo']] = train['PassengerId'].str.split(pat='_', expand=True)\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:24.940592Z","iopub.execute_input":"2022-08-09T23:55:24.941745Z","iopub.status.idle":"2022-08-09T23:55:24.986324Z","shell.execute_reply.started":"2022-08-09T23:55:24.941675Z","shell.execute_reply":"2022-08-09T23:55:24.985108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# now include a column that counts how many were in a group traveling together\ntrain['PartySize']=train.groupby('PassengerNo')['PassengerNo'].transform('count')\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:26.207398Z","iopub.execute_input":"2022-08-09T23:55:26.207920Z","iopub.status.idle":"2022-08-09T23:55:26.240767Z","shell.execute_reply.started":"2022-08-09T23:55:26.207873Z","shell.execute_reply":"2022-08-09T23:55:26.239500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.PartySize.value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:28.001687Z","iopub.execute_input":"2022-08-09T23:55:28.002186Z","iopub.status.idle":"2022-08-09T23:55:28.012452Z","shell.execute_reply.started":"2022-08-09T23:55:28.002146Z","shell.execute_reply":"2022-08-09T23:55:28.011280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train['PartySize'] = train['PartySize'].astype('category')","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:29.583143Z","iopub.execute_input":"2022-08-09T23:55:29.583637Z","iopub.status.idle":"2022-08-09T23:55:29.591762Z","shell.execute_reply.started":"2022-08-09T23:55:29.583596Z","shell.execute_reply":"2022-08-09T23:55:29.590525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12,7))\nax = sns.countplot(x='PartySize', hue='Transported', data=train);","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:30.799622Z","iopub.execute_input":"2022-08-09T23:55:30.800439Z","iopub.status.idle":"2022-08-09T23:55:31.210672Z","shell.execute_reply.started":"2022-08-09T23:55:30.800385Z","shell.execute_reply":"2022-08-09T23:55:31.209401Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Party sizes of 1 or 8 were transported at a lower rate while the sizes in between were transported at a higher rate. This time I will keep the partysize column. ","metadata":{}},{"cell_type":"markdown","source":"### Do the same for the test data.","metadata":{}},{"cell_type":"code","source":"test[['PassengerNo','GroupMemNo']] = test['PassengerId'].str.split(pat='_', expand=True)\ntest['PartySize']=test.groupby('PassengerNo')['PassengerNo'].transform('count')","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:38.032685Z","iopub.execute_input":"2022-08-09T23:55:38.033189Z","iopub.status.idle":"2022-08-09T23:55:38.054483Z","shell.execute_reply.started":"2022-08-09T23:55:38.033151Z","shell.execute_reply":"2022-08-09T23:55:38.053290Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test['PartySize'] = test['PartySize'].astype('category')","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:39.794190Z","iopub.execute_input":"2022-08-09T23:55:39.795445Z","iopub.status.idle":"2022-08-09T23:55:39.803932Z","shell.execute_reply.started":"2022-08-09T23:55:39.795385Z","shell.execute_reply":"2022-08-09T23:55:39.802803Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.Transported.value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:40.931641Z","iopub.execute_input":"2022-08-09T23:55:40.932136Z","iopub.status.idle":"2022-08-09T23:55:40.941531Z","shell.execute_reply.started":"2022-08-09T23:55:40.932095Z","shell.execute_reply":"2022-08-09T23:55:40.940520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"That's more of a sum which can be decieving. Let's look at percentages instead.","metadata":{}},{"cell_type":"code","source":"party_size = pd.DataFrame(train.groupby('PartySize')['Transported'].mean().reset_index())\nparty_size","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:42.755662Z","iopub.execute_input":"2022-08-09T23:55:42.756576Z","iopub.status.idle":"2022-08-09T23:55:42.774778Z","shell.execute_reply.started":"2022-08-09T23:55:42.756533Z","shell.execute_reply":"2022-08-09T23:55:42.773896Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Separate cabin components and keep first and last","metadata":{}},{"cell_type":"code","source":"train[['cabin1','cabin2','cabin3']] = train['Cabin'].str.split(pat='/', expand=True)\ntest[['cabin1','cabin2','cabin3']] = test['Cabin'].str.split(pat='/', expand=True)\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:45.091377Z","iopub.execute_input":"2022-08-09T23:55:45.092309Z","iopub.status.idle":"2022-08-09T23:55:45.146319Z","shell.execute_reply.started":"2022-08-09T23:55:45.092261Z","shell.execute_reply":"2022-08-09T23:55:45.145119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.catplot(x='cabin1', y='Transported', kind='bar', data=train, height=5, aspect=3);","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:46.037616Z","iopub.execute_input":"2022-08-09T23:55:46.038125Z","iopub.status.idle":"2022-08-09T23:55:46.589695Z","shell.execute_reply.started":"2022-08-09T23:55:46.038078Z","shell.execute_reply":"2022-08-09T23:55:46.588360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.catplot(x='cabin3', y='Transported', kind='bar', data=train, height=5, aspect=3);","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:47.067460Z","iopub.execute_input":"2022-08-09T23:55:47.068960Z","iopub.status.idle":"2022-08-09T23:55:47.406959Z","shell.execute_reply.started":"2022-08-09T23:55:47.068889Z","shell.execute_reply":"2022-08-09T23:55:47.405806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.cabin2.value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-09T23:55:47.982441Z","iopub.execute_input":"2022-08-09T23:55:47.982910Z","iopub.status.idle":"2022-08-09T23:55:47.995427Z","shell.execute_reply.started":"2022-08-09T23:55:47.982872Z","shell.execute_reply":"2022-08-09T23:55:47.994189Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.columns","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:02:54.120732Z","iopub.execute_input":"2022-08-10T00:02:54.121295Z","iopub.status.idle":"2022-08-10T00:02:54.129911Z","shell.execute_reply.started":"2022-08-10T00:02:54.121248Z","shell.execute_reply":"2022-08-10T00:02:54.128697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# keep PassengerId for submission\npassenger_id = test[['PassengerId']]\npassenger_id.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:02:55.280143Z","iopub.execute_input":"2022-08-10T00:02:55.280935Z","iopub.status.idle":"2022-08-10T00:02:55.295809Z","shell.execute_reply.started":"2022-08-10T00:02:55.280876Z","shell.execute_reply":"2022-08-10T00:02:55.294665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X = train.drop(['Transported','PassengerId','Cabin','Name','PassengerNo','GroupMemNo','cabin2'], axis=1)\ny = train['Transported']\nX_test = test.drop(['PassengerId','Cabin','Name','PassengerNo','GroupMemNo','cabin2'], axis=1)\nprint(X.shape, X_test.shape)","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:02:56.415736Z","iopub.execute_input":"2022-08-10T00:02:56.416216Z","iopub.status.idle":"2022-08-10T00:02:56.429480Z","shell.execute_reply.started":"2022-08-10T00:02:56.416174Z","shell.execute_reply":"2022-08-10T00:02:56.428249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Take care of missing data","metadata":{}},{"cell_type":"markdown","source":"Use the mode for categorical or boolean columns","metadata":{}},{"cell_type":"code","source":"mode_cols = ['HomePlanet', 'CryoSleep', 'Destination', 'VIP', 'cabin1', 'cabin3', 'PartySize']\nX[mode_cols].isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:02:59.028348Z","iopub.execute_input":"2022-08-10T00:02:59.029312Z","iopub.status.idle":"2022-08-10T00:02:59.047596Z","shell.execute_reply.started":"2022-08-10T00:02:59.029254Z","shell.execute_reply":"2022-08-10T00:02:59.046263Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X[mode_cols]=X[mode_cols].fillna(X.mode().iloc[0])\nX[mode_cols].isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:02:59.837041Z","iopub.execute_input":"2022-08-10T00:02:59.838125Z","iopub.status.idle":"2022-08-10T00:02:59.886215Z","shell.execute_reply.started":"2022-08-10T00:02:59.838073Z","shell.execute_reply":"2022-08-10T00:02:59.884995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Use the mean for floats","metadata":{}},{"cell_type":"code","source":"float_cols = [col for col in X.columns if X[col].dtype=='float64']\nX[float_cols].isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:03:01.108922Z","iopub.execute_input":"2022-08-10T00:03:01.111408Z","iopub.status.idle":"2022-08-10T00:03:01.124008Z","shell.execute_reply.started":"2022-08-10T00:03:01.111351Z","shell.execute_reply":"2022-08-10T00:03:01.122598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X[float_cols].mean()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:03:02.075782Z","iopub.execute_input":"2022-08-10T00:03:02.076296Z","iopub.status.idle":"2022-08-10T00:03:02.089658Z","shell.execute_reply.started":"2022-08-10T00:03:02.076251Z","shell.execute_reply":"2022-08-10T00:03:02.088440Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X[float_cols]=X[float_cols].fillna(X.mean().iloc[0])\nX[float_cols].isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:03:04.856237Z","iopub.execute_input":"2022-08-10T00:03:04.856717Z","iopub.status.idle":"2022-08-10T00:03:04.909333Z","shell.execute_reply.started":"2022-08-10T00:03:04.856679Z","shell.execute_reply":"2022-08-10T00:03:04.908127Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_test.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:03:05.634929Z","iopub.execute_input":"2022-08-10T00:03:05.635436Z","iopub.status.idle":"2022-08-10T00:03:05.646618Z","shell.execute_reply.started":"2022-08-10T00:03:05.635392Z","shell.execute_reply":"2022-08-10T00:03:05.645430Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_test[mode_cols]=X_test[mode_cols].fillna(X_test.mode().iloc[0])\nX_test[float_cols]=X_test[float_cols].fillna(X_test.mean().iloc[0])\nX_test.isna().any()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:03:06.420263Z","iopub.execute_input":"2022-08-10T00:03:06.421026Z","iopub.status.idle":"2022-08-10T00:03:06.465600Z","shell.execute_reply.started":"2022-08-10T00:03:06.420982Z","shell.execute_reply":"2022-08-10T00:03:06.464251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Make categorical features integers and scale the float features","metadata":{}},{"cell_type":"code","source":"X_oh = pd.get_dummies(X, columns = mode_cols)\nX_test_oh = pd.get_dummies(X_test, columns = mode_cols)\nX_oh.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:03:08.955770Z","iopub.execute_input":"2022-08-10T00:03:08.956275Z","iopub.status.idle":"2022-08-10T00:03:09.006282Z","shell.execute_reply.started":"2022-08-10T00:03:08.956232Z","shell.execute_reply":"2022-08-10T00:03:09.004904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_test_oh.shape","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:03:09.691919Z","iopub.execute_input":"2022-08-10T00:03:09.692428Z","iopub.status.idle":"2022-08-10T00:03:09.700552Z","shell.execute_reply.started":"2022-08-10T00:03:09.692384Z","shell.execute_reply":"2022-08-10T00:03:09.699327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import MinMaxScaler\n\nmmsc = MinMaxScaler()\n\nscaled_features = mmsc.fit_transform(X_oh)\nX_oh_scl = pd.DataFrame(scaled_features, index=X_oh.index, columns=X_oh.columns)\nX_oh_scl.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:03:10.686070Z","iopub.execute_input":"2022-08-10T00:03:10.686907Z","iopub.status.idle":"2022-08-10T00:03:10.793814Z","shell.execute_reply.started":"2022-08-10T00:03:10.686850Z","shell.execute_reply":"2022-08-10T00:03:10.792549Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Repeat for test data","metadata":{"execution":{"iopub.status.busy":"2022-08-07T22:30:42.782376Z","iopub.execute_input":"2022-08-07T22:30:42.783017Z","iopub.status.idle":"2022-08-07T22:30:42.815436Z","shell.execute_reply.started":"2022-08-07T22:30:42.782980Z","shell.execute_reply":"2022-08-07T22:30:42.814111Z"}}},{"cell_type":"code","source":"scaled_features = mmsc.transform(X_test_oh)\nX_test_oh_scl = pd.DataFrame(scaled_features, index=X_test_oh.index, columns=X_test_oh.columns)\nX_test_oh_scl.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:03:12.616275Z","iopub.execute_input":"2022-08-10T00:03:12.616756Z","iopub.status.idle":"2022-08-10T00:03:12.654034Z","shell.execute_reply.started":"2022-08-10T00:03:12.616718Z","shell.execute_reply":"2022-08-10T00:03:12.653121Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data before this is numeric and scaled","metadata":{}},{"cell_type":"markdown","source":"## This version will use hyperparameter tuning with GridSearchCV and a Support Vector Machine","metadata":{}},{"cell_type":"code","source":"from sklearn.svm import SVC  \nfrom sklearn.metrics import classification_report, confusion_matrix ","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:04:32.028156Z","iopub.execute_input":"2022-08-10T00:04:32.028643Z","iopub.status.idle":"2022-08-10T00:04:32.164547Z","shell.execute_reply.started":"2022-08-10T00:04:32.028605Z","shell.execute_reply":"2022-08-10T00:04:32.163384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split, GridSearchCV","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:08:41.048471Z","iopub.execute_input":"2022-08-10T00:08:41.048972Z","iopub.status.idle":"2022-08-10T00:08:41.055119Z","shell.execute_reply.started":"2022-08-10T00:08:41.048927Z","shell.execute_reply":"2022-08-10T00:08:41.053892Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# split the training data to get a validation portion\nX_train, X_valid, y_train, y_valid = train_test_split(X_oh_scl, y, test_size=0.3, random_state=43)\nprint(X_train.shape, X_valid.shape, len(y_train), len(y_valid))","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:18:47.264275Z","iopub.execute_input":"2022-08-10T00:18:47.264789Z","iopub.status.idle":"2022-08-10T00:18:47.276849Z","shell.execute_reply.started":"2022-08-10T00:18:47.264746Z","shell.execute_reply":"2022-08-10T00:18:47.275786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# create the parameters to try for the SVM\nparam_grid = {'C': [0.1,1, 10, 100], 'gamma': [1,0.1,0.01,0.001],'kernel': ['rbf', 'poly', 'sigmoid']}\n\nsvc = SVC()\ngrid = GridSearchCV(svc, param_grid=param_grid, refit=True, verbose=2)","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:27:15.780441Z","iopub.execute_input":"2022-08-10T00:27:15.780927Z","iopub.status.idle":"2022-08-10T00:27:15.787380Z","shell.execute_reply.started":"2022-08-10T00:27:15.780888Z","shell.execute_reply":"2022-08-10T00:27:15.786257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# not going through this before submission\n# grid.fit(X_train, y_train)","metadata":{"execution":{"iopub.status.busy":"2022-08-10T00:28:17.258894Z","iopub.execute_input":"2022-08-10T00:28:17.259404Z","iopub.status.idle":"2022-08-10T00:54:15.600066Z","shell.execute_reply.started":"2022-08-10T00:28:17.259365Z","shell.execute_reply":"2022-08-10T00:54:15.598997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# including the output here before commenting out\n# print(grid.best_estimator_)\n# output: SVC(C=100, gamma=0.1)","metadata":{"execution":{"iopub.status.busy":"2022-08-10T01:10:10.410321Z","iopub.execute_input":"2022-08-10T01:10:10.410795Z","iopub.status.idle":"2022-08-10T01:10:10.418376Z","shell.execute_reply.started":"2022-08-10T01:10:10.410754Z","shell.execute_reply":"2022-08-10T01:10:10.416739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print(grid.best_params_)\n# output: {'C': 100, 'gamma': 0.1, 'kernel': 'rbf'}","metadata":{"execution":{"iopub.status.busy":"2022-08-10T01:18:00.167001Z","iopub.execute_input":"2022-08-10T01:18:00.168392Z","iopub.status.idle":"2022-08-10T01:18:00.174908Z","shell.execute_reply.started":"2022-08-10T01:18:00.168325Z","shell.execute_reply":"2022-08-10T01:18:00.173451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"svc = SVC(C=100, gamma=0.1, kernel='rbf')","metadata":{"execution":{"iopub.status.busy":"2022-08-10T01:20:09.108307Z","iopub.execute_input":"2022-08-10T01:20:09.108820Z","iopub.status.idle":"2022-08-10T01:20:09.115076Z","shell.execute_reply.started":"2022-08-10T01:20:09.108779Z","shell.execute_reply":"2022-08-10T01:20:09.113798Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import accuracy_score","metadata":{"execution":{"iopub.status.busy":"2022-08-10T01:21:38.318724Z","iopub.execute_input":"2022-08-10T01:21:38.319229Z","iopub.status.idle":"2022-08-10T01:21:38.324917Z","shell.execute_reply.started":"2022-08-10T01:21:38.319190Z","shell.execute_reply":"2022-08-10T01:21:38.323817Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"svc.fit(X_train, y_train)\npreds = svc.predict(X_valid)\n\nprint(accuracy_score(preds,y_valid))","metadata":{"execution":{"iopub.status.busy":"2022-08-10T01:21:39.350864Z","iopub.execute_input":"2022-08-10T01:21:39.352004Z","iopub.status.idle":"2022-08-10T01:21:41.974762Z","shell.execute_reply.started":"2022-08-10T01:21:39.351953Z","shell.execute_reply":"2022-08-10T01:21:41.973378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Fit this SVC to the test data and submit","metadata":{}},{"cell_type":"code","source":"preds_svc = svc.predict(X_test_oh_scl)\npreds_svc[:10]","metadata":{"execution":{"iopub.status.busy":"2022-08-10T01:23:09.841602Z","iopub.execute_input":"2022-08-10T01:23:09.842840Z","iopub.status.idle":"2022-08-10T01:23:10.781990Z","shell.execute_reply.started":"2022-08-10T01:23:09.842788Z","shell.execute_reply":"2022-08-10T01:23:10.780770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = {\n    'PassengerId': np.array(passenger_id).reshape(len(passenger_id)),\n    'Transported': np.array(preds_svc)\n}\n\nsubmission_df = pd.DataFrame(data=data).reset_index()\nsubmission_df.drop(columns=['index'], inplace=True, axis=1)\nsubmission_df.to_csv(\"submission.csv\", index=False)\n\nsubmission_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-10T01:26:30.168499Z","iopub.execute_input":"2022-08-10T01:26:30.168979Z","iopub.status.idle":"2022-08-10T01:26:30.199239Z","shell.execute_reply.started":"2022-08-10T01:26:30.168942Z","shell.execute_reply":"2022-08-10T01:26:30.197870Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}