{"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"4.0.5"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Created by Burak Dilber**\n\n**Spaceship Titanic - EDA, Pre Processing and XGBoost**\n\n**14 August 2022**","metadata":{}},{"cell_type":"markdown","source":"#
Spaceship Titanic - EDA, Pre Processing and XGBoost
","metadata":{}},{"cell_type":"markdown","source":"# Table of Contents\n\n- [1. Introduction](#1)\n- [2. Imports](#2)\n- [3. Data Loading and Preperation](#3)\n - [3.1 Exploring Train Data](#3.1)\n - [3.2 Exploring Test Data](#3.2)\n - [3.3 Submission File](#3.3)\n - [3.4 Merge Data](#3.4)\n- [4. EDA](#4)\n - [4.1 Continuous Features](#4.1)\n - [4.2 Categorical Features](#4.2)\n- [5. Data Pre-Processing](#5)\n- [6. Modeling](#6)\n - [6.1 Hyperparameter Tuning](#6.1)\n - [6.2 Final Hyperparameter and Last Fit](#6.2)\n - [6.3 Test Data Prediction](#6.3)\n- [7. Submission](#7)","metadata":{}},{"cell_type":"markdown","source":"\n# **
Introduction
**","metadata":{}},{"cell_type":"markdown","source":"The competition is organised by `Kaggle` and is in the `GettingStarted Prediction Competition` series.\n\nIn this competition, you are supposed to predict predict which passengers were transported by the anomaly using records recovered from the spaceship’s damaged computer system.\n\nSubmissions are evaluated on `Classification Accuracy`.","metadata":{}},{"cell_type":"markdown","source":"\n# **
Imports
**","metadata":{"execution":{"iopub.status.busy":"2022-08-14T13:33:13.196626Z","iopub.execute_input":"2022-08-14T13:33:13.198688Z","iopub.status.idle":"2022-08-14T13:33:13.331984Z"}}},{"cell_type":"code","source":"library(tidyverse)\nlibrary(tidymodels)\nlibrary(class)\nlibrary(Boruta)\nlibrary(missMethods)\nlibrary(caret)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:30:57.541352Z","iopub.execute_input":"2022-08-14T14:30:57.543235Z","iopub.status.idle":"2022-08-14T14:30:57.567380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# **
Data Loading and Preparation
**","metadata":{}},{"cell_type":"code","source":"spaceship_train <- read_csv(\"../input/spaceship-titanic/train.csv\") %>%\n select(-Transported)\n\nspaceship_test <- read_csv(\"../input/spaceship-titanic/test.csv\")\n\nsubmission <- read_csv(\"../input/spaceship-titanic/sample_submission.csv\")\n\ntransported <- read_csv(\"../input/spaceship-titanic/train.csv\") %>%\n select(Transported)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:02.619243Z","iopub.execute_input":"2022-08-14T14:31:02.620900Z","iopub.status.idle":"2022-08-14T14:31:02.984701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Column Descriptions : \n\n\n- `PassengerId` - A unique Id for each passenger. Each Id takes the form gggg_pp where gggg indicates a group the passenger is travelling with and pp is their number within the group. People in a group are often family members, but not always.\n- `HomePlanet` - The planet the passenger departed from, typically their planet of permanent residence.\n- `CryoSleep` - Indicates whether the passenger elected to be put into suspended animation for the duration of the voyage. Passengers in cryosleep are confined to their cabins.\n- `Cabin` - The cabin number where the passenger is staying. Takes the form deck/num/side, where side can be either P for Port or S for Starboard.\n- `Destination` - The planet the passenger will be debarking to.\n- `Age` - The age of the passenger.\n- `VIP` - Whether the passenger has paid for special VIP service during the voyage.\n- `RoomService`, FoodCourt, ShoppingMall, Spa, VRDeck - Amount the passenger has billed at each of the Spaceship Titanic's many luxury amenities.\n- `Name` - The first and last names of the passenger.\n- `Transported` - Whether the passenger was transported to another dimension. This is the target, the column you are trying to predict.","metadata":{}},{"cell_type":"markdown","source":"\n## Exploring Train Data : ","metadata":{}},{"cell_type":"code","source":"spaceship_train %>% head()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:06.393481Z","iopub.execute_input":"2022-08-14T14:31:06.395043Z","iopub.status.idle":"2022-08-14T14:31:06.420952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dim(spaceship_train)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:07.441258Z","iopub.execute_input":"2022-08-14T14:31:07.442798Z","iopub.status.idle":"2022-08-14T14:31:07.458118Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glimpse(spaceship_train)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:08.231724Z","iopub.execute_input":"2022-08-14T14:31:08.233268Z","iopub.status.idle":"2022-08-14T14:31:08.256836Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spaceship_train %>% \n is.na() %>% \n sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:09.620421Z","iopub.execute_input":"2022-08-14T14:31:09.621965Z","iopub.status.idle":"2022-08-14T14:31:09.637209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spaceship_train %>%\n select(everything()) %>%\n summarise_all(funs(sum(is.na(.)))) %>%\n t()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:10.544835Z","iopub.execute_input":"2022-08-14T14:31:10.546370Z","iopub.status.idle":"2022-08-14T14:31:10.588460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spaceship_train %>% summary()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:10.925947Z","iopub.execute_input":"2022-08-14T14:31:10.927637Z","iopub.status.idle":"2022-08-14T14:31:10.955386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Exploring Test Data : ","metadata":{}},{"cell_type":"code","source":"spaceship_test %>% head()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:13.471654Z","iopub.execute_input":"2022-08-14T14:31:13.473490Z","iopub.status.idle":"2022-08-14T14:31:13.499507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dim(spaceship_test)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:15.893446Z","iopub.execute_input":"2022-08-14T14:31:15.895004Z","iopub.status.idle":"2022-08-14T14:31:15.910034Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glimpse(spaceship_test)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:16.807196Z","iopub.execute_input":"2022-08-14T14:31:16.808803Z","iopub.status.idle":"2022-08-14T14:31:16.840011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spaceship_test %>% \n is.na() %>% \n sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:18.209275Z","iopub.execute_input":"2022-08-14T14:31:18.210822Z","iopub.status.idle":"2022-08-14T14:31:18.228461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spaceship_test %>%\n select(everything()) %>%\n summarise_all(funs(sum(is.na(.)))) %>%\n t()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:18.738559Z","iopub.execute_input":"2022-08-14T14:31:18.740384Z","iopub.status.idle":"2022-08-14T14:31:18.782781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spaceship_test %>% summary()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:19.273865Z","iopub.execute_input":"2022-08-14T14:31:19.275522Z","iopub.status.idle":"2022-08-14T14:31:19.298113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Submission File : ","metadata":{}},{"cell_type":"code","source":"submission %>% head()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:20.681095Z","iopub.execute_input":"2022-08-14T14:31:20.682621Z","iopub.status.idle":"2022-08-14T14:31:20.705790Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Merge Data ","metadata":{}},{"cell_type":"code","source":"data_all <- rbind(spaceship_train %>% mutate(type = \"Train\"), spaceship_test %>% mutate(type = \"Test\"))","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:21.666236Z","iopub.execute_input":"2022-08-14T14:31:21.667845Z","iopub.status.idle":"2022-08-14T14:31:21.690433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"age <- data_all$Age","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:22.290308Z","iopub.execute_input":"2022-08-14T14:31:22.291831Z","iopub.status.idle":"2022-08-14T14:31:22.303321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# **
EDA
**","metadata":{}},{"cell_type":"markdown","source":"\n## Continuos Features ","metadata":{}},{"cell_type":"code","source":"data_all %>%\n ggplot(aes(x = Age, fill = type)) +\n geom_histogram(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Histogram of Age\") +\n xlab(\"Age\") +\n ylab(\"Count\")\n\ndata_all %>%\n ggplot(aes(x = RoomService, fill = type)) +\n geom_histogram(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Histogram of RoomService\") +\n xlab(\"RoomService\") +\n ylab(\"Count\")\n\ndata_all %>%\n ggplot(aes(x = FoodCourt, fill = type)) +\n geom_histogram(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Histogram of FoodCourt\") +\n xlab(\"FoodCourt\") +\n ylab(\"Count\")\n\ndata_all %>%\n ggplot(aes(x = ShoppingMall, fill = type)) +\n geom_histogram(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Histogram of ShoppingMall\") +\n xlab(\"ShoppingMall\") +\n ylab(\"Count\")\n\ndata_all %>%\n ggplot(aes(x = Spa, fill = type)) +\n geom_histogram(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Histogram of Spa\") +\n xlab(\"Spa\") +\n ylab(\"Count\")\n\ndata_all %>%\n ggplot(aes(x = VRDeck, fill = type)) +\n geom_histogram(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Histogram of VRDeck\") +\n xlab(\"VRDeck\") +\n ylab(\"Count\")","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:23.823226Z","iopub.execute_input":"2022-08-14T14:31:23.824967Z","iopub.status.idle":"2022-08-14T14:31:25.780636Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Categorical Features ","metadata":{}},{"cell_type":"code","source":"data_all %>%\n ggplot(aes(x = HomePlanet, fill = type)) +\n geom_bar(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Bar Plot of HomePlanet\") +\n xlab(\"HomePlanet\") +\n ylab(\"Count\")\n\ndata_all %>%\n ggplot(aes(x = CryoSleep, fill = type)) +\n geom_bar(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Bar Plot of CryoSleep\") +\n xlab(\"CryoSleep\") +\n ylab(\"Count\")\n\ndata_all %>%\n ggplot(aes(x = Destination, fill = type)) +\n geom_bar(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Bar Plot of Destination\") +\n xlab(\"Destination\") +\n ylab(\"Count\")\n\ndata_all %>%\n ggplot(aes(x = VIP, fill = type)) +\n geom_bar(color=\"#e9ecef\", alpha = 0.6) +\n labs(title = \"Bar Plot of VIP\") +\n xlab(\"VIP\") +\n ylab(\"Count\")","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:25.782919Z","iopub.execute_input":"2022-08-14T14:31:25.784257Z","iopub.status.idle":"2022-08-14T14:31:27.219053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# **
Data Pre-Processing
**","metadata":{}},{"cell_type":"code","source":"## Name Variable\n\nlastNameData <- data_all %>%\n separate(Name, c(\"FirstName\", \"LastName\"), \" \") %>%\n select(-\"FirstName\") \n\nFamilySize = as.data.frame(table(unique(lastNameData)$LastName)[as.character(lastNameData$LastName)])\n\ndata_all <- lastNameData %>% \n mutate(FamilySize = FamilySize$Freq) %>%\n select(-\"LastName\")","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:27.223036Z","iopub.execute_input":"2022-08-14T14:31:27.224676Z","iopub.status.idle":"2022-08-14T14:31:27.570706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## cabin variable\n\ndata_all <- data_all %>%\n separate(Cabin, c(\"Deck\", \"Num\", \"Side\"), \"/\") %>%\n mutate(across(where(is.character) | where(is.logical), as.factor)) %>%\n mutate(across(Num, as.integer)) %>%\n select(-\"PassengerId\")","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:27.574376Z","iopub.execute_input":"2022-08-14T14:31:27.576112Z","iopub.status.idle":"2022-08-14T14:31:28.155170Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glimpse(data_all)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:29.740438Z","iopub.execute_input":"2022-08-14T14:31:29.742116Z","iopub.status.idle":"2022-08-14T14:31:29.763909Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## Missing value imputation\ndata_all %>%\n select(everything()) %>%\n summarise_all(funs(sum(is.na(.)))) %>%\n t()\n\ndata_all <- data_all %>%\n mutate_if(is.numeric, function(x) ifelse(is.na(x), median(x, na.rm = T), x)) %>%\n impute_mode()\n\ndata_all %>%\n select(everything()) %>%\n summarise_all(funs(sum(is.na(.)))) %>%\n t()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:30.977592Z","iopub.execute_input":"2022-08-14T14:31:30.979109Z","iopub.status.idle":"2022-08-14T14:31:31.091790Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## Age group\ndata_all$Age[data_all$Age < 18] <- 1\ndata_all$Age[data_all$Age >= 18 & data_all$Age < 30] <- 2\ndata_all$Age[data_all$Age >= 30 & data_all$Age < 45] <- 3\ndata_all$Age[data_all$Age >= 45 & data_all$Age < 65] <- 4\ndata_all$Age[data_all$Age >= 65] <- 5\n\ndata_all$AgeGroup <- as.factor(data_all$Age)\ndata_all$Age <- age\n\ndata_all <- data_all %>%\n mutate_if(is.numeric, function(x) ifelse(is.na(x), median(x, na.rm = T), x)) %>%\n impute_mode()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:33.830814Z","iopub.execute_input":"2022-08-14T14:31:33.832471Z","iopub.status.idle":"2022-08-14T14:31:33.891123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glimpse(data_all)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:34.879300Z","iopub.execute_input":"2022-08-14T14:31:34.880851Z","iopub.status.idle":"2022-08-14T14:31:34.902279Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## Clustering\nspacekmc <- kmeans(x = data_all[9:13], centers = 5, algorithm = \"MacQueen\")\ndata_all$Cluster <- as.factor(spacekmc$cluster)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:37.496055Z","iopub.execute_input":"2022-08-14T14:31:37.497568Z","iopub.status.idle":"2022-08-14T14:31:37.520693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glimpse(data_all)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:31:38.128734Z","iopub.execute_input":"2022-08-14T14:31:38.130449Z","iopub.status.idle":"2022-08-14T14:31:38.152898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## log transformation\nRoomService_log <- log10(data_all$RoomService + 1)\nFoodCourt_log <- log10(data_all$FoodCourt + 1)\nShoppingMall_log <- log10(data_all$ShoppingMall + 1)\nSpa_log <- log10(data_all$Spa + 1)\nVRDeck_log <- log10(data_all$VRDeck + 1)\n\nhist(RoomService_log)\nhist(FoodCourt_log)\nhist(ShoppingMall_log)\nhist(Spa_log)\nhist(VRDeck_log)\n\ndata_all$RoomService <- RoomService_log\ndata_all$FoodCourt <- FoodCourt_log\ndata_all$ShoppingMall <- ShoppingMall_log\ndata_all$Spa <- Spa_log\ndata_all$VRDeck <- VRDeck_log","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:21.622102Z","iopub.execute_input":"2022-08-14T14:33:21.623986Z","iopub.status.idle":"2022-08-14T14:33:21.991034Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glimpse(data_all)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:23.703158Z","iopub.execute_input":"2022-08-14T14:33:23.704666Z","iopub.status.idle":"2022-08-14T14:33:23.726590Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# one hot encoding\ndummy_data <- dummyVars(\" ~ .\", data = data_all)\ndata_all <- as.data.frame(predict(dummy_data, data_all))","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:26.051999Z","iopub.execute_input":"2022-08-14T14:33:26.053674Z","iopub.status.idle":"2022-08-14T14:33:26.230960Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glimpse(data_all)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:29.590823Z","iopub.execute_input":"2022-08-14T14:33:29.592465Z","iopub.status.idle":"2022-08-14T14:33:29.621473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## Train and test data\ntrain_data <- data_all[data_all$type.Train == 1, ]\ntest_data <- data_all[data_all$type.Test == 1, ]\n\ntransported <- as.data.frame(transported)\n\ntrain_data <- train_data %>%\n select(-c(\"type.Train\", \"type.Test\"))\n\ntest_data <- test_data %>%\n select(-c(\"type.Train\", \"type.Test\"))\n\ntrain_data$Transported <- as.factor(transported$Transported)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:32.425786Z","iopub.execute_input":"2022-08-14T14:33:32.427323Z","iopub.status.idle":"2022-08-14T14:33:32.474906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glimpse(train_data)\nglimpse(test_data)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:34.771697Z","iopub.execute_input":"2022-08-14T14:33:34.773391Z","iopub.status.idle":"2022-08-14T14:33:34.821102Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## Feature Selection\n#select_feat <- Boruta(Transported ~., data = train_data)\n#select_feat$finalDecision","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:00:31.442649Z","iopub.execute_input":"2022-08-14T14:00:31.444199Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data <- train_data %>%\n select(-c(\"Deck.A\", \"Deck.T\", \"Deck.D\", \"VIP.FALSE\", \"VIP.TRUE\", \"AgeGroup.2\", \"AgeGroup.3\", \"AgeGroup.4\", \"AgeGroup.5\"))\n\ntest_data <- test_data %>%\n select(-c(\"Deck.A\", \"Deck.T\", \"Deck.D\", \"VIP.FALSE\", \"VIP.TRUE\", \"AgeGroup.2\", \"AgeGroup.3\", \"AgeGroup.4\", \"AgeGroup.5\"))","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:39.761248Z","iopub.execute_input":"2022-08-14T14:33:39.762716Z","iopub.status.idle":"2022-08-14T14:33:39.788840Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"glimpse(train_data)\nglimpse(test_data)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:42.045596Z","iopub.execute_input":"2022-08-14T14:33:42.047130Z","iopub.status.idle":"2022-08-14T14:33:42.088822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# **
Modelling
**","metadata":{}},{"cell_type":"code","source":"model_recipe <- \n recipe(Transported ~ ., data = train_data)\n\nspaceship_val <- validation_split(train_data, \n strata = Transported, \n prop = 0.80)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:46.964876Z","iopub.execute_input":"2022-08-14T14:33:46.966411Z","iopub.status.idle":"2022-08-14T14:33:47.013740Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"xgb_model <- boost_tree(mtry = tune(),\n min_n = tune(),\n tree_depth = tune(),\n learn_rate = tune(),\n loss_reduction = tune(),\n trees = 1000) %>% \n set_engine(\"xgboost\") %>% \n set_mode(\"classification\")\n\nxgb_wf <-\n workflow() %>%\n add_model(xgb_model) %>% \n add_recipe(model_recipe)\nxgb_wf","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:49.341536Z","iopub.execute_input":"2022-08-14T14:33:49.343010Z","iopub.status.idle":"2022-08-14T14:33:49.448186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Hyperparameter Tuning ","metadata":{}},{"cell_type":"code","source":"xgb_results <-\n xgb_wf %>% \n tune_grid(resamples = spaceship_val,\n grid = 25,\n control = control_grid(save_pred = TRUE),\n metrics = metric_set(accuracy)\n )","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:33:52.903118Z","iopub.execute_input":"2022-08-14T14:33:52.904699Z","iopub.status.idle":"2022-08-14T14:41:41.320419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"xgb_results %>% \n collect_predictions()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:41:54.598055Z","iopub.execute_input":"2022-08-14T14:41:54.602053Z","iopub.status.idle":"2022-08-14T14:41:55.010664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"xgb_results %>%\n collect_metrics()","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:42:01.298617Z","iopub.execute_input":"2022-08-14T14:42:01.300163Z","iopub.status.idle":"2022-08-14T14:42:01.375412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"xgb_results %>% \n show_best(metric = \"accuracy\")","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:42:04.983127Z","iopub.execute_input":"2022-08-14T14:42:04.985012Z","iopub.status.idle":"2022-08-14T14:42:05.053186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"param_final <- xgb_results %>%\n select_best(metric = \"accuracy\")\nparam_final","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:42:09.065073Z","iopub.execute_input":"2022-08-14T14:42:09.066581Z","iopub.status.idle":"2022-08-14T14:42:09.134349Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Final Hiperparameter and Last Fit ","metadata":{}},{"cell_type":"code","source":"\nxgb_model <- boost_tree(mtry = param_final$mtry,\n min_n = param_final$min_n,\n tree_depth = param_final$tree_depth,\n learn_rate = param_final$learn_rate,\n loss_reduction = param_final$loss_reduction,\n trees = 1000) %>% \n set_engine(\"xgboost\") %>% \n set_mode(\"classification\")\n\nlast_xgb_wf <- xgb_wf %>%\n update_model(xgb_model)\n\nlast_xgb_fit <- \n last_xgb_wf %>% \n parsnip::fit(train_data)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:42:19.619345Z","iopub.execute_input":"2022-08-14T14:42:19.620876Z","iopub.status.idle":"2022-08-14T14:42:49.360362Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n## Test Data Prediction ","metadata":{}},{"cell_type":"code","source":"test_pred <- predict(last_xgb_fit, test_data)\n\ntest_pred_new <- test_pred %>% \n mutate(.pred_class = str_to_title(.pred_class))\n\nsubmission$Transported <- test_pred_new$.pred_class","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:42:53.681617Z","iopub.execute_input":"2022-08-14T14:42:53.683180Z","iopub.status.idle":"2022-08-14T14:42:53.960439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# **
Submission
**","metadata":{}},{"cell_type":"code","source":"write_csv(submission, \"submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-08-14T14:42:58.888823Z","iopub.execute_input":"2022-08-14T14:42:58.890438Z","iopub.status.idle":"2022-08-14T14:42:58.926408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}