{"cells":[{"metadata":{"_uuid":"5b0b4c7427d5283b36c5e17c44552011b0db53a8"},"cell_type":"markdown","source":"# **1. INTRODUCTION ** \n\nPlayerUnknown's Battlegrounds (PUBG) is an online multiplayer battle royale game developed and published by PUBG Corporation, a subsidiary of South Korean video game company Bluehole. The game is based on previous mods that were created by Brendan \"PlayerUnknown\" Greene for other games using the film Battle Royale for inspiration, and expanded into a standalone game under Greene's creative direction. In the game, up to one hundred players parachute onto an island and scavenge for weapons and equipment to kill others while avoiding getting killed themselves. The available safe area of the game's map decreases in size over time, directing surviving players into tighter areas to force encounters. The last player or team standing wins the round.\n\nThe game was first released for Microsoft Windows via Steam's early access beta program in March 2017, with a full release on December 20, 2017. That same month, the game was released by Microsoft Studios for the Xbox One via its Xbox Game Preview program, and officially released in September 2018. The same year, a port for the PlayStation 4 was released, in addition to two different mobile versions for Android and iOS. The game is one of the best-selling of all time, with over fifty million sold across all platforms by June 2018. In addition, the Windows version holds a peak concurrent player count of over three million on Steam, which is an all-time high on the platform.\n\n# PROBLEM DESCRIPTION\n\nOver 65,000 games' worth of anonymized player data, split into training and testing sets, and asked to predict final placement from final in-game stats and initial player ratings. What's the best strategy to win in PUBG? Should you sit in one spot and hide your way into victory, or do you need to be the top shot? Let's let the data do the talking!"},{"metadata":{"_uuid":"8e007ae734141c717b34c25b04ddfd3854f2825d"},"cell_type":"markdown","source":"# **2. EXPLANATORY DATA ANALYSIS**"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"#Import Libraries\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sb\nimport gc\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.ensemble import RandomForestRegressor\nfrom sklearn.preprocessing import LabelEncoder, OneHotEncoder\nfrom mlxtend.evaluate import feature_importance_permutation\nfrom sklearn.model_selection import GridSearchCV\nimport gc\nimport os\nimport sys\n#Figures Inline and Visualization style\n%matplotlib inline\nsb.set()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/train_V2.csv')\ntest = pd.read_csv('../input/test_V2.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6b1ad6c463b1db251c25b07b48cf7028801fb9b1"},"cell_type":"markdown","source":"**Variables in the Dataset**\n* Id - Player’s Id\n* groupId - ID to identify a group within a match. If the same group of players plays in different matches, they will have a different groupId each time.\n* assists - Number of enemy players this player damaged that were killed by teammates.\n* boosts - Number of boost items used.\n* DBNOs - Number of enemy players knocked.\n* damageDealt - Total damage dealt. Note: Self inflicted damage is subtracted.\n* headshotKills - Number of enemy players killed with headshots.\n* heals - Number of healing items used.\n* killPlace - Ranking in match of number of enemy players killed.\n* killPoints - Kills-based external ranking of player. (Think of this as an Elo ranking where only kills matter.) If there is a value other than -1 in rankPoints, then any 0 in killPoints should be treated as a “None”.\n* killStreaks - Max number of enemy players killed in a short amount of time.\n* kills - Number of enemy players killed.\n* longestKill - Longest distance between player and player killed at time of death. This may be misleading, as downing a player and driving away may lead to a large longestKill stat.\n* matchDuration - Duration of match in seconds.\n* matchId - ID to identify match. There are no matches that are in both the training and testing set.\n* matchType - String identifying the game mode that the data comes from. The standard modes are “solo”, “duo”, “squad”, “solo-fpp”, “duo-fpp”, and “squad-fpp”; other modes are from events or custom matches.\n* rankPoints - Elo-like ranking of player. This ranking is inconsistent and is being deprecated in the API’s next version, so use with caution. Value of -1 takes place of “None”.\n* revives - Number of times this player revived teammates.\n* rideDistance - Total distance traveled in vehicles measured in meters.\n* roadKills - Number of kills while in a vehicle.\n* swimDistance - Total distance traveled by swimming measured in meters.\n* teamKills - Number of times this player killed a teammate.\n* vehicleDestroys - Number of vehicles destroyed.\n* walkDistance - Total distance traveled on foot measured in meters.\n* weaponsAcquired - Number of weapons picked up.\n* winPoints - Win-based external ranking of player. (Think of this as an Elo ranking where only winning matters.) If there is a value other than -1 in rankPoints, then any 0 in winPoints should be treated as a “None”.\n* numGroups - Number of groups we have data for in the match.\n* maxPlace - Worst placement we have data for in the match. This may not match with numGroups, as sometimes the data skips over placements.\n* winPlacePerc - The target of prediction. This is a percentile winning placement, where 1 corresponds to 1st place, and 0 corresponds to last place in the match. It is calculated off of maxPlace, not numGroups, so it is possible to have missing chunks in a match."},{"metadata":{"trusted":true,"scrolled":false,"_uuid":"24612b121f94a9a7584e4411b2dd16abed850666"},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"98bbe9b9c26752ef719c58045e17dd139d661905"},"cell_type":"code","source":"train['matchType'].value_counts().plot.bar(title='Match Type VS Count', figsize=(14,6))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3905550a5c96690b4de4a8697cb96c34a5528895"},"cell_type":"markdown","source":"**Take Away:**   Majority of players prefer squad  game followed by Duo. Solo game is the least preferred. "},{"metadata":{"trusted":true,"_uuid":"5a4347689fc428e8a026c5e17efd071059185359"},"cell_type":"code","source":"train.count()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c6e1e578b628d419faebb91de93b6f208f858094"},"cell_type":"markdown","source":"WinPlacePerc is the only column with missing value. Since the number of missing values is only 1 and the data is really big, simply delete the row."},{"metadata":{"trusted":true,"_uuid":"b77681a2a68c97be2260ec4103e81f45f3bc1121"},"cell_type":"code","source":"train.dropna(inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c5e9b8a41126dec15404e815035f5ff0fcb00345"},"cell_type":"code","source":"test.count()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0ad25407bd28bf2c3d42d0e8a982b57c308e2850","scrolled":true},"cell_type":"code","source":"train.describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2f2aa831df836bc01161d18b609239e04d19f27e"},"cell_type":"markdown","source":"**CORELATION MATRIX, HEAT MAP**"},{"metadata":{"trusted":true,"scrolled":false,"_uuid":"184945cfa3bbf67d692358d653e690b83cb188d5"},"cell_type":"code","source":"f,ax = plt.subplots(figsize=(20, 20))\nsb.heatmap(train.corr(), annot=True, linewidths=.5, fmt= '.1f',ax=ax, cmap=sb.color_palette(\"RdBu\", 20))\nax.set_xlabel('variables', size=14, color=\"black\")\nax.set_ylabel('variables', size=14, color=\"black\")\nax.set_title('Correlation Matrix', size=18, color=\"black\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e5c4c0f9ea7a595e78b944ba636816e9f66bdadf"},"cell_type":"markdown","source":"**Take Away** \n* WinPoints and RankPoints are the **most negatively corelated**\n* winPoints&killPoints, numGroups&maxPlace are the **most positively corelated variables**\n* walkDistance in the **most positively independent corelated** variable with the dependant variable winPlacePerc\n* killPlace is the **most negatively corelated independent** variable with the dependant variable winPlacePerc\n"},{"metadata":{"_uuid":"2d57f44a6b4c4c7c85d16ac7c96104f677fed4c6"},"cell_type":"markdown","source":"PUBG is all about Killing the opponents and surviving till the end. So let's  have a closer look at these variables."},{"metadata":{"trusted":true,"_uuid":"d0a6c70ae51fb6c3159681d72d23d9d63e0a4c97"},"cell_type":"code","source":"print(\"In the average, a person kills {:.4f} players, About 99% players have a kill count less than or equal  to {}, while the most kills ever recorded is {}.\".format(train['kills'].mean(),train['kills'].quantile(0.99), train['kills'].max()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"21ffc9073c5e951eebd54da4dbf38887b993f81a","scrolled":true},"cell_type":"code","source":"data = train.copy()\ndata.loc[data['kills'] > data['kills'].quantile(0.99)] = '8+'\nplt.figure(figsize=(15,8))\nsb.countplot(data['kills'].astype('str').sort_values())\nplt.title(\"Kill Count\",fontsize=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b0b748a6f4a1d7b42c5a17718ee2b1c17de8cc85"},"cell_type":"code","source":"data = train[['kills','winPlacePerc']].copy()\ndata.loc[data['kills'] > data['kills'].quantile(0.99), 'kills'] = '8+'\norder = data.groupby('kills').mean().reset_index()['kills']\nfig, ax = plt.subplots(figsize=(20,8))\na = sb.boxplot(x='kills', y='winPlacePerc', data=data, ax=ax, color=\"#2196F3\", order=order)\nax.set_xlabel('Kills', size=14, color=\"#263238\")\nax.set_ylabel('winPlacePerc', size=14, color=\"#263238\")\nax.set_title('Box Plot, Average winPlacePerc of Players VS kills', size=18, color=\"#263238\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"74fc196fcbbc919204227db9e5d772834e524f51"},"cell_type":"markdown","source":"**Take Away** Major Portion of the players in the train data  (Around 25lakh) finish the match without a single kill.  Around 9 lakh manages to get a kill. 99% Players have kills less than or equal to 7. Only 1% players managed to kill 8 or more than 8. The maximum kill recorded for a player is 72 in the train data.  Is it really possible to kill 72 out of maximum 100 players in a game? Let's have a look at the player data."},{"metadata":{"trusted":true,"_uuid":"e753d58bd0ca114b07988e001a1ad8c17ea28020","scrolled":false},"cell_type":"code","source":"train[train['kills']==72]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7206555b61bf191179174c96c6c730dde585fb75"},"cell_type":"markdown","source":"**Take Away ** The Player has 72 kills and 7 assists which means he himself has contributed to 79kills. His squad won the match. The Player has a damageDealt that is about 30x higher than the standard deviation, which means the player survived even after suffering heavy damage which is not possible in real case. 64 kills out of 72 are headShotKills. This opens two possibilities, He either used sniper for head shot kill from long distance or killed from point blank distance. The heavy damage that the player dealt indicates the second possibility.  Also the number of weapons acquired by the player is very high compared to the mean and standard deviation of the data. If the player was a sniper, Then he wouldn't have acquired this much weapons. So most likely the player has made some sort of cheating in the game. "},{"metadata":{"trusted":true,"_uuid":"8727d950c58c074677392539398483417e23fff2"},"cell_type":"code","source":"data = train[['kills','damageDealt']].copy()\ndata.loc[data['kills'] > data['kills'].quantile(0.99), 'kills'] = '8+'\nx_order = data.groupby('kills').mean().reset_index()['kills']\nfig, ax = plt.subplots(figsize=(20,8))\na = sb.boxplot(x='kills', y='damageDealt', data=data, ax=ax, color=\"#2196F3\", order=x_order)\nax.set_xlabel('Kills', size=14, color=\"#263238\")\nax.set_ylabel('damageDealt', size=14, color=\"#263238\")\nax.set_title('[Box Plot] Kills Vs Damage Dealt', size=18, color=\"#263238\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"894c38bf880497c764b3371ac41a542990d0d2a0"},"cell_type":"markdown","source":"**Take Away** kills increase proportionaly with damageDealt.  The relationship is understandable since more kills means more encounters with enemies and during these encounters, the player can receive a lot of damage.  The data contain many outliers. Most probably due to cheaters in the game.  DamageDealt can be used to detect hackers. Players with extremely high damageDealt might have did some sort of cheating in the game since they have very high winplaceprc and a bulk portion of these players won the game. "},{"metadata":{"trusted":true,"_uuid":"045c1e76affd1660a14c3509c152a2ef9a680f16","scrolled":true},"cell_type":"code","source":"data = train[['damageDealt','winPlacePerc']].copy()\n\nprint(\"99th percentile of damageDealt is {}\".format(data['damageDealt'].quantile(0.99)))\n\ncut = np.linspace(0,780,10)\ncut= np.append(cut, 6616)\n\ndata['damageDealtGrouping'] = pd.cut(data['damageDealt'],\n                                 cut,\n                                 labels=[\"{}-{}\".format(a, b) for a, b in zip(cut[:10], cut[1:])],\n                                 include_lowest=True\n                                )\n\nfig, ax = plt.subplots(figsize=(15,10))\nsb.boxplot(x=\"winPlacePerc\", y=\"damageDealtGrouping\", data=data, ax=ax, color=\"#2196F3\")\nax.set_xlabel('winPlacePerc', size=14, color=\"#263238\")\nax.set_ylabel('damageDealt Range Group', size=14, color=\"#263238\")\nax.set_title('Horizontal Box Plot, Win Place Percentile vs Damage Dealt', size=18, color=\"#263238\")\nplt.gca().xaxis.grid(True)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4e7453325c5fa8e4a333f45c8f2ac88746a0849c"},"cell_type":"markdown","source":"**Take Away** Players with high damageDealt have high winPlacePerc. Players who have survived till the end have higher chance for encounters and hence the damageDealt can be higher. "},{"metadata":{"trusted":true,"_uuid":"43ea5f129fa8a1d8538ad1a1820f49b05cd8db21"},"cell_type":"code","source":"data = train[['walkDistance','winPlacePerc']].copy()\n\nprint(\"99th percentile of walkDistance is {}\".format(data['walkDistance'].quantile(0.99)))\n\ncut = np.linspace(0,4400,10)\ncut= np.append(cut, 26000)\ndata['walkDistanceGrouping'] = pd.cut(data['walkDistance'],\n                                 cut,\n                                 labels=[\"{}-{}\".format(a, b) for a, b in zip(cut[:10], cut[1:])],\n                                 include_lowest=True\n                                )\n\nfig, ax = plt.subplots(figsize=(15,10))\nsb.boxplot(x=\"winPlacePerc\", y=\"walkDistanceGrouping\", data=data, ax=ax, color=\"#2196F3\")\nax.set_xlabel('winPlacePerc', size=14, color=\"#263238\")\nax.set_ylabel('walkDistance Range Group', size=14, color=\"#263238\")\nax.set_title('Horizontal Box Plot, Win Place Percentile vs Walk Distance', size=18, color=\"#263238\")\nplt.gca().xaxis.grid(True)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3ef9fa48cc5945f05e3bc11bce4579fceeaf387d"},"cell_type":"markdown","source":"**Take Away** Walk Distance definitely has positive corelation with WIn. As walk distance increases, Win Place also increases. "},{"metadata":{"trusted":true,"_uuid":"b01795a9730b1ec03907be770e1b065514df23a6","scrolled":true},"cell_type":"code","source":"data = train[['killPlace','winPlacePerc']].copy()\n\nprint(\"99th percentile of killPlace is {}\".format(data['killPlace'].quantile(0.99)))\n\ncut = np.linspace(0,101,10)\ndata['killPlaceGrouping'] = pd.cut(data['killPlace'],\n                                 cut,\n                                 labels=[\"{}-{}\".format(a, b) for a, b in zip(cut[:10], cut[1:])],\n                                 include_lowest=True\n                                )\n\nfig, ax = plt.subplots(figsize=(15,10))\nsb.boxplot(x=\"winPlacePerc\", y=\"killPlaceGrouping\", data=data, ax=ax, color=\"#2196F3\")\nax.set_xlabel('winPlacePerc', size=14, color=\"#263238\")\nax.set_ylabel('killPlace Range Group', size=14, color=\"#263238\")\nax.set_title('Horizontal Box Plot, Win Place Percentile vs killPlace', size=18, color=\"#263238\")\nplt.gca().xaxis.grid(True)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2011a2a5c1281a6798bebfa34486d80a295ace13"},"cell_type":"markdown","source":"**Take Away**  Increase in Killplace has a negative corelation with winplacePerc. "},{"metadata":{"trusted":true,"_uuid":"677bcba5d22abafe6bc62d4fa026bf97bb95b173","scrolled":true},"cell_type":"code","source":"data = train[['boosts','winPlacePerc']].copy()\ndata.loc[data['boosts'] >= 10, 'boosts'] = '10+'\norder = data.groupby('boosts').mean().reset_index()['boosts']\nfig, ax = plt.subplots(figsize=(20,8))\na = sb.boxplot(x='boosts', y='winPlacePerc', data=data, ax=ax, color=\"#2196F3\", order=order)\nax.set_xlabel('boosts', size=14, color=\"#263238\")\nax.set_ylabel('winPlacePerc', size=14, color=\"#263238\")\nax.set_title('Box Plot, Average winPlacePerc of Players VS boosts', size=18, color=\"#263238\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1b6c9eaf3ac01335e8dbb71a8d5eb7f988eae07a"},"cell_type":"markdown","source":"**Take Away** Players who used 10+ boosts has the highest win probability with a mean of .9. So players  should try to use more boosts inthe game as it will help increase damage Dealt and recover damage quicker.  "},{"metadata":{"trusted":true,"_uuid":"c736b41366ac69f2c7201986adbcae6f4f80bfc6"},"cell_type":"code","source":"data = train[['heals','winPlacePerc']].copy()\nprint(data['heals'].quantile(0.99))\ndata.loc[data['heals'] >= 12, 'heals'] = '12+'\norder = data.groupby('heals').mean().reset_index()['heals']\nfig, ax = plt.subplots(figsize=(20,8))\na = sb.boxplot(x='heals', y='winPlacePerc', data=data, ax=ax, color=\"#2196F3\", order=order)\nax.set_xlabel('heals', size=14, color=\"#263238\")\nax.set_ylabel('winPlacePerc', size=14, color=\"#263238\")\nax.set_title('Box Plot, Average winPlacePerc of Players VS heals', size=18, color=\"#263238\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bbb45a2ebb8717a34c869a9b501e3512144a7ecf"},"cell_type":"markdown","source":"**Take Away** Players who used 12+ heals has the highest win probability with a mean of about .9. These are players who survived for major portion of the game time and the probability for them to encounter enemies are higher. So chances for damage and knockdowns are higher which validates this relation as heals are used to recover damages. "},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"719dbf42f4c0be54edfd90499c35960fc32a8989"},"cell_type":"code","source":"data = train[['weaponsAcquired','winPlacePerc']].copy()\ndata.loc[data['weaponsAcquired'] >= 11, 'weaponsAcquired'] = '11+'\norder = data.groupby('weaponsAcquired').mean().reset_index()['weaponsAcquired']\nfig, ax = plt.subplots(figsize=(20,8))\na = sb.boxplot(x='weaponsAcquired', y='winPlacePerc', data=data, ax=ax, color=\"#2196F3\", order=order)\nax.set_xlabel('weaponsAcquired', size=14, color=\"#263238\")\nax.set_ylabel('winPlacePerc', size=14, color=\"#263238\")\nax.set_title('Box Plot, Average winPlacePerc of Players VS weaponsAcquired', size=18, color=\"#263238\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b1b9b9e59d4e6b5784d366d8a359ba443cc68031"},"cell_type":"markdown","source":"**Take Away**  Weapons are key to win this game. More the weapons a player has, the higher is the winPlacePerc till 10 weapons after which there is a slight dip."},{"metadata":{"trusted":true,"_uuid":"5aaef0d27270d9251fef61161ded6fb7a30025a5"},"cell_type":"code","source":"sb.set()\ncols = ['winPlacePerc', 'revives', 'swimDistance', 'numGroups', 'rankPoints', 'winPoints']\nsb.pairplot(train[cols], size = 2.5)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"74e7adfea67acf8f4c9d376c9b6009006550aeb1"},"cell_type":"markdown","source":"# 3.Base Model - Random Forest"},{"metadata":{"_uuid":"409ecdca915dfa659e92ee53baa2c3490bce8fcc"},"cell_type":"markdown","source":"Categorical Columns 'matchType' indicates that there are 3 type of game modes in PUBG\n\n<br>Solo : The player fights alone</br>\n<br>Duo : The player can pair with another player (Maximum one, Not compulsory)</br>\n<br>Squad : The player can pair up with atmost 3 other players.</br>\n<br>Categories like FPP, normal, TPP and other similar categories can be grouped together as follows</br>\n\nsolo <-- [solo,solo-fpp,normal-solo,normal-solo-fpp]\nduo <-- [duo,duo-fpp,normal-duo,normal-duo-fpp,crashfpp,crashtpp]\nsquad <-- [squad,squad-fpp,normal-squad,normal-squad-fpp,flarefpp,flaretpp]"},{"metadata":{"trusted":true,"_uuid":"8ef4d3036c01e5b8958158636c2916c9d686faf6"},"cell_type":"code","source":"train['matchType'].replace(['squad-fpp', 'normal-squad-fpp', 'normal-squad', 'flarefpp', 'flaretpp'], 'squad',inplace=True)\ntrain['matchType'].replace(['duo-fpp', 'normal-duo-fpp', 'normal-duo', 'crashfpp', 'crashtpp'], 'duo',inplace=True)\ntrain['matchType'].replace(['solo-fpp','normal-solo','normal-solo-fpp'], 'solo',inplace=True)\ntest['matchType'].replace(['squad-fpp', 'normal-squad-fpp', 'normal-squad', 'flarefpp', 'flaretpp'], 'squad',inplace=True)\ntest['matchType'].replace(['duo-fpp', 'normal-duo-fpp', 'normal-duo', 'crashfpp', 'crashtpp'], 'duo',inplace=True)\ntest['matchType'].replace(['solo-fpp','normal-solo','normal-solo-fpp'], 'solo',inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ffe3b615d6fd03a94247127231da6332fe34fbc6"},"cell_type":"markdown","source":"Since the given dataset is really huge, A subset is made with 2 lakh rows for training and is divided as Independent and dependent variables."},{"metadata":{"trusted":true,"_uuid":"02e70d3ecf76c10cb4119abc609011d81e7f3a3d"},"cell_type":"code","source":"train.drop(['Id', 'groupId', 'matchId'], axis=1, inplace = True)\nx = train.drop(['winPlacePerc'], axis=1).sample(180000).values\ny = train['winPlacePerc'].sample(180000).values","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"926c0c0a12e9470b07eaabea5ca332a6cca03874"},"cell_type":"markdown","source":"The dataset contains a categorical variable 'matchType' in it. Therefore, categorical encoding is performed"},{"metadata":{"trusted":true,"_uuid":"d6146f9aa42c3f269905fd39251717c774ddb79d"},"cell_type":"code","source":"x_linear = LabelEncoder()\nx[:, 12] = x_linear.fit_transform(x[:, 12])\nhot_enc = OneHotEncoder(categorical_features=[12])\nx = hot_enc.fit_transform(x).toarray()\n# To avoid Dummy trap\nx = x[:, 1:]\n# Split the data into train and test set\ntrain_x, test_x, train_y, test_y = train_test_split(x, y, test_size = .2, random_state = 0)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c04ff64c99bb27e8a4f008c7919a8c8e801e91ae"},"cell_type":"markdown","source":"The train set is fitted on to the Random Forest Regressor and the hyper parameters are tuned using GridSearchCV. The grid search code section is removed here to make the final commit faster. K Fold cross validation retured a mean absolute error of 0.06464 which is approximately 6.4%."},{"metadata":{"trusted":true,"_uuid":"f7c496292e3fccd7957f6aca3e004a0009fd7210"},"cell_type":"code","source":"regressor = RandomForestRegressor(n_estimators = 1000,\n                                  min_samples_split=5,\n                                  max_depth = 19,\n                                  random_state=101,\n                                  n_jobs = -1)\nregressor.fit(train_x,train_y)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"592140086bd745bcaa763c53fdf415c5db0665c7"},"cell_type":"markdown","source":"**Feature Importance for the Random Forest Model**"},{"metadata":{"_uuid":"2377dce4364f67c41efbabc4204b25af9cbbb38f"},"cell_type":"markdown","source":"Permutaition Importance test is performed on the random forest regression model and the result is interpreted as a histogram below. 'killPoints' is the most important variable in the model followed by 'walkDistance'. headShotKils, roadKills, vehiclesDestroyed have relatively no importance in the model's win prediction"},{"metadata":{"trusted":true,"_uuid":"4ed4750d9b1c71cd71694fd81748091986f18635"},"cell_type":"code","source":"imp_vals,_ = feature_importance_permutation(predict_method=regressor.predict,\n                                          X=test_x,\n                                          y=test_y, metric='r2',\n                                          num_rounds=1)\nfeature_importance = pd.DataFrame(imp_vals, index=['solo','squad','assists','boosts','damageDealt',\n                                  'DBNOs','headshotKills','heals','killPlace',\n                                  'killPoints','kills','killStreaks',\n                                  'longestKill','matchDuration','maxPlace',\n                                  'numGroups','rankPoints','revives',\n                                  'rideDistance','roadKills','swimDistance',\n                                  'teamKills','vehicleDestroys','walkDistance',\n                                  'weaponAcquired','winPoints'])\nfeature_importance.plot(kind='bar', figsize=(15,6),\n                       title='Features & Their Importance in Win Prediction')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2b790eccbfbe13e62c2f6ea2b28cb4bf30a3445a"},"cell_type":"markdown","source":"Removing non relevent columns(Cross Validation is performed with scoring MAE and adjusted R2 after removing each columns to find those columns that improve the model). However, the code is removed to prevent memory error and make commit faster. Refitting regressor,"},{"metadata":{"trusted":true,"_uuid":"82d553f0a2800a766d63dac3ffb44352a35f7aff"},"cell_type":"code","source":"x = train.drop(['headshotKills',\n                      'revives','swimDistance','teamKills','winPlacePerc'], axis=1).head(150000).values\ny = train['winPlacePerc'].head(150000).values\nx_linear = LabelEncoder()\nx[:, 11] = x_linear.fit_transform(x[:, 11])\nhot_enc = OneHotEncoder(categorical_features=[11])\nx = hot_enc.fit_transform(x).toarray()\nx_test = test.drop(['Id', 'groupId', 'matchId','headshotKills',\n                      'revives','swimDistance','teamKills'], axis=1).values\nx_linear_new = LabelEncoder()\nx_test[:, 11] = x_linear_new.fit_transform(x_test[:, 11])\nhot_enc_new = OneHotEncoder(categorical_features=[11])\nx_test = hot_enc_new.fit_transform(x_test).toarray()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3f72f5d2cd43e46616ae7ac6946eada9cd2f69ba"},"cell_type":"code","source":"regressor = RandomForestRegressor(n_estimators = 1000,\n                                  min_samples_split=5,\n                                  max_depth = 19,\n                                  random_state=101,\n                                  n_jobs = -1)\n\nregressor.fit(x,y)\ny_test_pred=regressor.predict(x_test)\nsubmissions = test['Id']\nsubmissions=pd.DataFrame(submissions)\nsubmissions['winPlacePerc']=pd.DataFrame(y_test_pred)\nsubmissions.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a7793de2b6c552f9c1f8359bf1357c3842a31a82"},"cell_type":"markdown","source":"# **The hyper parameter tuned and feature reduced base model returned an improved score of 0.0608.**\n\n# **Due to memory and time constraints in Kaggle kernels, the complete model is divided in to two sections. **"},{"metadata":{"_uuid":"a9abe87734b557715dcbfa1594d511a2b977f6a7"},"cell_type":"markdown","source":"# **For the second section containing Feature Engineering and LightGBM model with a much  better prediction, <a href=\"https://www.kaggle.com/iamarjunchandra/pubg-feature-engineering-lightgbm-explained\">Click Here!</a>**"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}