{"cells":[{"metadata":{"trusted":true,"_uuid":"12f519d145f90ccdfcc3f90029734dc7c09b2b68"},"cell_type":"markdown","source":"# NFL Punt Analytics Competition - Starter EDA\n\nIn this competition we are tasked with analzying punt plays for player safety and proposing rule changes."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pylab as plt\nimport os\nimport seaborn as sns\nsns.set_style('whitegrid')\n\n# For interactive plots\nfrom plotly import offline\nimport plotly.graph_objs as go\n\n\npd.set_option('max.columns', None)\noffline.init_notebook_mode()\nconfig = dict(showLink=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3c7259c74843ca0d1365a1e2a9503d74cc1861bf"},"cell_type":"code","source":"# Read the input data\nppd = pd.read_csv('../input/player_punt_data.csv')\ngd = pd.read_csv('../input/game_data.csv')\npprd = pd.read_csv('../input/play_player_role_data.csv')\nvr = pd.read_csv('../input/video_review.csv')\nvfi = pd.read_csv('../input/video_footage-injury.csv')\npi = pd.read_csv('../input/play_information.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5fc89f7807b7bc6624f00d15810ec3f33ca440b2"},"cell_type":"code","source":"# I want to create another variable that is concussive_games so that I can leave vr for later\n# I'm going to build on this variable\nconcussive_games = vr\nconcussive_games.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"926ba7f4d7f92c49caa0f2ed3a26db7cb34aa5eb"},"cell_type":"markdown","source":"There are only 37 games with a concussion but a lot of extra data. But we need to add more data regarding the games"},{"metadata":{"trusted":true,"_uuid":"1f15f9a9d940acb3f28acd5a6dc6cef86e2632a3"},"cell_type":"code","source":"# Here I'm going to merge games where someone got a concussion with the game data\n# game data has a lot more information: \n# Game level data that specifies the type of season (pre, reg, post), week, and hosting city and team. \n# Each game is uniquely identified across all seasons using GameKey.\nconGD = pd.merge(concussive_games, gd)\nconGD.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"28ae9f9d70abe4d40519fcbdf530034022f1a98e"},"cell_type":"code","source":"# lets verify we still have only 37 games to deal with\nconGD.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8e89ace66ddcc4f89a67f77fde26193af2310aa7"},"cell_type":"code","source":"#lets look at concussions by season year\nsns.factorplot('Season_Year',data=conGD,kind='count')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"39721034582093783abca8d7916180919c2885d9"},"cell_type":"code","source":"# lets look at the season year but by season type: preseason, regular season and post season\nsns.factorplot('Season_Year',data=conGD,hue='Season_Type', kind='count')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0fde701f9c4c8a256f0d03e2088cd4b9fb50836b"},"cell_type":"markdown","source":"There are no Post Season concussions during punts in 2016 or 2017. "},{"metadata":{"trusted":true,"_uuid":"1c28674aba4c1bac4bc2dd32eaf1b23a74202d66"},"cell_type":"code","source":"# let's look play information\n# this gives specific detail to the play\npi.tail()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ad96bb3fd69e4875de5f9bd046beee8e27d47978"},"cell_type":"code","source":"pi.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"230a63ab7c31149bdb1efccc6530b07bca30ac08"},"cell_type":"code","source":"# lets look at the punts by season year and season type: preseason, regular season and post season\nsns.factorplot('Season_Year',data=pi,hue='Season_Type', kind='count')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d2eb0bc79bb4711d3e20a5300842a5fcd559d0fa"},"cell_type":"markdown","source":"Interestingly, there were more punts in 2017 but yet there were less concussions."},{"metadata":{"trusted":true,"_uuid":"0d3c090a734856a3811da3e6dbd51a3218e6cf8b"},"cell_type":"code","source":"# after testing different combinations I found that there is duplicate GSISID data\n# however the player number was different so I'm looking at it here. \nppd.head(20).sort_values('GSISID')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cb22af11709fce3deff9cfd615ec145715b49dc2"},"cell_type":"code","source":"# here I want see what is duplicated\nppd[ppd.duplicated(['GSISID']) == True].sort_values('GSISID')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c7ad50869b3d8857ef9b08c677141822d48d69d5"},"cell_type":"code","source":"# FIND 33941 to see why it is duplicating when brought into concussive plays\nppd.loc[ppd['GSISID'] == 33941]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"45242af7218a857b0e3ab8e3ac02f9d60ac7ccdc"},"cell_type":"markdown","source":"What is going on? There are 817 rows of duplicated players. \nA closer look at the data, some players have letters in their numbers.\nBut in the case of 33941 they have two different numbers. "},{"metadata":{"trusted":true,"_uuid":"5401cee9067543c7c167f4eb50a070beec11de8d"},"cell_type":"code","source":"#remove the letters from the Numbers column\nimport re\nfor Number in ppd:\n    ppd['Number'] = [''.join(re.findall(\"\\d*\\.?\\d+\", item)) for item in ppd['Number']]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4ee5e22e11d8a897c7c9614876445d8e95d4a7cc"},"cell_type":"code","source":"# Lets see if we removed the characters from the Numbers column\nppd.loc[ppd['GSISID'] == 33941]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ae8d4cf5d5ce9b22eae0409f0cf39962ec668b9a"},"cell_type":"code","source":"# here I want see what is duplicated after cleanup\nppd[ppd.duplicated(['GSISID']) == True]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dfcff6b88ce977c761cd41bf839adce86eb797c6"},"cell_type":"markdown","source":"Great. But now we need to merge duplicated data so we can focus on solving"},{"metadata":{"trusted":true,"_uuid":"46bd2e52b2b47a11450faa78b62ebb58539e6d5a"},"cell_type":"code","source":"#There are still issues. Such as a 0 before 3, we need to remove that. \nppd['Number'] = ppd['Number'].str.replace('0','')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c785ad68a5d12f2fc8760da8c22e7f6ab2d6944e"},"cell_type":"code","source":"# Lets see if we removed the characters from the Numbers column\nppd.loc[ppd['GSISID'] == 33941]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f7a1492f13c03370fe0084c272aec1892ebb8ce3"},"cell_type":"code","source":"# After all that cleanup it is clear that numbrer is meaningless so lets remove the jersey number\nppdDrop = ppd.drop('Number', 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9b8c212742d406a6d5888d197f8a8006cc610195"},"cell_type":"code","source":"ppdDrop.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"64204aec9d8c1fa87dd9e090a61889f8dd922454"},"cell_type":"code","source":"# merge games where someone got a concussion with the game data\n# and merged the player data in hopes that we still get 38\nconPlayer = pd.merge(conGD, ppdDrop)\nconPlayer.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1f7a7365e444750de4333c2e4c7ce2b791e25b55"},"cell_type":"code","source":"conPlayer[conPlayer.duplicated(['PlayID','GSISID']) == True]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a54f127b5bfb6b58930913293db3d24615707994"},"cell_type":"markdown","source":"We have 57 duplications. What I need to do is remove the duplicates that are the same playID because there cannot be duplicate Plays."},{"metadata":{"trusted":true,"_uuid":"e9c250b0ccfd2551fe9e2bd2dde4e3f2ae0aadca"},"cell_type":"code","source":"conPlayer = conPlayer.dropna(subset=['PlayID']).drop_duplicates(['PlayID','GSISID'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2c542373dd9b28537c2eeab1e6b96582f3cf5a5d"},"cell_type":"code","source":"conPlayer.describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1e73846d871f9fd559669df25a2f84e972f8c7aa"},"cell_type":"markdown","source":"In my cleanup I found that there was one occasion where the same play had two players concussed. "},{"metadata":{"trusted":true,"_uuid":"9daaf59a48b912e9fa4c7b6fc29816345be50625"},"cell_type":"code","source":"sns.factorplot('Position',data=conPlayer,kind='count')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bdcc96687100c8b1417aff8b501dcfd556557689"},"cell_type":"markdown","source":"The numbers are small, so there doesn't seem to be a definitive position that gets injured.\nIt would be interesting to see if these players were only special teams players or if they played fulltime. \nFor example. Was the player just on defense?\nReason: Are they cold, as in not warmed up and coming right off the bench to perform\n\n\n"},{"metadata":{"trusted":true,"_uuid":"a1d3f78a5475a320477cfd6ba6568a4517d192b9"},"cell_type":"code","source":"sns.catplot(data=conPlayer, x=\"Position\",col=\"Player_Activity_Derived\", kind=\"count\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f905cc341023a585304169d40bc8b0aa17302591"},"cell_type":"markdown","source":"Player_Activity_Derived\t- player activity during primary injury causing event\nBlocked\tplayer was blocked\t\n\tBlocking\tplayer was blocking\t\n\tTackled\tplayer was tackled\t\n\tTackling\tplayer was tackling\t\n\tDiving/Leaping\tplayer was diving or leaping\t\n\tOther\tother activity"},{"metadata":{"trusted":true,"_uuid":"d5a799c040e3829c229c8e12de78b3db1d2017d8"},"cell_type":"code","source":"sns.catplot('Primary_Partner_Activity_Derived',data=conPlayer,kind='count')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bbec04059ca6f04b35468f98f620f59f992c08da"},"cell_type":"markdown","source":"Primary_Partner_Activity_Derived\t-categorical variable describing primary partner’s activity at the time of causing the concussion\nBlocked\tpartner was blocked\t\n\tBlocking\tpartner was blocking\t\n\tTackled\tpartner was tackled\t\n\tTackling\tpartner was tackling\t\n\tDiving/Leaping\tpartner was diving or leaping\t\n\tOther\tother activity"},{"metadata":{"trusted":true,"_uuid":"4620bbdfb4a94e25d9efdb2197cdd702d09f3935"},"cell_type":"code","source":"sns.catplot(data=conPlayer, x=\"Position\",col=\"Primary_Partner_Activity_Derived\", kind=\"count\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0d1958eade96c469061506cee1f0a310c0156bb5"},"cell_type":"markdown","source":"Player GSISID vs. primary partner GSISID\n- Can I see type of position of each?\nGSISID vs. Primary_Partner_GSISID\nWith Primary_Partner_Activity_Derived\n(Position type held in player_punt_data)\n"},{"metadata":{"trusted":true,"_uuid":"efaaad16217599acab580473a6d15041efd6f44d"},"cell_type":"code","source":"#conPlayer has primary partner GSISID\n# ppdDrop has the player positions\nconPlayer = conPlayer.rename(columns={'Position': 'Concussed_Position'})\nconPlayer.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ef2f26f2ad12f456c610d6babd222bb95c9aee89"},"cell_type":"code","source":"#NAme Position - Concussed Position\n#Name Position - Enemy Position\nconPlayer1 = conPlayer.rename(columns={'GSISID': 'Concussed_GSISID'})\nconPlayer1.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2c7baa2a32ad9606e3d5c43b1f3a13731d85c216"},"cell_type":"code","source":"conPlayer2 = conPlayer1.rename(columns={'Primary_Partner_GSISID': 'GSISID'})\nconPlayer2.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8786467f541835d648f9eb8689eb865a1c5f77fb"},"cell_type":"code","source":"# dropping NAN from GSISID because some or NAN becuase the ground can cause a concussion\nconPlayer2['GSISID'] = pd.to_numeric(conPlayer2['GSISID'], errors='coerce')\nconPlayer2 = conPlayer2.dropna(subset=['GSISID'])\nconPlayer2['GSISID']=conPlayer2['GSISID'].apply(int)\nconPlayer2.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8c7916291b4c1cee1e1ca2f5bad7c56594ea146f"},"cell_type":"code","source":"conPlayer2.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6c7055ce78cf50d03cb5b612e1fa3caf8cea3fb4"},"cell_type":"code","source":"# merged enemy player with their ID and position. They caused the concussion\nconPlayer3 = pd.merge(conPlayer2, ppdDrop)\nconPlayer3.head(20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1aa637631651860a917d98d90cb50ebf04ec5ca1"},"cell_type":"code","source":"conPlayer3.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e7d36e0bda57366c5aaf9963881a0783d0958812"},"cell_type":"code","source":"conPlayer4 = conPlayer3.dropna(subset=['PlayID']).drop_duplicates(['PlayID','Concussed_GSISID','GSISID'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"63404971d6c23e7f995c24beb7eb2481fae17f6e"},"cell_type":"code","source":"conPlayer4.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7ea81c53fc9a6bd726b4930d67257ccbc39a3adb"},"cell_type":"code","source":"# convert GSISID to Enemy_GSISID\nconPlayer5 = conPlayer4.rename(columns={'GSISID': 'Enemy_GSISID'})\nconPlayer5.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"790e659d293f396f8604c232d5f68aff71cd01d6"},"cell_type":"code","source":"# convert Position to Enemy_Position and drop number\nconPlayer5 = conPlayer4.rename(columns={'Position': 'Enemy_Position'})\n#conPlayer5 = conPlayer4.drop('Number', 1)\nconPlayer5.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e1428e42dae80813d4b5f2f3d9153d0e3734f5f5"},"cell_type":"code","source":"conPlayer5.describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"617aff12a12ada3e9e937d1c7a18875c261de2d8"},"cell_type":"markdown","source":"View Concussed Position vs. Enemy Position\n"},{"metadata":{"trusted":true,"_uuid":"1b83c087a094720327fe4027ff0458d144ad6412"},"cell_type":"code","source":"sns.catplot(data=conPlayer5, x=\"Concussed_Position\",col=\"Enemy_Position\", kind=\"count\", col_wrap=4 )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f9ce5f1fcf6818dc1d7aef3337ee583afec3cb5c"},"cell_type":"code","source":"sns.catplot(data=conPlayer5, x=\"Primary_Partner_Activity_Derived\",col=\"Enemy_Position\", kind=\"count\", col_wrap=4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1587a2b301b81e6682aaf897f67edde848a64d8a"},"cell_type":"code","source":"sns.catplot(\"Primary_Impact_Type\", data=conPlayer5,kind=\"count\")\n#sns.catplot('Primary_Partner_Activity_Derived',data=conPlayer,kind='count')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4828ff5990acd076e9032ba47fa23cd5fb882ee2"},"cell_type":"markdown","source":"The conversion from 1 yard/second to miles/hour is 1 * 2.04545. Since each row is a tenth of a second, we need to also multiply by 10.\n\nThe main fuction below reads in one of the csv's and filters for one of the specific punt plays were an injury occured. It then converts the distance field to MPH and returns the max and mean speeds for the player injured [1st row] and primary partner [2nd row]"},{"metadata":{"trusted":true,"_uuid":"e706b8cb654e82b15449836cd09b773c6809d703"},"cell_type":"code","source":"conPlayer5.head()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}