{"cells":[{"metadata":{"_uuid":"373b7564d0a59c49978a1f5ed21ba80cdfb74e8c"},"cell_type":"markdown","source":"**The Punt Play in Football**\n\nIn football, the punt is an **offensive** play where an offensive player kicks the ball down the field. The **defending** team punt returner can either call a fair catch, catch the ball and attempt to run it up field, or let it bounce and not touch it. If the punt returner calls fair catch, the **returning team** takes over possession of the ball where the ball is caught. Otherwise, the **returning team** takes over possession where either the returner is tackled, or the ball is first touched by the **kicking team**. The goal of punting is to gain a field position advantage over the other team, hence the returning team has an incentive to do the action that best gives them a chance to get the ball toward their goal, and the kicking team has an incentive to get to the returner/ball as fast as possible to prevent them from moving forward. Good punting is an essential part of a winning a football game. \n\nA normal punt play looks like the following:\n"},{"metadata":{"_uuid":"e3be6aa424e75408ce100ee9fa09db267ca37d46"},"cell_type":"markdown","source":"**Common Punting Practices and Rules**\n\nAccording to NFL rule 9-1-2: Only the end men or an eligible receiver aligned behind the line of scrimmage is permitted to advance one yard beyond the line of scrimmage before the ball is kicked. So in the image above, only GR, GL, PRW, PLW, and PPR would be eligble to run down the field before the ball is kicked. Hence, in order to maximize the chances of getting to the ball, the punting team usually has two players GR and GL immedietly run toward the PR at full speed to attempt to down the ball. PLW, PRW, and PPR are also allowed down the field, but ocasionally they may perform a chip block on one of the end men before running down field at full speed. PDR1, PDR2, PDR3, PDL3, PDL2, and PDL1 usually line up in the gaps created by the offensive linemen (PLT, PLG, PLS, PRG, and PRT) in order to try and rush through to block the kick. The offensive linemen block the defensive linemen from getting to the punter and then make their way downfield to tackle the returner/down the ball. PLR usually drops back in order to block a player rushing downfield to down the ball. VL(i,o) and VR(i,o) are players designated to specifically block the GL and the GR. They perform blocks to prevent the gunner from reaching the PR at full speed. They usually run with the gunner and attempt to force them toward the sidelines so that they have a lesser angle toward the ball. \n\nAccording to NFL rule 9-1-3, a defensive team (returning team) player who is within one yard of the line of scrimmage must lineup outside the shoulders of the snapper. This is to protect the long snapper who has his head down completely in order to snap the 15 yards to the punter. This is the only restriction on the defense when it comes to formations, and the offense has no offensive restrictions other than NFL rule 7-5-1(a) where the offensive team must have 7 players on the line of scrimmage. The NFL doesn't place too many restrictions on formations and we believe that our rule change should maintain the integrity of this principle. \n\nOn an aside, members of the special teams/punt unit are usually not players from normal offensive personnel. Rather, the \"offensive linemen\" on a punt team are usually comprised of players smaller than true offensive linemen (so that they can run downfield fast to down the ball) but large enough to lay blocks on defensive linemen. These players are usually linebackers and defensive backs (as we see in the video review, the numbers 30-59 are incredibly common on punt downs for the offense). We believe that this is a strategy used to 1) prevent injuries to offensive and defensive starters, and 2) so that the players on the punt coverage are more athletic. This is important because players on the field during punt downs are all usually very fast and very strong meaning collisions happen with huge momentum. \n\nIt is important to note that punting is an offensive down, and **generally** happens on 4th down. Teams generally use the first 3 downs to try and make the yard to gain, and use the 4th down in order to kick the ball away and force worse field position for opposition. While this is the case, an incredibly important feature of football is misdirection, and teams may decide to perform a **fake** punt or even punt on early downs to catch the opposition off guard. \n\n''\n\nThese kind of fake punt plays are very successful sometimes because the opposition is not ready for it or the opposition did not expect it. It is a major part of football and its essential that we do not make significant rule changes that make punting a *non offensive play* so that the excitement and strategy of fake punts is not removed. Hence it is vital we do not make restrictive formation requirements or remove the punt completely. We will attempt to use the data provided in order to create a rule that reduces potential concussions on punting while preserving as much of the excitement, strategy, and appeal a punt play can potentially provide. "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nprint(os.listdir(\"../input\"))\n","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"**Basic Experimental Data Analysis**"},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"46b49e4ebf07104c929a50880f9433c025c6d731"},"cell_type":"code","source":"# Loading video_review.csv data\nvr = pd.read_csv('../input/video_review.csv')\nvr.head(10)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ec050f220a79ad0d16851aa1757e214501e771b2"},"cell_type":"markdown","source":"**Histogram for type of impact in our 37 concussion causing plays.**\n\nHelmet-to-helmet hits are illegal in NFL plays, yet we see that concussions still happen from these hits. "},{"metadata":{"trusted":true,"_uuid":"5a4367df6cc3872d81284f436f9ea56b21fdc643"},"cell_type":"code","source":"primary_impact_type_hist = vr['Primary_Impact_Type'].value_counts().plot(kind='bar')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ec457f940f6f23fbfa9ec2aae95e59ba7c8ff9b4"},"cell_type":"markdown","source":"**Most common type of activity of players at the time of receiving concussion.**\n\nAccording to the data, tackling is the most common method of receiving a concussion."},{"metadata":{"trusted":true,"_uuid":"f3166264ec0b13415481777e5a83124ad88ba1c3"},"cell_type":"code","source":"player_activity_derived_hist = vr['Player_Activity_Derived'].value_counts().plot(kind='bar')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"35e3cfed542073b72633be98cc96dd8483eb3707"},"cell_type":"markdown","source":"**Most common type of activity of partner of players at the time of causing concussion.**\n\nInterestingly, the most common activity of the person giving a concussion is being blocked and tackling."},{"metadata":{"trusted":true,"_uuid":"958c64d8f017519732e16bb842d8c1b983e8088f"},"cell_type":"code","source":"primary_partner_activity_derived = vr['Primary_Partner_Activity_Derived'].value_counts().plot(kind='bar')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f04962b6dca595042046517458ef56d1c327f9e8"},"cell_type":"markdown","source":"**Retrieving Punt Play Formations**\n\nThe following two sections present exact formations of all concussion plays. We will refer back to these formations later on."},{"metadata":{"trusted":true,"_uuid":"e2a1116bc5a7a77bd17441ed76dc86615abebb8b","_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# Create play_player_role_data dataframe.\npprd = pd.read_csv('../input/play_player_role_data.csv')\n# Create play_information dataframe.\nplay_inf = pd.read_csv('../input/play_information.csv')\n# Create game_inf dataframe.\ngame_inf = pd.read_csv('../input/game_data.csv')\n# Create video_review dataframe\nvr = pd.read_csv('../input/video_review.csv')\n\n# Extract roles column from play_player_role_data.csv and check if player is on punting team.\npunting_pos = {\"PLS\",\"PLG\", \"PLT\", \"PLW\",\"PRG\",\"PRT\",\"PRW\",\"PC\",\"PPR\", \"PPL\",\"P\",\"GL\",\"GR\"}\nrole_col = pprd['Role']\npunting_team_col = [x in punting_pos for x in role_col.tolist()]\n\n# Adding punting_team_col to pprd dataframe.\npprd['punting_team'] = punting_team_col\n\n# Function that merges GameKey and PlayID given a row.\ndef merge(row):\n return str(row['GameKey']) + '_' + str(row['PlayID'])\n\n# Combine GameKey and PlayID to make unique Key that organizes formations.\npprd_merged_col = pprd.apply(lambda row: merge(row), axis=1)\nplay_inf_merged_col = play_inf.apply(lambda row: merge(row), axis=1)\nvr_merged_col = vr.apply(lambda row: merge(row), axis=1)\n\n# Adds merged_col to pprd dataframe.\npprd['game_play_key'] = pprd_merged_col\nplay_inf['game_play_key'] = play_inf_merged_col\nvr['game_play_key'] = vr_merged_col\n\n# Groupby game_play_key and punting_team.\npprd_formations = pprd.groupby(['game_play_key','punting_team']).apply(lambda x: sorted(x['Role'].values.tolist()))\npprd_formations = pprd_formations.rename(\"form\")\npunting_team_boolean = [x[0] in punting_pos for x in pprd_formations.tolist()]\npprd_formations = pprd_formations.apply(lambda x: ', '.join(x)).to_frame()\npprd_formations['punting_team'] = punting_team_boolean\n# Sorted formations\n#print(pprd_formations.keys())\npprd_and_playinf = play_inf.merge(pprd_formations, on=['game_play_key'], how='outer')\npprd_and_playinf = pprd_and_playinf.drop(columns=['Season_Year', 'GameKey', 'PlayID'])\n\n# Merge play_information.csv with play_player_role_data.csv by game_play_key.\nmerged = pprd_and_playinf.merge(vr, on=['game_play_key'], how='outer')\nreturnTeamForms = merged[merged['punting_team'] == False]['form']\n\nconcuss = merged[merged['Turnover_Related'].notnull()]\nconcussReturnTeamsForms = concuss[concuss['punting_team'] == False]['form']\n\ncount_formations_punting = concuss.loc[merged['punting_team'] == False, 'form']\ncount_formations_recv = concuss.loc[merged['punting_team'] == True, 'form']\n# print(count_formations_punting)\na = count_formations_punting.value_counts()\nb = count_formations_recv.value_counts()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9d8385c51eecd9a2504c6a6575d8679aa32d0112"},"cell_type":"code","source":"print(\"Types of formations for punting team, counts\")\na.head(7)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"81324c89e889a3c3470244cbc25fa2904162530c"},"cell_type":"code","source":"print(\"Types of formations for receiving team, counts\")\nb.head(7)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"5ee800bdfdc0a9a8c0194147f861ca3165ded110"},"cell_type":"code","source":"\nimport time\nstart = time.time()\nNGSDICT = {\n \"pre-2016\" : pd.read_csv('../input/NGS-2016-pre.csv'),\n \"pre-2017\" : pd.read_csv('../input/NGS-2017-pre.csv'),\n \"post-2016\" : pd.read_csv('../input/NGS-2016-post.csv'),\n \"post-2017\" : pd.read_csv('../input/NGS-2017-post.csv'),\n \n \"reg-2016-wk1-6\" : pd.read_csv('../input/NGS-2016-reg-wk1-6.csv'),\n \"reg-2016-wk7-12\" : pd.read_csv('../input/NGS-2016-reg-wk7-12.csv'),\n \"reg-2016-wk13-17\" : pd.read_csv('../input/NGS-2016-reg-wk13-17.csv'),\n \n \"reg-2017-wk1-6\" : pd.read_csv('../input/NGS-2017-reg-wk1-6.csv'),\n \"reg-2017-wk7-12\" : pd.read_csv('../input/NGS-2017-reg-wk7-12.csv'),\n \"reg-2017-wk13-17\" : pd.read_csv('../input/NGS-2017-reg-wk13-17.csv')\n}\nend = time.time()\nprint(\"Time to load:\")\nprint(end- start)\n\n\ndef getNGSName(season,week,season_type):\n if season_type == 'Pre':\n return 'pre-' + str(season)\n elif season_type == 'Post':\n return 'post-' + str(season) \n else:\n if(week <= 6):\n return 'reg-' + str(season)+ '-wk1-6'\n if(week >= 7 and week <=12):\n return 'reg-' + str(season)+ '-wk7-12'\n else:\n return 'reg-' + str(season)+ '-wk13-17'","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c62b99016933d31c898fa70bcca8c51c2b2fd1f0"},"cell_type":"markdown","source":"**Counting the number of injuries on punting team vs receiving team**\n\nWe can see from the chart above, that the punting teams recieved signficantly more injuries than the kicking team. This is interesting considering the most obvious candidate for a concussion would be the punt returner (PR) as the focus of the entire kicking team is to tackle him. "},{"metadata":{"trusted":true,"_uuid":"7441260f9a6dbb8e8b405d35da451896a4bd2a0c","_kg_hide-input":true},"cell_type":"code","source":"#print(pprd.keys())\nPUNTPOS = {\"PLS\",\"PLG\", \"PLT\", \"PLW\",\"PRG\",\"PRT\",\"PRW\",\"PC\",\"PPR\",\"P\",\"GL\",\"GR\"}\nRETURNPOS = {\"PDR3\",\"PDR2\",\"PDR1\",\"PDL1\",\"PDL2\",\"PDL3\",\"PLM\",\"PLL\",\"PLR\",\"PFB\",\"PR\",\"VRi\",\"VRo\",\"VLi\",\"VLo\",\"VR\",\"VL\"}\n\ncc = vr['game_play_key'].tolist()\ngsi = vr['GSISID'].tolist()\n#print(cc)\n#print(gsi)\n# print(pprd['GSISID'] == gsi[0])\nc = pd.DataFrame()\nfor x in range(len(cc)):\n curr = (pprd[(pprd['game_play_key'] == cc[x]) & (pprd['GSISID'] == gsi[x])])\n c = c.append(curr)\n#print(c)\nc['punting_team'] = c['punting_team'].replace(True,'Punting Team')\nc['punting_team'] = c['punting_team'].replace(False, 'Recv Team')\ncount = c['punting_team'].value_counts().plot(title =\"Concussions per Team\",kind='bar')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b1ae933112146143206399fcefe13d75a8b6920c"},"cell_type":"markdown","source":"**Feature Generation & Feature Selection**\n\nIn order to determine the best rule change, it is first important to understand what exactly is causing concussions. We want to know what the features are of a play involving a concussion vs the a play without one. We will attempt to find these through feature generation and a mutual information based feature selection. For each feature, we find a mutual information score. The mutual information score is a measure of dependence between two variables. The score quantifies how much each feature tells us about whether a play will result in a concussion. From here we can analyze the plays themselves while focusing on our selected features to see how they contribute to a concussion. \n\nWe must first generate the features of each play. Each play has associated game data containing game day information (venue, stadium type, turf type, visiting team, weather, temperature, etc) and associated play data (game clock, quarter, yardline, possession, score, etc). Now using the Next Gen Stats (NGS) data, we can generate some more features for each play. For each player, we calculate their average speed, average direction, and average orientation throughout the play. These values are also features that we will use. Using the NGS data we also found the hang time of the punt and used that as a feature. \n\nWe want to see which features are most indicative of a concussion and we can do this by looking at which features provide the most mutual information. In the following code sample, we generate said features."},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"e1dae253628e3f2ff874c2beecf991097ce38bd0"},"cell_type":"code","source":"from multiprocessing import Queue, Process\nfrom threading import Thread\nfeatureVectors = Queue()\nfvs = []\npuntplays = play_inf.query(\"Play_Type == 'Punt'\")\ndef makeFeatureVector(lo,hi,qq):\n #print(lo,hi)\n localcount = 0\n for rr in range(lo,hi):\n s = time.time()\n currow = puntplays.iloc[rr]\n nam = getNGSName(currow[\"Season_Year\"],currow[\"Week\"],currow[\"Season_Type\"])\n \n #if(localcount > 0):\n #break\n ngsTotal = NGSDICT[nam]\n #print(currow[\"GameKey\"],currow[\"PlayID\"])\n thisplay = ngsTotal.query('Season_Year == %d and GameKey == %d and PlayID == %d' %(currow[\"Season_Year\"], currow['GameKey'],currow['PlayID']))\n if(len(thisplay) > 0):\n puntinfo = thisplay.query(\"Event == 'punt'\")\n prinfo = thisplay.query(\"Event == 'punt_received' or Event == 'fair_catch'\")\n puntinfo = pd.to_datetime(puntinfo['Time'],format='%Y-%m-%d %H:%M:%S.%f')\n prinfo = pd.to_datetime(prinfo['Time'],format='%Y-%m-%d %H:%M:%S.%f')\n ht = 0\n if(len(puntinfo) > 0 and len(prinfo) > 0):\n kicked = min(puntinfo)\n gotten = min(prinfo)\n ht = (gotten-kicked).total_seconds()\n gamedata = game_inf.query(\"Season_Year == %d and GameKey == %d\" % (currow[\"Season_Year\"],currow[\"GameKey\"]))\n smallp = pprd.query('Season_Year == %d and GameKey == %d and PlayID == %d' %(currow[\"Season_Year\"], currow['GameKey'],currow['PlayID']))\n gamedata = gamedata.iloc[0]\n featureVector = {}\n featureVector['HangTime'] = ht\n featureVector['Week'] = gamedata[\"Week\"]\n featureVector['Season_Type'] = gamedata[\"Season_Type\"]\n featureVector['Game_Day'] = gamedata[\"Game_Day\"]\n featureVector['Game_Site'] = gamedata[\"Game_Site\"]\n featureVector['HomeTeamCode'] = gamedata[\"HomeTeamCode\"]\n featureVector['VisitTeamCode'] = gamedata[\"VisitTeamCode\"]\n featureVector['Turf'] = gamedata[\"Turf\"]\n featureVector['GameWeather'] = gamedata[\"GameWeather\"]\n featureVector['Temperature'] = gamedata[\"Temperature\"]\n featureVector['OutdoorWeather'] = gamedata[\"OutdoorWeather\"]\n featureVector['Stadium'] = gamedata[\"Stadium\"]\n featureVector['StadiumType'] = gamedata[\"StadiumType\"]\n featureVector['Start_Time'] = gamedata[\"Start_Time\"]\n featureVector['Game_Clock'] = currow[\"Game_Clock\"]\n featureVector['YardLine'] = currow[\"YardLine\"]\n featureVector['Quarter'] = currow[\"Quarter\"]\n featureVector['GameKey'] = currow[\"GameKey\"]\n featureVector['PlayID'] = currow[\"PlayID\"]\n featureVector['Season_Year'] = currow[\"Season_Year\"]\n featureVector['Play_Type'] = currow[\"Play_Type\"]\n featureVector['Poss_Team'] = currow[\"Poss_Team\"]\n featureVector['HomeScore'] = currow[\"Score_Home_Visiting\"].split(' - ')[0]\n featureVector['AwayScore'] = currow[\"Score_Home_Visiting\"].split(' - ')[1]\n \n for x in PUNTPOS:\n featureVector[x + '_' + 'AVG SPEED'] = \"NA\"\n featureVector[x + '_' + 'AVG DIR'] = \"NA\"\n featureVector[x + '_' + 'AVG O'] = \"NA\"\n \n for x in RETURNPOS:\n featureVector[x + '_' + 'AVG SPEED'] = \"NA\"\n featureVector[x + '_' + 'AVG DIR'] = \"NA\"\n featureVector[x + '_' + 'AVG O'] = \"NA\"\n found = 0\n for ttt in set(thisplay['GSISID']):\n try:\n k = smallp.query('GSISID == %d' %(ttt))\n except:\n continue\n if(len(k) > 0):\n localcount += 1\n legalplayermovement = thisplay[thisplay['GSISID']== ttt]\n #print(legalplayermovement['Event'].unique())\n rl = k.iloc[0][\"Role\"]\n split = len(legalplayermovement)//3\n featureVector[rl + '_' + 'AVG DIR'] = legalplayermovement['dir'].mean()\n featureVector[rl + '_' + 'AVG O'] = legalplayermovement['o'].mean()\n wind = 0\n '''\n for x in range(0,len(legalplayermovement),split):\n wind+= 1\n tempdf = legalplayermovement[x:] if x+split >= len(legalplayermovement) else legalplayermovement[x:x+split]\n featureVector[rl + '_' + 'AVG DIR_' + str(wind) +'-WINDOW' ] = legalplayermovement['dir'].mean()\n featureVector[rl + '_' + 'AVG O' + str(wind) +'-WINDOW'] = legalplayermovement['o'].mean()\n speeds = legalplayermovement[\"dis\"].apply(lambda x: x * 20.5)\n featureVector[rl + '_' + 'AVG SPEED'+ str(wind) +'-WINDOW'] = speeds.mean()\n '''\n speeds = legalplayermovement[\"dis\"].apply(lambda x: x * 20.5)\n\n featureVector[rl + '_' + 'AVG SPEED'] = speeds.mean() \n found += 1\n\n qq.put(featureVector)\n #print(thisplay)\n e = time.time()\n #print(e-s)\n print(\"done with the process\")\n return\n\ndef empty_queue(q):\n while True:\n k = q.get()\n fvs.append(k)\n\ndef makeFeatures(threads):\n ss = time.time()\n ts = []\n numthreads = threads\n workperthread = int(len(puntplays)/numthreads)\n for rr in range(0,len(puntplays),workperthread):\n #if rr >= 100: break\n if(rr + workperthread >= len(puntplays)):\n curt = Process(target = makeFeatureVector, args = (rr,len(puntplays),featureVectors,))\n else:\n curt = Process(target = makeFeatureVector, args = (rr,rr+workperthread,featureVectors,))\n ts.append(curt)\n for t in ts:\n t.start()\n cc = 0\n monit = Thread(target=empty_queue, args=(featureVectors,))\n monit.start()\n for t in ts:\n print(\"waiting for %d\" % cc)\n cc+= 1\n t.join()\n \n \n ee = time.time()\n print(\"Time Taken\")\n print(ee - ss)\n print(\"Made this many feature vectors:\")\n print(len(fvs))\n return ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"ad1105e017e4b29a327d053ba5b3b46f28c1f382","_kg_hide-output":true},"cell_type":"code","source":"# Make the features in parallel using 10 processes. \nmakeFeatures(10)\nprint(len(fvs))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"f4c338b64c73e8a87b9996542d1102a664490f1d"},"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nfrom sklearn.feature_selection import SelectKBest\nfrom sklearn.feature_selection import mutual_info_classif\nimport matplotlib.pyplot as plt \nfrom imblearn.over_sampling import SMOTE\nimport copy\nimport matplotlib.pyplot as plt\n\nvf = pd.read_csv('../input/video_footage-injury.csv')\nconcussionset = set()\nfor x in range(len(vr)):\n cr = vr.iloc[x]\n idenstring = str(cr['Season_Year'])+ '_' + str(cr['GameKey']) + '_' + str(cr['PlayID'])\n concussionset.add(idenstring)\n\nfvscopy = copy.deepcopy(fvs)\nfvscopy2 = copy.deepcopy(fvs)\nconfvs = []\nregularfvs = []\n\nconcussionFEATS= []\nregularFEATS = []\nfor x in fvscopy2:\n fviden = str(x['Season_Year'])+ '_' + str(x['GameKey']) + '_' + str(x['PlayID'])\n if(fviden in concussionset):\n concussionFEATS.append(x)\n else:\n regularFEATS.append(x)\n \nfor x in fvscopy:\n fviden = str(x['Season_Year'])+ '_' + str(x['GameKey']) + '_' + str(x['PlayID'])\n jon_check1, jon_check2 = x['GameKey'],x['PlayID']\n del x['GameKey']\n del x['Season_Year']\n del x['PlayID']\n del x['Play_Type']\n #print(x[\"HangTime\"])\n if(fviden in concussionset and not (jon_check1 == 274 and jon_check2 == 3609)):\n confvs.append(x)\n else:\n regularfvs.append(x)\n\nconsdf = pd.DataFrame(confvs)\nregsdf = pd.DataFrame(regularfvs)\nconcussionFEATSdf = pd.DataFrame(concussionFEATS)\nregularFEATSdf = pd.DataFrame(regularFEATS)\ndef subsetMean(c,n):\n comps = {}\n for k in c.keys():\n comps[k] = {}\n #print(k)\n ccworked = False\n ncworked = False\n try:\n if('O' in k or 'DIR' in k):\n comps[k]['Concussion'] = c[k].replace('NA',0).fillna(0).mean() - 180\n #cc.append(consdf[k].replace('NA',0).fillna(0).mean() - 180)\n ccworked = True\n else:\n comps[k]['Concussion'] = c[k].replace('NA',0).fillna(0).mean()\n #cc.append(consdf[k].replace('NA',0).fillna(0).mean())\n ccworked = True\n #print(k, consdf[k].mean(),'c')\n except: pass\n try:\n if('O' in k or 'DIR' in k):\n comps[k]['No Concussion'] = n[k].replace('NA',0).fillna(0).mean() -180\n #nc.append(regsdf[k].replace('NA',0).fillna(0).mean())\n ncworked = True\n else:\n comps[k]['No Concussion'] = n[k].replace('NA',0).fillna(0).mean()\n #nc.append(regsdf[k].replace('NA',0).fillna(0).mean())\n ncworked = True\n except: pass\n #print(k, regsdf[k].mean(),'nc')\n return comps\n\ncomps = subsetMean(consdf,regsdf)\ncompsdf = pd.DataFrame(comps)\ndef plot_comparisons(word,word2):\n subsetDIR = []\n subsetO = []\n subsetSPEED = []\n\n subsetDIRR = []\n subsetOR = []\n subsetSPEEDR = []\n\n for x in PUNTPOS:\n if(x == \"PC\" or x == \"PPR\"): continue\n subsetDIR.append(x+'_AVG' +' DIR')\n subsetO.append(x+'_AVG' +' O')\n subsetSPEED.append(x+'_AVG' +' SPEED')\n\n for x in RETURNPOS:\n if(x == \"PC\" or x == \"PPR\"): continue\n subsetDIRR.append(x+'_AVG' +' DIR')\n subsetOR.append(x+'_AVG' +' O')\n subsetSPEEDR.append(x+'_AVG' +' SPEED')\n smoldfDir = compsdf[subsetDIR]\n smoldfO = compsdf[subsetO]\n smoldfSPEED = compsdf[subsetSPEED]\n\n smoldfDirR = compsdf[subsetDIRR]\n smoldfOR = compsdf[subsetOR]\n smoldfSPEEDR = compsdf[subsetSPEEDR]\n if(word == \"punt\"):\n if(word2 == 'directions'):\n plt.figure()\n smoldfDir.T.plot.bar(title='Punt Team Directions (degrees)',figsize=(12,8))\n if(word2 == 'o'):\n plt.figure()\n smoldfO.T.plot.bar( title='Punt Team Orientations (degrees)',figsize=(12,8))\n if(word2 == 'speed'):\n plt.figure()\n smoldfSPEED.T.plot.bar( title='Punt Team Speeds (mph)',figsize=(12,8))\n else:\n if(word2 == 'directions'):\n plt.figure()\n smoldfDirR.T.plot.bar(title='Return Team Directions (degrees)',figsize=(12,8))\n if(word2 == 'o'):\n plt.figure()\n smoldfOR.T.plot.bar( title='Return Team Orientations (degrees)',figsize=(12,8))\n if(word2 == 'speed'):\n plt.figure()\n smoldfSPEEDR.T.plot.bar( title='Return Team Speeds (mph)',figsize=(12,8))\nprint(len(fvs))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d06287e8ff78c153c94ee8c64e2a812e980f0d49"},"cell_type":"markdown","source":"**Pre-Feature Selection Analysis**\n\nBefore selecting the most important features, we want to visualize our features better. We plot a bar chart that compares the average speed of positions in plays with and without concussions. \n\nWe can see from \"Punt Team Speeds\" and \"Receiving Team Speeds\", speeds for different players are similar between concussion and non-concussion plays. "},{"metadata":{"trusted":true,"_uuid":"a108f18f5d7e51a04d4e8127e21fb6d5af9aa9be"},"cell_type":"code","source":"plot_comparisons(\"punt\",'speed')\nplot_comparisons(\"recv\",'speed')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8c357e15cc13457f90d07c92c0f4dc29f12e0b8f"},"cell_type":"markdown","source":"In the field diagram below, the horizontal line in the middle represents a degree of 0. If a player is standing above the line their angle relative to the 0 increase as they move further from the line. Conversly, if a player is standing below the line, the angle decreases negatively as they move further below the line. The players on the punting team would start the play with an orientation of 0 (facing the right endzone), and the players of the receiving team would start with an orientation of 180 (facing the left endzone)."},{"metadata":{"trusted":true,"_uuid":"0d271dd7b002477014b082333f8ded523a5a6aa0"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"578f39b8a457b010870a8bd7da06f69f8e160f46"},"cell_type":"markdown","source":"The following charts compare the average direction of player motion between plays with and without concussions. It is interesting to see the punting team players' direction of motion varies depending whether or not the play resulted in a concussion, whereas this variation does not seem to exist for receving team players. In this case when we review concussion play video later on, we will pay close attention to the players' direction of motion and how that contributes to concussions. "},{"metadata":{"trusted":true,"_uuid":"d418a7b1d5172cfa95534b6afb578f54d358161f"},"cell_type":"code","source":"plot_comparisons(\"punt\",'directions')\nplot_comparisons(\"recv\",'directions')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6d09059338335b0566046f83724bcb7212373b1e"},"cell_type":"markdown","source":"We can see the differences in the average directions of motions of the punting team between the plays involving concussions and normal plays. It is interesting to note that the GL and the GR in regular plays have motions reflected across the 0 line, toward the PR who is in the middle of the field. Furthermore, we see that the VRs and the VLs should have a steeper angle toward the punt returner to block the gunners from getting to him. All of the players on the return team generally move towards the punt returner so they can make blocks. The punt returner generally moves toward the line of scrimmage. "},{"metadata":{"_uuid":"21a0f08634f125126d71328566af32267ef610d7"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"397893907b2bdd3fe6877277aa6d22be8bf0064c"},"cell_type":"markdown","source":"Now looking at the orientations of the players. This is significant because often times, players are not oriented in the direction they are moving. This could lead to dangerous plays such as blindside blocks.\n\nWe see that player orientations in \"Return Team Orientations\" are not significantly different between plays with and without concussions. However, there is a significant difference in the punt team players' average orientation. In a normal play, the average orientations of GL and GR should be reflections across the horizontal. Referring to \"Punt Team Orientations\", we see that this is indeed the case. However, in plays with concussions the GL and GR have the same degree of orientation. Furthermore, the average orientation of three other positions (PRW, PLT, PLS) are also inverted. Because of this inconsistency in orientation between plays with and without concussions, we suggest there is some relationship between the orientation of a player and concussions. "},{"metadata":{"trusted":true,"_uuid":"43a8845c25fa28769b9d6687937b64c333600b87"},"cell_type":"code","source":"plot_comparisons(\"punt\",'o')\nplot_comparisons(\"recv\",'o')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6256b9eb1e64ee88fd51759f011139bb29a3c5b8"},"cell_type":"markdown","source":"**Feature Selection**\n\nNow we would like to see what features of a play lead to a concussion. We will be using Sci-Kit Learn's SelectKBest method to perform a feature selection based on mutual information score. We will also be using the SMOTE algorithm in order to interpolate more concussion examples. \n\nWe use SMOTE in order to generate simulated concussion samples by randomly sampling instances of concussion plays and interpolating new examples from this random set. We randomly generate a training set consisting (on average) one-fourth concussions and three-fourths regular plays. SMOTE then interpolates the remaining concussion plays. After 10 iterations of mutual information based feature selection using Sci-Kit Learn's SelectKBest, we rank the most important features in determining a concussion."},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true,"_uuid":"24c2bb9b02ff9cd21cf5b79db7016ae360b263c1"},"cell_type":"code","source":"def randomSet(concs, notconcus):\n import random\n testset = []\n results = []\n cs,ncs = 0,0\n nex = 0\n for x in range(500):\n rando = random.randint(0,500)\n if(rando % 4 == 1):\n ind = random.randint(0,len(concs)-1)\n testset.append(concs[nex % len(concs)-1])\n nex += 1\n results.append(1)\n cs += 1\n else:\n ind = random.randint(0,len(notconcus)-1)\n testset.append(notconcus[ind])\n results.append(0)\n ncs += 1\n \n print(\"set contains %d concussions and %d regular plays\" % (cs,ncs))\n return testset,results\n\ndef doSelect():\n ts,rs = randomSet(confvs,regularfvs)\n setdf = pd.DataFrame(ts)\n setdf = setdf.fillna(0)\n for s in setdf:\n setdf[s]=setdf[s].astype('str')\n #print(setdf)\n sm = SMOTE(random_state=2)\n fixed = setdf.apply(LabelEncoder().fit_transform)\n X_train_res, y_train_res = sm.fit_sample(fixed, rs)\n print(len(X_train_res))\n feats = SelectKBest(mutual_info_classif,k=50).fit(X_train_res,y_train_res)\n new_features = [] # The list of your K best features\n feature_names = list(setdf.columns.values)\n mask = feats.get_support()#indices=True)\n for bool, feature in zip(mask, feature_names):\n if bool:\n new_features.append(feature)\n cols = feats.get_support(indices=True)\n mutualInfoVal = dict(sorted(zip(fixed.columns.values,feats.scores_),key=lambda x: x[1]))\n ordered = reversed(list(mutualInfoVal))\n #print(new_features)\n for m in ordered:\n try:\n selectResults[m] += mutualInfoVal[m]\n except:\n selectResults[m] = mutualInfoVal[m]\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"ed55bbb450b5d49f39bbe3c9d8001a46246e61c2"},"cell_type":"code","source":"selectResults = {}\nfor x in range(10):\n doSelect()\nprint('done selections')\n\nfeaturesCount = 0\n#print(selectResults)\nfor sR in reversed(sorted(selectResults,key=selectResults.get)):\n if(featuresCount > 50):break\n print(sR, selectResults[sR]/10)\n featuresCount += 1","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3fce99140b0dc3e734fa7a52bb9e3139c55c8d17"},"cell_type":"markdown","source":"**Most Important Features Found**\n\nWe find that average orientation of punting team players and average direction of punting team players are some of the most important features in determining a concussion. We see that the yard line is also important in determining concussions. This makes sense because if a team is backed up into their own side of the field, the punter tries to kick the ball further, leading to more hang time, and a higher possibility of a return. Furthermore, a punter running/having an orientation that isn't downfield is a solid indication there is a return on the play because the punter is now moving to make a tackle on the returner. \n\nTo reiterate, the most important features in determing a concussion are the average directions and orientations of the punting team. From the previous chart \"Concussions per Team\", we see that there are significantly more injuries to the punting team, we want to make a rule that makes it safer for the players on that side of the ball. \n\nIn order to see why the orientations and directions are so different for concussion plays and regular plays, we should first see how starting formations are different. We will define the formations based on the number of outside defenders (jammers). The following formation has 3 jammers.\n\n"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"eb0ed5dd666802e99c48ebf6a7d18697f6dd03a4"},"cell_type":"code","source":"count = 0\njammerIDS = {}\ndef countJammerFormations(df,s):\n jammercounts = {}\n for x in range(len(df)):\n forms = df.iloc[x].split(', ')\n j = 0\n for y in forms:\n if y.startswith(\"V\"):\n j += 1\n if(j == 5):\n j = 4\n #print(df.iloc[x].split(', '),j)\n try: jammercounts[j] +=1\n except: jammercounts[j] = 1\n for n in sorted(jammercounts):\n print(\"The formation with %d jammers appears %d times %s\" % (n,jammercounts[n],s))\n #count += 1\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"76fdd467f0770bf6d9b9216a3ba185bade6849b9"},"cell_type":"code","source":"countJammerFormations(returnTeamForms,'overall')\nprint()\ncountJammerFormations(concussReturnTeamsForms,'in the concussion set')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d557c7982f702245751f1f299727e74ef30e9e87"},"cell_type":"markdown","source":"We can see that concussions happened on formations with 2 jammers, 3 jammers, and 4 jammers, which are also the most common formations overall. Now we want to see what kind players are getting concussions in each of these formation types. "},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"4110e7333807ecb61ad045b31be65494038156e3"},"cell_type":"code","source":"def getFormationInjury(df):\n formationsInjury = {}\n for x in range(len(df)):\n rr = df.iloc[x]\n #print(rr.keys())\n forms = rr[\"form\"].split(', ')\n j=0\n for y in forms:\n if y.startswith(\"V\"):\n j += 1\n\n primary = rr['GSISID']\n g,p = rr['game_play_key'].split(\"_\")\n prole = pprd.query(\"GameKey == %d and PlayID == %d and GSISID == %d\" %(int(g),int(p),int(primary))).iloc[0]['Role']\n\n try:formationsInjury[j].append(prole)\n except:\n formationsInjury[j] = []\n formationsInjury[j].append(prole)\n \n #print(prole)\n \n for x in sorted(formationsInjury):\n injuredOffense = 0\n print(\"In the formations with %d jammers, the following positions received concussions:\" % x)\n for y in formationsInjury[x]:\n if(y in PUNTPOS):\n injuredOffense += 1\n \n \n print(\"\\t \", formationsInjury[x])\n print('\\t Percent of injuries to the punting team: %f' % (injuredOffense*100/len(formationsInjury[x]) ))\n \n\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a0a191e164ae22b9d90080c3792b0a5eddcc5288"},"cell_type":"code","source":"getFormationInjury(concuss[concuss['punting_team'] == False])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5e8e6932057dfc4b3ac4ef64af9e8d65b8c3002d"},"cell_type":"markdown","source":"The interesting observation to point out here is that in 4 jammer formations, mostly players on the offensive line (PRW, PRG, PRT, PLS, PLT, PLG, PLW) were injured. Only two defensive players were injured on this play. Lets look at some video footage to understand what is happening in these formations. \n\n\nIn this play, we see there are two sets of two jammers on either side of the field. As the ball is snapped and the play goes on, both jammers block the gunners and they are effectively taken out of the play; neither of them really get close to the ball. This means the kicking team now has numerous unblocked players running down field at the time of the kick (all linemen, the punter didn't move). The returner is the legendery Devin Hester, and the Bengals (kicking team) are clearly respecting his return ability, but they do not force or fill properly, and Hester makes 3 people miss and turns up field. Now there are 3 Bengals behind Devin Hester, chasing him upfield (their orientations are upfield and direction of motion is mostly upfield). As Hester turns the edge, number 42 of the return team runs downfield (orientation and direction of motion are mostly opposite that of the trailing Bengals, and toward his own goal) blocks one Bengal, ramming him into the trailing player. This is a legal block according to the NFL, as the player was not hit in the head or neck area. The concussion happens at this point in time. \n\nWe see that this formation took two fast, athletic players out of the play and resulted in a really good punt returner gaining substantial yards up field and the subsequent blocks toward their own goal caused a concussion. A similar kind of play happens in the next video. \n\n\nIn this play, the return team again has 4 jammers. As the play happens, the gunners try to avoid getting jammed by taking a hard inside release. This means they try to run past the jammers by cutting sharply toward the middle of the field. This vacates space on the outsides of the field. The jammers end up blocking the gunners, who are now unable to get close to the returner by the time he catches the ball. The returner then runs into the space vacated by the gunners and then turns up field. At this point, the returner has managed to get into open space so the kicking team players (Steelers) turn back towards their own goal to try and chase the returner down. However, the defensive linemen that initially tried to block the punt are now running towards their own goal. One of them lays a block toward his own goal. The concussion happens here. \n\nWe can see the implications of formation on the return. The punt returner is much more likely to make a play if there are no gunners running at him as he has much more space to work with. The following play has the same 4 jammer formation, but in this case, the jammers don't succeed on the left side. \n\n\nIn this play, the right gunner makes his way downfield, albeit with a hard outside release, so he is running down the sideline. The gunner runs past the returner and effectively takes himself out of the play, so the returner now turns across the field and makes his way up the field. The punting long snapper (PLS) number 49, had gone past the returners current yardline so he turns back up field and across the field to chase after the returner. Again his orientation is toward his own goal and the direction of motion is similar to that of the returners. At this point he is blocked by a defensive linemen who is running downfield to make the block. The contact happens in the legal strike zone, and hence is not illegal, but still gives the opposition player a concussion. The direction of motion and orientation of the impacted player is opposite that of the player who blocked him, but both are oriented toward their own goal lines. \n\nThe above three plays show offensive line injuries, but interestingly, we see there are two defensive linemen injuries (in the 4 jammer formation, referring back to the previous code block injuries based on jammer formation). The following video is a play where the defensive end (number 33) gets injured. This is, again, a 4 jammer formation. \n\n\nIn this play, the gunners are limited in their way down the field, and the returner makes quite a few people miss. As the kicking team players turn to chase the returner, number 33 comes running downfield and tries to make a block toward his own goal on the opposing number 78 (Dolphins). In trying to make this block, 33 himself gets a concussion. \n\n\n\nSimilar play, same formation type, the defensive lineman on the Chiefs (second from the left on the defensive line) attempts to block the punt. He is not succesful and then proceeds to turn around in order to make a block for the punt returner. Tyreek Hill (a dangerous punt returner) sees the area vacated by the jammed gunners and attempts to run up that sideline. The defensive line player then comes in (direction of motion and orientation are opposite that of Hill and the players behind Hill at this point) and makes helmet-to-helmet contact with Giants number 51. The defensive end gave himself a concussion with that attempted block toward his own goal. Even though this was an illegal block, this video reveals the mindset of players on the returning team. They are more incentivized to make more dangerous blocks which would suggest why most concussions belong to the kicking team. \n\nWe can see a pattern beginning to emerge. Kicking team players attempting to make a tackle are in dangerous positions because players on the returning team are coming back towards them at high speeds to make blocks. Additionally, these blocks are dangerous for both parties as they happen at high speeds.\n\nLet's see generally what happens in this kind of play, the following shows how the play develops in stages:\n\nThe play starts out as a normal play with 2 sets of 2 jammers (a).\n\n\n\n"},{"metadata":{"_uuid":"6f87b7c08665f179e58eeba83d6a4513470e9f21"},"cell_type":"markdown","source":"As the play starts, the general motion of the players is shown (b).\n\n"},{"metadata":{"_uuid":"8e80df8c98ca6a9e3043c6130ac2b3f8f890f564"},"cell_type":"markdown","source":"As some of the players on the defensive line drop toward their own endline to make blocks, others try to block the punt. The offensive linemen block the defensive linemen and then proceed to make their way downfield. The gunners are stopped by the jammers and the returner can make his way toward one of those lanes vacated by the blocked gunner (c).\n\n"},{"metadata":{"_uuid":"aa691fda77ef85ab69c98a359270ff7c7db14ef7"},"cell_type":"markdown","source":"As the blocks are made, the punt returner finds a lane near one of the sidelines and runs toward it, his motion and orientation are opposite that of many of the tacklers and some of his own teammates (PDR1, PDR2, PDL3, PDL1). After making some of the players on the blue team miss, the PR runs into the gap, and the blue team players now turn back upfield. toward their own goal, in order to make the tackle (d).\n\n"},{"metadata":{"_uuid":"8fd0e8a6949002bab93b22da11e1f443df6e2a77"},"cell_type":"markdown","source":"Now we can see that PRT, PRG, PLS, and even PLW are in danger of getting blocked toward their own goal. Their orientation is turning up field so that they can meet the path of PR, but PDL1, PDL3, PDR2, PDR1 can now make a block on any of these players while running at high speed toward their own goal (on the right of the image below) (e).\n\n"},{"metadata":{"_uuid":"695249d4cd140ed6f834758529f1e8a0880e7ff3"},"cell_type":"markdown","source":"The area highlighted between the red lines in the image below is a danger zone for the kicking team players. This is because as they turn to chase the returner, they are likely to be blindsided by blockers on the returning team. This danger area needs to be the place where punting team members are protected so that they do not receive concussions at the rate that they do on these kinds of plays. \n\n"},{"metadata":{"_uuid":"77efcc644f5141e6ddf2c79909d61ac7ffee4776"},"cell_type":"markdown","source":"Now we see why the 4 jammer formation is dangerous for the punting team. The area in the middle of the field is generally occupied and the punt returner usually has a lane up the sideline for a return. Now the two jammer and the three jammer formation also have more punting team concussions than returning. From our feature selection we see that the orientation and direction of the players on the offensive team are still really important in determing if there is a concussion or not, hence we will again look at the problem from that perspective. The first question that we have to answer is, since there are potentially two extra non-jammers in the two jammer and three jammer formation, what positions do these extra men take? We answer this by looking at the number of defensive linemen and linebackers for the 2 jammer and 3 jammer formations. "},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"c4de131cfbc21d8e6e704235a33bb84581be1121"},"cell_type":"code","source":"count = 0\njammerIDSBackers = {}\ndef countBackerFormations(df,s):\n jammercounts = {}\n for x in range(len(df)):\n forms = df.iloc[x].split(', ')\n j = 0\n backer = 0\n dl = 0\n for y in forms:\n if(y == \"PR\" or y == \"PFB\"): continue\n if y.startswith(\"V\"):\n j += 1\n continue\n if(\"DL\" in y or \"DR\" in y):\n dl += 1\n else:\n backer += 1\n if(j == 5):\n j = 4\n #print(df.iloc[x].split(', '),j)\n if(j == 4): continue\n \n try: jammercounts[(backer,dl)] +=1\n except: jammercounts[(backer,dl)] = 1\n #for n in sorted(jammercounts):\n #print(\"The formation with %d linebackers and %d linemen appears %d times %s\" % (n[0],n[1],jammercounts[n],s))\n #print()\n return jammercounts\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"236a16c7426e299806ec7c1d27b2fd75404fd4bb","scrolled":false},"cell_type":"code","source":"a = countBackerFormations(returnTeamForms,'overall')\nb = countBackerFormations(concussReturnTeamsForms,'in the concussion set')\nfor x in b:\n print(\"The formation with %d linebackers and %d linemen has concussion percentage %f\" % (x[0],x[1],(b[x]*100/a[x])))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6961fb2a248bf2f804db1ad0c1ae34c9f3cb092a"},"cell_type":"markdown","source":"Interestingly, we see that the formation with 1 linebacker and 5 linemen has a concussion percentage of 5% as opposed to the less than 1 percent for the other formations that had concussion. Since we know there are at most 3 jammers and a punt returner, the remaining position MUST be a PFB, or a punt full back. This formation was only used 60 times throughout the year, but there were 3 concussions on this formation. In order to understand why, we can look at 2 of the plays a fullback was used. \n''\nWe see in this play that there is a left jammer and two right jammers. The full back drops facing the away from the punt returner as the ball is kicked in the air. Then, when the ball is caught, the full back changes his direction of motion toward the closest covering player, in this case is the gunner on the side with only one jammer. He then blocks the player, and the returner is able to run toward the side with two jammers for a decent gain. We see that in this play, the full back acted as a kind of free-roam blocker, but the only player downfield fast enough for him to block is the gunner. By drawing the gunner in, and having a personal blocker for the punt returner, this gave the returner an interesting angle in his return and allowed him to run upfield. He is eventually hit by a helmet-to-helmet collision (illegal) which caused a concussion. \n''\\\nWe see on this play that the full back acts as a linebacker and then drops straight back, oriented away from the punt returner and toward the oncoming tacklers. As he drops, he does not make a block until the punt returner catches the ball, after which the PFB attempts to make a block toward his own goal. Blocking towards ones own goal again causes a concussion for a player on the punting team. \n\nWe see that this formation causes problems because players are drawn into the danger zone where they can potentially be blindsided by a block towards the blocker's own goal. \n\nFurthermore, if we look at the role of the linebackers in a punting formation we can understand where there is an orientation and direction issue. In a 3 jammer formation, the linebackers have freedom to make blocks as they wish. \n\n"},{"metadata":{"_uuid":"0b37951469cd89807ac2c643b098ad59fb454605"},"cell_type":"markdown","source":"Referring to the image above, the PLR can drop to block the PLW or PPL (if there is one) and the PLL can drop to either block the gunner (GR) toward his own goal or block the PRW. The linebackers have the freedom to block the people with the greatest threat of getting downfield. Sometimes the linebackers can even rush the punter to try and get a blocked punt, but usually they drop and try to set up blocks for the punt returners.\n\nThe more linebackers there are, the more blocking the PR has set up for him to follow, but fewer players rush the punter. Usually this is an attempt to set up a return down the middle as the players are being forced outside in the example above. We can see that the collisions that happen during the blocking are intense and happen at high speeds and head on. If the PLW beats the PLR outside, and attempts to come back upfield, he is in the danger zone. Essentially, players turning up field to chase the returner are in the danger zone. \n\nThe starting formations of the play imply a lot about the behavior of the return team players. We can use this information in order to define a rule change in accordance to the features we found through the video review and feature selection. "},{"metadata":{"_uuid":"c069987c368ea520717b5a94a4f5af22b683a27b"},"cell_type":"markdown","source":"**Proposed Rule Change**\n\nWe are proposing 2 rule changes.\n\n*Any player blocking toward their own end zone is assessed an unnecessary roughness penalty of 15 yards.*\n\nWe believe the NFL should make blocking towards your own end zone illegal during kicking plays in order to decrease the number of concussions. The main reasons are as follows:\n\n1. There are 16 plays with players blocking toward their own goal resulting in concussions. (16/37) concussions could be prevented with this rule change. \n2. Blocking toward your own goal is more likely to be a head-on or blindsided collisions. This maximizes the force of impact thus increasing the chances of a concussion and eliminates the danger area. Through our data analysis we have found that orientation and direction are two important features associated with concussions. Our proposed rule tackles (pun-intended) these two features directly. The rule will decrease the number of concussion causing punt plays.\n3. This rule change is simple to integrate and easy to learn. Referees need to be aware of any players blocking towards their own goal.\n4. This rule preserves the integrity of the sport. We believe that it would not decentivize punt returns, which we believe is an integral part of the sport. We will address how our proposed changes could introduce new risks to player safety in the next paragraph. Punting is a major part of football and many teams can gain a huge advantage through their punt return teams and punt coverage teams. Changing the rules too drastically would remove the importance this play type. \n\nSince players are not allowed to block toward their own end zone, they may be more inclined to play more off coverage, giving the PR a chance to make a return. Longer returns could lead to greater possilbilities of concussions. As the defensive linemen would not be able to come back down the field and make a block, teams may chose to use the defensive linemen more aggresively in rushing the punter, potentially leading to dangers for the offensive line and the punter. \n\n*Any player intiating helmet-to-helmet contact during a punt play is assessed an unnecessary roughness penalty of 15 yards and disqualified from the contest.*\n\nAdditionally, we think that intentional helmet-to-helmet contact should result in immediate ejection. The main reasons are as follows:\n\n1. There are 16 plays with helmet-to-helmet contact resulting in concussions. (16/37) concussions could be prevented with this rule change. \n2. In a punt play, since players are moving downfield at high speeds in many directions and orientations, a helmet-to-helmet contact would be especially dangerous. While there is a rule in place that makes helmet-to-helmet contact illegal, we see that it still occurs. By making the punishment for this dangerous play even harsher, we strongly incentivize players to not commit this infraction. \n3. This rule change is simple to integrate. Referees need to be aware of any players initiating head-to-head contact.\n4. This rule change preserves the integrity of the sport. The NFL has added rules in place attempting to make this contact illegal for many years, the harsher punishment would make sure this happens. \n\nIt is possible that players who have learned over the years to hit in one way may have to change their tackling technique in order to make sure they do not get disqualified by this rule. This can potentially lead to players injuring other parts of their body. This could be dangerous for both parties involved in the tackles. "},{"metadata":{"_uuid":"7d78ca5ba77e3f4d4fe8b36e5042ed299429cc44"},"cell_type":"markdown","source":"**Rebuttal to Other Rule Changes**\n\nIn addition to our proposed rule changes we also considered other rules:\n\n*Adding restrictions on starting formations*\n\nWe decided that adding restrictions on the starting formations would take away from the integrity of the sport. Punting is an offensive play, which means fake punt passes and fake punt throws are valid plays; we did not want to restrict the creativity of the teams on these plays. Furthermore, from our data analysis we see that formations are not the major feature causing concussions, rather it is the actions taken by the players in those formations. Making those actions illegal is safer than trying to restrict the formation.\n\n*Incentivising fair catches*\n\nWe decided that incentivizing fair catches strongly impacts the integrity of the game. Adding some number of yards to the end of the fair catch would give the receiving team an advantage. It would allow the returning team to rush the punter without worrying about blocking for the returner. The kicking team is at a disadvantage regardless of whether the returner fair catches.\n\n*Moving the Touchback from the 20 to the 15*\n\nThis would incentivize the kicking team to kick through the end zone on every punt, because they would guarantee bad field position for the other team. This changes the integrity of the game by significantly decreasing the chances of a return. We think that returns are one of the most essential and exciting aspects of football. "},{"metadata":{"trusted":true,"_uuid":"90b495a1ae295667fa5c64ef889c8cf5351998b1"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}
![](\"https://i.imgur.com/jmzZgvs.png\")
"},{"metadata":{"_uuid":"e3be6aa424e75408ce100ee9fa09db267ca37d46"},"cell_type":"markdown","source":"**Common Punting Practices and Rules**\n\nAccording to NFL rule 9-1-2: Only the end men or an eligible receiver aligned behind the line of scrimmage is permitted to advance one yard beyond the line of scrimmage before the ball is kicked. So in the image above, only GR, GL, PRW, PLW, and PPR would be eligble to run down the field before the ball is kicked. Hence, in order to maximize the chances of getting to the ball, the punting team usually has two players GR and GL immedietly run toward the PR at full speed to attempt to down the ball. PLW, PRW, and PPR are also allowed down the field, but ocasionally they may perform a chip block on one of the end men before running down field at full speed. PDR1, PDR2, PDR3, PDL3, PDL2, and PDL1 usually line up in the gaps created by the offensive linemen (PLT, PLG, PLS, PRG, and PRT) in order to try and rush through to block the kick. The offensive linemen block the defensive linemen from getting to the punter and then make their way downfield to tackle the returner/down the ball. PLR usually drops back in order to block a player rushing downfield to down the ball. VL(i,o) and VR(i,o) are players designated to specifically block the GL and the GR. They perform blocks to prevent the gunner from reaching the PR at full speed. They usually run with the gunner and attempt to force them toward the sidelines so that they have a lesser angle toward the ball. \n\nAccording to NFL rule 9-1-3, a defensive team (returning team) player who is within one yard of the line of scrimmage must lineup outside the shoulders of the snapper. This is to protect the long snapper who has his head down completely in order to snap the 15 yards to the punter. This is the only restriction on the defense when it comes to formations, and the offense has no offensive restrictions other than NFL rule 7-5-1(a) where the offensive team must have 7 players on the line of scrimmage. The NFL doesn't place too many restrictions on formations and we believe that our rule change should maintain the integrity of this principle. \n\nOn an aside, members of the special teams/punt unit are usually not players from normal offensive personnel. Rather, the \"offensive linemen\" on a punt team are usually comprised of players smaller than true offensive linemen (so that they can run downfield fast to down the ball) but large enough to lay blocks on defensive linemen. These players are usually linebackers and defensive backs (as we see in the video review, the numbers 30-59 are incredibly common on punt downs for the offense). We believe that this is a strategy used to 1) prevent injuries to offensive and defensive starters, and 2) so that the players on the punt coverage are more athletic. This is important because players on the field during punt downs are all usually very fast and very strong meaning collisions happen with huge momentum. \n\nIt is important to note that punting is an offensive down, and **generally** happens on 4th down. Teams generally use the first 3 downs to try and make the yard to gain, and use the 4th down in order to kick the ball away and force worse field position for opposition. While this is the case, an incredibly important feature of football is misdirection, and teams may decide to perform a **fake** punt or even punt on early downs to catch the opposition off guard. \n\n''\n\nThese kind of fake punt plays are very successful sometimes because the opposition is not ready for it or the opposition did not expect it. It is a major part of football and its essential that we do not make significant rule changes that make punting a *non offensive play* so that the excitement and strategy of fake punts is not removed. Hence it is vital we do not make restrictive formation requirements or remove the punt completely. We will attempt to use the data provided in order to create a rule that reduces potential concussions on punting while preserving as much of the excitement, strategy, and appeal a punt play can potentially provide. "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nprint(os.listdir(\"../input\"))\n","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"**Basic Experimental Data Analysis**"},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"46b49e4ebf07104c929a50880f9433c025c6d731"},"cell_type":"code","source":"# Loading video_review.csv data\nvr = pd.read_csv('../input/video_review.csv')\nvr.head(10)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ec050f220a79ad0d16851aa1757e214501e771b2"},"cell_type":"markdown","source":"**Histogram for type of impact in our 37 concussion causing plays.**\n\nHelmet-to-helmet hits are illegal in NFL plays, yet we see that concussions still happen from these hits. "},{"metadata":{"trusted":true,"_uuid":"5a4367df6cc3872d81284f436f9ea56b21fdc643"},"cell_type":"code","source":"primary_impact_type_hist = vr['Primary_Impact_Type'].value_counts().plot(kind='bar')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ec457f940f6f23fbfa9ec2aae95e59ba7c8ff9b4"},"cell_type":"markdown","source":"**Most common type of activity of players at the time of receiving concussion.**\n\nAccording to the data, tackling is the most common method of receiving a concussion."},{"metadata":{"trusted":true,"_uuid":"f3166264ec0b13415481777e5a83124ad88ba1c3"},"cell_type":"code","source":"player_activity_derived_hist = vr['Player_Activity_Derived'].value_counts().plot(kind='bar')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"35e3cfed542073b72633be98cc96dd8483eb3707"},"cell_type":"markdown","source":"**Most common type of activity of partner of players at the time of causing concussion.**\n\nInterestingly, the most common activity of the person giving a concussion is being blocked and tackling."},{"metadata":{"trusted":true,"_uuid":"958c64d8f017519732e16bb842d8c1b983e8088f"},"cell_type":"code","source":"primary_partner_activity_derived = vr['Primary_Partner_Activity_Derived'].value_counts().plot(kind='bar')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f04962b6dca595042046517458ef56d1c327f9e8"},"cell_type":"markdown","source":"**Retrieving Punt Play Formations**\n\nThe following two sections present exact formations of all concussion plays. We will refer back to these formations later on."},{"metadata":{"trusted":true,"_uuid":"e2a1116bc5a7a77bd17441ed76dc86615abebb8b","_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# Create play_player_role_data dataframe.\npprd = pd.read_csv('../input/play_player_role_data.csv')\n# Create play_information dataframe.\nplay_inf = pd.read_csv('../input/play_information.csv')\n# Create game_inf dataframe.\ngame_inf = pd.read_csv('../input/game_data.csv')\n# Create video_review dataframe\nvr = pd.read_csv('../input/video_review.csv')\n\n# Extract roles column from play_player_role_data.csv and check if player is on punting team.\npunting_pos = {\"PLS\",\"PLG\", \"PLT\", \"PLW\",\"PRG\",\"PRT\",\"PRW\",\"PC\",\"PPR\", \"PPL\",\"P\",\"GL\",\"GR\"}\nrole_col = pprd['Role']\npunting_team_col = [x in punting_pos for x in role_col.tolist()]\n\n# Adding punting_team_col to pprd dataframe.\npprd['punting_team'] = punting_team_col\n\n# Function that merges GameKey and PlayID given a row.\ndef merge(row):\n return str(row['GameKey']) + '_' + str(row['PlayID'])\n\n# Combine GameKey and PlayID to make unique Key that organizes formations.\npprd_merged_col = pprd.apply(lambda row: merge(row), axis=1)\nplay_inf_merged_col = play_inf.apply(lambda row: merge(row), axis=1)\nvr_merged_col = vr.apply(lambda row: merge(row), axis=1)\n\n# Adds merged_col to pprd dataframe.\npprd['game_play_key'] = pprd_merged_col\nplay_inf['game_play_key'] = play_inf_merged_col\nvr['game_play_key'] = vr_merged_col\n\n# Groupby game_play_key and punting_team.\npprd_formations = pprd.groupby(['game_play_key','punting_team']).apply(lambda x: sorted(x['Role'].values.tolist()))\npprd_formations = pprd_formations.rename(\"form\")\npunting_team_boolean = [x[0] in punting_pos for x in pprd_formations.tolist()]\npprd_formations = pprd_formations.apply(lambda x: ', '.join(x)).to_frame()\npprd_formations['punting_team'] = punting_team_boolean\n# Sorted formations\n#print(pprd_formations.keys())\npprd_and_playinf = play_inf.merge(pprd_formations, on=['game_play_key'], how='outer')\npprd_and_playinf = pprd_and_playinf.drop(columns=['Season_Year', 'GameKey', 'PlayID'])\n\n# Merge play_information.csv with play_player_role_data.csv by game_play_key.\nmerged = pprd_and_playinf.merge(vr, on=['game_play_key'], how='outer')\nreturnTeamForms = merged[merged['punting_team'] == False]['form']\n\nconcuss = merged[merged['Turnover_Related'].notnull()]\nconcussReturnTeamsForms = concuss[concuss['punting_team'] == False]['form']\n\ncount_formations_punting = concuss.loc[merged['punting_team'] == False, 'form']\ncount_formations_recv = concuss.loc[merged['punting_team'] == True, 'form']\n# print(count_formations_punting)\na = count_formations_punting.value_counts()\nb = count_formations_recv.value_counts()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9d8385c51eecd9a2504c6a6575d8679aa32d0112"},"cell_type":"code","source":"print(\"Types of formations for punting team, counts\")\na.head(7)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"81324c89e889a3c3470244cbc25fa2904162530c"},"cell_type":"code","source":"print(\"Types of formations for receiving team, counts\")\nb.head(7)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"5ee800bdfdc0a9a8c0194147f861ca3165ded110"},"cell_type":"code","source":"\nimport time\nstart = time.time()\nNGSDICT = {\n \"pre-2016\" : pd.read_csv('../input/NGS-2016-pre.csv'),\n \"pre-2017\" : pd.read_csv('../input/NGS-2017-pre.csv'),\n \"post-2016\" : pd.read_csv('../input/NGS-2016-post.csv'),\n \"post-2017\" : pd.read_csv('../input/NGS-2017-post.csv'),\n \n \"reg-2016-wk1-6\" : pd.read_csv('../input/NGS-2016-reg-wk1-6.csv'),\n \"reg-2016-wk7-12\" : pd.read_csv('../input/NGS-2016-reg-wk7-12.csv'),\n \"reg-2016-wk13-17\" : pd.read_csv('../input/NGS-2016-reg-wk13-17.csv'),\n \n \"reg-2017-wk1-6\" : pd.read_csv('../input/NGS-2017-reg-wk1-6.csv'),\n \"reg-2017-wk7-12\" : pd.read_csv('../input/NGS-2017-reg-wk7-12.csv'),\n \"reg-2017-wk13-17\" : pd.read_csv('../input/NGS-2017-reg-wk13-17.csv')\n}\nend = time.time()\nprint(\"Time to load:\")\nprint(end- start)\n\n\ndef getNGSName(season,week,season_type):\n if season_type == 'Pre':\n return 'pre-' + str(season)\n elif season_type == 'Post':\n return 'post-' + str(season) \n else:\n if(week <= 6):\n return 'reg-' + str(season)+ '-wk1-6'\n if(week >= 7 and week <=12):\n return 'reg-' + str(season)+ '-wk7-12'\n else:\n return 'reg-' + str(season)+ '-wk13-17'","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c62b99016933d31c898fa70bcca8c51c2b2fd1f0"},"cell_type":"markdown","source":"**Counting the number of injuries on punting team vs receiving team**\n\nWe can see from the chart above, that the punting teams recieved signficantly more injuries than the kicking team. This is interesting considering the most obvious candidate for a concussion would be the punt returner (PR) as the focus of the entire kicking team is to tackle him. "},{"metadata":{"trusted":true,"_uuid":"7441260f9a6dbb8e8b405d35da451896a4bd2a0c","_kg_hide-input":true},"cell_type":"code","source":"#print(pprd.keys())\nPUNTPOS = {\"PLS\",\"PLG\", \"PLT\", \"PLW\",\"PRG\",\"PRT\",\"PRW\",\"PC\",\"PPR\",\"P\",\"GL\",\"GR\"}\nRETURNPOS = {\"PDR3\",\"PDR2\",\"PDR1\",\"PDL1\",\"PDL2\",\"PDL3\",\"PLM\",\"PLL\",\"PLR\",\"PFB\",\"PR\",\"VRi\",\"VRo\",\"VLi\",\"VLo\",\"VR\",\"VL\"}\n\ncc = vr['game_play_key'].tolist()\ngsi = vr['GSISID'].tolist()\n#print(cc)\n#print(gsi)\n# print(pprd['GSISID'] == gsi[0])\nc = pd.DataFrame()\nfor x in range(len(cc)):\n curr = (pprd[(pprd['game_play_key'] == cc[x]) & (pprd['GSISID'] == gsi[x])])\n c = c.append(curr)\n#print(c)\nc['punting_team'] = c['punting_team'].replace(True,'Punting Team')\nc['punting_team'] = c['punting_team'].replace(False, 'Recv Team')\ncount = c['punting_team'].value_counts().plot(title =\"Concussions per Team\",kind='bar')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b1ae933112146143206399fcefe13d75a8b6920c"},"cell_type":"markdown","source":"**Feature Generation & Feature Selection**\n\nIn order to determine the best rule change, it is first important to understand what exactly is causing concussions. We want to know what the features are of a play involving a concussion vs the a play without one. We will attempt to find these through feature generation and a mutual information based feature selection. For each feature, we find a mutual information score. The mutual information score is a measure of dependence between two variables. The score quantifies how much each feature tells us about whether a play will result in a concussion. From here we can analyze the plays themselves while focusing on our selected features to see how they contribute to a concussion. \n\nWe must first generate the features of each play. Each play has associated game data containing game day information (venue, stadium type, turf type, visiting team, weather, temperature, etc) and associated play data (game clock, quarter, yardline, possession, score, etc). Now using the Next Gen Stats (NGS) data, we can generate some more features for each play. For each player, we calculate their average speed, average direction, and average orientation throughout the play. These values are also features that we will use. Using the NGS data we also found the hang time of the punt and used that as a feature. \n\nWe want to see which features are most indicative of a concussion and we can do this by looking at which features provide the most mutual information. In the following code sample, we generate said features."},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"e1dae253628e3f2ff874c2beecf991097ce38bd0"},"cell_type":"code","source":"from multiprocessing import Queue, Process\nfrom threading import Thread\nfeatureVectors = Queue()\nfvs = []\npuntplays = play_inf.query(\"Play_Type == 'Punt'\")\ndef makeFeatureVector(lo,hi,qq):\n #print(lo,hi)\n localcount = 0\n for rr in range(lo,hi):\n s = time.time()\n currow = puntplays.iloc[rr]\n nam = getNGSName(currow[\"Season_Year\"],currow[\"Week\"],currow[\"Season_Type\"])\n \n #if(localcount > 0):\n #break\n ngsTotal = NGSDICT[nam]\n #print(currow[\"GameKey\"],currow[\"PlayID\"])\n thisplay = ngsTotal.query('Season_Year == %d and GameKey == %d and PlayID == %d' %(currow[\"Season_Year\"], currow['GameKey'],currow['PlayID']))\n if(len(thisplay) > 0):\n puntinfo = thisplay.query(\"Event == 'punt'\")\n prinfo = thisplay.query(\"Event == 'punt_received' or Event == 'fair_catch'\")\n puntinfo = pd.to_datetime(puntinfo['Time'],format='%Y-%m-%d %H:%M:%S.%f')\n prinfo = pd.to_datetime(prinfo['Time'],format='%Y-%m-%d %H:%M:%S.%f')\n ht = 0\n if(len(puntinfo) > 0 and len(prinfo) > 0):\n kicked = min(puntinfo)\n gotten = min(prinfo)\n ht = (gotten-kicked).total_seconds()\n gamedata = game_inf.query(\"Season_Year == %d and GameKey == %d\" % (currow[\"Season_Year\"],currow[\"GameKey\"]))\n smallp = pprd.query('Season_Year == %d and GameKey == %d and PlayID == %d' %(currow[\"Season_Year\"], currow['GameKey'],currow['PlayID']))\n gamedata = gamedata.iloc[0]\n featureVector = {}\n featureVector['HangTime'] = ht\n featureVector['Week'] = gamedata[\"Week\"]\n featureVector['Season_Type'] = gamedata[\"Season_Type\"]\n featureVector['Game_Day'] = gamedata[\"Game_Day\"]\n featureVector['Game_Site'] = gamedata[\"Game_Site\"]\n featureVector['HomeTeamCode'] = gamedata[\"HomeTeamCode\"]\n featureVector['VisitTeamCode'] = gamedata[\"VisitTeamCode\"]\n featureVector['Turf'] = gamedata[\"Turf\"]\n featureVector['GameWeather'] = gamedata[\"GameWeather\"]\n featureVector['Temperature'] = gamedata[\"Temperature\"]\n featureVector['OutdoorWeather'] = gamedata[\"OutdoorWeather\"]\n featureVector['Stadium'] = gamedata[\"Stadium\"]\n featureVector['StadiumType'] = gamedata[\"StadiumType\"]\n featureVector['Start_Time'] = gamedata[\"Start_Time\"]\n featureVector['Game_Clock'] = currow[\"Game_Clock\"]\n featureVector['YardLine'] = currow[\"YardLine\"]\n featureVector['Quarter'] = currow[\"Quarter\"]\n featureVector['GameKey'] = currow[\"GameKey\"]\n featureVector['PlayID'] = currow[\"PlayID\"]\n featureVector['Season_Year'] = currow[\"Season_Year\"]\n featureVector['Play_Type'] = currow[\"Play_Type\"]\n featureVector['Poss_Team'] = currow[\"Poss_Team\"]\n featureVector['HomeScore'] = currow[\"Score_Home_Visiting\"].split(' - ')[0]\n featureVector['AwayScore'] = currow[\"Score_Home_Visiting\"].split(' - ')[1]\n \n for x in PUNTPOS:\n featureVector[x + '_' + 'AVG SPEED'] = \"NA\"\n featureVector[x + '_' + 'AVG DIR'] = \"NA\"\n featureVector[x + '_' + 'AVG O'] = \"NA\"\n \n for x in RETURNPOS:\n featureVector[x + '_' + 'AVG SPEED'] = \"NA\"\n featureVector[x + '_' + 'AVG DIR'] = \"NA\"\n featureVector[x + '_' + 'AVG O'] = \"NA\"\n found = 0\n for ttt in set(thisplay['GSISID']):\n try:\n k = smallp.query('GSISID == %d' %(ttt))\n except:\n continue\n if(len(k) > 0):\n localcount += 1\n legalplayermovement = thisplay[thisplay['GSISID']== ttt]\n #print(legalplayermovement['Event'].unique())\n rl = k.iloc[0][\"Role\"]\n split = len(legalplayermovement)//3\n featureVector[rl + '_' + 'AVG DIR'] = legalplayermovement['dir'].mean()\n featureVector[rl + '_' + 'AVG O'] = legalplayermovement['o'].mean()\n wind = 0\n '''\n for x in range(0,len(legalplayermovement),split):\n wind+= 1\n tempdf = legalplayermovement[x:] if x+split >= len(legalplayermovement) else legalplayermovement[x:x+split]\n featureVector[rl + '_' + 'AVG DIR_' + str(wind) +'-WINDOW' ] = legalplayermovement['dir'].mean()\n featureVector[rl + '_' + 'AVG O' + str(wind) +'-WINDOW'] = legalplayermovement['o'].mean()\n speeds = legalplayermovement[\"dis\"].apply(lambda x: x * 20.5)\n featureVector[rl + '_' + 'AVG SPEED'+ str(wind) +'-WINDOW'] = speeds.mean()\n '''\n speeds = legalplayermovement[\"dis\"].apply(lambda x: x * 20.5)\n\n featureVector[rl + '_' + 'AVG SPEED'] = speeds.mean() \n found += 1\n\n qq.put(featureVector)\n #print(thisplay)\n e = time.time()\n #print(e-s)\n print(\"done with the process\")\n return\n\ndef empty_queue(q):\n while True:\n k = q.get()\n fvs.append(k)\n\ndef makeFeatures(threads):\n ss = time.time()\n ts = []\n numthreads = threads\n workperthread = int(len(puntplays)/numthreads)\n for rr in range(0,len(puntplays),workperthread):\n #if rr >= 100: break\n if(rr + workperthread >= len(puntplays)):\n curt = Process(target = makeFeatureVector, args = (rr,len(puntplays),featureVectors,))\n else:\n curt = Process(target = makeFeatureVector, args = (rr,rr+workperthread,featureVectors,))\n ts.append(curt)\n for t in ts:\n t.start()\n cc = 0\n monit = Thread(target=empty_queue, args=(featureVectors,))\n monit.start()\n for t in ts:\n print(\"waiting for %d\" % cc)\n cc+= 1\n t.join()\n \n \n ee = time.time()\n print(\"Time Taken\")\n print(ee - ss)\n print(\"Made this many feature vectors:\")\n print(len(fvs))\n return ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"ad1105e017e4b29a327d053ba5b3b46f28c1f382","_kg_hide-output":true},"cell_type":"code","source":"# Make the features in parallel using 10 processes. \nmakeFeatures(10)\nprint(len(fvs))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"f4c338b64c73e8a87b9996542d1102a664490f1d"},"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nfrom sklearn.feature_selection import SelectKBest\nfrom sklearn.feature_selection import mutual_info_classif\nimport matplotlib.pyplot as plt \nfrom imblearn.over_sampling import SMOTE\nimport copy\nimport matplotlib.pyplot as plt\n\nvf = pd.read_csv('../input/video_footage-injury.csv')\nconcussionset = set()\nfor x in range(len(vr)):\n cr = vr.iloc[x]\n idenstring = str(cr['Season_Year'])+ '_' + str(cr['GameKey']) + '_' + str(cr['PlayID'])\n concussionset.add(idenstring)\n\nfvscopy = copy.deepcopy(fvs)\nfvscopy2 = copy.deepcopy(fvs)\nconfvs = []\nregularfvs = []\n\nconcussionFEATS= []\nregularFEATS = []\nfor x in fvscopy2:\n fviden = str(x['Season_Year'])+ '_' + str(x['GameKey']) + '_' + str(x['PlayID'])\n if(fviden in concussionset):\n concussionFEATS.append(x)\n else:\n regularFEATS.append(x)\n \nfor x in fvscopy:\n fviden = str(x['Season_Year'])+ '_' + str(x['GameKey']) + '_' + str(x['PlayID'])\n jon_check1, jon_check2 = x['GameKey'],x['PlayID']\n del x['GameKey']\n del x['Season_Year']\n del x['PlayID']\n del x['Play_Type']\n #print(x[\"HangTime\"])\n if(fviden in concussionset and not (jon_check1 == 274 and jon_check2 == 3609)):\n confvs.append(x)\n else:\n regularfvs.append(x)\n\nconsdf = pd.DataFrame(confvs)\nregsdf = pd.DataFrame(regularfvs)\nconcussionFEATSdf = pd.DataFrame(concussionFEATS)\nregularFEATSdf = pd.DataFrame(regularFEATS)\ndef subsetMean(c,n):\n comps = {}\n for k in c.keys():\n comps[k] = {}\n #print(k)\n ccworked = False\n ncworked = False\n try:\n if('O' in k or 'DIR' in k):\n comps[k]['Concussion'] = c[k].replace('NA',0).fillna(0).mean() - 180\n #cc.append(consdf[k].replace('NA',0).fillna(0).mean() - 180)\n ccworked = True\n else:\n comps[k]['Concussion'] = c[k].replace('NA',0).fillna(0).mean()\n #cc.append(consdf[k].replace('NA',0).fillna(0).mean())\n ccworked = True\n #print(k, consdf[k].mean(),'c')\n except: pass\n try:\n if('O' in k or 'DIR' in k):\n comps[k]['No Concussion'] = n[k].replace('NA',0).fillna(0).mean() -180\n #nc.append(regsdf[k].replace('NA',0).fillna(0).mean())\n ncworked = True\n else:\n comps[k]['No Concussion'] = n[k].replace('NA',0).fillna(0).mean()\n #nc.append(regsdf[k].replace('NA',0).fillna(0).mean())\n ncworked = True\n except: pass\n #print(k, regsdf[k].mean(),'nc')\n return comps\n\ncomps = subsetMean(consdf,regsdf)\ncompsdf = pd.DataFrame(comps)\ndef plot_comparisons(word,word2):\n subsetDIR = []\n subsetO = []\n subsetSPEED = []\n\n subsetDIRR = []\n subsetOR = []\n subsetSPEEDR = []\n\n for x in PUNTPOS:\n if(x == \"PC\" or x == \"PPR\"): continue\n subsetDIR.append(x+'_AVG' +' DIR')\n subsetO.append(x+'_AVG' +' O')\n subsetSPEED.append(x+'_AVG' +' SPEED')\n\n for x in RETURNPOS:\n if(x == \"PC\" or x == \"PPR\"): continue\n subsetDIRR.append(x+'_AVG' +' DIR')\n subsetOR.append(x+'_AVG' +' O')\n subsetSPEEDR.append(x+'_AVG' +' SPEED')\n smoldfDir = compsdf[subsetDIR]\n smoldfO = compsdf[subsetO]\n smoldfSPEED = compsdf[subsetSPEED]\n\n smoldfDirR = compsdf[subsetDIRR]\n smoldfOR = compsdf[subsetOR]\n smoldfSPEEDR = compsdf[subsetSPEEDR]\n if(word == \"punt\"):\n if(word2 == 'directions'):\n plt.figure()\n smoldfDir.T.plot.bar(title='Punt Team Directions (degrees)',figsize=(12,8))\n if(word2 == 'o'):\n plt.figure()\n smoldfO.T.plot.bar( title='Punt Team Orientations (degrees)',figsize=(12,8))\n if(word2 == 'speed'):\n plt.figure()\n smoldfSPEED.T.plot.bar( title='Punt Team Speeds (mph)',figsize=(12,8))\n else:\n if(word2 == 'directions'):\n plt.figure()\n smoldfDirR.T.plot.bar(title='Return Team Directions (degrees)',figsize=(12,8))\n if(word2 == 'o'):\n plt.figure()\n smoldfOR.T.plot.bar( title='Return Team Orientations (degrees)',figsize=(12,8))\n if(word2 == 'speed'):\n plt.figure()\n smoldfSPEEDR.T.plot.bar( title='Return Team Speeds (mph)',figsize=(12,8))\nprint(len(fvs))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d06287e8ff78c153c94ee8c64e2a812e980f0d49"},"cell_type":"markdown","source":"**Pre-Feature Selection Analysis**\n\nBefore selecting the most important features, we want to visualize our features better. We plot a bar chart that compares the average speed of positions in plays with and without concussions. \n\nWe can see from \"Punt Team Speeds\" and \"Receiving Team Speeds\", speeds for different players are similar between concussion and non-concussion plays. "},{"metadata":{"trusted":true,"_uuid":"a108f18f5d7e51a04d4e8127e21fb6d5af9aa9be"},"cell_type":"code","source":"plot_comparisons(\"punt\",'speed')\nplot_comparisons(\"recv\",'speed')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8c357e15cc13457f90d07c92c0f4dc29f12e0b8f"},"cell_type":"markdown","source":"In the field diagram below, the horizontal line in the middle represents a degree of 0. If a player is standing above the line their angle relative to the 0 increase as they move further from the line. Conversly, if a player is standing below the line, the angle decreases negatively as they move further below the line. The players on the punting team would start the play with an orientation of 0 (facing the right endzone), and the players of the receiving team would start with an orientation of 180 (facing the left endzone)."},{"metadata":{"trusted":true,"_uuid":"0d271dd7b002477014b082333f8ded523a5a6aa0"},"cell_type":"markdown","source":"![](\"https://i.imgur.com/bNOEjqF.png\")
"},{"metadata":{"_uuid":"578f39b8a457b010870a8bd7da06f69f8e160f46"},"cell_type":"markdown","source":"The following charts compare the average direction of player motion between plays with and without concussions. It is interesting to see the punting team players' direction of motion varies depending whether or not the play resulted in a concussion, whereas this variation does not seem to exist for receving team players. In this case when we review concussion play video later on, we will pay close attention to the players' direction of motion and how that contributes to concussions. "},{"metadata":{"trusted":true,"_uuid":"d418a7b1d5172cfa95534b6afb578f54d358161f"},"cell_type":"code","source":"plot_comparisons(\"punt\",'directions')\nplot_comparisons(\"recv\",'directions')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6d09059338335b0566046f83724bcb7212373b1e"},"cell_type":"markdown","source":"We can see the differences in the average directions of motions of the punting team between the plays involving concussions and normal plays. It is interesting to note that the GL and the GR in regular plays have motions reflected across the 0 line, toward the PR who is in the middle of the field. Furthermore, we see that the VRs and the VLs should have a steeper angle toward the punt returner to block the gunners from getting to him. All of the players on the return team generally move towards the punt returner so they can make blocks. The punt returner generally moves toward the line of scrimmage. "},{"metadata":{"_uuid":"21a0f08634f125126d71328566af32267ef610d7"},"cell_type":"markdown","source":"![](\"https://i.imgur.com/K0RLft4.png\")
"},{"metadata":{"_uuid":"397893907b2bdd3fe6877277aa6d22be8bf0064c"},"cell_type":"markdown","source":"Now looking at the orientations of the players. This is significant because often times, players are not oriented in the direction they are moving. This could lead to dangerous plays such as blindside blocks.\n\nWe see that player orientations in \"Return Team Orientations\" are not significantly different between plays with and without concussions. However, there is a significant difference in the punt team players' average orientation. In a normal play, the average orientations of GL and GR should be reflections across the horizontal. Referring to \"Punt Team Orientations\", we see that this is indeed the case. However, in plays with concussions the GL and GR have the same degree of orientation. Furthermore, the average orientation of three other positions (PRW, PLT, PLS) are also inverted. Because of this inconsistency in orientation between plays with and without concussions, we suggest there is some relationship between the orientation of a player and concussions. "},{"metadata":{"trusted":true,"_uuid":"43a8845c25fa28769b9d6687937b64c333600b87"},"cell_type":"code","source":"plot_comparisons(\"punt\",'o')\nplot_comparisons(\"recv\",'o')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6256b9eb1e64ee88fd51759f011139bb29a3c5b8"},"cell_type":"markdown","source":"**Feature Selection**\n\nNow we would like to see what features of a play lead to a concussion. We will be using Sci-Kit Learn's SelectKBest method to perform a feature selection based on mutual information score. We will also be using the SMOTE algorithm in order to interpolate more concussion examples. \n\nWe use SMOTE in order to generate simulated concussion samples by randomly sampling instances of concussion plays and interpolating new examples from this random set. We randomly generate a training set consisting (on average) one-fourth concussions and three-fourths regular plays. SMOTE then interpolates the remaining concussion plays. After 10 iterations of mutual information based feature selection using Sci-Kit Learn's SelectKBest, we rank the most important features in determining a concussion."},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true,"_uuid":"24c2bb9b02ff9cd21cf5b79db7016ae360b263c1"},"cell_type":"code","source":"def randomSet(concs, notconcus):\n import random\n testset = []\n results = []\n cs,ncs = 0,0\n nex = 0\n for x in range(500):\n rando = random.randint(0,500)\n if(rando % 4 == 1):\n ind = random.randint(0,len(concs)-1)\n testset.append(concs[nex % len(concs)-1])\n nex += 1\n results.append(1)\n cs += 1\n else:\n ind = random.randint(0,len(notconcus)-1)\n testset.append(notconcus[ind])\n results.append(0)\n ncs += 1\n \n print(\"set contains %d concussions and %d regular plays\" % (cs,ncs))\n return testset,results\n\ndef doSelect():\n ts,rs = randomSet(confvs,regularfvs)\n setdf = pd.DataFrame(ts)\n setdf = setdf.fillna(0)\n for s in setdf:\n setdf[s]=setdf[s].astype('str')\n #print(setdf)\n sm = SMOTE(random_state=2)\n fixed = setdf.apply(LabelEncoder().fit_transform)\n X_train_res, y_train_res = sm.fit_sample(fixed, rs)\n print(len(X_train_res))\n feats = SelectKBest(mutual_info_classif,k=50).fit(X_train_res,y_train_res)\n new_features = [] # The list of your K best features\n feature_names = list(setdf.columns.values)\n mask = feats.get_support()#indices=True)\n for bool, feature in zip(mask, feature_names):\n if bool:\n new_features.append(feature)\n cols = feats.get_support(indices=True)\n mutualInfoVal = dict(sorted(zip(fixed.columns.values,feats.scores_),key=lambda x: x[1]))\n ordered = reversed(list(mutualInfoVal))\n #print(new_features)\n for m in ordered:\n try:\n selectResults[m] += mutualInfoVal[m]\n except:\n selectResults[m] = mutualInfoVal[m]\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"ed55bbb450b5d49f39bbe3c9d8001a46246e61c2"},"cell_type":"code","source":"selectResults = {}\nfor x in range(10):\n doSelect()\nprint('done selections')\n\nfeaturesCount = 0\n#print(selectResults)\nfor sR in reversed(sorted(selectResults,key=selectResults.get)):\n if(featuresCount > 50):break\n print(sR, selectResults[sR]/10)\n featuresCount += 1","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3fce99140b0dc3e734fa7a52bb9e3139c55c8d17"},"cell_type":"markdown","source":"**Most Important Features Found**\n\nWe find that average orientation of punting team players and average direction of punting team players are some of the most important features in determining a concussion. We see that the yard line is also important in determining concussions. This makes sense because if a team is backed up into their own side of the field, the punter tries to kick the ball further, leading to more hang time, and a higher possibility of a return. Furthermore, a punter running/having an orientation that isn't downfield is a solid indication there is a return on the play because the punter is now moving to make a tackle on the returner. \n\nTo reiterate, the most important features in determing a concussion are the average directions and orientations of the punting team. From the previous chart \"Concussions per Team\", we see that there are significantly more injuries to the punting team, we want to make a rule that makes it safer for the players on that side of the ball. \n\nIn order to see why the orientations and directions are so different for concussion plays and regular plays, we should first see how starting formations are different. We will define the formations based on the number of outside defenders (jammers). The following formation has 3 jammers.\n\n![](\"https://i.imgur.com/8nnckKA.png\"/)
"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"eb0ed5dd666802e99c48ebf6a7d18697f6dd03a4"},"cell_type":"code","source":"count = 0\njammerIDS = {}\ndef countJammerFormations(df,s):\n jammercounts = {}\n for x in range(len(df)):\n forms = df.iloc[x].split(', ')\n j = 0\n for y in forms:\n if y.startswith(\"V\"):\n j += 1\n if(j == 5):\n j = 4\n #print(df.iloc[x].split(', '),j)\n try: jammercounts[j] +=1\n except: jammercounts[j] = 1\n for n in sorted(jammercounts):\n print(\"The formation with %d jammers appears %d times %s\" % (n,jammercounts[n],s))\n #count += 1\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"76fdd467f0770bf6d9b9216a3ba185bade6849b9"},"cell_type":"code","source":"countJammerFormations(returnTeamForms,'overall')\nprint()\ncountJammerFormations(concussReturnTeamsForms,'in the concussion set')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d557c7982f702245751f1f299727e74ef30e9e87"},"cell_type":"markdown","source":"We can see that concussions happened on formations with 2 jammers, 3 jammers, and 4 jammers, which are also the most common formations overall. Now we want to see what kind players are getting concussions in each of these formation types. "},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"4110e7333807ecb61ad045b31be65494038156e3"},"cell_type":"code","source":"def getFormationInjury(df):\n formationsInjury = {}\n for x in range(len(df)):\n rr = df.iloc[x]\n #print(rr.keys())\n forms = rr[\"form\"].split(', ')\n j=0\n for y in forms:\n if y.startswith(\"V\"):\n j += 1\n\n primary = rr['GSISID']\n g,p = rr['game_play_key'].split(\"_\")\n prole = pprd.query(\"GameKey == %d and PlayID == %d and GSISID == %d\" %(int(g),int(p),int(primary))).iloc[0]['Role']\n\n try:formationsInjury[j].append(prole)\n except:\n formationsInjury[j] = []\n formationsInjury[j].append(prole)\n \n #print(prole)\n \n for x in sorted(formationsInjury):\n injuredOffense = 0\n print(\"In the formations with %d jammers, the following positions received concussions:\" % x)\n for y in formationsInjury[x]:\n if(y in PUNTPOS):\n injuredOffense += 1\n \n \n print(\"\\t \", formationsInjury[x])\n print('\\t Percent of injuries to the punting team: %f' % (injuredOffense*100/len(formationsInjury[x]) ))\n \n\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a0a191e164ae22b9d90080c3792b0a5eddcc5288"},"cell_type":"code","source":"getFormationInjury(concuss[concuss['punting_team'] == False])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5e8e6932057dfc4b3ac4ef64af9e8d65b8c3002d"},"cell_type":"markdown","source":"The interesting observation to point out here is that in 4 jammer formations, mostly players on the offensive line (PRW, PRG, PRT, PLS, PLT, PLG, PLW) were injured. Only two defensive players were injured on this play. Lets look at some video footage to understand what is happening in these formations. \n\n\nIn this play, we see there are two sets of two jammers on either side of the field. As the ball is snapped and the play goes on, both jammers block the gunners and they are effectively taken out of the play; neither of them really get close to the ball. This means the kicking team now has numerous unblocked players running down field at the time of the kick (all linemen, the punter didn't move). The returner is the legendery Devin Hester, and the Bengals (kicking team) are clearly respecting his return ability, but they do not force or fill properly, and Hester makes 3 people miss and turns up field. Now there are 3 Bengals behind Devin Hester, chasing him upfield (their orientations are upfield and direction of motion is mostly upfield). As Hester turns the edge, number 42 of the return team runs downfield (orientation and direction of motion are mostly opposite that of the trailing Bengals, and toward his own goal) blocks one Bengal, ramming him into the trailing player. This is a legal block according to the NFL, as the player was not hit in the head or neck area. The concussion happens at this point in time. \n\nWe see that this formation took two fast, athletic players out of the play and resulted in a really good punt returner gaining substantial yards up field and the subsequent blocks toward their own goal caused a concussion. A similar kind of play happens in the next video. \n\n\nIn this play, the return team again has 4 jammers. As the play happens, the gunners try to avoid getting jammed by taking a hard inside release. This means they try to run past the jammers by cutting sharply toward the middle of the field. This vacates space on the outsides of the field. The jammers end up blocking the gunners, who are now unable to get close to the returner by the time he catches the ball. The returner then runs into the space vacated by the gunners and then turns up field. At this point, the returner has managed to get into open space so the kicking team players (Steelers) turn back towards their own goal to try and chase the returner down. However, the defensive linemen that initially tried to block the punt are now running towards their own goal. One of them lays a block toward his own goal. The concussion happens here. \n\nWe can see the implications of formation on the return. The punt returner is much more likely to make a play if there are no gunners running at him as he has much more space to work with. The following play has the same 4 jammer formation, but in this case, the jammers don't succeed on the left side. \n\n\nIn this play, the right gunner makes his way downfield, albeit with a hard outside release, so he is running down the sideline. The gunner runs past the returner and effectively takes himself out of the play, so the returner now turns across the field and makes his way up the field. The punting long snapper (PLS) number 49, had gone past the returners current yardline so he turns back up field and across the field to chase after the returner. Again his orientation is toward his own goal and the direction of motion is similar to that of the returners. At this point he is blocked by a defensive linemen who is running downfield to make the block. The contact happens in the legal strike zone, and hence is not illegal, but still gives the opposition player a concussion. The direction of motion and orientation of the impacted player is opposite that of the player who blocked him, but both are oriented toward their own goal lines. \n\nThe above three plays show offensive line injuries, but interestingly, we see there are two defensive linemen injuries (in the 4 jammer formation, referring back to the previous code block injuries based on jammer formation). The following video is a play where the defensive end (number 33) gets injured. This is, again, a 4 jammer formation. \n\n\nIn this play, the gunners are limited in their way down the field, and the returner makes quite a few people miss. As the kicking team players turn to chase the returner, number 33 comes running downfield and tries to make a block toward his own goal on the opposing number 78 (Dolphins). In trying to make this block, 33 himself gets a concussion. \n\n\n\nSimilar play, same formation type, the defensive lineman on the Chiefs (second from the left on the defensive line) attempts to block the punt. He is not succesful and then proceeds to turn around in order to make a block for the punt returner. Tyreek Hill (a dangerous punt returner) sees the area vacated by the jammed gunners and attempts to run up that sideline. The defensive line player then comes in (direction of motion and orientation are opposite that of Hill and the players behind Hill at this point) and makes helmet-to-helmet contact with Giants number 51. The defensive end gave himself a concussion with that attempted block toward his own goal. Even though this was an illegal block, this video reveals the mindset of players on the returning team. They are more incentivized to make more dangerous blocks which would suggest why most concussions belong to the kicking team. \n\nWe can see a pattern beginning to emerge. Kicking team players attempting to make a tackle are in dangerous positions because players on the returning team are coming back towards them at high speeds to make blocks. Additionally, these blocks are dangerous for both parties as they happen at high speeds.\n\nLet's see generally what happens in this kind of play, the following shows how the play develops in stages:\n\nThe play starts out as a normal play with 2 sets of 2 jammers (a).\n![](\"https://i.imgur.com/7s2B4jm.png\")
\n\n\n"},{"metadata":{"_uuid":"6f87b7c08665f179e58eeba83d6a4513470e9f21"},"cell_type":"markdown","source":"As the play starts, the general motion of the players is shown (b).\n\n![](\"https://i.imgur.com/cgSWDaX.png\")
"},{"metadata":{"_uuid":"8e80df8c98ca6a9e3043c6130ac2b3f8f890f564"},"cell_type":"markdown","source":"As some of the players on the defensive line drop toward their own endline to make blocks, others try to block the punt. The offensive linemen block the defensive linemen and then proceed to make their way downfield. The gunners are stopped by the jammers and the returner can make his way toward one of those lanes vacated by the blocked gunner (c).\n\n![](\"https://i.imgur.com/gZQLX1E.png\")
"},{"metadata":{"_uuid":"aa691fda77ef85ab69c98a359270ff7c7db14ef7"},"cell_type":"markdown","source":"As the blocks are made, the punt returner finds a lane near one of the sidelines and runs toward it, his motion and orientation are opposite that of many of the tacklers and some of his own teammates (PDR1, PDR2, PDL3, PDL1). After making some of the players on the blue team miss, the PR runs into the gap, and the blue team players now turn back upfield. toward their own goal, in order to make the tackle (d).\n\n![](\"https://i.imgur.com/hRgl8va.png\"/)
"},{"metadata":{"_uuid":"8fd0e8a6949002bab93b22da11e1f443df6e2a77"},"cell_type":"markdown","source":"Now we can see that PRT, PRG, PLS, and even PLW are in danger of getting blocked toward their own goal. Their orientation is turning up field so that they can meet the path of PR, but PDL1, PDL3, PDR2, PDR1 can now make a block on any of these players while running at high speed toward their own goal (on the right of the image below) (e).\n\n![](\"https://i.imgur.com/pgM2GM9.png\")
"},{"metadata":{"_uuid":"695249d4cd140ed6f834758529f1e8a0880e7ff3"},"cell_type":"markdown","source":"The area highlighted between the red lines in the image below is a danger zone for the kicking team players. This is because as they turn to chase the returner, they are likely to be blindsided by blockers on the returning team. This danger area needs to be the place where punting team members are protected so that they do not receive concussions at the rate that they do on these kinds of plays. \n\n![](\"https://i.imgur.com/04e6y1l.png\")
"},{"metadata":{"_uuid":"77efcc644f5141e6ddf2c79909d61ac7ffee4776"},"cell_type":"markdown","source":"Now we see why the 4 jammer formation is dangerous for the punting team. The area in the middle of the field is generally occupied and the punt returner usually has a lane up the sideline for a return. Now the two jammer and the three jammer formation also have more punting team concussions than returning. From our feature selection we see that the orientation and direction of the players on the offensive team are still really important in determing if there is a concussion or not, hence we will again look at the problem from that perspective. The first question that we have to answer is, since there are potentially two extra non-jammers in the two jammer and three jammer formation, what positions do these extra men take? We answer this by looking at the number of defensive linemen and linebackers for the 2 jammer and 3 jammer formations. "},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"c4de131cfbc21d8e6e704235a33bb84581be1121"},"cell_type":"code","source":"count = 0\njammerIDSBackers = {}\ndef countBackerFormations(df,s):\n jammercounts = {}\n for x in range(len(df)):\n forms = df.iloc[x].split(', ')\n j = 0\n backer = 0\n dl = 0\n for y in forms:\n if(y == \"PR\" or y == \"PFB\"): continue\n if y.startswith(\"V\"):\n j += 1\n continue\n if(\"DL\" in y or \"DR\" in y):\n dl += 1\n else:\n backer += 1\n if(j == 5):\n j = 4\n #print(df.iloc[x].split(', '),j)\n if(j == 4): continue\n \n try: jammercounts[(backer,dl)] +=1\n except: jammercounts[(backer,dl)] = 1\n #for n in sorted(jammercounts):\n #print(\"The formation with %d linebackers and %d linemen appears %d times %s\" % (n[0],n[1],jammercounts[n],s))\n #print()\n return jammercounts\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"236a16c7426e299806ec7c1d27b2fd75404fd4bb","scrolled":false},"cell_type":"code","source":"a = countBackerFormations(returnTeamForms,'overall')\nb = countBackerFormations(concussReturnTeamsForms,'in the concussion set')\nfor x in b:\n print(\"The formation with %d linebackers and %d linemen has concussion percentage %f\" % (x[0],x[1],(b[x]*100/a[x])))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6961fb2a248bf2f804db1ad0c1ae34c9f3cb092a"},"cell_type":"markdown","source":"Interestingly, we see that the formation with 1 linebacker and 5 linemen has a concussion percentage of 5% as opposed to the less than 1 percent for the other formations that had concussion. Since we know there are at most 3 jammers and a punt returner, the remaining position MUST be a PFB, or a punt full back. This formation was only used 60 times throughout the year, but there were 3 concussions on this formation. In order to understand why, we can look at 2 of the plays a fullback was used. \n''\nWe see in this play that there is a left jammer and two right jammers. The full back drops facing the away from the punt returner as the ball is kicked in the air. Then, when the ball is caught, the full back changes his direction of motion toward the closest covering player, in this case is the gunner on the side with only one jammer. He then blocks the player, and the returner is able to run toward the side with two jammers for a decent gain. We see that in this play, the full back acted as a kind of free-roam blocker, but the only player downfield fast enough for him to block is the gunner. By drawing the gunner in, and having a personal blocker for the punt returner, this gave the returner an interesting angle in his return and allowed him to run upfield. He is eventually hit by a helmet-to-helmet collision (illegal) which caused a concussion. \n''\\\nWe see on this play that the full back acts as a linebacker and then drops straight back, oriented away from the punt returner and toward the oncoming tacklers. As he drops, he does not make a block until the punt returner catches the ball, after which the PFB attempts to make a block toward his own goal. Blocking towards ones own goal again causes a concussion for a player on the punting team. \n\nWe see that this formation causes problems because players are drawn into the danger zone where they can potentially be blindsided by a block towards the blocker's own goal. \n\nFurthermore, if we look at the role of the linebackers in a punting formation we can understand where there is an orientation and direction issue. In a 3 jammer formation, the linebackers have freedom to make blocks as they wish. \n\n![](\"https://i.imgur.com/LMOtcis.png\")
"},{"metadata":{"_uuid":"0b37951469cd89807ac2c643b098ad59fb454605"},"cell_type":"markdown","source":"Referring to the image above, the PLR can drop to block the PLW or PPL (if there is one) and the PLL can drop to either block the gunner (GR) toward his own goal or block the PRW. The linebackers have the freedom to block the people with the greatest threat of getting downfield. Sometimes the linebackers can even rush the punter to try and get a blocked punt, but usually they drop and try to set up blocks for the punt returners.\n\nThe more linebackers there are, the more blocking the PR has set up for him to follow, but fewer players rush the punter. Usually this is an attempt to set up a return down the middle as the players are being forced outside in the example above. We can see that the collisions that happen during the blocking are intense and happen at high speeds and head on. If the PLW beats the PLR outside, and attempts to come back upfield, he is in the danger zone. Essentially, players turning up field to chase the returner are in the danger zone. \n\nThe starting formations of the play imply a lot about the behavior of the return team players. We can use this information in order to define a rule change in accordance to the features we found through the video review and feature selection. "},{"metadata":{"_uuid":"c069987c368ea520717b5a94a4f5af22b683a27b"},"cell_type":"markdown","source":"**Proposed Rule Change**\n\nWe are proposing 2 rule changes.\n\n*Any player blocking toward their own end zone is assessed an unnecessary roughness penalty of 15 yards.*\n\nWe believe the NFL should make blocking towards your own end zone illegal during kicking plays in order to decrease the number of concussions. The main reasons are as follows:\n\n1. There are 16 plays with players blocking toward their own goal resulting in concussions. (16/37) concussions could be prevented with this rule change. \n2. Blocking toward your own goal is more likely to be a head-on or blindsided collisions. This maximizes the force of impact thus increasing the chances of a concussion and eliminates the danger area. Through our data analysis we have found that orientation and direction are two important features associated with concussions. Our proposed rule tackles (pun-intended) these two features directly. The rule will decrease the number of concussion causing punt plays.\n3. This rule change is simple to integrate and easy to learn. Referees need to be aware of any players blocking towards their own goal.\n4. This rule preserves the integrity of the sport. We believe that it would not decentivize punt returns, which we believe is an integral part of the sport. We will address how our proposed changes could introduce new risks to player safety in the next paragraph. Punting is a major part of football and many teams can gain a huge advantage through their punt return teams and punt coverage teams. Changing the rules too drastically would remove the importance this play type. \n\nSince players are not allowed to block toward their own end zone, they may be more inclined to play more off coverage, giving the PR a chance to make a return. Longer returns could lead to greater possilbilities of concussions. As the defensive linemen would not be able to come back down the field and make a block, teams may chose to use the defensive linemen more aggresively in rushing the punter, potentially leading to dangers for the offensive line and the punter. \n\n*Any player intiating helmet-to-helmet contact during a punt play is assessed an unnecessary roughness penalty of 15 yards and disqualified from the contest.*\n\nAdditionally, we think that intentional helmet-to-helmet contact should result in immediate ejection. The main reasons are as follows:\n\n1. There are 16 plays with helmet-to-helmet contact resulting in concussions. (16/37) concussions could be prevented with this rule change. \n2. In a punt play, since players are moving downfield at high speeds in many directions and orientations, a helmet-to-helmet contact would be especially dangerous. While there is a rule in place that makes helmet-to-helmet contact illegal, we see that it still occurs. By making the punishment for this dangerous play even harsher, we strongly incentivize players to not commit this infraction. \n3. This rule change is simple to integrate. Referees need to be aware of any players initiating head-to-head contact.\n4. This rule change preserves the integrity of the sport. The NFL has added rules in place attempting to make this contact illegal for many years, the harsher punishment would make sure this happens. \n\nIt is possible that players who have learned over the years to hit in one way may have to change their tackling technique in order to make sure they do not get disqualified by this rule. This can potentially lead to players injuring other parts of their body. This could be dangerous for both parties involved in the tackles. "},{"metadata":{"_uuid":"7d78ca5ba77e3f4d4fe8b36e5042ed299429cc44"},"cell_type":"markdown","source":"**Rebuttal to Other Rule Changes**\n\nIn addition to our proposed rule changes we also considered other rules:\n\n*Adding restrictions on starting formations*\n\nWe decided that adding restrictions on the starting formations would take away from the integrity of the sport. Punting is an offensive play, which means fake punt passes and fake punt throws are valid plays; we did not want to restrict the creativity of the teams on these plays. Furthermore, from our data analysis we see that formations are not the major feature causing concussions, rather it is the actions taken by the players in those formations. Making those actions illegal is safer than trying to restrict the formation.\n\n*Incentivising fair catches*\n\nWe decided that incentivizing fair catches strongly impacts the integrity of the game. Adding some number of yards to the end of the fair catch would give the receiving team an advantage. It would allow the returning team to rush the punter without worrying about blocking for the returner. The kicking team is at a disadvantage regardless of whether the returner fair catches.\n\n*Moving the Touchback from the 20 to the 15*\n\nThis would incentivize the kicking team to kick through the end zone on every punt, because they would guarantee bad field position for the other team. This changes the integrity of the game by significantly decreasing the chances of a return. We think that returns are one of the most essential and exciting aspects of football. "},{"metadata":{"trusted":true,"_uuid":"90b495a1ae295667fa5c64ef889c8cf5351998b1"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}