{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"**An Analysis of injury factors in NFL**\n\n\nThe main purpose of this analysis is to find out what factors cause injury to NFL athletes through the analysis of the data provided. It not only contributes to the effects of synthetic turn or natural turn, but also to the analysis of various factors to help prevent injury in advance"},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\nsns.set_style('whitegrid')\n\ninjury_df = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/InjuryRecord.csv')\nplayer_df = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayList.csv')\nplayer_track_info_df = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayerTrackData.csv')\ninjury_df.shape, player_df.shape, player_track_info_df.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# column names lower for convenience\ninjury_df.columns = injury_df.columns.str.lower()\nplayer_df.columns = player_df.columns.str.lower()\nplayer_track_info_df.columns = player_track_info_df.columns.str.lower()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**_EDA_**"},{"metadata":{"trusted":true},"cell_type":"code","source":"# check any missing value from Injury data\ninjury_df.isnull().sum(), player_df.isnull().sum(), player_track_info_df.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We have only 105 injury data. I think, this is not enough data to design effective model to predict. But still, we can go over the data which factors give negative or positive impact for the injury even though it is not significant. And there are 28 missing value in playkey. This would be fiiled with the value of the Playlist. For rest of missing value in other data tables, I tried not to fill out because most of them are categorical values and there is a lack of evidence to estimate the value."},{"metadata":{"trusted":true},"cell_type":"code","source":"injury_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# extract gameid with no playkey from injury_df\ngameid_no_playkey = injury_df[injury_df['playkey'].isnull()]['gameid']\ngameid_no_playkey\nplaykey_=[]\nfor id in gameid_no_playkey:\n    playkey_.append(player_df[player_df['gameid']==id].iloc[len(player_df[player_df['gameid']==id])-1,2])\n\nii=0\nfor i in gameid_no_playkey.index:\n    injury_df.loc[i,'playkey']= playkey_[ii]\n    ii=ii+1\ninjury_df.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Injury bodypart Frequency Check"},{"metadata":{"trusted":true},"cell_type":"code","source":"injury_df.groupby('bodypart').count()['playerkey'].sort_values(ascending=False).plot(kind='bar',\n                                                                     figsize=(15,5),\n                                                                     title=\"# of injuries by BodyPart\")\nfor a, b in enumerate(injury_df.groupby('bodypart').count()['playerkey'].sort_values(ascending=False)):\n    plt.text(a,b,str(b))\n    \nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can easily find, Knee and Ankle are mostly injured. Let's take a look fieldtype which causes injury more. "},{"metadata":{"trusted":true},"cell_type":"code","source":"injury_df.groupby('surface')['playerkey'].count().sort_values(ascending=False)\\\n.plot(kind='bar', figsize=(6,6), title=\"# of injuries on different fieldtype\")\nfor a, b in enumerate(injury_df.groupby('surface').count()['playerkey'].sort_values(ascending=False)):\n    plt.text(a,b,str(b))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can find out that injuries happen on Synthetic turf more times than natural turf. Let's see the percentage of body part injured on different field types. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# % of body injuried on diff surfaces\npd.set_option('display.float_format', '{:.2f}'.format)\ninjury_df.groupby(['surface','bodypart'])['playerkey'].count().unstack('bodypart').T.sort_values('Natural').T.sort_values('Ankle').plot(kind='bar', figsize=(15,5))\nprevious_r=injury_df.groupby(['surface','bodypart'])['playerkey'].count()\nprevious_s=injury_df.groupby('surface')['playerkey'].count()\nprevious_p=previous_r/previous_s*100\nprevious_p.unstack('bodypart').T.sort_values('Natural').T.sort_values('Ankle').plot(kind='bar', figsize=(15,5))\nprevious_p","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nOn both Natural and Synthetic turf, knee and ankle injuries accounted for about 85% of all injuries. However, Foot injuries on natural turf, toe injuries on synthetic turf seem to be more environmental effects. \n\n\n**Determine When to occur**"},{"metadata":{"trusted":true},"cell_type":"code","source":"inj_detailed = injury_df.merge(player_df)\n# find out when to occur\np_d_i =inj_detailed.groupby('playergame')['playerkey'].count()\np_day_inj = pd.DataFrame(p_d_i)\np_day_inj.columns = ['inj_cnt']\np_day_inj.reset_index(inplace=True)\n#fig = plt.figure()\n#ax = plt.axes()\np_day_inj.plot(x='playergame', y='inj_cnt',  figsize=(12,5))\nplt.ylabel('Number of Injuries'), plt.xlabel('number of games'), plt.title('Injury Trend')\nplt.legend(['Injury count'])\np_day_inj['inj_p_gday']=(p_day_inj['inj_cnt']/p_day_inj['inj_cnt'].sum())*100\np_day_inj['inj_p_acu']=p_day_inj['inj_p_gday'].cumsum()\np_day_inj['inj_c_acu']=p_day_inj['inj_cnt'].cumsum()\np_day_inj.plot(x='playergame', y='inj_p_acu',  figsize=(12,5), color='orange')\nplt.ylabel('Accumulated percentage of Injuries'), plt.xlabel('number of games'), plt.title('Injury Trend')\nplt.legend(['Accumulated injury (%)'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As shown in the above results, there have been about 70/105 injuries in 10 games, accounting for about 67% of the total injuries."},{"metadata":{"trusted":true},"cell_type":"code","source":"p_p_d=player_df.groupby(['playergame'])['playerkey'].nunique()\np_p_day = pd.DataFrame(p_p_d)\np_p_day.columns = ['nofplayers']\np_p_day.reset_index(inplace=True)\ninj_pct_by_time=pd.merge(p_day_inj, p_p_day, left_on='playergame', right_on='playergame')\ninj_pct_by_time['inj_pct']= inj_pct_by_time['inj_cnt']/inj_pct_by_time['nofplayers']*100\ninj_pct_by_time['inj_pct_player_acu']=inj_pct_by_time['inj_pct'].cumsum()\ninj_pct_by_time.plot('playergame', 'inj_pct', figsize=(12,5))\nplt.title('injury % in total players'), plt.xlabel('number of games'), plt.ylabel('injury percentage')\nplt.legend(['Percentage / players'])\ninj_pct_by_time.plot('playergame', 'inj_pct_player_acu', figsize=(12,5),color='orange')\nplt.title('Accumulated injury % in total players'), plt.xlabel('number of games'), plt.ylabel('Accumulated injury percentage')\nplt.legend(['Accumulated percentage / players'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above results indicate the number of injuries per player. In the graph below, about 30 percent of 250 players in 10 games were injured.\n\n\nSmall result from EDA of Injury data: \n   - Most injuries occurred in ankle and knee\n   - More injuries on synthetic turf\n   - 10 more ankle injuries on the synthetic turf\n   - The percentage of injuries on each field type show similar features\n     However, foot injuries on the natural turf and toe injuries on the synthetic turf\n     are more concerned\n   - about 67% of injuries in 10 gamies were occured\n   - about 30 % of 250 players in 10 games were injured"},{"metadata":{},"cell_type":"markdown","source":"Combine Injury data and Playerlist data to identify the bodypart of injury according to the type of play"},{"metadata":{"trusted":true},"cell_type":"code","source":"inj_detailed = injury_df.merge(player_df)\n# percentage of injury body part on each different fieldtypes \ninj_body=inj_detailed.groupby(['surface','playtype','bodypart'])['playerkey'].count().unstack('bodypart')\ninj_tot_body=inj_detailed.groupby(['surface','playtype'])['playerkey'].count()\ninj_p_body=pd.DataFrame()\nfor i in inj_body.columns:\n      inj_p_body[i]=inj_body[i]/inj_tot_body\n\ninj_p_body.loc['Natural'].plot(kind='bar', figsize=(10,4), title='Natural',stacked=True)\n\ninj_p_body.loc['Synthetic'].plot(kind='bar', figsize=(10,4), title='Synthetic',stacked=True)\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nIn this result, what I can see is that specially, on the synthetic turf, ankle injuries were more common in the kicking playtype."},{"metadata":{},"cell_type":"markdown","source":"By the position, injury percentage on each different fieldtype"},{"metadata":{"trusted":true},"cell_type":"code","source":"#  injury percentage on each position\ninj_detailed.groupby(['rosterposition','surface']).count().unstack('surface')['playerkey']\\\n    .T.apply(lambda x : x/x.sum()).sort_values('surface').T.sort_values('Natural')\\\n    .plot(kind='barh',figsize=(15,5), title='injury percentage(%) on each position', stacked=True)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The result of separating the total number of injury occurrences for each position according to the field type is that in the case of cornerbacks, more injuries occurred on synthetic turf."},{"metadata":{"trusted":true},"cell_type":"code","source":"#injury percentage on each fieldtype\ninj_detailed.groupby(['surface','rosterposition','bodypart']).count()['playerkey']\ninj_rosterposition=inj_detailed.groupby(['surface','rosterposition']).count()['playerkey']\ninj_surface=inj_detailed.groupby(['surface']).count()['playerkey']\ninj_ros_sur_p = inj_rosterposition/inj_surface\ninj_ros_sur_p.unstack('rosterposition').T.sort_values('Natural').T.sort_values('Cornerback').plot(kind='bar',figsize=(15,5), stacked=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"However, if you divide injuries by the environment and look at the percentage of injuries caused by the environment, you can see slightly different results. On the Natural turf, especially the Linebacker position has more injuries than on the synthetic turf. This accounts for more than 25% of all injuries on natural turf. On the Synthetic turf, you can see that the injury of a cornerback position occurs noticeably more than that of natural turf. Compared to the result of the natural turf, there is about a 15% difference. How do they play? Let's check their play type."},{"metadata":{"trusted":true},"cell_type":"code","source":"inj_detailed[inj_detailed['rosterposition'].isin(['Cornerback','Linebacker'])].groupby(['rosterposition','surface','playtype'])['playerkey'].count().unstack('playtype').plot(kind='bar', figsize=(10,4), title='Injuries by playtype',stacked=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above results show that Cornerbacks were particularly prone to injuries by pass play on synthetic turf. The Linebackers were injured by various plays. But in the case of punts, more injuries were reported on the natural turf. So, where do they get injured? "},{"metadata":{"trusted":true},"cell_type":"code","source":"# injury bodypart by position(%)\ninj_bodypart_freq = inj_detailed.groupby(['surface','rosterposition','bodypart'])['bodypart'].count()\ninj_bodypart_pct_nat=inj_bodypart_freq['Natural']/inj_bodypart_freq['Natural'].sum()*100\ninj_bodypart_pct_syn=inj_bodypart_freq['Synthetic']/inj_bodypart_freq['Synthetic'].sum()*100\n\ninj_bodypart_pct_nat.unstack('bodypart').plot(kind='bar', figsize=(10,4), title='Natural Turf',stacked=True)\ninj_bodypart_pct_syn.unstack('bodypart').plot(kind='bar', figsize=(10,4),title='Synthetic Turf',stacked=True)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above results show that on the Synthetic turf, Cornerbacks suffered injuries to their ankle, knee, and foot. Linebackers suffered ankle and knee injuries in both environments. The results also show that athletes in various positions are injured in toes on synthetic turf, regardless of their position. This seems to be caused by environmental factors. Then, how serious are the injuries?"},{"metadata":{"trusted":true},"cell_type":"code","source":"# grouping by fieldtype\ninj_sep =inj_detailed.groupby(['surface','dm_m1','dm_m7','dm_m28','dm_m42'])['playerkey'].count()\ninj_sep.unstack(['dm_m1','dm_m7','dm_m28','dm_m42']).plot(kind='bar', figsize=(10,4), stacked=True)\nplt.legend(['1d','7d','28d','42d'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This result shows that injuries on synthetic turf seem more serious. This is because more cases have been registered as injured for longer than 28 days on synthetic turf. "},{"metadata":{"trusted":true},"cell_type":"code","source":"inj_detailed['light_inj']= inj_detailed['dm_m1']+inj_detailed['dm_m7']\ninj_detailed['light_inj']=inj_detailed['light_inj'].map((lambda x: True if x>0 else False)).astype(np.int)\ninj_detailed['heavy_inj']= inj_detailed['dm_m28']+inj_detailed['dm_m42']\ninj_detailed['heavy_inj']=inj_detailed['heavy_inj'].map((lambda x: True if x>0 else False)).astype(np.int)\n\npd.set_option('display.float_format','{:.2f}'. format)\ninj_heavy =inj_detailed.groupby('surface')['heavy_inj'].sum()\ninj_surface=inj_detailed.groupby('surface')['heavy_inj'].count()\ninj_heavy_surface_p=round(inj_heavy/inj_surface*100,1)\nfor a,b in enumerate(inj_heavy_surface_p):\n    plt.text(a,b,str(b))\ninj_heavy_surface_p.plot(kind='bar', stacked=True)\nplt.title('Percentage of Severe injury on each different field type')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Heavy injuries by position\ninj_detailed.groupby(['rosterposition'])['heavy_inj'].sum().sort_values(ascending=False).plot(kind='bar', figsize=(10,4))\nplt.title('Severe injuries by play position')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Severe injuries are more apparent in Linebackers and Wide Receivers.\n\n** Data Preprocessing 2**"},{"metadata":{"trusted":true},"cell_type":"code","source":"#merge injury with playlist table\n\ntotal_player=player_df.groupby(['gameid'])['playkey'].count()\ninjured_player=injury_df.groupby(['gameid'])['playkey'].count()\n\ntest_frame = pd.DataFrame(total_player)\ntest_frame.columns=['tot_cnt_p_game']\ntest_frame.reset_index(level=0,inplace=True)\n\ninjury_player_frame = pd.DataFrame(injured_player)\ninjury_player_frame.columns=['injury_cnt_p_game']\ninjury_player_frame.reset_index(level=0, inplace=True)\n\nt_i_rate=pd.merge(test_frame,injury_player_frame, left_on=['gameid'], right_on=['gameid'], how='left')\nt_i_rate.fillna(0,inplace=True)\n\n# data trim(replacement, generalization)\n#stadium data\nst_map={'Outdoor':'outdoor_nc', 'Indoors':'indoor_wc', 'Oudoor':'outdoor_nc', 'Outdoors':'outdoor_nc', 'Open':'indoor_nc', 'Closed Dome':'indoor_wc',\n       'Domed, closed':'indoor_wc', 'Dome':'indoor_wc', 'Indoor':'indoor_wc', 'Domed':'indoor_wc',\n       'Retr. Roof-Closed':'indoor_wc', 'Outdoor Retr Roof-Open':'indoor_nc', 'Retractable Roof':'indoor_nc',\n       'Ourdoor':'outdoor_nc', 'Indoor, Roof Closed':'indoor_wc', 'Retr. Roof - Closed':'indoor_wc', 'Bowl':'outdoor_nc',\n       'Outddors':'outdoor_nc', 'Retr. Roof-Open':'indoor_nc', 'Dome, closed':'indoor_wc', 'Indoor, Open Roof':'indoor_nc',\n       'Domed, Open':'indoor_nc', 'Domed, open':'indoor_nc', 'Heinz Field':'outdoor_nc', 'Cloudy':'outdoor_nc',\n       'Retr. Roof - Open':'indoor_nc', 'Retr. Roof Closed':'indoor_wc', 'Outdor':'outdoor_nc', 'Outside':'outdoor_nc'}\n\n\n# weather data \nweather_map={'Clear and warm':'clear', 'Mostly Cloudy':'cloudy', 'Sunny':'sunny', 'Clear':'clear', 'Cloudy':'cloudy',\n       'Cloudy, fog started developing in 2nd quarter':'fog', 'Rain':'rain',\n       'Partly Cloudy':'cloudy', 'Mostly cloudy':'cloudy', 'Cloudy and cold':'cloudy',\n       'Cloudy and Cool':'cloudy', 'Rain Chance 40%':'cloudy', 'Controlled Climate':'clear',\n       'Sunny and warm':'sunny', 'Partly cloudy':'cloudy', 'Clear and Cool':'clear',\n       'Clear and cold':'clear', 'Sunny and cold':'sunny', 'Indoor':'clear', 'Partly Sunny':'sunny',\n       'N/A (Indoors)':'clear', 'Mostly Sunny':'sunny', 'Indoors':'clear', 'Clear Skies':'clear',\n       'Partly sunny':'sunny', 'Showers':'rain', 'N/A Indoor':'clear', 'Sunny and clear':'clear', 'Snow':'snow',\n       'Scattered Showers':'rain', 'Party Cloudy':'cloudy', 'Clear skies':'clear',\n       'Rain likely, temps in low 40s.':'rain', 'Hazy':'cloudy', 'Partly Clouidy':'cloudy',\n       'Sunny Skies':'sunny', 'Overcast':'cloudy', 'Cloudy, 50% change of rain':'rain', 'Fair':'cloudy',\n       'Light Rain':'rain', 'Partly clear':'clear', 'Mostly Coudy':'cloudy', '10% Chance of Rain':'cloudy',\n       'Cloudy, chance of rain':'cloudy', 'Heat Index 95':'sunny',\n       'Sunny, highs to upper 80s':'sunny', 'Sun & clouds':'cloudy',\n       'Heavy lake effect snow':'snow', 'Mostly sunny':'sunny', 'Cloudy, Rain':'rain',\n       'Sunny, Windy':'sunny', 'Mostly Sunny Skies':'sunny', 'Rainy':'rain',\n       '30% Chance of Rain':'cloudy', 'Cloudy, light snow accumulating 1-3\"':'snow',\n       'cloudy':'cloudy', 'Clear and Sunny':'sunny', 'Coudy':'cloudy', 'Clear and sunny':'sunny',\n       'Clear to Partly Cloudy':'cloudy',\n       'Cloudy with periods of rain, thunder possible. Winds shifting to WNW, 10-20 mph.':'rain',\n       'Rain shower':'rain', 'Cold':'cold'}\n\n\nplayer_df2=player_df.copy() # don't want to touch original data\nplayer_df2['weather']=player_df2['weather'].map(weather_map)\nplayer_df2['stadiumtype']=player_df['stadiumtype'].map(st_map)\n\n#remove duplication based on gameid and temperature missinng value fill out by mean value it won't affect much.\nplayer_df3=player_df2.drop_duplicates('gameid')\nplayer_df_nodup=player_df3.copy()\nplayer_df_nodup[(player_df_nodup['temperature']!=-999)].groupby('stadiumtype').mean()\ntemp_indoornc_index=player_df_nodup[(player_df_nodup['temperature']==-999) & (player_df_nodup['stadiumtype']=='indoor_nc')].temperature.index\ntemp_indoorwc_index=player_df_nodup[(player_df_nodup['temperature']==-999) & (player_df_nodup['stadiumtype']=='indoor_wc')].temperature.index\ntemp_outdoornc_index=player_df_nodup[(player_df_nodup['temperature']==-999) & (player_df_nodup['stadiumtype']=='outdoor_nc')].temperature.index\n\nplayer_df_nodup.loc[temp_indoornc_index,'temperature']=72\nplayer_df_nodup.loc[temp_indoorwc_index,'temperature']=67\nplayer_df_nodup.loc[temp_outdoornc_index,'temperature']=59\n\n# temperature categorize.\nplayer_df_nodup['temperature'].unique()\ntemp_bins=[0, 10, 20, 30,40,50,60,70,80,90,100]\nplayer_df_nodup['tempe_cat'] = pd.cut(player_df_nodup['temperature'],temp_bins)\n\nplay_inj_df=pd.merge(player_df_nodup,t_i_rate, left_on=['gameid'], right_on=['gameid'], how='left')\n\ninj_pick=inj_detailed[['gameid','bodypart','heavy_inj']]\ntot_df=pd.merge(play_inj_df,inj_pick, left_on=['gameid'], right_on=['gameid'], how='left')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Injury percentage per game on each different stadium"},{"metadata":{"trusted":true},"cell_type":"code","source":"a=tot_df.groupby('stadiumtype')['playerkey'].count()\na=pd.DataFrame(a)\na.columns=['tot_game']\na.reset_index(inplace=True)\nb=tot_df.groupby('stadiumtype')['injury_cnt_p_game'].sum()\nb=pd.DataFrame(b)\nb.columns=['inj_cnt']\nb.reset_index(inplace=True)\nstadium_inj_pct=pd.merge(a,b, left_on='stadiumtype', right_on='stadiumtype')\nstadium_inj_pct['pct']= round(stadium_inj_pct['inj_cnt']/stadium_inj_pct['tot_game']*100,2)\nstadium_inj_pct.plot('stadiumtype','pct', kind=\"bar\", figsize=(10,4), title='Injury percentage')\nplt.legend([])\ndel a,b\nfor a, b in enumerate(stadium_inj_pct['pct']):\n    plt.text(a,b, str(b))\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Definition of stadium type :** \n     - indoor_nc  : Indoor stadium without weather control(open)\n     - indoor_wc  : Indoor stadium with weather control(closed)\n     - outdoor_nc : Outdoor stadium, no weather control\n\nThe above results show that the percentage of injuries occurring in indoor stadiums is higher than that of outdoor stadiums, even though there is about 1% difference. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# number of games in different stadium types\ntot_df.groupby(['stadiumtype'])['playerkey'].count().plot(kind='bar',figsize=(10,5))\nfor a, b in enumerate(tot_df.groupby(['stadiumtype'])['playerkey'].count()):\n    plt.text(a,b, str(b))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As shown above, the number of games held in the outdoor_nc was about 3 times more than the number of games held in the inoor, and about 10 times more than that of the inoor_nc."},{"metadata":{"trusted":true},"cell_type":"code","source":"# injury frequency and percentage by stadium type \nst_bd_cnt =tot_df.groupby(['stadiumtype','bodypart'])['playerkey'].count() # 부상부위 빈도수\nst_bd_cnt_frame=pd.DataFrame(st_bd_cnt)\nst_bd_cnt_frame.columns = ['bd_cnt']\nst_bd_cnt_frame.reset_index(inplace=True)\nst_cnt=tot_df.groupby(['stadiumtype'])['playerkey'].count()# stadium 별 부상 빈도\n##st_bd_p=st_bd_cnt.copy()\nst_bd_p_inc=(st_bd_cnt['indoor_nc']/st_cnt['indoor_nc'])*100\nst_bd_p_iwc=(st_bd_cnt['indoor_wc']/st_cnt['indoor_wc'])*100\nst_bd_p_onc=(st_bd_cnt['outdoor_nc']/st_cnt['outdoor_nc'])*100\nst_bd_p_inwc=pd.concat([st_bd_p_inc,st_bd_p_iwc], axis=0)\nst_bd_p=pd.concat([st_bd_p_inwc,st_bd_p_onc], axis=0)\nst_bd_p_frame=pd.DataFrame(st_bd_p)\nst_bd_p_frame.columns = ['pct']\nst_bd_p_frame.reset_index(inplace=True)\n#st_bd_p_frame.drop('bodypart', axis=1,inplace=True)\nst_bd_cnt_frame['pct']=st_bd_p_frame['pct'].copy()\na=st_bd_cnt_frame.pivot(index='bodypart',columns='stadiumtype',values='pct')\nb=st_bd_cnt_frame.pivot(index='bodypart',columns='stadiumtype',values='bd_cnt')\n#st_bd_cnt_frame.plot('bodypart','pct',kind=\"bar\", figsize=(3,4), title='Injury percentage')\nb.plot(kind='bar',figsize=(10,4)) \nplt.ylabel('Frequency/game')\na.plot(kind='bar',figsize=(10,4)) \nplt.ylabel('%/game')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From the above results, the number of injuries sustained in the outdoor_nc was the largest in ankle and knee, but the percentage of injuries per game was the highest in the indoor_nc, with more than 1%. The rate of injury in the indoor stadium is more than 5 % and higher than outdoor's."},{"metadata":{"trusted":true},"cell_type":"code","source":"# injury frequency and percentage by fieldtype\n\nplayer_play_fieldtype_freq=tot_df.groupby(['fieldtype','bodypart'])['playerday'].count()\npp_fieldtype_freq=pd.DataFrame(player_play_fieldtype_freq)\npp_fieldtype_freq.columns = ['play_cnt']\npp_fieldtype_freq.reset_index(inplace=True)\nft_cnt=tot_df.groupby(['fieldtype'])['playerkey'].count()\n# bodypart injury percentage per game on each different fieldtype\nft_bd_p_na=(player_play_fieldtype_freq['Natural']/ft_cnt['Natural'])*100\nft_bd_p_sy=(player_play_fieldtype_freq['Synthetic']/ft_cnt['Synthetic'])*100\n\nft_bd_p=pd.concat([ft_bd_p_na,ft_bd_p_sy], axis=0)\nft_bd_p_frame=pd.DataFrame(ft_bd_p)\nft_bd_p_frame.columns = ['pct']\nft_bd_p_frame.reset_index(inplace=True)\npp_fieldtype_freq['pct']=ft_bd_p_frame['pct'].copy()\na=pp_fieldtype_freq.pivot(index='bodypart',columns='fieldtype',values='pct')\nb=pp_fieldtype_freq.pivot(index='bodypart',columns='fieldtype',values='play_cnt')\n\nfig, axes = plt.subplots(3)\nb.plot(kind='bar',figsize=(15,20),title='Number of Injury', ax=axes[0]) \naxes[0].set_ylabel('Frequency')\na.plot(kind='bar',figsize=(15,20),title='Percentage of Injury',ax=axes[1]) \naxes[1].set_ylabel('Percentage(%)')\n\npp_fieldtype_inj_pct=pd.read_csv('/kaggle/input/pp-fieldtype-inj-freq/pp_fieldtype_inj_freq.csv')\n\n\n\npp_fieldtype_inj_pct.groupby('fieldtype')['inj_pct'].mean().plot(kind='bar',figsize=(15,20),ax=axes[2])\nplt.title('Injury percentage on each field type')\naxes[2].set_ylabel('Percentage(%)')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Based on that results above, on synthetic turf, ankle injury rate is twice that of natural turf and toes injury rate is more than 8 times.Finally, the graph shows the rate of injury per player. Athletes have an injury rate of about 5 percent on synthetic turf and about 3 percent on natural turf.\n\n**Weather/Temperature**"},{"metadata":{"trusted":true},"cell_type":"code","source":"# injury percentage by weather on each different fieldtype \na=tot_df.groupby(['weather','fieldtype'])['injury_cnt_p_game'].sum()\nwe_fi_freq=pd.DataFrame(a)\nwe_fi_freq.columns = ['inj_cnt']\nwe_fi_freq.reset_index(inplace=True)\nwe_cnt=tot_df.groupby(['weather','fieldtype'])['playerkey'].count()\nb=pd.DataFrame(we_cnt)\nb.columns=['game_cnt']\nb.reset_index(inplace=True)\nwe_fi_freq['game_cnt']=b['game_cnt']\nwe_fi_freq['pct']=(we_fi_freq['inj_cnt']/we_fi_freq['game_cnt'])*100\n\na=we_fi_freq.pivot(index='weather',columns='fieldtype',values='pct')\nb=we_fi_freq.pivot(index='weather',columns='fieldtype',values='inj_cnt')\n\nfig, axes = plt.subplots(2)\nb.plot(kind='bar', ax=axes[0]) \naxes[0].set(ylabel='Frequency/game', title='Number of injury / game')\n\na.plot(kind='bar',figsize=(10,13),ax=axes[1]) \naxes[1].set(ylabel='%/game', title='Percentage of injury /game')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above results show the rate of injury per game for each weather condition. From the results, the rate of injuries on cold days is the highest at about 33 %, with injuries occurring in 3 out of 9 games held. It is important to note that all of this occurred on natural turf."},{"metadata":{"trusted":true},"cell_type":"code","source":"# injury percentage by temperature on each different fieldtype \na=tot_df.groupby(['tempe_cat','fieldtype'])['injury_cnt_p_game'].sum()\ntp_fi_freq=pd.DataFrame(a)\ntp_fi_freq.columns = ['inj_cnt']\ntp_fi_freq.reset_index(inplace=True)\ntp_cnt=tot_df.groupby(['tempe_cat','fieldtype'])['playerkey'].count()\nb=pd.DataFrame(tp_cnt)\nb.columns=['game_cnt']\nb.reset_index(inplace=True)\ntp_fi_freq['game_cnt']=b['game_cnt']\ntp_fi_freq['pct']=(tp_fi_freq['inj_cnt']/tp_fi_freq['game_cnt'])*100\n\na=tp_fi_freq.pivot(index='tempe_cat',columns='fieldtype',values='pct')\nb=tp_fi_freq.pivot(index='tempe_cat',columns='fieldtype',values='inj_cnt')\n\nfig, axes = plt.subplots(2)\nb.plot(kind='bar', ax=axes[0]) \naxes[0].set(ylabel='Frequency/game', title='Number of injury / game')\n\na.plot(kind='bar',figsize=(10,13),ax=axes[1]) \naxes[1].set(ylabel='%/game', title='Percentage of injury /game')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Looking first at the environment of synthetic turf, the most injuries occurred at temperatures between 60 and 70 degrees, but more than 4% were injured at temperatures between 70 and 80 degrees. In particular, it is surprising that about 9% of injuries per game occurred at temperatures higher than 60 degrees. Natural turf showed the highest incidence at low temperatures of 30-40 degrees and at high temperatures of 80-90 degrees, and the lowest in the 40-50 degrees range.\nOverall, 40-60 degrees showed low injury rates per game regardless of the field type.\nThe point to note here is between 60 and 80 degrees. In this temperature range, the incidence of injuries per game in synthetic turf was more than double that of natural turf."},{"metadata":{},"cell_type":"markdown","source":"Speed matter?\n\n"},{"metadata":{"_cell_guid":"","_uuid":"","trusted":true},"cell_type":"code","source":"import pandas as pd\npt_inj_expt = pd.read_csv(\"/kaggle/input/pt-inj-expt/pt_inj_expt.csv\")\ninj_detailed_df2=pd.read_csv('/kaggle/input/inj-detailed-df2/inj_detailed_df2.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"a=inj_detailed_df2.groupby(['playerkey'])['s'].mean()\na_frame=pd.DataFrame(a)\na_frame.columns =['inj_avg_s']\na_frame.reset_index(inplace=True)\nb=pt_inj_expt.groupby('playerkey')['s'].mean()\nb_frame=pd.DataFrame(b)\nb_frame.columns =['norm_avg_s']\nb_frame.reset_index(inplace=True)\n\n\n\nc=pd.merge(a_frame,b_frame, left_on='playerkey', right_on='playerkey', how='left' )# injured\nd=pd.merge(b_frame,a_frame, left_on='playerkey', right_on='playerkey', how='left' )# total","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Average speed of same athletes in both cases: Injured vs. Not injured  \nfig, axes=plt.subplots(2, figsize=(15,12))\nc.boxplot(column=['inj_avg_s','norm_avg_s'],vert=True,notch=True, ax=axes[0])\naxes[0].set_title('Average speed of athletes when Injured vs. Not injured ')\naxes[0].set_ylabel('speed')\nd.boxplot(column=['inj_avg_s','norm_avg_s'], notch=True,patch_artist=True,vert= True, ax=axes[1])\naxes[1].set_title('Average speed Injured vs. Not injured')\naxes[1].set_ylabel('speed')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above first results show the average speed of the injured player in the injury game and the average speed in the normal game. As the results show, the average speed in the injured game was about 0.7 y/s slower. The speed at which no injuries are sustained is about the same as that of other players."},{"metadata":{"trusted":true},"cell_type":"code","source":"# avg speed on each different field type injured vs. not injured\nf =player_df.groupby(['playkey','fieldtype'])['gameid'].count()\nff=pd.DataFrame(f)\nff.reset_index(inplace=True)\nff.drop('gameid', axis=1, inplace=True)\n#del f\npt_inj_expt_df2=pd.merge(pt_inj_expt,ff, left_on='playkey',right_on='playkey', how='left')\n\nf=pt_inj_expt_df2.groupby('fieldtype')['s'].mean()\nf_frame=pd.DataFrame(f)\nf_frame.columns =['norm_avg_s']\nf_frame.reset_index(inplace=True)\ng=inj_detailed_df2.groupby(['fieldtype'])['s'].mean()\ng_frame=pd.DataFrame(g)\ng_frame.columns =['inj_avg_s']\ng_frame.reset_index(inplace=True)\nf_field=pd.merge(g_frame, f_frame, left_on='fieldtype', right_on='fieldtype', how='left')\nf_field.plot(x='fieldtype',kind='bar',title='Average speed by fieldtype',figsize=(15,7))\n#f_field.set_title('Average speed by fieldtype')\nplt.ylabel('speed')\nplt.legend(['Injury','Normal'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The following are the average speed results of the injured and the normal athletes for each field type. As shown in the above results, the average speed of the injured players, regardless of their field type, was about 0.25 to 0.5y/s slow. The average speed of players on synthetic turf is slightly faster, but speed gap between injured and not injured  are bigger."},{"metadata":{"trusted":true},"cell_type":"code","source":"import scipy.stats as stats\ninj_playkey_list=injury_df['playkey'].tolist()\n#fig, axes = plt.subplots(1,2)\nfig = plt.figure(figsize=(7,9))\na=player_track_info_df.query('playkey in @inj_playkey_list')['s']\nsp_a= stats.gaussian_kde(a)\nn,x, _ = plt.hist(a, bins=10, histtype=u'step', density=True, color='blue')\nplt.title('distribution of player speed injured vs. not injured')\nplt.plot(sp_a(x))\nb=player_track_info_df.query('playkey not in @inj_playkey_list')['s'].sample(1000000)\nsp_b= stats.gaussian_kde(b)\nn,x1, _ = plt.hist(b, bins=10, histtype=u'step', density=True, color='orange')\nplt.plot(sp_b(x1))\nplt.legend(['Injured', 'Not injured'])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nThe above results are based on a Gaussian distribution of the speed of the injured and the uninjured athletes. The distribution showed a difference in the highest value, but the distribution of speed is not very different."},{"metadata":{"trusted":true},"cell_type":"code","source":"import scipy.stats as stats\n#inj_playkey_list=injury_df['playkey'].tolist()\n#fig, axes = plt.subplots(1,2)\nfig = plt.figure(figsize=(7,9))\na=player_track_info_df.query('playkey in @inj_playkey_list')['dis']\nsp_a= stats.gaussian_kde(a)\nn,x, _ = plt.hist(a, bins=10, histtype=u'step', density=True, color='blue')\nplt.title('distribution of player distance injured vs. not injured')\nplt.plot(sp_a(x))\nb=player_track_info_df.query('playkey not in @inj_playkey_list')['dis'].sample(1000000)\nsp_b= stats.gaussian_kde(b)\nn,x1, _ = plt.hist(b, bins=10, histtype=u'step', density=True, color='orange')\nplt.plot(sp_b(x1))\nplt.legend(['Injured', 'Not injured'])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above results are based on a Gaussian distribution of distance between injured and uninjured athletes. The distribution appears to be more diverse in the case of the injured. Diversity can increase the probability of injury. However, as an athlete, diversity does not seem to be a factor that needs to be removed to avoid injury. The injury sample size is also too small, so there may be other results when more data is available.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Creating master table considering relationship\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\nsns.set_style('whitegrid')\n\ntot_df_test =tot_df.drop(['playerkey', 'gameid','playkey','playergameplay',\n                          'position','positiongroup','bodypart',\n                         'heavy_inj'], axis=1)\n\ntot_df_test['injury_cnt_p_game']=tot_df_test['injury_cnt_p_game'].astype(np.int64)\n\ntot_df_test_num = pd.get_dummies(tot_df_test)\n\ntest =['fieldtype_Natural', 'fieldtype_Synthetic', 'playtype_Kickoff', 'playtype_Punt','playtype_Pass','playtype_Rush','rosterposition_Cornerback','rosterposition_Linebacker',\n        'stadiumtype_indoor_wc','stadiumtype_indoor_nc','stadiumtype_outdoor_nc', \n        'weather_clear','weather_cold','tempe_cat_(30, 40]','tempe_cat_(60, 70]','tempe_cat_(70, 80]',\n       'tempe_cat_(80, 90]','injury_cnt_p_game']\ncorr = tot_df_test_num[test].corr()\nfig = plt.figure(figsize=(14,12))\nsns.heatmap(corr, annot=True, cmap=sns.diverging_palette(220,20, as_cmap=True))\nplt.title('Correlation Heatmap')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nThe correlation map above shows that field type shows opposite characteristics for injury. Natural turf seems to help prevent injury, and the kicking style play seems to increase injury. Cold weather, indoor statium type seems to have a bad effect on injury. At temperature, 70-80 range seem to have the most adverse effect on injuries.\n\n**FEATURE SELECTION BY ML(Garbage In Garbage Out)**"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.feature_selection import SelectKBest, f_classif, SelectFromModel, RFE\nfrom sklearn.ensemble import RandomForestRegressor\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import  train_test_split\nimport statsmodels.api as sm\n\n\n\n\nX = tot_df_test_num[tot_df_test_num.columns.difference(['injury_cnt_p_game'])]\ny = tot_df_test_num['injury_cnt_p_game']\n\n# 1.\nlogistic = LogisticRegression(C=0.2, penalty=\"l1\", random_state=0).fit(X,y)\nmodel = SelectFromModel(logistic, prefit=True)\nX_new = model.transform(X)\n\n\n# 2.\n#model = SelectKBest(f_classif, k=10)\n#X_new = model.fit_transform(X, y)\n\n\n\n#for 1,2\nselected_features = pd.DataFrame(model.inverse_transform(X_new),\n                                index=X.index, columns=X.columns)\nselected_columns = selected_features.columns[selected_features.var()!=0]\nselected_columns = selected_columns.insert(0,'injury_cnt_p_game')\nfor i in range(0,len(selected_columns)):\n    if selected_columns[i]=='playerday':\n        selected_columns = selected_columns.delete(i)\n        break\n        \ncor=tot_df_test_num[selected_columns].corr()# correlation map\n\nfig = plt.figure(figsize=(10,8))\nsns.heatmap(cor, annot=True, cmap=sns.diverging_palette(220,20, as_cmap=True))\nplt.title('Correlation Heatmap by Logistic')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above correlation map is a logistic regression that classifies features that affect injury more among many features. Let’s look at some of the different features I’ve found in our exploratory data analysis(EDA). If looked at the features classified by the logistic regression algorithm, the playergame, the defensive/offensive Lineman position, and the cloud/sunny in weather were newly classified as influencing features."},{"metadata":{"trusted":true},"cell_type":"code","source":"# 2.\nmodel = SelectKBest(f_classif, k=10)\nX_new = model.fit_transform(X, y)\n\n\nselected_features = pd.DataFrame(model.inverse_transform(X_new),\n                                index=X.index, columns=X.columns)\nselected_columns = selected_features.columns[selected_features.var()!=0]\nselected_columns = selected_columns.insert(0,'injury_cnt_p_game')\nfor i in range(0,len(selected_columns)):\n    if selected_columns[i]=='playerday':\n        selected_columns = selected_columns.delete(i)\n        break\n        \ncor=tot_df_test_num[selected_columns].corr()# correlation map\n\nfig = plt.figure(figsize=(10,8))\nsns.heatmap(cor, annot=True, cmap=sns.diverging_palette(220,20, as_cmap=True))\nplt.title('Correlation Heatmap by SelectKBest')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above result is a correlation heatmap drawn in features that affect injuries found by other algorithms. Cornerback position and weather feature differ from those obtained by logistic regression algorithm. However, the importance of features found by this algorithm is more similar to those identified by EDA."},{"metadata":{"trusted":true},"cell_type":"code","source":"# linear regression\nfrom sklearn.feature_selection import SelectKBest, f_classif, SelectFromModel, RFE\nfrom sklearn.ensemble import RandomForestRegressor\nfrom sklearn.linear_model import LogisticRegression, LinearRegression\nfrom sklearn.model_selection import  train_test_split\nimport statsmodels.api as sm\n\n\nX = tot_df_test_num[tot_df_test_num.columns.difference(['injury_cnt_p_game'])]\ny = tot_df_test_num['injury_cnt_p_game']\n\n# 4.\ncols = list(X.columns)\nmodel = LinearRegression()\n\n#Initializing RFE model\nrfe = RFE(model,10)\nX_rfe = rfe.fit_transform(X,y)\nmodel.fit(X_rfe,y)\ntemp=pd.Series(rfe.support_, index=cols)\nselected_features_rfe = temp[temp==True].index\nprint(selected_features_rfe)\n\n##############################\n\nselected_columns = selected_features_rfe.insert(0,'injury_cnt_p_game')      \ncor=tot_df_test_num[selected_columns].corr()# correlation map\n\nfig = plt.figure(figsize=(10,8))\nsns.heatmap(cor, annot=True, cmap=sns.diverging_palette(220,20, as_cmap=True))\nplt.title('Correlation Heatmap by Linear Regression')\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nThe above result is a choice of 10 important features through linear regression. Another difference in algorithms is the importance of categorized temperature features. In addition, compared to other algorithms, more factors were chosen to prevent injury than to cause injury."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Lasso\nfrom sklearn.feature_selection import SelectKBest, f_classif, SelectFromModel, RFE\nfrom sklearn.ensemble import RandomForestRegressor\nfrom sklearn.linear_model import LogisticRegression, LinearRegression, LassoCV\nfrom sklearn.model_selection import  train_test_split\nimport statsmodels.api as sm\nimport matplotlib\n\n\nX = tot_df_test_num[tot_df_test_num.columns.difference(['injury_cnt_p_game'])]\ny = tot_df_test_num['injury_cnt_p_game']\n\n# Lasso regression()\nreg=LassoCV()\nreg.fit(X,y)\ncoef=pd.Series(reg.coef_, index = X.columns)\nimp_coef = coef.sort_values()\nmatplotlib.rcParams['figure.figsize']=(8.0,10.0)\nimp_coef.plot(kind='barh')\nplt.title('feature importance using Lasso Model')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nLasso regression automates certain parts of model selection, like feature selection or parameter elimination, and is a good algorithm for simplification. According to the above results, the algorithm chose six features of player play per game, number of games, field type, rush, and temperature as important features for the model to identify the player's injury. \nIt's not surprising that the player's rush play is a factor that affects the occurrence of the injury. However, it is hard to understand that the higher the temperature, the clearer the weather is the cause of injury.\nDue to the high temperature, or the weather being too clear, poor preparation or lack of tension over injuries may be a factor.\nIt is even more difficult to understand that the more games an athlete plays, or the more time they play, the less risk of injury. Perhaps, they are not causal data about risk, but result data.\nIt is not surprising that natural grass helps prevent injury"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}