{"cells":[{"metadata":{"colab_type":"text","id":"Ef_-6Q7hAtWE"},"cell_type":"markdown","source":"# Сompetition task:\nThe main goal is to characterize any differences in player movement between the playing surfaces and to identify specific variables (e.g., field surface, weather, position, play type, etc.) that may influence player movement and the risk of injury."},{"metadata":{"colab_type":"text","id":"W4oE2vxS4sMh"},"cell_type":"markdown","source":"# Main findings:\n**According to research the main factors influencing injury risks are as follows:** </br>\n1.\t**Turf type.** Injuries on synthetic turf happen 1.7 times more often than on natural turf. \n2.\t**Play scenario.** Injury risks vary depending on play scenario ( a combination of roster position, position, group position, play type) and the turf type. A play scenario “Wide Receiver, WR, WR, Kickoff” on synthetic turn has the highest injury risk. The chance of injury in such a scenario is about 0.9%. The highest risk injury on natural turf happens in a “Linebacker, OLB, LB, Punt” scenario. The chance of injury in such a scenario is about 0.4%.\n3.\t**Temperature.** It is likely that injury risk on synthetic turf increases with higher temperatures (approximately above 70F). One can suggest that some physical properties of synthetic turf start changing at these temperatures. Natural turf is seemingly not temperature-sensitive, as there were no indications of such dependencies found.\n4.\t**Player dynamics.** It has been found that athletes who got injured on average run faster and have more rapid accelerations than those who did not got  injured. This difference does not depend on turf type.\n\nMost discussion of analisys results are in PDF report. In Notebooks just brief points."},{"metadata":{"colab_type":"text","id":"wm14jVJnb31k"},"cell_type":"markdown","source":"# Content:"},{"metadata":{"colab_type":"text","id":"D9a6R8hEVu2U"},"cell_type":"markdown","source":"Work represents in five main parts:\n\n\n1.   Basic data exploration.\n2. Influence of game scenario.\n3. Influence of game conditions.\n4.   Athlete movement analisys.\n5. Conclusions\n\n"},{"metadata":{"colab_type":"text","id":"N5Uu-1bXi2QW"},"cell_type":"markdown","source":"**Personal preface:**\nAs luck would have it, I myself experienced a lower limb injury in the end of November, just when the challenge started. I broke my tibia and sprained my ankle as a result of not my best front flip on a trampoline. So being fully immersed and having experienced all the consequences of lower limb injury first-hand, I tried my best to help reduce such risks.\n"},{"metadata":{"colab_type":"text","id":"mFK-xm4Sbo_-"},"cell_type":"markdown","source":"# Imports:"},{"metadata":{"colab":{},"colab_type":"code","id":"qL6msgazQux4","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport warnings\nwarnings.simplefilter('ignore')","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"1S1ZqIQPXRJR"},"cell_type":"markdown","source":"# 1. Basic data exploration."},{"metadata":{"colab_type":"text","id":"hkvaESRgXUBH"},"cell_type":"markdown","source":"First fings first. Start with analysing datasets with injuryes and play list:"},{"metadata":{"colab":{},"colab_type":"code","id":"KkJmMFXiVkoe","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Import datasets:\ninjury_record = pd.read_csv(\"/kaggle/input/nfl-playing-surface-analytics/InjuryRecord.csv\")\nplay_list = pd.read_csv(\"/kaggle/input/nfl-playing-surface-analytics/PlayList.csv\")\n\ngame_id_with_injuryes = injury_record['GameID'].unique()\ngame_id_without_injuryes = play_list.query('GameID not in @game_id_with_injuryes')['GameID'].unique()\n\nnum_games_with_inj = game_id_with_injuryes.shape[0]\nnum_games_safe = game_id_without_injuryes.shape[0]\n\ninj_ratio = num_games_with_inj / num_games_safe*100\n\nprint('Percent of GameID with at least one injury: {0:.2f}%'.format(inj_ratio))","execution_count":null,"outputs":[]},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":68},"colab_type":"code","id":"rfMkT0zo4r8b","outputId":"dfb29e4c-6e86-47cf-bcae-162765214693","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"play_keys_inj = injury_record['PlayKey'].unique().shape[0]\nplay_keys_safe = play_list['PlayKey'].unique().shape[0] - play_keys_inj\n\nprint(f'Avaliable play keys for safe play: {play_keys_safe}')\nprint(f'Avaliable play keys for play with injury: {play_keys_inj}')\nprint('Percent of plays with injury: {0:.2f}%'.format(play_keys_inj / play_keys_safe *100))","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"FvjVN0mRYcD-"},"cell_type":"markdown","source":"**The good news** is that non-contact injuries happen not so frequently: only about 0.04% of plays end with some non-contact injury. <br>\nThe challenge is that there is a huge data imbalance between GameIDs and PlayKeys so it can be quite tricky to train models and draw statistically meaningful conclusions. Statistical significance of the findings should be double-checked. \n\n\n"},{"metadata":{"colab_type":"text","id":"YOrN3Xgt5BBb"},"cell_type":"markdown","source":"Let’s dig further and check is there is difference between natural and synthetic turf:"},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":360},"colab_type":"code","id":"6nzsuOQws_L9","outputId":"71b16927-6c9a-4ed9-9461-2b553c000d4d","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"inj_nat = injury_record.groupby('Surface').count()['PlayerKey']['Natural']\ninj_synt = injury_record.groupby('Surface').count()['PlayerKey']['Synthetic']\n\ngames_nat = play_list.groupby('FieldType').count()['PlayerKey']['Natural']\ngames_synt = play_list.groupby('FieldType').count()['PlayerKey']['Synthetic']\n\nnatural_inj_rate = inj_nat / games_nat * 100\nsynt_inj_rate = inj_synt / games_synt * 100\n\nsns.set_context(\"talk\", font_scale=0.8) \nsns.set_palette(\"Blues_d\")\nfig, ax = plt.subplots(1, 2, figsize=(15, 5))\n\ng1 = sns.barplot(x=['Synthetic', 'Natural'], y=[inj_synt, inj_nat], \n                 alpha=0.7, ax=ax[0])\ng2 = sns.barplot(x=['Synthetic', 'Natural'], y=[synt_inj_rate, natural_inj_rate],\n                 alpha=0.7, ax=ax[1])\n\nax[0].set_title('Absolute value of injuries', fontsize=14)\nax[1].set_title('Frequency of injury occurrence', fontsize=14)\nax[1].set_ylabel('%', fontsize=14)\n\nsns.despine()  \nfig.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"-sm-hDo5ZIHE"},"cell_type":"markdown","source":"Difference in absolute values doesn’t look significant, but one should take into account that there were more games played on natural turf. Thus comparing the relation between games resulting in injury with the total amount of games on synthetic and natural turf is more informative. Here one can see a bigger difference which supports the idea of the turf type influence."},{"metadata":{"colab_type":"text","id":"f-jNjU044Sww"},"cell_type":"markdown","source":"Lets see injury severity on different surfaces:"},{"metadata":{"colab":{},"colab_type":"code","id":"XXe_RVxi5JDg","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"inj_nat = injury_record[injury_record['Surface'] == 'Natural']\nnat_size = inj_nat.shape[0]\nnat_42 = inj_nat['DM_M42'].sum() \nnat_28 = inj_nat['DM_M28'].sum() - nat_42\nnat_7 = inj_nat['DM_M7'].sum() - nat_42 - nat_28\nnat_1 = inj_nat['DM_M1'].sum() - nat_42 - nat_28 - nat_7\n\ninj_synt = injury_record[injury_record['Surface'] == 'Synthetic']\nsynt_size = inj_synt.shape[0]\nsynt_42 = inj_synt['DM_M42'].sum()\nsynt_28 = inj_synt['DM_M28'].sum() - synt_42\nsynt_7 = inj_synt['DM_M7'].sum() - synt_42 - synt_28\nsynt_1 = inj_synt['DM_M1'].sum() - synt_42 - synt_28 - synt_7\n\nks = 100/57 \nkn = 100/48\n\nnat = [nat_1*kn, nat_7*kn, nat_28*kn, nat_42*kn] \nsynt = [synt_1*ks, synt_7*ks, synt_28*ks, synt_42*ks] \ninj_days = ['1', '1-7', '7-28', '42+']\n\nfig, ax = plt.subplots(1, 2, figsize=(15, 5))\n\ng1 = sns.barplot(x=inj_days, y=nat, order=inj_days, ax=ax[0])\ng2 = sns.barplot(x=inj_days, y=synt, order=inj_days, ax=ax[1])\n\nax[0].set_title('Natural', fontsize=14)\nax[1].set_title('Synthetic', fontsize=14)\nax[0].set_ylim([0, 50])\nax[1].set_ylim([0, 50])\nax[0].set_xlabel('Days to recover')\nax[1].set_xlabel('Days to recover')\nax[0].set_ylabel('%')\nax[1].set_ylabel('%')\nsns.despine()  \nfig.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"BGLgQv3S_m4q"},"cell_type":"markdown","source":"Distribution between recovery days after each injury looks quite similar for natural and synthetic turf. One can notice that injuries with recovering time of 28-42 days are less frequent than others. If we divide injuries into “short recovery” (1-7days to recover) and ”long recovery” (7+ days to recover) injuries, synthetic turf can be assumed to be leading to slightly bigger injury severity, as ”long recovery” injuries are about 10% more frequent."},{"metadata":{"colab_type":"text","id":"Yx1R_LutbGdn"},"cell_type":"markdown","source":"Lets discover injuries in more depth:"},{"metadata":{"colab":{},"colab_type":"code","id":"_N9Ag2QuKQXr","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"natural = injury_record[injury_record['Surface'] == 'Natural']\nsynthetic = injury_record[injury_record['Surface'] == 'Synthetic']\n\nnatural_bp = natural.groupby('BodyPart').count()['GameID']\nsynt_bp = synthetic.groupby('BodyPart').count()['GameID']\nsynt_bp['Heel'] = 0\n\nfeet_part = pd.merge(natural_bp, synt_bp, on='BodyPart')\nfeet_part.sort_values('GameID_x', inplace=True, ascending=False)\nfeet_part.columns = ['Natural', 'Synthetic']\n\nfeet_part['Natural'] = feet_part['Natural']  / 48* 100\nfeet_part['Synthetic'] = feet_part['Synthetic']  / 57 * 100\n\nnatural = injury_record[injury_record['Surface'] == 'Natural']\nsynthetic = injury_record[injury_record['Surface'] == 'Synthetic']\n\n# Injured parts\nbody_parts = injury_record['BodyPart'].unique()\n\nfig, ax = plt.subplots(1, 2, figsize=(15, 5))\nsns.barplot(x=feet_part.index, y=feet_part['Natural'], ax=ax[0])\nsns.barplot(x=feet_part.index, y=feet_part['Synthetic'], ax=ax[1])\n\nax[0].set_title('Natural')\nax[1].set_title('Synthetic')\nax[0].set_ylabel('%')\nax[1].set_ylabel('%')\n\nax[0].set_ylim([0, 60])\nax[1].set_ylim([0, 60])\n\nsns.despine() \nfig.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"3Ci57arHc-HE"},"cell_type":"markdown","source":"We can see that most frequently injured parts are ankles and knees on both types of turf. Heel injuries are rare. One can suggest that a foot is more frequently injured on natural turf and a toe is more frequently injured on synthetic turf. "},{"metadata":{"colab_type":"text","id":"r6YL1bMfXs2k"},"cell_type":"markdown","source":"**Main takeaways:**\n<li> Injuries on synthetic turf are approximately 1.7 times more frequent than on natural turf.\n<li> Toe injury received on synthetic turf is likely to be more frequent.\n<li> One can suspect that knee and foot injuries received on natural turf are more frequent.\n"},{"metadata":{"colab_type":"text","id":"poT02pWthz5F"},"cell_type":"markdown","source":"# 2. Influence of game scenario."},{"metadata":{"colab_type":"text","id":"m9EGrMX06s1Q"},"cell_type":"markdown","source":"Let us consider a distribution between the risk of injury and different roster positions. "},{"metadata":{"colab":{},"colab_type":"code","id":"1k3m9pdjD_UO","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Import datasets:\ninjury_record = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/InjuryRecord.csv')\nplay_list = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayList.csv')\n\npk_with_injuryes = injury_record['PlayKey'].unique()\ninjured = play_list.query('PlayKey in @pk_with_injuryes')\n\ninjured_rp_sorted = list(injured['RosterPosition'].value_counts().sort_values(ascending=False).index)\n\n# getting roster position that never injured (safe_rp)\nsafe_rp = []\nfor rp in play_list['RosterPosition'].unique():\n  if rp not in injured_rp_sorted:\n    safe_rp.append(rp)\n\nfor sp in safe_rp:\n  injured_rp_sorted.append(sp)\n\n# calculating ratio of injuries in relation for total plays in according roster position\ninj_rp = injured['RosterPosition'].value_counts()\nfor rp in safe_rp:\n  inj_rp[rp] = 0\ntotal_rp = play_list['RosterPosition'].value_counts()\nrp_inj_ratio = inj_rp / total_rp * 100\n\nplay_list_natural = play_list[play_list['FieldType']=='Natural']\nplay_list_synt = play_list[play_list['FieldType']=='Synthetic']\n\ninjured_natural = play_list_natural.query('PlayKey in @pk_with_injuryes')\ninjured_synthetic = play_list_synt.query('PlayKey in @pk_with_injuryes')\n\ninj_rp_natural = injured_natural['RosterPosition'].value_counts()\ninj_rp_synt = injured_synthetic['RosterPosition'].value_counts()\ntotal_rp_natural = play_list_natural['RosterPosition'].value_counts()\ntotal_rp_synt = play_list_synt['RosterPosition'].value_counts()\n\nrp_inj_ratio_natural = inj_rp_natural / total_rp_natural * 100\nrp_inj_ratio_synt = inj_rp_synt / total_rp_synt * 100\n\nfig, ax = plt.subplots(1, 2, figsize=(15, 5))\nsns.set_palette(\"Blues_d\")\nsns.countplot(y=injured['RosterPosition'],\n              order=rp_inj_ratio.sort_values(ascending=False).index, \n              ax=ax[0], palette=\"Blues_d\")\n\nsns.barplot(x=rp_inj_ratio.sort_values(ascending=False).values,\n            y=rp_inj_ratio.sort_values(ascending=False).index,\n              ax=ax[1], palette=\"Blues_d\")\n\nax[0].set_title('Absolute values')\nax[1].set_title('Relative values')\nax[1].set_xlabel('frequency of injury, %')\nsns.despine() \nfig.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"3tpCOoysIJSb"},"cell_type":"markdown","source":"Most injuries happen in a Linebacker position, but most frequent injuries happen in a Running Back position. <br> The safest positions are quarterback and kicker positions. Such a dependence feels quite natural as kickers and quarterbacks are likely to run less than a running back or a linebacker. "},{"metadata":{"colab_type":"text","id":"x6LBDBhP6tJs"},"cell_type":"markdown","source":"Next, let us consider a turf type. "},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":452},"colab_type":"code","id":"BxDYd2ai1bMu","outputId":"70ebae53-b0a8-47d3-bf4c-7fb500b94f4c","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize = (12, 6))\n\nd_rate = rp_inj_ratio_natural - rp_inj_ratio_synt\n\nsns.barplot(x=d_rate.sort_values(ascending=False).values,\n            y=d_rate.sort_values(ascending=False).index,\n            label='Synthetic',\n              ax=ax, palette=\"Blues_d\", edgecolor='w')\nax.set_title('Difference between natural and synthetic injury frequences.')\nax.set_xlabel('%')\nsns.despine()\nplt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"js3qeM3d69JK"},"cell_type":"markdown","source":"Most roster positions injuries occur more frequently on synthetic turf, except for an offensive lineman, who is more likely to get an injury on natural turf. The more player moves during the game, the higher the risk of getting an injury on synthetic turf is and the stronger turf influence is. It seems valid as an offensive lineman isn’t likely to move as much as other players. So it fits with a proposed idea of a cleats-turf interaction"},{"metadata":{"colab_type":"text","id":"mzcpmwqZ_DmZ"},"cell_type":"markdown","source":"**Next we performe same analisys for play type:**"},{"metadata":{"colab":{},"colab_type":"code","id":"l6BLrtDk_Cv5","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"injured_pt_sorted = list(injured['PlayType'].value_counts().sort_values(ascending=False).index)\n\n# getting roster position that never injured (safe_rp)\nsafe_pt = []\nfor pt in play_list['PlayType'].unique():\n  if pt not in injured_pt_sorted:\n    safe_pt.append(pt)\n\nfor pt in safe_pt:\n  if pt not in ['0', 0 , np.nan, 'nan']:\n    injured_pt_sorted.append(pt)\n\n# calculating ratio of injuries in relation for total plays in according roster position\ninj_pt = injured['PlayType'].value_counts()\nfor pt in safe_pt:\n  inj_pt[pt] = 0\ntotal_pt = play_list['PlayType'].value_counts()\npt_inj_ratio = (inj_pt / total_pt * 100).drop(index='0')\n\nfig, ax = plt.subplots(1, 2, figsize=(15, 5))\n\nsns.countplot(y=injured['PlayType'],\n              order=pt_inj_ratio.sort_values(ascending=False).index, \n              ax=ax[0], palette=\"Blues_d\")\n\nsns.barplot(x=pt_inj_ratio.sort_values(ascending=False).values,\n            y=pt_inj_ratio.sort_values(ascending=False).index,\n              ax=ax[1], palette=\"Blues_d\")\n\nax[0].set_title('Absolute values')\nax[1].set_title('Relative values')\nax[1].set_xlabel('frequency of injury, %')\nsns.despine()\nfig.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"STJqV2E9AHQO"},"cell_type":"markdown","source":"And dividing by turf type:"},{"metadata":{"colab":{},"colab_type":"code","id":"qiB_UY0rAJ4b","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"play_list_natural = play_list[play_list['FieldType']=='Natural']\nplay_list_synt = play_list[play_list['FieldType']=='Synthetic']\n\ninjured_natural = play_list_natural.query('PlayKey in @pk_with_injuryes')\ninjured_synthetic = play_list_synt.query('PlayKey in @pk_with_injuryes')\n\ninj_pt_natural = injured_natural['PlayType'].value_counts()\ninj_pt_synt = injured_synthetic['PlayType'].value_counts()\ntotal_pt_natural = play_list_natural['PlayType'].value_counts()\ntotal_pt_synt = play_list_synt['PlayType'].value_counts()\n\npt_inj_ratio_natural = inj_pt_natural / total_pt_natural * 100\npt_inj_ratio_synt = inj_pt_synt / total_pt_synt * 100\n\nfig, ax = plt.subplots(figsize = (12, 6))\n\nd_rate = (pt_inj_ratio_natural.fillna(0) - pt_inj_ratio_synt.fillna(0)).drop(index='0')\n\nsns.barplot(x=d_rate.sort_values(ascending=False).values,\n            y=d_rate.sort_values(ascending=False).index,\n            label='Synthetic',\n              ax=ax, palette=\"Blues_d\", edgecolor='w')\nax.set_title('Difference between natural and synthetic injury frequences.')\nax.set_xlabel('%')\nsns.despine()\nplt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"YaCvQeUsTQd5"},"cell_type":"markdown","source":"Lets break down position and position group:"},{"metadata":{"colab":{},"colab_type":"code","id":"dc_hbvFETZfT","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"injured_pos_sorted = list(injured['Position'].value_counts().sort_values(ascending=False).index)\n\n# getting position that never injured \nsafe_pos = []\nfor pos in play_list['Position'].unique():\n  if pos not in injured_pos_sorted:\n    safe_pos.append(pos)\n\nfor pos in safe_pos:\n  if pos != 'Missing Data':\n    injured_pos_sorted.append(pos)\n\n# calculating ratio of injuries in relation for total plays in according to position\ninj_pos = injured['Position'].value_counts()\nfor pos in safe_pos:\n  inj_pos[pos] = 0\ntotal_pos = play_list['Position'].value_counts()\npos_inj_ratio = (inj_pos / total_pos * 100).drop(index='Missing Data')\n\ninjured_posg_sorted = list(injured['PositionGroup'].value_counts().sort_values(ascending=False).index)\n\n# getting position group that never injured \nsafe_posg = []\nfor posg in play_list['PositionGroup'].unique():\n  if posg not in injured_posg_sorted:\n    safe_posg.append(posg)\n\nfor posg in safe_posg:\n  if posg != 'Missing Data':\n    injured_posg_sorted.append(posg)\n\n# calculating ratio of injuries in relation for total plays in according to position group\ninj_posg = injured['PositionGroup'].value_counts()\nfor posg in safe_posg:\n  inj_posg[posg] = 0\ntotal_posg = play_list['PositionGroup'].value_counts()\nposg_inj_ratio = (inj_posg / total_posg * 100).drop(index='Missing Data')\n\nfig, ax = plt.subplots(2, 2, figsize=(15, 10))\n\nsns.countplot(y=injured['Position'],\n              order=pos_inj_ratio.sort_values(ascending=False).index, \n              ax=ax[0][0], palette=\"Blues_d\")\n\nsns.barplot(x=pos_inj_ratio.sort_values(ascending=False).values,\n            y=pos_inj_ratio.sort_values(ascending=False).index,\n              ax=ax[0][1], palette=\"Blues_d\")\n\nsns.countplot(y=injured['PositionGroup'],\n              order=posg_inj_ratio.sort_values(ascending=False).index, \n              ax=ax[1][0], palette=\"Blues_d\")\n\nsns.barplot(x=posg_inj_ratio.sort_values(ascending=False).values,\n            y=posg_inj_ratio.sort_values(ascending=False).index,\n              ax=ax[1][1], palette=\"Blues_d\")\n\nax[0][0].set_title('Absolute values')\nax[0][1].set_title('Relative values')\nax[0][1].set_xlabel('frequency of injury, %')\nax[1][1].set_xlabel('frequency of injury, %')\nsns.despine()\nfig.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"ZNccYvYma9nZ"},"cell_type":"markdown","source":"Most injuries happen at a WR position and LB group positions, but the most frequent injuries happen at DB position and RB group positions."},{"metadata":{"colab_type":"text","id":"M_AsXZaGElhR"},"cell_type":"markdown","source":"**Factor combinations:**\nIn order to group plays by similarity in terms of roster position, position, group position and play type, data was represented in a dendrogram with cosine similarity. Dendrograms were created for injured plays, separately for natural and synthetic turf."},{"metadata":{"colab":{},"colab_type":"code","id":"YyudFVYtzYIM","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"injury_record = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/InjuryRecord.csv')\nplay_list = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayList.csv')\n\ninjured_natural = injured[injured['FieldType'] == 'Natural']\ninjured_synth = injured[injured['FieldType'] == 'Synthetic']\n\ninjured_rp_sort_nat = injured_natural['RosterPosition']\ninjured_pos_sort_nat = injured_natural['Position']\ninjured_posg_sort_nat = injured_natural['PositionGroup']\ninjured_pt_sort_nat = injured_natural['PlayType']\n\ninjured_rp_sort_synt = injured_synth['RosterPosition']\ninjured_pos_sort_synt = injured_synth['Position']\ninjured_posg_sort_synt = injured_synth['PositionGroup']\ninjured_pt_sort_synt = injured_synth['PlayType']\n\n\ninj_groups_nat = pd.concat([injured_rp_sort_nat, injured_pos_sort_nat, \n                            injured_posg_sort_nat, injured_pt_sort_nat], axis=1)\ninj_groups_nat_d = pd.get_dummies(inj_groups_nat)\n\ninj_groups_synt = pd.concat([injured_rp_sort_synt, injured_pos_sort_synt, \n                             injured_posg_sort_synt, injured_pt_sort_synt], axis=1)\ninj_groups_synt_d = pd.get_dummies(inj_groups_synt)","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"fmYGe8yIFf4G"},"cell_type":"markdown","source":"With threshold = 0.1 we can obtain clusters with same 4 common positions:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from scipy.cluster.hierarchy import fcluster, dendrogram, linkage\n\ntreshold = 0.1\n\nplt.subplots(figsize=(12, 7))\n\nZ_nat = linkage(inj_groups_nat_d, method='average', metric='cosine')\ndend_nat = dendrogram(Z_nat)\n\nplt.axhline(y=treshold, c='grey', lw=2, linestyle='dashed')\nplt.tick_params(axis='x', which='major', labelsize=15)\nplt.title('Natural')\nsns.despine()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"treshold = 0.1\n\nplt.subplots(figsize=(12, 7))\n\nZ_synt = linkage(inj_groups_synt_d, method='average', metric='cosine')\ndend_synt = dendrogram(Z_synt)\n\nplt.axhline(y=treshold, c='grey', lw=2, linestyle='dashed')\nplt.tick_params(axis='x', which='major', labelsize=15)\nplt.title('Synthetic')\nsns.despine()","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"WAQi4oAD7j75","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_clusters(labels, groups):\n  clusters = {key: ([],[]) for key in range(1, labels.max()+1)}\n  for label, player in zip(labels, groups.index):\n    clusters[label][1].append(player)\n  for key in clusters.keys():\n    try:\n      clusters[key][0].append(\n          np.dot(groups.loc[clusters[key][1][0]],\n                groups.loc[clusters[key][1][1]])\n      )\n    except IndexError:\n      clusters[key][0].append(0)\n  return clusters\n\nlabels_nat = fcluster(Z_nat, treshold, criterion='distance') \nlabels_synt = fcluster(Z_synt, treshold, criterion='distance') \n\nclusters_nat = get_clusters(labels_nat, inj_groups_nat_d)\nclusters_synt = get_clusters(labels_synt, inj_groups_synt_d)","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"bhwmOl2G839S","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def calc_groups_stat(clusters, groups):\n  cols = ['RosterPosition', 'Position', 'PositionGroup', 'PlayType', 'sum']\n  groups_stat = pd.DataFrame(columns=cols)\n\n  idx = 0\n  for key in clusters.keys():\n    inj_sum = len(clusters[key][1])\n    if clusters[key][0][0] == 4:\n      rp = groups.loc[clusters[key][1][0]]['RosterPosition'] \n      pos = groups.loc[clusters[key][1][0]]['Position']\n      pg = groups.loc[clusters[key][1][0]]['PositionGroup']\n      pt = groups.loc[clusters[key][1][0]]['PlayType'] \n    else:\n      continue\n    col = [rp, pos, pg, pt, inj_sum]\n    groups_stat.loc[idx] = col\n    idx += 1\n  return groups_stat\n\ngroups_stat_nat = calc_groups_stat(clusters_nat, inj_groups_nat)\ngroups_stat_synt = calc_groups_stat(clusters_synt, inj_groups_synt)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Groups for natural field:"},{"metadata":{"colab":{},"colab_type":"code","id":"g9WwJOJk9M_h","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"groups_stat_nat.sort_values(by='sum', ascending=False, inplace=True)\ngroups_stat_nat.reset_index(drop=True, inplace=True)\ngroups_stat_nat","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"raw","source":"Groups for synthetic field:"},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":297},"colab_type":"code","id":"uOILHLSr2oxw","outputId":"bb4bd7c1-9f58-4cef-db98-40ae77aea965","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"groups_stat_synt.sort_values(by='sum', ascending=False, inplace=True)\ngroups_stat_synt.reset_index(drop=True, inplace=True)\ngroups_stat_synt","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"play_list_nat = play_list[play_list['FieldType']=='Natural']\nplay_list_synt = play_list[play_list['FieldType']=='Synthetic']\n\ncount_synt = pd.DataFrame(columns=['sum_total'])\n\nfor i in groups_stat_synt.index:\n  rp = groups_stat_synt.loc[i]['RosterPosition']\n  pos = groups_stat_synt.loc[i]['Position']\n  pg = groups_stat_synt.loc[i]['PositionGroup']\n  pt = groups_stat_synt.loc[i]['PlayType']\n\n  subset = play_list_synt[play_list_synt['RosterPosition'] == rp] \\\n            [play_list_synt['Position'] == pos] \\\n            [play_list_synt['PositionGroup'] == pg] \\\n            [play_list_synt['PlayType'] == pt]\n  count_synt.loc[i] = [subset.shape[0]]\n    \ncount_nat = pd.DataFrame(columns=['sum_total'])\n\nfor i in groups_stat_nat.index:\n  rp = groups_stat_nat.loc[i]['RosterPosition']\n  pos = groups_stat_nat.loc[i]['Position']\n  pg = groups_stat_nat.loc[i]['PositionGroup']\n  pt = groups_stat_nat.loc[i]['PlayType']\n\n  subset = play_list_nat[play_list_nat['RosterPosition'] == rp] \\\n            [play_list_nat['Position'] == pos] \\\n            [play_list_nat['PositionGroup'] == pg] \\\n            [play_list_nat['PlayType'] == pt]\n  count_nat.loc[i] = [subset.shape[0]]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For natural turf:"},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":266},"colab_type":"code","id":"QnHvjg9GUvs5","outputId":"a1dadbed-1273-4a66-98c0-7a5269f0b83a","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"groups_stat_nat_tot = pd.merge(groups_stat_nat, count_nat, left_on=groups_stat_nat.index, right_on=count_nat.index).drop('key_0', axis=1)\ngroups_stat_nat_tot['Ratio'] = groups_stat_nat_tot['sum'] / groups_stat_nat_tot['sum_total'] * 100\ngroups_stat_nat_tot.sort_values(by='Ratio', ascending=False, inplace=True)\npd.options.display.float_format = '{:.3f}'.format\ngroups_stat_nat_tot","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"raw","source":"For synthetic turf:"},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":297},"colab_type":"code","id":"y1yB7SmR3wO1","outputId":"34408837-a1f0-450d-b774-ecfbd9f5be12","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"groups_stat_synt_tot = pd.merge(groups_stat_synt, count_synt, left_on=groups_stat_synt.index, right_on=count_synt.index).drop('key_0', axis=1)\ngroups_stat_synt_tot['Ratio'] = groups_stat_synt_tot['sum'] / groups_stat_synt_tot['sum_total'] * 100\ngroups_stat_synt_tot.sort_values(by='Ratio', ascending=False, inplace=True)\npd.options.display.float_format = '{:.3f}'.format\ngroups_stat_synt_tot","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"ZjR3dI1Xac3F"},"cell_type":"markdown","source":"We can see that a scenario “Wide Receiver, WR, WR, Kickoff” on synthetic turf leads to the highest injury frequency from all current observations. A chance of injury in such a scenario is about 0.9%. \n<br> A scenario “Linebacker, OLB, LB, Punt” on natural turf results in a chance of injury of 0.4%. Rest of the scenarios had much lower frequency.\n"},{"metadata":{"colab_type":"text","id":"1kCKsVSAZIxm"},"cell_type":"markdown","source":"**Main takeaways:**\n<li>\tIf one considers factors separately, a running back roster position leads to highest injury risk, about 0.05% of athletes in this position were injured. Also about 0.16% of punts resulted in athletes getting injured, 0.08% of athletes in a DB position and 0.05% of athletes in a RB group position got injured.\n<li>The highest risk injury on synthetic turf happens in a “Wide Receiver, WR, WR, Kickoff” scenario. The chance of injury in such a scenario is about 0.9%. \n<li>The highest risk injury on natural turf happens in a “Linebacker, OLB, LB, Punt” scenario. The chance of injury in such a scenario is about 0.4%.\n<li>\tAthletes in positions with more movement get injured on synthetic turf more frequently.\n"},{"metadata":{"colab_type":"text","id":"Bhd1LhwiX5Bt"},"cell_type":"markdown","source":"# 3. Influence of game conditions."},{"metadata":{"colab":{},"colab_type":"code","id":"nKkmvt5diKun","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"injury_record = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/InjuryRecord.csv')\nplay_list = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayList.csv')","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"MZVM3_iRI9Ds"},"cell_type":"markdown","source":"First of all corresponding data should be cleaned as there are typos in “Weather” and “StadiumType” fields. "},{"metadata":{"colab_type":"text","id":"1MSB_4SXKu8p"},"cell_type":"markdown","source":"Stadium types will be represented as::\n1.   Open\n2.   Closed\n"},{"metadata":{"colab":{},"colab_type":"code","id":"dAvplWsvJHER","trusted":true,"_kg_hide-input":false},"cell_type":"code","source":"# Renaming open stadiums\nopen_stadiums = ['Outdoor', 'Oudoor', 'Outdoors', 'Open', \n       'Outdoor Retr Roof-Open', 'Ourdoor', 'Bowl', \n       'Outddors', 'Retr. Roof-Open', 'Indoor, Open Roof',\n       'Domed, Open', 'Domed, open', 'Heinz Field',\n       'Retr. Roof - Open', 'Outdor', 'Outside']\n\nplay_list.replace(open_stadiums, 'Open', inplace=True)\n\n# Renaming closed stadiums\nclosed_stadiums = ['Indoors', 'Closed Dome', 'Domed, closed', \n                   'Dome', 'Indoor', 'Domed', 'Retr. Roof-Closed', \n                   'Retractable Roof', 'Indoor, Roof Closed', \n                   'Retr. Roof - Closed', 'Dome, closed', 'Retr. Roof Closed']\n\nplay_list.replace(closed_stadiums, 'Closed', inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"SiGoptCuK8SP","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# dropping 'Cloudy' data (outlier)\nidx_to_drop = play_list[play_list['StadiumType'] == 'Cloudy'].index\nplay_list.drop(idx_to_drop, inplace=True)\n\n# Dropping nan data\nplay_list['StadiumType'].dropna(axis=0, inplace=True)\n\ngames_total = play_list['GameID'].unique().shape[0]","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"RT971fzpLlXV"},"cell_type":"markdown","source":"Weather will be represented as:\n\n1. Clear\n2. Cloudy\n3. Rainy\n4. Snowy"},{"metadata":{"colab":{},"colab_type":"code","id":"qCuYCDEFLHFs","trusted":true},"cell_type":"code","source":"clear = ['Clear and warm', 'Sunny', 'Clear',\n       'Controlled Climate', 'Sunny and warm', 'Clear and Cool',\n       'Clear and cold', 'Sunny and cold', 'Closed', 'Partly Sunny',\n       'Mostly Sunny', 'Clear Skies', 'Partly sunny',\n       'Sunny and clear', 'Clear skies', 'Sunny Skies',\n       'Fair', 'Partly clear', 'Heat Index 95', \n       'Sunny, highs to upper 80s', 'Sun & clouds',\n       'Mostly sunny', 'Sunny, Windy', 'Mostly Sunny Skies',\n       'Clear and Sunny', 'Clear and sunny',\n       'Clear to Partly Cloudy', 'Cold', 'N/A Indoor', 'N/A (Indoors)']\n\ncloudy = ['Mostly Cloudy', 'Cloudy',\n       'Partly Cloudy', 'Mostly cloudy', 'Cloudy and cold',\n       'Cloudy and Cool', 'Partly cloudy',\n       'Party Cloudy', 'Partly Clouidy', 'Overcast',\n       'Mostly Coudy', 'cloudy', 'Coudy']\n\nrainy = ['Cloudy, fog started developing in 2nd quarter', 'Rain',\n       'Rain Chance 40%', 'Showers', 'Scattered Showers', 'Hazy',\n       'Rain likely, temps in low 40s.', 'Cloudy, 50% change of rain', \n       'Light Rain', '10% Chance of Rain', 'Cloudy, chance of rain',\n       'Cloudy, Rain', 'Rainy', '30% Chance of Rain',\n       'Cloudy with periods of rain, thunder possible. Winds shifting to WNW, 10-20 mph.',\n       'Rain shower']\n\nsnowy = ['Snow', 'Heavy lake effect snow', \n                'Cloudy, light snow accumulating 1-3\"']\n\nplay_list['Weather'].replace(clear, 'Clear', inplace=True)\nplay_list['Weather'].replace(cloudy, 'Cloudy', inplace=True)\nplay_list['Weather'].replace(rainy, 'Rainy', inplace=True)\nplay_list['Weather'].replace(snowy, 'Snowy', inplace=True)\nplay_list['Weather'].dropna(inplace=True)\n\n# drop outliers from temperature\nplay_list = play_list[play_list['Temperature'] != -999]","execution_count":null,"outputs":[]},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":68},"colab_type":"code","id":"bzM18tWKmVjD","outputId":"4bafe619-30d3-4e9d-9abf-600e52b1a57a","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# selecting all unique games with correspondind conditions\ngame_conditions = play_list.drop(['PlayerKey', 'PlayKey', 'RosterPosition',\n                                               'PlayerDay', 'PlayerGame', 'PlayType', 'PlayerGamePlay',\n                                               'Position', 'PositionGroup'], axis=1).copy()\ngame_conditions.drop_duplicates(inplace=True)\nprint(f'Total amount of GameID: {games_total}')\nprint(f'Total amount of GameID with full information about conditions: {game_conditions.shape[0]}')\nprint('Percent of data losed after cleaning: {:.2f}%'.format(100 - game_conditions.shape[0] / games_total *100))","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"Ado7pzDxGnxk"},"cell_type":"markdown","source":"Less then 10% of data was loosed after cleaning, not so bad. <br>\nLets start with weather conditions:"},{"metadata":{"colab":{},"colab_type":"code","id":"3dAgNT0OlFoe","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"gi_with_injuryes = injury_record['GameID'].unique()\n\ngame_conditions_natural = game_conditions[game_conditions['FieldType']=='Natural']\ngame_conditions_synt = game_conditions[game_conditions['FieldType']=='Synthetic']\n\ninjured_natural = game_conditions_natural.query('GameID in @gi_with_injuryes')\ninjured_synthetic = game_conditions_synt.query('GameID in @gi_with_injuryes')\n\ninj_st_natural = injured_natural['StadiumType'].value_counts()\ninj_st_synt = injured_synthetic['StadiumType'].value_counts()\ntotal_st_natural = game_conditions_natural['StadiumType'].value_counts()\ntotal_st_synt = game_conditions_synt['StadiumType'].value_counts()\n\nst_inj_ratio_natural = inj_st_natural / total_st_natural * 100\nst_inj_ratio_synt = inj_st_synt / total_st_synt * 100\n\nsns.set_context(\"talk\", font_scale=0.8) \nsns.set_palette(\"Blues_d\")\n\nfig, ax = plt.subplots(3, 2, figsize=(15, 7))\n\nsns.barplot(x=total_st_natural.sort_values(ascending=False).values,\n            y=total_st_natural.sort_values(ascending=False).index,\n              ax=ax[0][0], palette=\"Blues_d\")\nsns.barplot(x=total_st_synt.sort_values(ascending=False).values,\n            y=total_st_synt.sort_values(ascending=False).index,\n              ax=ax[0][1], palette=\"Blues_d\")\n\nsns.barplot(x=inj_st_natural.sort_values(ascending=False).values,\n            y=inj_st_natural.sort_values(ascending=False).index,\n              ax=ax[1][0], palette=\"Blues_d\")\nsns.barplot(x=inj_st_synt.sort_values(ascending=False).values,\n            y=inj_st_synt.sort_values(ascending=False).index,\n              ax=ax[1][1], palette=\"Blues_d\")\n\nsns.barplot(x=st_inj_ratio_natural.sort_values(ascending=True).values,\n            y=st_inj_ratio_natural.sort_values(ascending=True).index,\n              ax=ax[2][0], palette=\"Blues_d\")\nsns.barplot(x=st_inj_ratio_synt.sort_values(ascending=True).values,\n            y=st_inj_ratio_synt.sort_values(ascending=True).index,\n              ax=ax[2][1], palette=\"Blues_d\")\n\nax[2][0].set_xlim([0, 4])\nax[2][1].set_xlim([0, 4])\nax[1][0].set_xlim([0, 50])\nax[1][1].set_xlim([0, 50])\nax[0][0].set_xlim([0, 3100])\nax[0][1].set_xlim([0, 3100])\nax[0][0].set_title('Natural')\nax[0][1].set_title('Synthetic')\nax[0][0].set_xlabel('Amount of plays')\nax[0][1].set_xlabel('Amount of plays')\nax[1][0].set_xlabel('Amount of plays with injury')\nax[1][1].set_xlabel('Amount of plays with injury')\nax[2][0].set_xlabel('percent of injury, %')\nax[2][1].set_xlabel('percent of injury, %')\nsns.despine() \nfig.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"Ae26kX0BQ2DS"},"cell_type":"markdown","source":"Lets consider weather conditions:"},{"metadata":{"colab":{},"colab_type":"code","id":"m2JUg47UlFZ3","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"inj_wt_natural = injured_natural['Weather'].value_counts()\ninj_wt_synt = injured_synthetic['Weather'].value_counts()\ntotal_wt_natural = game_conditions_natural['Weather'].value_counts()\ntotal_wt_synt = game_conditions_synt['Weather'].value_counts()\n\nwt_inj_ratio_natural = inj_wt_natural / total_wt_natural * 100\nwt_inj_ratio_synt = inj_wt_synt / total_wt_synt * 100\n\nfig, ax = plt.subplots(3, 2, figsize=(15, 7))\n\nsns.barplot(x=total_wt_natural.sort_values(ascending=False).values,\n            y=total_wt_natural.sort_values(ascending=False).index,\n            order=total_wt_synt.sort_values(ascending=False).index,\n              ax=ax[0][0], palette=\"Blues_d\")\nsns.barplot(x=total_wt_synt.sort_values(ascending=False).values,\n            y=total_wt_synt.sort_values(ascending=False).index,\n            order=total_wt_synt.sort_values(ascending=False).index,\n              ax=ax[0][1], palette=\"Blues_d\")\n\nsns.barplot(x=inj_wt_natural.sort_values(ascending=False).values,\n            y=inj_wt_natural.sort_values(ascending=False).index,\n            order=total_wt_synt.sort_values(ascending=False).index,\n              ax=ax[1][0], palette=\"Blues_d\")\nsns.barplot(x=inj_wt_synt.sort_values(ascending=False).values,\n            y=inj_wt_synt.sort_values(ascending=False).index,\n            order=total_wt_synt.sort_values(ascending=False).index,\n              ax=ax[1][1], palette=\"Blues_d\")\n\nsns.barplot(x=wt_inj_ratio_natural.sort_values(ascending=False).values,\n            y=wt_inj_ratio_natural.sort_values(ascending=False).index,\n            order=total_wt_synt.sort_values(ascending=False).index,\n              ax=ax[2][0], palette=\"Blues_d\")\nsns.barplot(x=wt_inj_ratio_synt.sort_values(ascending=False).values,\n            y=wt_inj_ratio_natural.sort_values(ascending=False).index,\n            order=total_wt_synt.sort_values(ascending=False).index,\n              ax=ax[2][1], palette=\"Blues_d\")\n\nax[2][0].set_xlim([0, 3])\nax[2][1].set_xlim([0, 3])\nax[1][0].set_xlim([0, 25])\nax[1][1].set_xlim([0, 25])\nax[0][0].set_xlim([0, 1500])\nax[0][1].set_xlim([0, 1500])\nax[0][0].set_title('Natural')\nax[0][1].set_title('Synthetic')\nax[0][0].set_xlabel('Amount of games')\nax[0][1].set_xlabel('Amount of games')\nax[1][0].set_xlabel('Amount of games with injury')\nax[1][1].set_xlabel('Amount of games with injury')\nax[2][0].set_xlabel('percent of injury, %')\nax[2][1].set_xlabel('percent of injury, %')\nsns.despine() \nfig.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"He0cGK6ETFXs"},"cell_type":"markdown","source":"**Main takeaways:**\n\n\n1.   Most of the games were played at clear or cloudy weather.\n2.   **8%** of games were played in rain conditions is same proportion for natural and synthetic turf.\n3. Chance of injury in rain weather are highterst between rest weather positions, and it slightly higher for synthetic turf (**3%** vs **2.2%**).\n4. There are no injures in snowy weather, probably as there are quite few games at such weather conditions (**0.8%**).\n\n"},{"metadata":{"colab_type":"text","id":"TDa5Nz2IVs_9"},"cell_type":"markdown","source":"Now lets explore temperature influence:"},{"metadata":{"colab":{},"colab_type":"code","id":"0_uKRWNCpGXj","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Left graph (Injured split by natural and synthetic)\ninj_temp_natural = injured_natural['Temperature']\ninj_temp_synt = injured_synthetic['Temperature']\n\n# Right graph (all split by injured and not)\ninjured_games = game_conditions.query('GameID in @gi_with_injuryes')\nnot_injured_games = game_conditions.query('GameID not in @gi_with_injuryes')\ninj_temp = injured_games['Temperature']\nnot_inj_temp = not_injured_games['Temperature']\n\nfig, ax = plt.subplots(1, 2, figsize=(15, 5))\n\nsns.distplot(inj_temp_natural, label='Natural', color='g', bins=10,  ax=ax[0])\nsns.distplot(inj_temp_synt, label='Synthetic', bins=10, ax=ax[0])\n\nsns.distplot(not_inj_temp, label='Not injured', color='g', bins=10,ax=ax[1])\nsns.distplot(inj_temp, label='Injured', color='r', bins=10,  ax=ax[1])\n\nax[0].set_title('Games with injury')\nax[1].set_title('All games')\nax[0].legend()\nax[1].legend()\nsns.despine()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"Y5jmDvD8Z_31"},"cell_type":"markdown","source":"On visual inspection, one can suspect a slight difference between distributions.\nIn order to be more precise, let us conduct Kolmogorov-Smirnow test to compare distributions."},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":51},"colab_type":"code","id":"E1BTmqB6wcNY","outputId":"c3293779-0056-441c-cbae-1feb29fdcfc7","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from scipy.stats import ks_2samp\n\ninj_games_test = ks_2samp(inj_temp_natural, inj_temp_synt)\nall_games_test = ks_2samp(not_inj_temp, inj_temp)\nprint('p-value that temperatures distribution differs between \\\nsynthetic and natural turf for games with at leas one injury: {:.3f}%'.format(inj_games_test[1]*100))\nprint('p-value that temperatures distribution differs between\\\n games there are at least one injury happens and games without injury: {:.3f}%'.format(all_games_test[1]*100))","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"ROqdLCeUcpAh"},"cell_type":"markdown","source":"Distributions on the left aren’t likely to have a statistically significant difference, but distributions on the right differ. "},{"metadata":{"colab_type":"text","id":"QM2lSo42nhhA"},"cell_type":"markdown","source":"**Now lets consider factors combination:**"},{"metadata":{"colab":{},"colab_type":"code","id":"wYMXq80wLG8t","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"game_id_with_injuryes = injury_record['GameID'].unique()\ngame_id_without_injuryes = play_list.query('GameID not in @game_id_with_injuryes')['GameID'].unique()\n\ninjured_games = play_list.query('GameID in @game_id_with_injuryes')\ninjured_games_conditions = injured_games.drop(['PlayerKey', 'PlayKey', 'RosterPosition',\n                                               'PlayerDay', 'PlayerGame', 'PlayType', 'PlayerGamePlay',\n                                               'Position', 'PositionGroup'], axis=1)\n\ninjured_games_conditions.drop_duplicates(inplace=True)\ninjured_games_conditions.dropna(inplace=True)\ninjured_games_conditions.drop('GameID', axis=1, inplace=True)\n\nmax_temp = play_list['Temperature'].max()\n# Normalize tempereture\ninjured_games_conditions['Temperature'] = injured_games_conditions['Temperature'] / injured_games_conditions['Temperature'].max()\n\ninjured_games_conditions_one_hot = pd.get_dummies(injured_games_conditions)","execution_count":null,"outputs":[]},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":389},"colab_type":"code","id":"pgnZqSpAX1sj","outputId":"a387b2ea-7f7d-4338-93b3-6eecabad5167","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from scipy.cluster.hierarchy import fcluster, dendrogram, linkage\n\n# selecting most similar\ntreshold = 0.1\n\nplt.subplots(figsize=(18, 6))\n\nZ = linkage(injured_games_conditions_one_hot, method='average', metric='cosine')\ndend = dendrogram(Z)\n\nplt.axhline(y=treshold, c='grey', lw=2, linestyle='dashed')\nplt.tick_params(axis='x', which='major', labelsize=15)\nsns.despine()","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"UnFbkw7zYTxW","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"labels = fcluster(Z, treshold, criterion='distance') \n\nclusters = {key: ([],[]) for key in range(1, labels.max() + 1)}\nfor label, player in zip(labels, injured_games_conditions_one_hot.index):\n  clusters[label][1].append(player)\n\nfor key in clusters.keys():\n  try:\n    clusters[key][0].append(\n        np.dot(injured_games_conditions_one_hot.loc[clusters[key][1][0]],\n              injured_games_conditions_one_hot.loc[clusters[key][1][1]])\n    )\n  except IndexError:\n    clusters[key][0].append(0)","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"GjDaekxgZLlK","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"cols = ['StadiumType', 'FieldType', 'Weather', 'Temperature', 'sum']\ncondition_stat = pd.DataFrame(columns=cols)\n\nidx = 0\nfor key in clusters.keys():\n  inj_sum = len(clusters[key][1])\n  if clusters[key][0][0] > 3:\n    st = injured_games_conditions.loc[clusters[key][1][0]]['StadiumType'] \n    ft = injured_games_conditions.loc[clusters[key][1][0]]['FieldType']\n    wt = injured_games_conditions.loc[clusters[key][1][0]]['Weather']\n    # tmp as average\n    tmp = 0\n    for cond in clusters[key][1]:\n      tmp += injured_games_conditions.loc[cond]['Temperature'] \n    tmp /= len(clusters[key][1])\n  else:\n    continue\n\n  col = [st, ft, wt, tmp, inj_sum]\n  condition_stat.loc[idx] = col\n  idx += 1\n\ncondition_stat.sort_values(by='sum', ascending=False, inplace=True)\ncondition_stat.reset_index(drop=True, inplace=True)\ncondition_stat","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"ebEluqxGd6jq","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"count = pd.DataFrame(columns=['mean_tmp', 'sum_total'])\n\nfor idx in condition_stat.index:\n  st = condition_stat.loc[idx]['StadiumType']\n  ft = condition_stat.loc[idx]['FieldType']\n  wt = condition_stat.loc[idx]['Weather']\n  subset = game_conditions[game_conditions['StadiumType'] == st] \\\n            [game_conditions['FieldType'] == ft] \\\n            [game_conditions['Weather'] == wt]\n\n  tmp = subset['Temperature'].mean() / max_temp\n  count.loc[idx] = [tmp, subset.shape[0]]\n    \ncondition_stat_tot = pd.merge(condition_stat, count, left_on=condition_stat.index, right_on=count.index)\ncondition_stat_tot['Ratio'] = condition_stat_tot['sum'] / condition_stat_tot['sum_total'] * 100\ncondition_stat_tot.drop('key_0', axis=1, inplace=True)\n\ncondition_stat_tot['Temperature'] *= max_temp\ncondition_stat_tot['mean_tmp'] *= max_temp\ncondition_stat_tot.sort_values(by='Ratio', ascending=False, inplace=True)\ncondition_stat_tot.reset_index(drop=True, inplace=True)\ncondition_stat_tot","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"k9jzZEAU3LKN"},"cell_type":"markdown","source":"As one can see, the highest risk of injury can be observed in closed stadiums with synthetic turf. Synthetic turf has again proved to be the riskiest in terms of getting injured. Top 4 riskiest combinations include synthetic turf. Let us point out that the injury risk on a closed stadium with synthetic turf is higher than the injury risk in rainy weather on an open stadium with natural turf (3.1% vs 2.3%). <br>\nAlso it is of a great interest that synthetic turf at a relatively cold temperature in rainy weather seems to lead to a lower injury risk than in dry weather but with a higher temperature. <br>\nOne more interesting finding here is that the risk of injury on synthetic turf seems to decrease with decreasing temperature. <br> It also correlates with single temperature observations in the previous section. Even rainy weather seems to be a less important factor. Comparing observation #3 and #6 in Table we can see that cloudy weather with lower temperature results in even lesser risk than rainy weather with higher temperature with the all the other identical factors. According to table injury risk with a temperature of about 75 F is approximately twice higher than with a temperature of 50 F on synthetic turf. \n"},{"metadata":{"colab_type":"text","id":"qBbJdoAq3g3P"},"cell_type":"markdown","source":"**Into another perspective:**"},{"metadata":{"colab":{},"colab_type":"code","id":"sJ62mOvy3f6Q","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"injury_record = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/InjuryRecord.csv')\nplay_list = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayList.csv')\n\n# leave only data with correct temperature\nplay_list = play_list[play_list['Temperature'] != -999]\nplay_list['Temperature'].dropna(inplace=True)\n# leave only synthetic turf\nplay_list_synt = play_list[play_list['FieldType']=='Synthetic']\nplay_list_nat = play_list[play_list['FieldType']=='Natural']\n\n# leave only unique GameIDs\nplay_list_synt.drop_duplicates('GameID', inplace=True)\nplay_list_nat.drop_duplicates('GameID', inplace=True)\n\n# total amount of games by temperatures\ntotal_synt = play_list_synt.groupby('Temperature').count()['PlayKey']\ntotal_nat = play_list_nat.groupby('Temperature').count()['PlayKey']\n\n# selecting injured GameIDs on synthetyc surface\ninj_synt = injury_record[injury_record['Surface']=='Synthetic']\nsynt_inj_id = inj_synt['GameID']\ninj_nat = injury_record[injury_record['Surface']=='Natural']\nnat_inj_id = inj_nat['GameID']\n\n# selecting injured from synthetic play list\nplay_list_synt_inj = play_list_synt.query('GameID in @synt_inj_id')\nplay_list_nat_inj = play_list_nat.query('GameID in @nat_inj_id')\n\n# total amount of games by temperatures\ninj_synt = play_list_synt_inj.groupby('Temperature').count()['PlayKey']\ninj_nat = play_list_nat_inj.groupby('Temperature').count()['PlayKey']","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"3ZuZCe2k5f2G","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def group_by_t(data, dt=10):\n  res = {t: 0 for t in range(10, 110, dt)}\n  for t in data.index:\n    if t % dt == 0:\n      res[t] += data[t]\n    else:\n      key = (t // dt + 1) * dt\n      res[key] += data[t]\n  return res\n\ninj_synt_g = group_by_t(inj_synt)\nall_synt_g = group_by_t(total_synt)\n\ninj_nat_g = group_by_t(inj_nat)\nall_nat_g = group_by_t(total_nat)\n\nres_synt = []\nfor key in list(inj_nat_g.keys())[:-1]:\n  res_synt.append(inj_synt_g[key] / all_synt_g[key]*100)\n\nres_nat = []\nfor key in list(inj_nat_g.keys())[:-1]:\n  res_nat.append(inj_nat_g[key] / all_nat_g[key]*100)\n\nt = range(10, 100, 10)\n\nsynt_fit = np.polyfit(t[3:], res_synt[3:], 1)\nnat_fit = np.polyfit(t[3:], res_nat[3:], 1)\n\nsns.set_context(\"talk\", font_scale=0.8) \nsns.set_palette(\"Blues_d\")\nfig, ax = plt.subplots(figsize=(12, 6))\n\nsns.scatterplot(t, res_synt, label='Synthetic', color='r', ax=ax)\nax.plot(t[3:], synt_fit[0]*t[3:] + synt_fit[1], '--', color='r', alpha=0.8)\nsns.scatterplot(t, res_nat, label='Natural', color='g', marker='s', ax=ax)\nax.plot(t[3:], nat_fit[0]*t[3:] + nat_fit[1], '--', color='g', alpha=0.8)\n\nax2 = ax.twinx()  # instantiate a second axes that shares the same x-axis\ncolor = 'tab:gray'\nax2.set_ylabel('Total amount of games', color=color)  \nplay_list_synt['Temperature'].hist(bins=35, label='Synthetic', color='r', alpha=0.2, ax=ax2)\nplay_list_nat['Temperature'].hist(bins=35, label='Natural', color='g',  alpha=0.2, ax=ax2)\nax2.tick_params(axis='y', labelcolor=color)\n\nax.set_ylabel('Percent of injuries, %')\nax.set_xlabel('Temperature, F')\nax2.set_ylim([0, 350])\nsns.despine()\nax.legend(loc='upper left', facecolor='white', framealpha=1)\nfig.tight_layout()\n","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"1sgaWsx9bmah"},"cell_type":"markdown","source":"There is not so much data to draw a solid conclusion, but it’s likely that the temperature may influence injury risk on synthetic turf. Such an influence feels valid as there is a simple physical model which can be applied to describe it. Physical properties of synthetic turf and their dependencies on temperature should be investigated. If that is the case, changing of synthetic turf properties can lead to reducing injury risks."},{"metadata":{"colab_type":"text","id":"6hCrbwqgejLZ"},"cell_type":"markdown","source":"**Summirizung categorical features:**"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"play_list = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/PlayList.csv')\n\nclear = ['Clear and warm', 'Sunny', 'Clear',\n       'Controlled Climate', 'Sunny and warm', 'Clear and Cool',\n       'Clear and cold', 'Sunny and cold', 'Closed', 'Partly Sunny',\n       'Mostly Sunny', 'Clear Skies', 'Partly sunny',\n       'Sunny and clear', 'Clear skies', 'Sunny Skies',\n       'Fair', 'Partly clear', 'Heat Index 95', \n       'Sunny, highs to upper 80s', 'Sun & clouds',\n       'Mostly sunny', 'Sunny, Windy', 'Mostly Sunny Skies',\n       'Clear and Sunny', 'Clear and sunny',\n       'Clear to Partly Cloudy', 'Cold', 'N/A Indoor', 'N/A (Indoors)']\n\ncloudy = ['Mostly Cloudy', 'Cloudy',\n       'Partly Cloudy', 'Mostly cloudy', 'Cloudy and cold',\n       'Cloudy and Cool', 'Partly cloudy',\n       'Party Cloudy', 'Partly Clouidy', 'Overcast',\n       'Mostly Coudy', 'cloudy', 'Coudy']\n\nrainy = ['Cloudy, fog started developing in 2nd quarter', 'Rain',\n       'Rain Chance 40%', 'Showers', 'Scattered Showers', 'Hazy',\n       'Rain likely, temps in low 40s.', 'Cloudy, 50% change of rain', \n       'Light Rain', '10% Chance of Rain', 'Cloudy, chance of rain',\n       'Cloudy, Rain', 'Rainy', '30% Chance of Rain',\n       'Cloudy with periods of rain, thunder possible. Winds shifting to WNW, 10-20 mph.',\n       'Rain shower']\n\nsnowy = ['Snow', 'Heavy lake effect snow', \n                'Cloudy, light snow accumulating 1-3\"']\n\nplay_list['Weather'].replace(clear, 'Clear', inplace=True)\nplay_list['Weather'].replace(cloudy, 'Cloudy', inplace=True)\nplay_list['Weather'].replace(rainy, 'Rainy', inplace=True)\nplay_list['Weather'].replace(snowy, 'Snowy', inplace=True)\nplay_list['Weather'].dropna(inplace=True)\n\n# drop outliers from temperature\nplay_list = play_list[play_list['Temperature'] != -999]\n\ninjury_record = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/InjuryRecord.csv')\nplay_list_clean = play_list\nplay_list_clean.dropna(axis=0, inplace=True)\nplay_list_clean = play_list_clean[play_list_clean['PlayType'] != '0']\nplay_list_clean = play_list_clean[play_list_clean['PositionGroup'] != 'Missing Data']\n\ninjured_game_id = injury_record['GameID'].unique()\n\nplay_list_injured = play_list_clean.query('GameID in @injured_game_id').copy()\nplay_list_not_injured = play_list_clean.query('GameID not in @injured_game_id').copy()\n\nplay_list_injured.drop_duplicates(subset=['GameID'], inplace=True)\nplay_list_not_injured.drop_duplicates(subset=['GameID'], inplace=True)\n\nplay_list_not_injured_subset = play_list_not_injured.sample(400)\n\nplay_list_injured['Injury'] = np.ones((play_list_injured.shape[0], 1))\nplay_list_not_injured_subset['Injury'] = np.zeros((play_list_not_injured_subset.shape[0], 1))\n\nplay_list_subset = pd.concat([play_list_injured, play_list_not_injured_subset])\n\nplay_list_subset['Temperature'] /= play_list_subset['Temperature'].max()\nplay_list_subset['PlayerDay'] /= play_list_subset['PlayerDay'].max() \nplay_list_subset['PlayerGame'] /= play_list_subset['PlayerGame'].max() \nplay_list_subset.drop(['PlayerKey', 'GameID', 'PlayKey', 'PlayerGamePlay'], axis=1, inplace=True)\n\nplay_list_X = play_list_subset[play_list_subset.columns[:-1]]\ninjury_y = play_list_subset['Injury']\n\nplay_list_subs_one_hot = pd.get_dummies(play_list_X)\nplay_list_subs_one_hot = pd.concat([play_list_subs_one_hot, injury_y], axis=1)\n\nimport seaborn as sns\n\n#get correlations of each features in dataset\ncorrmat = play_list_subs_one_hot.corr()\ntop_corr_features = corrmat.index\nplt.figure(figsize=(20,20))\n#plot heat map\nsns.heatmap(play_list_subs_one_hot[top_corr_features].corr(), annot=False, cmap=\"RdBu_r\")\n#plt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"bd7B8cbYb6Ac"},"cell_type":"markdown","source":"Most of the relations have already been observed in the previous sections, for example that natural turf correlated with open stadiums. Also we can see some natural dependencies like slight correlation between temperature and stadium type or with weather conditions. Play type “rush” had a strong negative correlation with “pass”, as they are opposing play types. There are strong correlations between positions that reflect most frequent player combinations. Focusing on injuries, there is a weak negative correlation with player’s game, which likely reflects that injured athletes take part in less games. Other correlations with injuries are negligible. "},{"metadata":{"colab_type":"text","id":"iaWV7mF4cE9D"},"cell_type":"markdown","source":"Attempt to create prediction model by features above:"},{"metadata":{"colab":{},"colab_type":"code","id":"Vt9GKCfKcD9s","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"play_list_X.drop(['PlayerDay', 'PlayerGame'], axis=1, inplace=True)\nX = pd.get_dummies(play_list_X)\n\nfrom sklearn.ensemble import GradientBoostingClassifier\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import accuracy_score, recall_score\n\nX_train, X_test, y_train, y_test = train_test_split(X, injury_y, \n                                                    test_size=0.25, \n                                                    stratify=injury_y,\n                                                    random_state=0)","execution_count":null,"outputs":[]},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":435},"colab_type":"code","id":"PocfnWaiH4TM","outputId":"d9661c6a-0aff-4678-b8a7-5976dc007b00","trusted":false,"_kg_hide-input":true},"cell_type":"code","source":"from sklearn.metrics import roc_curve, auc\n\nclf = GradientBoostingClassifier(min_samples_leaf=2,\n                                 max_depth=5,\n                                 max_features=8,\n                                 learning_rate=0.2,\n                                 n_estimators=100)\n\ny_score_lr = clf.fit(X_train, y_train).decision_function(X_test)\nfpr_lr, tpr_lr, _ = roc_curve(y_test, y_score_lr)\nroc_auc_lr = auc(fpr_lr, tpr_lr)\n\nfig, ax = plt.subplots(figsize=(6, 15))\nsns.set_context(\"talk\", font_scale=0.8) \nax.set_xlim([-0.01, 1.00])\nax.set_ylim([-0.01, 1.01])\nax.plot(fpr_lr, tpr_lr, lw=3, label='LogRegr ROC curve (area = {:0.2f})'.format(roc_auc_lr))\nax.set_xlabel('False Positive Rate', fontsize=16)\nax.set_ylabel('True Positive Rate', fontsize=16)\nax.set_title('ROC curve', fontsize=16)\nax.legend(loc='lower right', fontsize=13)\nax.plot([0, 1], [0, 1], color='navy', lw=3, linestyle='--')\nax.set_aspect('equal')\nsns.despine()\nfig.tight_layout()","execution_count":0,"outputs":[]},{"metadata":{"colab_type":"text","id":"HlMkXNmYA73W"},"cell_type":"markdown","source":"Model performance is poor, feature importance analisys became pointless."},{"metadata":{"colab_type":"text","id":"Fb7B98cCptxa"},"cell_type":"markdown","source":"# 4. Athlete movement analisys."},{"metadata":{"colab":{},"colab_type":"code","id":"YfAI2_khpyKf","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"injury_record = pd.read_csv('/kaggle/input/nfl-playing-surface-analytics/InjuryRecord.csv')\nplay_list = pd.read_csv(\"../input/nfl-playing-surface-analytics/PlayList.csv\")\ntrack_data = pd.read_csv(\"../input/nfl-playing-surface-analytics/PlayerTrackData.csv\")","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"a97Fh-Lop_t2","trusted":false,"_kg_hide-input":true},"cell_type":"code","source":"def get_deriv(data, dt=0.1, threshold=300):\n  '''\n  Deriviate that handle with outliers that appears in angle variables, due to 360 rotation.\n  Outliers are replaced by mean of left and right neightbours.\n  '''\n  delta = np.array(data[1:]) - np.array(data[:-1])\n  outliers = np.where(np.abs(delta) > threshold)\n  mean_nei = np.array(outliers)\n  for i in outliers:\n    try:\n      delta[i] = (delta[i - 1] + delta[i + 1]) / 2\n    except:\n      continue\n  return delta / dt\n\ndef smooth(x, window_len=11, window='hanning'):\n    \"\"\"smooth the data using a window with requested size.\n    This method is based on the convolution of a scaled window with the signal.\n    The signal is prepared by introducing reflected copies of the signal \n    (with the window size) in both ends so that transient parts are minimized\n    in the begining and end part of the output signal.\n    input:\n        x: the input signal \n        window_len: the dimension of the smoothing window; should be an odd integer\n        window: the type of window from 'flat', 'hanning', 'hamming', 'bartlett', 'blackman'\n            flat window will produce a moving average smoothing.\n    output:\n        the smoothed signal  \n    \"\"\"\n    if x.ndim != 1:\n        raise ValueError('smooth only accepts 1 dimension arrays')\n    if x.size < window_len:\n        raise ValueError(\"Input vector needs to be bigger than window size.\")\n    if window_len < 3:\n        return x\n    if not window in ['flat', 'hanning', 'hamming', 'bartlett', 'blackman']:\n        raise ValueError(\"Window is on of 'flat', 'hanning', 'hamming', 'bartlett', 'blackman'\")\n    s = np.r_[x[window_len-1: 0: -1], x, x[-2: -window_len-1 :-1]]\n    if window == 'flat': #moving average\n        w = np.ones(window_len, 'd')\n    else:\n        w = eval('np.' + window + '(window_len)')\n    y=np.convolve(w/w.sum(), s, mode='valid')\n    return y\n\ndef eng_difference(fi1, fi2):\n  fi1 = (fi1 + 90)*2*np.pi / 360\n  fi2 = (fi2 + 90)*2*np.pi / 360\n  dp = np.abs(np.sin(fi1) - np.sin(fi2))\n  dfi = np.arccos(1 - dp)\n  return dfi*360/2/np.pi","execution_count":0,"outputs":[]},{"metadata":{"colab_type":"text","id":"BenG_IYoBe3X"},"cell_type":"markdown","source":"Data is represented as time series and consists of coordinates, speed, orientation and direction. Average duration of such an interval is about 30s with discretisation frequency 10Hz. Here are the examples of one such plays:"},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":504},"colab_type":"code","id":"MIEtVrdXxuu5","outputId":"9bacb1dd-8cbb-4f71-8d37-02698bf9206a","trusted":false,"_kg_hide-input":true},"cell_type":"code","source":"track = track_data[track_data['PlayKey'] == '36621-13-58']\n\nsns.set_context(\"talk\", font_scale=0.8) \nsns.set_palette(\"Blues_d\")\nfig, ax = plt.subplots(2, 2, figsize=(15, 7))\n\nax[0][0].plot(track['x'], track['y'], marker='.', ms=8,)\nax[0][1].plot(track['time'], track['x'], marker='.', ms=8, label='x')\nax[0][1].set_label('x')\nax2 = ax[0][1].twinx()  # instantiate a second axes that shares the same x-axis\ncolor = 'tab:blue'\nax2.set_ylabel('y', color=color)  \nax2.plot(track['time'], track['y'], marker='.', ms=8, color=color, label='y')\nax2.tick_params(axis='y', labelcolor=color)\n\nax[1][0].plot(track['time'], track['o'], marker='.', ms=8, label='orientation')\nax[1][0].plot(track['time'], track['dir'], marker='.', ms=8, label='direction', color='tab:blue')\n\nax[1][1].plot(track['time'], track['s'], marker='.', ms=8, label='direction')\n\nax[0][0].set_title('Trajectory')\nax[0][1].set_title('x(t), y(t)')\nax[1][0].set_title('Orientation and direction')\nax[1][1].set_title('Speed')\nax[1][0].legend(fontsize=12)\nax[0][1].legend(fontsize=12)\nax2.legend(fontsize=12)\nax[0][0].grid()\nax[1][0].grid()\nax[0][1].grid()\nax[1][1].grid()\nsns.despine()\n\nax[0][0].set_xlabel('$x, yd$')\nax[0][0].set_ylabel('$y, yd$')\nax[0][1].set_xlabel('$t, s$')\nax[0][1].set_ylabel('$x, yd$')\nax2.set_ylabel('$y, yd$')\nax[1][0].set_xlabel('$t, s$')\nax[1][0].set_ylabel('$angle, deg$')\nax[1][1].set_xlabel('$t, s$')\nax[1][1].set_ylabel('$V, yd/s$')\n\nfig.tight_layout()","execution_count":0,"outputs":[]},{"metadata":{"colab_type":"text","id":"OMB--LIZc0s5"},"cell_type":"markdown","source":"**Main ideas for feature extraction:**\n<li>\tIt is likely that injuries happen at moments of rapid acceleration or sudden stops, i.e. during big positive and negative linear accelerations.\n<li>\tSudden direction changes may lead to injury.\n<li>\tRunning fast may increase the chance of positioning feet in such a manner that may lead to injury.\n<li>\tRunning when having a big difference between orientation and direction may lead to injury.\n"},{"metadata":{"colab_type":"text","id":"GYzFNxILc-Cu"},"cell_type":"markdown","source":"Based on the above-mentioned assumptions, the following features were extracted:\n1.\tLinear acceleration (peak value)\n2.\tLinear deceleration (peak value)\n3.\tLinear speed (peak value)\n4.\tAngle speed of orientation (peak value)\n5.\tAngle speed of direction (peak value)\n6.\tDifference between direction and orientation (peak value)\n7.\tDifference between direction and orientation in the moment with maximum speed\n8.\tDifference between direction and orientation in the  moment with maximum acceleration\n9.\tDifference between direction and orientation in the  moment with maximum deceleration\n10.\tSpeed in the moment of maximum difference between direction and orientation\n11.\tAcceleration in the moment of maximum difference between direction and orientation\n12.\tAverage speed of side running (when the difference between direction and orientation is 80-100 degrees)\n13.\tTotal distance per play\n14.\tAverage speed of running backwards (when difference between direction and orientation is >100 degrees)\n"},{"metadata":{"colab":{},"colab_type":"code","id":"WSjGuj0a1VQu","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_global_peaks_extended(play_keys, min_sample_size=50):\n\n  cols = ['a_max', 'a_min', 'v_max', 'vo_max', 'vdir_max',\n          'd_max', 'd_Vmax', 'd_Amax', 'd_Amin', \n          'maxD_v', ' maxD_a',\n          'siderun', 'distance', 'back_run']\n  data = pd.DataFrame(columns=cols)\n\n  win_len = 7\n  col = 0\n  for pk in play_keys:\n    track = track_data[track_data['PlayKey'] == pk]\n    vx = get_deriv(track['x'])\n    vy = get_deriv(track['y'])\n    v = np.sqrt(vx**2 + vy**2)\n    o = track['o']\n    dir_ = track['dir']\n    dist = track['dis']\n\n    if v.shape[0] + o.shape[0] + dir_.shape[0]  > 3*min_sample_size:\n      vf = smooth(v, win_len, 'hanning')\n      a = get_deriv(vf)\n      af = smooth(a, win_len, 'hanning')\n      vo = get_deriv(o)\n      vof = smooth(vo, win_len-2, 'hanning')\n\n      norm = np.array([1 if vi > 0.05*vf.max() else 0 for vi in v])\n      d = np.abs(eng_difference(o[:-1], dir_[:-1])) * norm\n\n      vdir = get_deriv(dir_) * norm\n      vdir = smooth(vdir, win_len, 'hanning') \n\n      d = np.abs(eng_difference(o[:-1], dir_[:-1])) * norm\n\n      a_max = af.max()\n      a_min = af.min()\n      v_max = vf.max()\n      vo_max = vof.max()\n      vdir_max = vdir.max()\n      d_max = d.max()\n\n      sr = side_run(d, vf)\n      dist = dist.sum()\n      bck_run = back_run(vf, d)\n\n      try:\n        d_Vmax = d[np.where(vf==v_max)[0][0]-1]\n        d_Amax = d[np.where(af==a_max)[0][0]-1]\n        d_Amin = d[np.where(af==a_min)[0][0]-1]\n        maxD_v = vf[np.where(d==d.max())[0][0]-1]\n        maxD_a = af[np.where(d==d.max())[0][0]-1]\n      except:\n        continue\n      \n      values = [a_max, a_min, v_max, vo_max, vdir_max,\n                d_max, d_Vmax, d_Amax, d_Amin,\n                maxD_v, maxD_a,\n                sr, dist, bck_run]\n      data.loc[col] = values\n      col += 1     \n    else:\n      continue\n\n  return data\n\ndef back_run(v, d):\n  d.index = range(len(d.index))\n  back_r = d[d > 100]\n  if back_r.shape[0] > 1:\n    avg_v = v[back_r.index[:-1]].mean()\n    return avg_v\n  return 0\n\ndef side_run(d, v):\n  d.index = range(len(d.index))\n  side_r = d[(d > 80) & (d < 100)]\n  if side_r.shape[0] > 1:\n    v = v[side_r.index[:-1]].mean()\n    return v\n  return 0","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"wQabigQOzTv0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"inj_pk = injury_record['PlayKey'].dropna()\npk = play_list['PlayKey'].dropna()\n\nhealthy_pk = play_list.query('PlayKey not in @inj_pk')\nhealthy_pk = healthy_pk['PlayKey'].dropna()\n\n# all pk without injury\nhealthy_pk = healthy_pk.unique()\n\nsubs = np.array(healthy_pk)\nnp.random.shuffle(subs)\n\nsubset_size = 400\nhealthy_pk_subset = subs[:subset_size]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"injured_metrics = pd.read_csv(\"../input/metrics-for-nfl/InjMetrics_1.csv\")\nnot_injured_metrics = pd.read_csv(\"../input/metrics-for-nfl/NotInjMetrics_1.csv\")\n\ninjured_metrics['Injury'] = np.ones((injured_metrics.shape[0], 1))\nnot_injured_metrics['Injury'] = np.zeros((not_injured_metrics.shape[0], 1))\n\ndata = pd.concat([injured_metrics, not_injured_metrics])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In order to deal with class imbalance between plays where athletes got injured and plays where athletes did not get injured, plays without injuries were randomly downsampled to 400. Finally, all the defined features were extracted from the dataset of 77 plays where athletes got injured and 400 plays where athletes did not get injured. To see the overall picture, it’s convenient to look at the correlation heat map:"},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":556},"colab_type":"code","id":"uYxCLyQysElV","outputId":"2437812f-a27d-4fa4-f73b-77ba68eb18e4","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"#get correlations of each features in dataset\ncorrmat = data.corr()\ntop_corr_features = corrmat.index\nplt.figure(figsize=(10,8))\n#plot heat map\nsns.heatmap(data[top_corr_features].corr(), annot=False, cmap=\"RdBu_r\")\n#plt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"colab":{},"colab_type":"code","id":"63aOFBah9pNf","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from scipy.stats import ks_2samp\n\n\nvo_ks_test = ks_2samp(injured_metrics['vo_max'], not_injured_metrics['vo_max'])[1]\nvdir_ks_test = ks_2samp(injured_metrics['vdir_max'], not_injured_metrics['vdir_max'])[1]\nd_ks_test = ks_2samp(injured_metrics['d_max'], not_injured_metrics['d_max'])[1]\n\ndVmax_ks_test = ks_2samp(injured_metrics['d_Vmax'], not_injured_metrics['d_Vmax'])[1]\ndAmax_ks_test = ks_2samp(injured_metrics['d_Amax'], not_injured_metrics['d_Amax'])[1]\naAmin_ks_test = ks_2samp(injured_metrics['d_Amin'], not_injured_metrics['d_Amin'])[1]\nmaxDv_ks_test = ks_2samp(injured_metrics['maxD_v'], not_injured_metrics['maxD_v'])[1]\n\ndistance_ks_test = ks_2samp(injured_metrics['distance'], not_injured_metrics['distance'])[1]\nsiderun_ks_test = ks_2samp(injured_metrics['siderun'], not_injured_metrics['siderun'])[1]\nbkrun_ks_test = ks_2samp(injured_metrics['back_run'], not_injured_metrics['back_run'])[1]\n\na_ks_test = ks_2samp(injured_metrics['a_max'], not_injured_metrics['a_max'])[1]\nd_ks_test = ks_2samp(injured_metrics['a_min'], not_injured_metrics['a_min'])[1]\nv_ks_test = ks_2samp(injured_metrics['v_max'], not_injured_metrics['v_max'])[1]\nvo_ks_test = ks_2samp(injured_metrics['vo_max'], not_injured_metrics['vo_max'])[1]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Extracted features do not have a strong correlation with the risk of injury. Distributions of metrics for injured and not injured athletes were compared with a view to finding differences. To be more precise about metrics difference, Kolmohorov-Smirnov test was conducted. P-value of the test is represented in the graphs."},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":448},"colab_type":"code","id":"qIYkDL8ur0_q","outputId":"0687bd9c-bd6f-4fd5-d954-e40501266a2a","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig, ax = plt.subplots(1, 3, figsize=(20, 6))\nsns.set_context(\"talk\", font_scale=0.8) \nsns.set_palette(\"Blues_d\")\n\ng1 = sns.distplot(injured_metrics['vo_max'], label='Injured', ax=ax[0], color='r')\nsns.distplot(not_injured_metrics['vo_max'], label='Not injured', ax=ax[0], color='g')\ng1.text(500, 0.004, 'K-S test \\np_val={:.1f}%'.format(vo_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\ng2 = sns.distplot(injured_metrics['vdir_max'], label='Injured', ax=ax[1], color='r')\nsns.distplot(not_injured_metrics['vdir_max'], label='Not injured', ax=ax[1], color='g')\ng2.text(600, 0.0024, 'K-S test \\np_val={:.1f}%'.format(vdir_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\ng3 = sns.distplot(injured_metrics['d_max'], label='Injured', ax=ax[2], color='r')\nsns.distplot(not_injured_metrics['d_max'], label='Not injured', ax=ax[2], color='g')\ng3.text(60, 0.012, 'K-S test \\np_val={:.1f}%'.format(d_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\nfig.suptitle('Global peaks', fontsize=16)\nax[0].set_title('Maximum orientation speed')\nax[1].set_title('Maximum direction speed')\nax[2].set_title('Maximum difference between orientaion and direction')\nax[0].set_xlabel('$deg/s$')\nax[1].set_xlabel('$deg/s$')\nax[2].set_xlabel('$deg$')\nax[0].legend()\nax[1].legend()\nax[2].legend()\nax[0].set_xlim(0, 500)\nax[1].set_xlim(0, 1000)\nsns.despine()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The maximum difference between orientation and direction and maximum direction rotation speed had no significant difference between injured and non-injured samples. Formally the maximum orientation speed was different for injured and non-injured populations, but it is likely due to outliers on the tail of the distribution. There is some difference in total distance and “side run” features:"},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":422},"colab_type":"code","id":"SqIO10V8FG9l","outputId":"731a9d27-224e-447f-b3bb-90605f237b88","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig, ax = plt.subplots(1, 3, figsize=(20, 6))\nsns.set_context(\"talk\", font_scale=0.8) \nsns.set_palette(\"Blues_d\")\n\ng1 = sns.distplot(injured_metrics['distance'], label='Injured', ax=ax[0], color='r')\nsns.distplot(not_injured_metrics['distance'], label='Not injured', ax=ax[0], color='g')\ng1.text(100, 0.013, 'K-S test \\np_val={:.1f}%'.format(distance_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\ng2 = sns.distplot(injured_metrics['siderun'], label='Injured', ax=ax[1], color='r')\nsns.distplot(not_injured_metrics['siderun'], label='Not injured', ax=ax[1], color='g')\ng2.text(4, 0.3, 'K-S test \\np_val={:.1f}%'.format(siderun_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\ng3 = sns.distplot(injured_metrics['back_run'], label='Injured', ax=ax[2], color='r')\nsns.distplot(not_injured_metrics['back_run'], label='Not injured', ax=ax[2], color='g')\ng3.text(4, 0.3, 'K-S test \\np_val={:.1f}%'.format(bkrun_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\nax[0].set_title('Distance')\nax[1].set_title('Side run')\nax[2].set_title('Back run')\nax[0].set_xlabel('$yd$')\nax[1].set_xlabel('$yd/s$')\nax[2].set_xlabel('$yd/s$')\nax[0].legend()\nax[1].legend()\nax[2].legend()\nsns.despine()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Comparing such metrics as the difference between direction and orientation in the moment with maximum speed, the difference between direction and orientation in the moment with maximum acceleration, the difference between direction and orientation in the moment with maximum deceleration, and speed in the moment of the maximum difference between direction and orientation has not shown no significant difference:"},{"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":448},"colab_type":"code","id":"kxnaT2GNDGir","outputId":"27597213-87fc-4ecf-b372-add1368a7332","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig, ax = plt.subplots(1, 4, figsize=(20, 6))\nsns.set_context(\"talk\", font_scale=0.8) \nsns.set_palette(\"Blues_d\")\n\ng1 = sns.distplot(injured_metrics['d_Vmax'], label='Injured', ax=ax[0], color='r')\nsns.distplot(not_injured_metrics['d_Vmax'], label='Not injured', ax=ax[0], color='g')\ng1.text(125, 0.006, 'K-S test \\np_val={:.1f}%'.format(dVmax_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\ng2 = sns.distplot(injured_metrics['d_Amax'], label='Injured', ax=ax[1], color='r')\nsns.distplot(not_injured_metrics['d_Amax'], label='Not injured', ax=ax[1], color='g')\ng2.text(130, 0.0065, 'K-S test \\np_val={:.1f}%'.format(dAmax_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\ng3 = sns.distplot(injured_metrics['d_Amin'], label='Injured', ax=ax[2], color='r')\nsns.distplot(not_injured_metrics['d_Amin'], label='Not injured', ax=ax[2], color='g')\ng3.text(120, 0.007, 'K-S test \\np_val={:.1f}%'.format(aAmin_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\ng4 = sns.distplot(injured_metrics['maxD_v'], label='Injured', ax=ax[3], color='r')\nsns.distplot(not_injured_metrics['maxD_v'], label='Not injured', ax=ax[3], color='g')\ng4.text(3, 0.55, 'K-S test \\np_val={:.1f}%'.format(maxDv_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\nfig.suptitle('Global peaks', fontsize=16)\nax[0].set_title('|dir - o| when v_max')\nax[1].set_title('|dir - o| when a_max')\nax[2].set_title('|dir - o| when a_min')\nax[3].set_title('Speed in moment of max |dir - o|')\nax[0].set_xlabel('$deg$')\nax[1].set_xlabel('$deg$')\nax[2].set_xlabel('$deg$')\nax[3].set_xlabel('$yd/s$')\nax[0].legend()\nax[1].legend()\nax[2].legend()\nax[3].legend()\nsns.despine()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"And the most significant difference is seen between speed and acceleration features:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig, ax = plt.subplots(1, 3, figsize=(20, 6))\n\ng1 = sns.distplot(injured_metrics['a_max'], label='Injured', ax=ax[0], color='r')\nsns.distplot(not_injured_metrics['a_max'], label='Not injured', ax=ax[0], color='g')\ng1.text(10, 0.125, 'K-S test \\np_val={:.1f}%'.format(a_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\ng2 = sns.distplot(injured_metrics['a_min'], label='Injured', ax=ax[1], color='r')\nsns.distplot(not_injured_metrics['a_min'], label='Not injured', ax=ax[1], color='g')\ng2.text(-11, 0.23, 'K-S test \\np_val={:.1f}%'.format(d_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\ng3 = sns.distplot(injured_metrics['v_max'], label='Injured', ax=ax[2], color='r')\nsns.distplot(not_injured_metrics['v_max'], label='Not injured', ax=ax[2], color='g')\ng3.text(10, 0.14, 'K-S test \\np_val={:.1f}%'.format(v_ks_test*100), horizontalalignment='left', \n        size='medium', color='black', weight='semibold', fontsize=12)\n\nfig.suptitle('Global peaks', fontsize=16)\nax[0].set_title('Maximum linear accelerations')\nax[1].set_title('Maximum linear decelerations')\nax[2].set_title('Maximum linear speed')\nax[0].set_xlabel('$yd/s^2$')\nax[1].set_xlabel('$yd/s^2$')\nax[2].set_xlabel('$yd/s$')\nax[0].legend()\nax[1].legend()\nax[2].legend()\n\nax[0].set_xlim(0, 20)\nax[1].set_xlim(-15, 0)\nax[2].set_xlim(0, 15)\nsns.despine()","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"text","id":"Ru0ywpoldWYv"},"cell_type":"markdown","source":"We can see that the distribution of injured and not injured athletes differs, and according to Kolmohorov-Smirnov test the difference can be treated as statistically significant. Difference seems logical: injured players on average had more powerful accelerations and decelerations. Average maximum speed of injured population is approximately 20% higher. That may well be the reason of difference in distance and “side run” between injured and non-injured populations, as both of metrics correlate with speed."},{"metadata":{"colab_type":"text","id":"sbyf59FrdjPk"},"cell_type":"markdown","source":"# General conclusions:\nInjuries are of stochastic nature. Some important factors may have been out of the scope, for example, health condition of athletes, i.e. some of them can be more prone to injuries (due to joint dysfunction, repetitive traumas, or some chronic diseases, etc.). It seems fair to suggest that different athletes may have different injury risks while being exposed to the same conditions. A strong single factor or a combination of factors, which alteration may diminish the risk of injury was not found in this work. At the same time it has been found that there are a lot of factors and combinations that slightly influence injury risk. The main factors influencing injury risks are as follows: \n1.\tTurf type. Injuries on synthetic turf happen 1.7 times more often than on natural turf. \n2.\tPlay scenario. Injury risks vary depending on play scenario ( a combination of roster position, position, group position, play type) and the turf type. A play scenario “Wide Receiver, WR, WR, Kickoff” on synthetic turn has the highest injury risk. The chance of injury in such a scenario is about 0.9%. The highest risk injury on natural turf happens in a “Linebacker, OLB, LB, Punt” scenario. The chance of injury in such a scenario is about 0.4%.\n3.\tTemperature. It is likely that injury risk on synthetic turf increases with higher temperatures (approximately above 70F). One can suggest that some physical properties of synthetic turf start changing at these temperatures. Natural turf is seemingly not temperature-sensitive, as there were no indications of such dependencies found.\n4.\tPlayer dynamics. It has been found that athletes who got injured on average run faster and have more rapid accelerations than those who did not got  injured. This difference does not depend on turf type.\n"}],"metadata":{"colab":{"collapsed_sections":["mFK-xm4Sbo_-","1S1ZqIQPXRJR","poT02pWthz5F","Bhd1LhwiX5Bt","Fb7B98cCptxa"],"name":"NFL_report_submission.ipynb","provenance":[]},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.3"}},"nbformat":4,"nbformat_minor":1}