{"cells":[{"metadata":{},"cell_type":"markdown","source":"<font size=\"5\">  A simple analysis of NFL 1st and Future Analytics. Final Submission.</font> \n"},{"metadata":{},"cell_type":"markdown","source":"Let us load the libraries and set the path. I will be using data.table for fast reading of the data and also sqldf for executing some sql queries. There are three csv files \n'InjuryRecord.csv' 'PlayerTrackData.csv' 'PlayList.csv'"},{"metadata":{"trusted":true},"cell_type":"code","source":"library(tidyverse)\nlibrary(data.table)\nlibrary(sqldf)\nlibrary(caret)\nlibrary(magrittr)\nrequire(zoo)\n#library(RANN)\nlist.files(path = '../input/nfl-playing-surface-analytics')\n#data = read_csv(\"../input/nfl-playing-surface-analytics/injuryrecords.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let us read the data and see how much times it takes to read the data. I find that player track data is the largest file with  > 76m records which will be a big challenge. It also takes the largest time to load.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"T1 <- Sys.time()\n#InjuryRecord <- read.csv('../input/nfl-playing-surface-analytics/InjuryRecord.csv', , na.strings=c(\"\",\" \",\"NA\"))\nInjuryRecord <- data.table::fread(\"../input/nfl-playing-surface-analytics/InjuryRecord.csv\", stringsAsFactors = F )\nT2 <- Sys.time()\n\nInjuryRecord  <- as.data.frame(\n                lapply(InjuryRecord,  function(x) {ifelse(x == \"\", NA, x)})\n                )\n                               \nprint(T2-T1)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"T3 <- Sys.time()\n#PlayerTrackData <- read.csv('../input/nfl-playing-surface-analytics/PlayerTrackData.csv', , na.strings=c(\"\",\" \",\"NA\"))\nPlayerTrackData <-  data.table::fread(\"../input/nfl-playing-surface-analytics/PlayerTrackData.csv\", stringsAsFactors = F)\nT4 <- Sys.time()\nPlayerTrackData  <- as.data.frame(\n                lapply(PlayerTrackData ,  function(x) {ifelse(x == \"\", NA, x)})\n                )\n\nprint(T4-T3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"T5 <- Sys.time()\n#PlayList <- read.csv('../input/nfl-playing-surface-analytics/PlayList.csv',  na.strings=c(\"\",\" \",\"NA\"))\nPlayList <- data.table::fread(\"../input/nfl-playing-surface-analytics/PlayList.csv\", stringsAsFactors = F)\n\nPlayList  <- as.data.frame(\n                lapply(PlayList ,  function(x) {ifelse(x == \"\", NA, x)})\n                )\n    \nT6 <- Sys.time()\nprint(T6-T5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let us look at the number of rows in each data set."},{"metadata":{"trusted":true},"cell_type":"code","source":"nrow(InjuryRecord) #105\nnrow(PlayerTrackData) #76366748\nnrow(PlayList) #267005","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"InjuryRecord analysis"},{"metadata":{},"cell_type":"markdown","source":"There are 28 'NA' records in InjuryRecord$PlayKey.\nThere are 105 records in InjuryRecord table accross 9 columns. Column PlayKey as some blank records. In columns DM_M1 to  DM_M42, records  have been given as one hot encoding. We will convert it to a single column in subsequent code."},{"metadata":{"trusted":true},"cell_type":"code","source":"summary(InjuryRecord)\nstr(InjuryRecord)\nnrow(InjuryRecord)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There are 74526875 NA records in PlayerTrackData#event. Also there are few NA records in 'dir' and 'o' columns."},{"metadata":{"trusted":true},"cell_type":"code","source":"summary(PlayerTrackData)\nstr(PlayerTrackData)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PlayList data set has good number of NA records in StadiumType, Weather, PlayType and PositionGroup fields."},{"metadata":{"trusted":true},"cell_type":"code","source":"summary(PlayList)\nstr(PlayList)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let us see the relationsip between Body part and Surface in InjuryRecord data and check if they are releated. We can see that injuries are more on Synthetic surface. And injuries are more in Ankle and Knee followed by Toes."},{"metadata":{"trusted":true},"cell_type":"code","source":" ggplot(data=InjuryRecord, aes(x = BodyPart, fill = Surface)) +\n  geom_bar() + ggtitle(\"Releationship between Injury, Boday part and Surface on which it occured.\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Converting DM_M1, DM_M7, DM_28 and DM_42 to single records with one column and removing these columns"},{"metadata":{"trusted":true},"cell_type":"code","source":"InjuryRecord <- InjuryRecord %>% mutate(DurSum = rowSums(.[6:9]))\nInjuryRecord","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"InjuryRecord$InjDuration <- ifelse(InjuryRecord$DurSum == 4,42,\n                                   (ifelse(InjuryRecord$DurSum == 3,28,\n                                           ifelse(InjuryRecord$DurSum == 2,7,1)\n                                          )\n                                          )\n                                    )\n                                  \n#Remove extra variables\nInjuryRecord$DM_M1 <- NULL\nInjuryRecord$DM_M7 <- NULL\nInjuryRecord$DM_M28 <- NULL\nInjuryRecord$DM_M42 <- NULL\nInjuryRecord$DurSum <- NULL\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#head(InjuryRecord, n = 10)\ns1 <- sqldf('select BodyPart, sum(InjDuration) Duration from InjuryRecord group by BodyPart')\ns1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As can be seen below, injury lasts maximum for Knee followed by Ankle and Foot."},{"metadata":{"trusted":true},"cell_type":"code","source":" ggplot(data=s1, aes(x = BodyPart,y = Duration, fill=BodyPart)) +\n  geom_bar(stat=\"identity\") + ggtitle(\"How long the injury lasts for each Body Part.\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#head(InjuryRecord, n = 10)\ns2 <- sqldf('select Surface, sum(InjDuration) Duration from InjuryRecord group by Surface')\ns2","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As can be seen below, injury lasts maximum for Synthetic surface."},{"metadata":{"trusted":true},"cell_type":"code","source":"ggplot(data=s2, aes(x = Surface,y = Duration, fill=Surface)) +\n  geom_bar(stat=\"identity\") + ggtitle(\"How long the injury lasts on each surface.\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let us join InjuryRecord and PlayList."},{"metadata":{"trusted":true},"cell_type":"code","source":"                                \nInjury_PlayList <- inner_join(InjuryRecord, PlayList, by = c('PlayerKey', 'GameID'))\n\nInjury_PlayList <- Injury_PlayList %>% select (\"PlayerKey\",\"GameID\",\"BodyPart\", \"Surface\",\"InjDuration\",\n                                              \"RosterPosition\", \"PlayerDay\", \"PlayerGame\",\"StadiumType\" ,\"FieldType\",\n                                               \"Temperature\", \"Weather\" ,\"PlayType\", \"PlayerGamePlay\", \"Position\", \"PositionGroup\")\n\n\nnrow(Injury_PlayList)\n#sum(is.na(Injury_PlayList))\nhead(Injury_PlayList, n = 15)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"summary(Injury_PlayList)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Missing values treatment in Injury_PlayList"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nInjury_PlayList <- transform(Injury_PlayList, Weather = na.locf(Weather))\nInjury_PlayList <- transform(Injury_PlayList, StadiumType = na.locf(StadiumType))\nInjury_PlayList <- transform(Injury_PlayList, PositionGroup = na.locf(PositionGroup))\nInjury_PlayList <- transform(Injury_PlayList, PlayType = na.locf(PlayType))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"summary(Injury_PlayList)\nsum(is.na(Injury_PlayList))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From data given below, injuries seem to be more at Wide Receiver, Safety, LineBacker and Cornerback positions."},{"metadata":{"trusted":true},"cell_type":"code","source":"s3 <- sqldf('select distinct  RosterPosition , count( RosterPosition) RosterPosCnt from Injury_PlayList group by  RosterPosition  order by count( RosterPosition ) desc')\ns3","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class(s3)\ns3 <- s3 %>%\n  arrange(desc(RosterPosition)) %>%\n  mutate(lab.ypos = cumsum(RosterPosCnt) - 0.5*RosterPosCnt)\n\n\ns3","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ggplot(data=s3, aes(x = 2, y = RosterPosCnt, fill = RosterPosition)) + \n geom_bar(width=1,stat = \"identity\", color = \"white\") +\n  coord_polar(theta = \"y\", start = 0) +\n    #scale_fill_manual(values = mycols) +\n  theme_void() +\n   xlim(0.5, 2.5) +\nlabs(subtitle=\"Roaster Position Wise injuries\", \n            title=\"Donuts Chart\" ) +\ngeom_text(aes(y = lab.ypos, label = RosterPosCnt), color = \"white\") ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There doesn't seem to be any impact of weather conditions on injuries as per the data given below. Injuries seem to be equal for clear and cloudy weather."},{"metadata":{"trusted":true},"cell_type":"code","source":"s4 <- sqldf('select distinct Weather, count(Weather) Count from Injury_PlayList group by Weather order by count(Weather) desc')\ns4","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ggplot(s4, aes(x = Weather)) +\n  geom_col(aes(y = Count), fill = \"#5d8402\") +\n    coord_polar() +\nlabs(subtitle=\"Injuries during different types of weather\", \n            title=\"Radial, mirrored Chart\" ) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let us see different stadium types. There seems to be many. So we need to do some sort of feature engineering. I will convert them into two categories - Outdoor and Indoor. For cloudy , light rain weather, stadium should be indoor, so will replace all such weather with Indoor. Where both weather and stadium type are null we will also make them Indoor.\n "},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\nlevels( Injury_PlayList$StadiumType)\n#'' 'Bowl' 'Closed Dome' 'Cloudy' 'Dome' 'Dome, closed' 'Domed' 'Domed, closed' 'Domed, open' 'Domed, Open' \n#'Heinz Field' 'Indoor' 'Indoor, Open Roof' 'Indoor, Roof Closed' 'Indoors' 'Open' 'Oudoor' 'Ourdoor' 'Outddors' 'Outdoor'\n#'Outdoor Retr Roof-Open' 'Outdoors' 'Outdor' 'Outside' \n#Retr. Roof - Closed' 'Retr. Roof - Open' 'Retr. Roof Closed' 'Retr. Roof-Closed' 'Retr. Roof-Open' 'Retractable Roof'\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"StadiumTypeGrp <- Injury_PlayList %>% group_by(StadiumType) %>%   summarise(count = n()) %>% arrange(desc(count))\nStadiumTypeGrp\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# feature engineering for statiumtype. We will keep only two types - Outdoor and Indoor\nOutdoor <- c(\"Outdoor\", \"Outdoors\",  \"Retr. Roof - Open\",  \"Open\", \"Heinz Field\",\"Outddors\", \"Oudoor\",  \"Open Roof\" ,\n            \"Bowl\", \"Outdor\", \"Outdoor Retr Roof-Open\", \"Outside\")\nIndoor <- c(\"Indoors\", \"Indoor\", \"Retractable Roof\",\"Dome\", \"Indoor\", \"Roof Closed\", \"Domed\", \"closed\", \"Closed Dome\",\"Retr. Roof - Closed\",\n           \"Closed Dome\", \"Cloudy\", \"Domed, open\", \"Domed, Open\", \"Indoor, Roof Closed\", \"Domed, closed\", \"Retr. Roof-Closed\", \"Indoor, Open Roof\")\nInjury_PlayList$StadiumType[Injury_PlayList$StadiumType %in% Outdoor]  <- 'Outdoor'\nInjury_PlayList$StadiumType[Injury_PlayList$StadiumType %in% Indoor]  <- 'Indoor'\n\nweather <- c(\"Cloudy\",\"Partly Cloudy\",\"Light Rain\")\nInjury_PlayList$StadiumType[is.na(Injury_PlayList$StadiumType) & (Injury_PlayList$Weather %in% weather)]  <- 'Indoor'\nInjury_PlayList$StadiumType[is.na(Injury_PlayList$StadiumType)]  <- 'Indoor'\nsummary(Injury_PlayList)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\nStadiumTypeGrp <- Injury_PlayList %>% group_by(StadiumType) %>%   summarise(count = n()) %>% arrange(desc(count))\nStadiumTypeGrp\nlevels( Injury_PlayList$StadiumType)\nsummary(Injury_PlayList)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#StadiumType count\nStadiumTypeGrp <- StadiumTypeGrp %>%\n  arrange(desc(StadiumType)) %>%\n  mutate(lab.ypos = cumsum(count) - 0.5*count)\n\n\nggplot(data=StadiumTypeGrp, aes(x = 2, y = count, fill = StadiumType)) + \n geom_bar(width=1,stat = \"identity\", color = \"white\") +\n  coord_polar(theta = \"y\", start = 0) +\n    #scale_fill_manual(values = mycols) +\n  theme_void() +\n   xlim(0.5, 2.5) +\nlabs(subtitle=\"Sdatium type with injuries\", \n            title=\"Donuts Chart\" ) +\ngeom_text(aes(y = lab.ypos, label = StadiumType), color = \"white\") ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Above chart clearly states that injuries are more when stadium type is Outdoor. **"},{"metadata":{},"cell_type":"markdown","source":"**Let us see the injuries Playtype wise. We will see the unique values presebt in playtype field. We find that below are the unique values and some have zero values as well.\n\n'0' 'Extra Point' 'Field Goal' 'Kickoff' 'Kickoff Not Returned' 'Kickoff Returned' 'Pass' 'Punt' 'Punt Not Returned' 'Punt Returned' 'Rush'\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"levels( Injury_PlayList$PlayType)\ns5 <- sqldf('select * from Injury_PlayList where PlayType = 0')\ns5\ns6 <- sqldf('select  PlayType, count( PlayType) from Injury_PlayList group by PlayType')\ns6\nInjury_PlayList$PlayType[Injury_PlayList$PlayType == '0']        <- NA\nInjury_PlayList <- transform(Injury_PlayList, PlayType = na.locf(PlayType))\ns7 <- sqldf('select * from Injury_PlayList where PlayType = 0')\ns7","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{},"cell_type":"markdown","source":"**Let us look into Player Track Data which is a very large data set.******"},{"metadata":{"trusted":true},"cell_type":"code","source":"PlayTypeGrp <- Injury_PlayList %>% group_by(PlayType) %>%   summarise(count = n()) %>% arrange(desc(count))\nPlayTypeGrp\nsummary(PlayTypeGrp)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let us see the relation between PlayType and injuries"},{"metadata":{"trusted":true},"cell_type":"code","source":"ggplot(PlayTypeGrp, aes(x = PlayType )) +\n  geom_col(aes(y = count), fill = \"blue\") +\n    coord_polar() +\nlabs(subtitle=\"Injuries during different types of Playtypes\", \n            title=\"Radial, mirrored Chart\" ) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From above chart, it is clear that injuries occur mostly during Pass and Rush playtypes."},{"metadata":{},"cell_type":"markdown","source":"**Let us see the relationship between Position Group and injures**"},{"metadata":{"trusted":true},"cell_type":"code","source":"PositionGrp <- Injury_PlayList %>% group_by(PositionGroup) %>%   summarise(count = n()) %>% arrange(desc(count))\nPositionGrp\nsummary(PositionGrp)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ggplot(PositionGrp, aes(x = PositionGroup )) +\n  geom_col(aes(y = count), fill = \"red\") +\n    coord_polar() +\nlabs(subtitle=\"Injuries by position groups\", \n            title=\"Radial, mirrored Chart\" )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From above chart it is clear that most injuries occur during DB, WR and LB position groups."},{"metadata":{},"cell_type":"markdown","source":"**Time to exlpore Player Track Data. Let's join this with injury data**"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nPlayerTrackData <- PlayerTrackData %>% \n  mutate(HasInjury = PlayKey %in% InjuryRecord$PlayKey) \n#diff <- sqldf('select * from PlayerTrackData,InjuryRecord where  PlayerTrackData.PlayKey = InjuryRecord.PlayKey')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"summary(PlayerTrackData)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Missing values treatment for events"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nPlayerTrackData <- transform(PlayerTrackData, event = na.locf(event))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"TrackDataWithInjury <- filter(PlayerTrackData, HasInjury == 'TRUE')\nnrow(filter(TrackDataWithInjury, HasInjury == 'TRUE'))\n#levels(TrackDataWithInjury$PlayKey )\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"s8 <- sqldf('select x.*, y.Surface from TrackDataWithInjury x, InjuryRecord y where x.PlayKey = y.PlayKey')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"nrow(s8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"s9 <- sqldf('select  substr(PlayKey,1,5), count(substr(PlayKey,1,5))  from TrackDataWithInjury group by substr(PlayKey,1,5) \n        order by count(substr(PlayKey,1,5)) desc')\ns9","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let us see the movements of few players in x and y coordinate"},{"metadata":{"trusted":true},"cell_type":"code","source":"#38228 44449\ns10 <- sqldf('select * from TrackDataWithInjury where substr(PlayKey,1,5) = \"38228\"')\n#sql1\n\ns11 <- sqldf('select * from TrackDataWithInjury where substr(PlayKey,1,5) = \"44449\"')\n#sql2","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":" ggplot(data=s10, aes(x = x, y = y, col = PlayKey)) +\n   geom_point(shape = 8, size = 2) +  ggtitle(\"Movement of player 38228\")\n ggplot(data=s11, aes(x = x, y = y, col = PlayKey)) +\n   geom_point(shape = 8, size = 2) +  ggtitle(\"Movement of player 44449\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Let us see injuries data by event**"},{"metadata":{"trusted":true},"cell_type":"code","source":"TrackDataWithInjury_q <-  TrackDataWithInjury %>% group_by(event) %>%   summarise(count = n()) %>% arrange(desc(count)) %>% top_n(20)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"TrackDataWithInjury_q","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ggplot(TrackDataWithInjury_q, aes(x = event )) +\n  geom_col(aes(y = count), fill = \"#00FF00\") +\n    coord_polar() +\nlabs(subtitle=\"Injuries by event\", \n            title=\"Radial, mirrored Chart\" ) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**From chart it is clear that most of the  injuries are occur during tackle, line_set, huddle_break_offence, pass_outcome_incomplete and penalty_flag.**"},{"metadata":{"trusted":true},"cell_type":"code","source":"summary(s8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\navg_speed <- s8 %>% group_by(Surface) %>%\n    summarise(AverageSpeed = mean(s))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"avg_speed","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ggplot(data=avg_speed, aes(x = Surface,y = AverageSpeed, fill=Surface)) +\n  geom_bar(stat=\"identity\") + ggtitle(\"Average S.\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Average speed on Natural surface is more than for injured players than on Synthetic surface"}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}