{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"# Awesome Data Science Resources\n\n An open source Data Science Kernel to learn and apply towards solving real world problems.\n\n### Table of contents\n\n* [Motivation](#motivation)\n* [Infographic](#infographic)\n* [What is Data Science?](#what-is-data-science)\n* [Colleges](#colleges)\n* [MOOC's](#moocs)\n* [Data Sets ](#data-sets)\n* [Bloggers](#bloggers)\n* [Podcasts](#podcasts)\n* [Books](#books)\n* [Facebook Accounts](#facebook-accounts)\n* [Twitter Accounts ](#twitter-accounts )\n* [YouTube Videos & Channels](#youtube-videos--channels)\n* [Toolboxes - Environment](#toolboxes---environment)\n* [Journals, Publications and Magazines](#journals-publications-and-magazines)\n* [Presentations](#presentations)\n* [Data Science Competitions](#competitions)\n* [Comics](#comics)\n* [Tutorials](#tutorials)\n\n\n## Motivation\n\n*This part is for dummies who are new to Data Science*\n\nThis is a shortcut path to start studying **Data Science**. Just follow the steps to answer the questions, \"What is Data Science and what should I study to learn Data Science?\"\n\nFirst of all, Data Science is one of the hottest topics on the Computer and Internet farmland nowadays. People have gathered data from applications and systems until today and now is the time to analyze them. The next steps are producing suggestions from the data and creating predictions about the future. [Here](https://www.quora.com/Data-Science/What-is-data-science) you can find the biggest question for **Data Science** and hundreds of answers from experts. Our favorite data scientist is [Clare Corthell](https://twitter.com/clarecorthell). She is an expert in data-related systems and a hacker, and has been working on a company as a data scientist. [Clare's blog](http://datasciencemasters.org/). This website helps you to understand the exact way to study as a professional data scientist.\n\nSecondly, Our favorite programming language is *Python* nowadays for #DataScience. Python's - [Pandas](http://pandas.pydata.org/) library has full functionality for collecting and analyzing data. We use [Anaconda](https://www.continuum.io/downloads) to play with data and to create applications. \n\n## Infographic\n\nPreview | Description\n------------ | -------------\n[](https://s3.amazonaws.com/assets.datacamp.com/blog_assets/DataScienceEightSteps_Full.png) | A visual guide to Becoming a Data Scientist in 8 Steps by [DataCamp](https://www.datacamp.com) [(img)](https://s3.amazonaws.com/assets.datacamp.com/blog_assets/DataScienceEightSteps_Full.png)\n[](http://i.imgur.com/FxsL3b8.png) | Mindmap on required skills ([img](http://i.imgur.com/FxsL3b8.png))\n[](http://nirvacana.com/thoughts/wp-content/uploads/2013/07/RoadToDataScientist1.png) | Swami Chandrasekaran made a [Curriculum via Metro map](http://nirvacana.com/thoughts/becoming-a-data-scientist/).\n[](http://i.imgur.com/4ZBBvb0.png) | by [@kzawadz](https://twitter.com/kzawadz) via [twitter](https://twitter.com/MktngDistillery/status/538671811991715840), [MarketingDistillery.com](http://www.marketingdistillery.com/2014/11/29/is-data-science-a-buzzword-modern-data-scientist-defined/)\n[](http://i.imgur.com/4e705Q4.png) | And a male version, from another article by [MarketingDistillery.com](http://www.marketingdistillery.com/2014/08/30/data-science-skill-set-explained/)\n[](http://i.imgur.com/xLY3XZn.jpg) | By [Data Science Central](http://www.datasciencecentral.com/)\n[](http://i.imgur.com/aoz1BJy.jpg) | From [this article](http://berkeleysciencereview.com/how-to-become-a-data-scientist-before-you-graduate/) by Berkeley Science Review.\n[](http://i.imgur.com/0TydZ4M.png) | Data Science Wars: R vs Python\n[](http://i.imgur.com/HnRwlce.png) | How to select statistical or machine learning techniques\n[](http://scikit-learn.org/stable/_static/ml_map.png) | Choosing the Right Estimator\n[](http://i.imgur.com/uEqMwZa.png) | The Data Science Industry: Who Does What\n[](http://i.imgur.com/RsHqY84.png) | Data Science Venn Diagram\n[](https://www.springboard.com/blog/wp-content/uploads/2016/03/20160324_springboard_vennDiagram.png) | Different Data Science Skills and Roles from [this article](https://www.springboard.com/blog/data-science-career-paths-different-roles-industry/) by Springboard\n[\"Data](https://data-literacy.geckoboard.com/poster/) | A simple and friendly way of teaching your non-data scientist/non-statistician colleagues [how to avoid mistakes with data](https://data-literacy.geckoboard.com/poster/). From Geckoboard's [Data Literacy Lessons](https://data-literacy.geckoboard.com/).\n\n\n## What is Data Science?\n\n* [What is Data Science @ O'reilly](https://www.oreilly.com/ideas/what-is-data-science)\n* [What is Data Science @ Quora](https://www.quora.com/Data-Science/What-is-data-science)\n* [The sexiest job of 21st century](https://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century)\n* [What is data science](http://www.datascientists.net/what-is-data-science)\n* [What is a data scientist](http://www.becomingadatascientist.com/2014/02/14/what-is-a-data-scientist/)\n* [Wikipedia](https://en.wikipedia.org/wiki/Data_science)\n* [a very short history of #datascience](http://www.forbes.com/sites/gilpress/2013/05/28/a-very-short-history-of-data-science/)\n* [An Introduction to Data Science, PDF](https://ischool.syr.edu/media/documents/2012/3/DataScienceBook1_1.pdf).\n* [Data Science Methodology by John Rollins PhD](http://www.ibmbigdatahub.com/blog/why-we-need-methodology-data-science)\n* [A Day in the Life of a Data Scientist by Rutgers University](http://online.rutgers.edu/resources/articles/a-day-in-the-life-of-a-data-scientist/)\n\n## COLLEGES\n\n* [A list of colleges and universities offering degrees in data science.](https://github.com/ryanswanstrom/awesome-datascience-colleges)\n* [Data Science Degree @ Berkeley](https://datascience.berkeley.edu/)\n* [Data Science Degree @ UVA](https://dsi.virginia.edu/)\n* [Data Science Degree @ Wisconsin](http://datasciencedegree.wisconsin.edu/)\n* [Master of Information @ Rutgers](http://online.rutgers.edu/master-library-info/)\n* [MS in Computer Information Systems @ Boston University](http://cisonline.bu.edu/)\n* [MS in Business Analytics @ ASU Online](http://asuonline.asu.edu/online-degree-programs/graduate/master-science-business-analytics/)\n* [Data Science Engineer @ BTH](https://www.bth.se/nyheter/bth-startar-sveriges-forsta-civilingenjorsprogram-inom-data-science/)\n* [MS in Applied Data Science @ Syracuse](https://ischool.syr.edu/academics/graduate/masters-degrees/ms-in-applied-data-science/)\n* [M.S. Management & Data Science @ Leuphana](https://www.leuphana.de/en/graduate-school/master/course-offerings/management-data-science.html)\n* [Master of Data Science @ Melbourne University](https://science-courses.unimelb.edu.au/study/degrees/master-of-data-science/overview#overview)\n* [Msc in Data Science @ The University of Edinburgh](https://www.ed.ac.uk/studying/postgraduate/degrees/index.php?r=site/view&id=902)\n\n## MOOC's\n\n* [Coursera Introduction to Data Science](https://www.coursera.org/specializations/data-science)\n* [Data Science - 9 Steps Courses, A Specialization on Coursera](https://www.coursera.org/specializations/jhu-data-science)\n* [Data Mining - 5 Steps Courses, A Specialization on Coursera](https://www.coursera.org/specialization/datamining)\n* [Machine Learning – 5 Steps Courses, A Specialization on Coursera](https://www.coursera.org/specializations/machine-learning)\n* [CS 109 Data Science](http://cs109.github.io/2015/)\n* [Schoolofdata](https://schoolofdata.org/)\n* [OpenIntro](https://www.openintro.org/)\n* [Data science MOOC](http://datascience.sg/categories/MOOC/)\n* [CS 171 Visualization](http://www.cs171.org/#!index.md)\n* [Process Mining: Data science in Action](https://www.coursera.org/learn/process-mining)\n* [Oxford Deep Learning](http://www.cs.ox.ac.uk/projects/DeepLearn/)\n* [Oxford Deep Learning - video](https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPoTuPcqmOV53Fu)\n* [Oxford Machine Learning](http://www.cs.ox.ac.uk/activities/machinelearning/)\n* [UBC Machine Learning - video](http://www.cs.ubc.ca/~nando/540-2013/lectures.html)\n* [Data Science Specialization](https://github.com/DataScienceSpecialization/courses)\n* [Coursera Big Data Specialization](https://www.coursera.org/specializations/big-data)\n* [Data Science and Analytics in Context by Edx](https://www.edx.org/xseries/data-science-analytics-context)\n* [Big Data University by IBM](https://bigdatauniversity.com/)\n* [Udacity - Deep Learning](https://www.udacity.com/course/deep-learning--ud730)\n* [Keras in Motion](https://www.manning.com/livevideo/keras-in-motion) \n* [Microsoft Professional Program for Data Science](https://academy.microsoft.com/en-us/professional-program/tracks/data-science/)\n\n## Data Sets\n\n* [Academic Torrents](http://academictorrents.com/)\n* [hadoopilluminated.com](http://hadoopilluminated.com/hadoop_illuminated/Public_Bigdata_Sets.html)\n* [data.gov](https://catalog.data.gov/dataset) - The home of the U.S. Government's open data\n* [United States Census Bureau](http://www.census.gov/)\n* [usgovxml.com](http://usgovxml.com/)\n* [enigma.com](http://enigma.com/) - Navigate the world of public data - Quickly search and analyze billions of public records published by governments, companies and organizations.\n* [datahub.io](https://datahub.io/)\n* [aws.amazon.com/datasets](https://aws.amazon.com/datasets/)\n* [databib.org](http://databib.org/)\n* [datacite.org](https://www.datacite.org)\n* [quandl.com](https://www.quandl.com/) - Get the data you need in the form you want; instant download, API or direct to your app.\n* [figshare.com](https://figshare.com/)\n* [GeoLite Legacy Downloadable Databases](http://dev.maxmind.com/geoip/legacy/geolite/)\n* [Quora's Big Datasets Answer](https://www.quora.com/Where-can-I-find-large-datasets-open-to-the-public)\n* [Public Big Data Sets](http://hadoopilluminated.com/hadoop_illuminated/Public_Bigdata_Sets.html)\n* [Houston Data Portal](http://data.ohouston.org/)\n* [Kaggle Data Sources](https://www.kaggle.com/wiki/DataSources)\n* [Kaggle Datasets](https://www.kaggle.com/datasets)\n* [A Deep Catalog of Human Genetic Variation](http://www.internationalgenome.org/data)\n* [A community-curated database of well-known people, places, and things](https://developers.google.com/freebase/)\n* [Google Public Data](http://www.google.com/publicdata/directory)\n* [World Bank Data](http://data.worldbank.org/)\n* [NYC Taxi data](http://chriswhong.github.io/nyctaxi/)\n* [Open Data Philly](https://www.opendataphilly.org/) Connecting people with data for Philadelphia\n* [A list of useful sources](http://ahmetkurnaz.net/en/statistical-data-sources/) A blog post includes many data set databases\n* [grouplens.org](https://grouplens.org/datasets/) Sample movie (with ratings), book and wiki datasets\n* [UC Irvine Machine Learning Repository](http://archive.ics.uci.edu/ml/) - contains data sets good for machine learning\n* [research-quality data sets](http://web.archive.org/web/20150320022752/https://bitly.com/bundles/hmason/1) by [Hilary Mason](http://web.archive.org/web/20150501033715/https://bitly.com/u/hmason/bundles)\n* [National Climatic Data Center - NOAA](https://www.ncdc.noaa.gov/)\n* [ClimateData.us](http://www.climatedata.us/) (related: [U.S. Climate Resilience Toolkit](https://toolkit.climate.gov/))\n* [r/datasets](https://www.reddit.com/r/datasets/)\n* [MapLight](http://maplight.org/data) - provides a variety of data free of charge for uses that are freely available to the general public. Click on a data set below to learn more\n* [GHDx](http://ghdx.healthdata.org/) - Institute for Health Metrics and Evaluation - a catalog of health and demographic datasets from around the world and including IHME results\n* [St. Louis Federal Reserve Economic Data - FRED](https://fred.stlouisfed.org/)\n* [New Zealand Institute of Economic Research – Data1850](https://data1850.nz/)\n* [Dept. of Politics @ New York University](http://www.nyu.edu/projects/politicsdatalab/datasupp_datasources.html)\n* [Open Data Sources](https://github.com/datasciencemasters/data)\n* [UNICEF Statistics and Monitoring](https://www.unicef.org/statistics/index_24287.html)\n* [UNICEF Data](https://data.unicef.org/)\n* [undata](http://data.un.org/)\n* [NASA SocioEconomic Data and Applications Center - SEDAC](http://sedac.ciesin.columbia.edu/)\n* [The GDELT Project](http://gdeltproject.org/)\n* [Sweden, Statistics](http://www.scb.se/en/)\n* [Github free data source list](http://www.datasciencecentral.com/profiles/blogs/great-github-list-of-public-data-sets)\n* [StackExchange Data Explorer](http://data.stackexchange.com) - an open source tool for running arbitrary queries against public data from the Stack Exchange network.\n* [San Fransisco Government Open Data](https://data.sfgov.org/)\n* [IBM Blog abour open data](http://www.datasciencecentral.com/profiles/blogs/the-free-big-data-sources-everyone-should-know)\n* [Open data Index](http://index.okfn.org/)\n* [Liver Tumor Segmentation Challenge Dataset](http://www.lits-challenge.com/)\n* [Public Git Archive](https://github.com/src-d/datasets/tree/master/PublicGitArchive)\n* [GHTorrent](http://ghtorrent.org/)\n* [Microsoft Research Open Data](https://msropendata.com/)\n* [Open Government Data Platform India](https://data.gov.in/)\n\n## Bloggers\n\n- [No Free Hunch The Official Blog of Kaggle](http://blog.kaggle.com)\n- [Wes McKinney](http://wesmckinney.com/archives.html) - Wes McKinney Archives.\n- [Matthew Russell](https://miningthesocialweb.com/) - Mining The Social Web.\n- [Greg Reda](http://www.gregreda.com/) - Greg Reda Personal Blog\n- [Kevin Davenport](http://kldavenport.com/) - Kevin Davenport Personal Blog\n- [Julia Evans](http://jvns.ca/) - Recurse Center alumna\n- [Hakan Kardas](https://www.cse.unr.edu/~hkardes/) - Personal Web Page\n- [Sean J. Taylor](http://seanjtaylor.com/) - Personal Web Page\n- [Drew Conway](http://drewconway.com/) - Personal Web Page\n- [Hilary Mason](https://hilarymason.com/) - Personal Web Page\n- [Noah Iliinsky](http://complexdiagrams.com/) - Personal Blog\n- [Matt Harrison](http://hairysun.com/) - Personal Blog\n- [Data Science Renee](http://www.becomingadatascientist.com/) Documenting my path from \"SQL Data Analyst pursuing an Engineering Master's Degree\" to \"Data Scientist\"\n- [Vamshi Ambati](https://allthingsds.wordpress.com/) - AllThings Data Sciene\n- [Prash Chan](http://www.mdmgeek.com/) - Tech Blog on Master Data Management And Every Buzz Surrounding It\n- [Clare Corthell](http://datasciencemasters.org/) - The Open Source Data Science Masters\n- [Paul Miller](http://cloudofdata.com/) Based in the UK and working globally, Cloud of Data's consultancy services help clients understand the implications of taking data and more to the Cloud.\n- [Data Science London](http://datasciencelondon.org/) Data Science London is a non-profit organization dedicated to the free, open, dissemination of data science.\nWe are the largest data science community in Europe.\nWe are more than 3,190 data scientists and data geeks in our community.\n- [Datawrangling](http://datawrangling.com/) by Peter Skomoroch. MACHINE LEARNING, DATA MINING, AND MORE\n- [John Myles White](http://www.johnmyleswhite.com/) Personal Blog\n- [Quora Data Science](https://www.quora.com/Data-Science) - Data Science Questions and Answers from experts\n- [Siah](https://openresearch.wordpress.com/) a PhD student at Berkeley\n- [Data Science Report](http://datasciencereport.com/) MDS, Inc. Helps Build Careers in Data Science, Advanced Analytics, Big Data Architecture, and High Performance Software Engineering\n- [Louis Dorard](http://www.louisdorard.com/blog/) a technology guy with a penchant for the web and for data, big and small\n- [Machine Learning Mastery](http://machinelearningmastery.com/) about helping professional programmers to confidently apply machine learning algorithms to address complex problems.\n- [Daniel Forsyth](http://www.danielforsyth.me/) - Personal Blog\n- [Data Science Weekly](https://www.datascienceweekly.org/) - Weekly News Blog\n- [Revolution Analytics](http://blog.revolutionanalytics.com/) - Data Science Blog\n- [R Bloggers](https://www.r-bloggers.com/) - R Bloggers\n- [The Practical Quant](https://practicalquant.blogspot.com/) Big data\n- [Micheal Le Gal](http://www.mickaellegal.com/) a data enthusiast who gets hooked on solving intriguing problems and crafting beautiful stories and visualizations with data. Over the past 5 years, He haas applied statistics to solve problems in government, brain sciences, and most recently, retail.\n- [Datascope Analytics](https://datascopeanalytics.com/) data-driven consulting and design\n- [Yet Another Data Blog](http://yet-another-data-blog.blogspot.com.tr/) Yet Another Data Blog\n- [Spenczar](http://spenczar.com/) a data scientist at _Twitch_. I handle the whole data pipeline, from tracking to model-building to reporting.\n- [KD Nuggets](http://www.kdnuggets.com/) Data Mining, Analytics, Big Data, Data, Science not a blog a portal\n- [Meta Brown](http://www.metabrown.com/blog/) - Personal Blog\n- [Data Scientist](http://www.datascientists.net/) is building the data scientist culture.\n- [WhatSTheBigData](https://whatsthebigdata.com/) is some of, all of, or much more than the above and this blog explores its impact on information technology, the business world, government agencies, and our lives.\n- [Mic Farris](http://www.micfarris.com/) Focusing on science, datascience, business, technology, and channeling inner geekness!\n- [Tevfik Kosar](http://magnus-notitia.blogspot.com.tr/) - Magnus Notitia\n- [New Data Scientist](http://newdatascientist.blogspot.com/) How a Social Scientist Jumps into the World of Big Data\n- [Harvard Data Science](http://harvarddatascience.com/) - Thoughts on Statistical Computing and Visualization\n- [Data Science 101](http://101.datascience.community/) - Learning To Be A Data Scientist\n- [Kaggle Past Solutions](http://www.chioka.in/kaggle-competition-solutions/)\n- [DataScientistJourney](https://datascientistjourney.wordpress.com/category/data-science/)\n- [NYC Taxi Visualization Blog](http://chriswhong.github.io/nyctaxi/)\n- [Learning Lover](http://learninglover.com/blog/)\n- [Dataists](http://www.dataists.com/)\n- [Data-Mania](http://www.data-mania.com/)\n- [Data-Magnum](http://data-magnum.com/)\n- [Map Reduce Blog](https://www.mapr.com/blog)\n- [FastML Blog](http://fastml.com/)\n- [P-value](http://www.p-value.info/) - Musings on data science, machine learning and stats.\n- [datascopeanalytics](https://datascopeanalytics.com/blog/)\n- [Digital transformation](http://tarrysingh.com/)\n- [datascientistjourney](https://datascientistjourney.wordpress.com/category/data-science/)\n- [Data Mania Blog](http://www.data-mania.com/blog/)\n- [The File Drawer](http://chris-said.io/) - Chris Said's science blog\n- [Emilio Ferrara's web page](http://www.emilio.ferrara.name/)\n- [DataNews](http://datanews.tumblr.com/)\n- [Reddit TextMining](https://www.reddit.com/r/textdatamining/)\n- [Periscopic](http://www.periscopic.com/#/news)\n- [Hilary Parker](https://hilaryparker.com/)\n- [Data Stories](http://datastori.es/)\n- [Data Science Lab](https://datasciencelab.wordpress.com/)\n- [Meaning of](http://www.kennybastani.com/)\n- [Adventures in Data Land]( http://blog.smola.org)\n- [DATA MINERS BLOG](http://blog.data-miners.com/)\n- [Dataclysm](https://theblog.okcupid.com/)\n- [FlowingData](http://flowingdata.com/) - Visualization and Statistics\n- [Calculated Risk](http://www.calculatedriskblog.com/)\n- [O'reilly Learning Blog](https://www.oreilly.com/learning)\n- [Dominodatalab](https://blog.dominodatalab.com/)\n- [i am trask](http://iamtrask.github.io/) - A Machine Learning Craftsmanship Blog\n- [Vademecum of Practical Data Science](https://datasciencevademecum.wordpress.com/) - Handbook and recipes for data-driven solutions of real-world problems\n- [Dataconomy](http://dataconomy.com/) - A blog on the new emerging data economy\n- [Springboard](https://springboard.com/blog) - A blog with resources for data science learners\n- [Analytics Vidhya](https://www.analyticsvidhya.com/) - A full-fledged website about data science and analytics study material. \n- [Occam's Razor](https://www.kaushik.net/avinash/) - Focused on Web Analytics.\n- [Data School](http://www.dataschool.io/) - Data science tutorials for beginners!\n- [Colah's Blog](http://colah.github.io) - Blog for understanding Neural Networks!\n- [Sebastian's Blog](http://sebastianruder.com/#open) - Blog for NLP and transfer learning!\n- [Distill](http://distill.pub) - Dedicated to clear explanations of machine learning!\n- [Chris Albon's Website](https://chrisalbon.com/) - Data Science and AI notes \n\n## Podcasts\n\n- [Adversarial Learning](http://adversariallearning.com/)\n- [Becoming a Data Scientist](https://www.becomingadatascientist.com/category/podcast/)\n- [Data Crunch](http://vaultanalytics.com/datacrunch/)\n- [Data Skeptic](https://dataskeptic.com/)\n- [Data Stories](http://datastori.es/)\n- [Learning Machines 101](http://www.learningmachines101.com/)\n- [Linear Digressions](http://lineardigressions.com/)\n- [Not So Standard Deviations](https://soundcloud.com/nssd-podcast)\n- [Partially Derivative](http://partiallyderivative.com/)\n- [Superdatascience](https://www.superdatascience.com/podcast/)\n- [What's The Point](https://fivethirtyeight.com/tag/whats-the-point/)\n\n## Books\n- [Data Science at Scale with Python and Dask](https://www.manning.com/books/data-science-at-scale-with-python-and-dask)\n- [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/)\n- [The Data Science Handbook](http://www.thedatasciencehandbook.com/)\n- [The Art of Data Usability](https://www.manning.com/books/the-art-of-data-usability) - Early access\n- [Think Like a Data Scientist](https://www.manning.com/books/think-like-a-data-scientist)\n- [R in Action, Second Edition](https://www.manning.com/books/r-in-action-second-edition)\n- [Introducing Data Science](https://www.manning.com/books/introducing-data-science)\n- [Practical Data Science with R](https://www.manning.com/books/practical-data-science-with-r)\n- [Exploring Data Science](https://www.manning.com/books/exploring-data-science) - free eBook sampler\n- [Exploring the Data Jungle](https://www.manning.com/books/exploring-the-data-jungle) - free eBook sampler\n- [Python® for R Users: A Data Science Approach](https://onlinelibrary.wiley.com/doi/book/10.1002/9781119126805)\n- [Classic Computer Science Problems in Python](https://www.manning.com/books/classic-computer-science-problems-in-python) \n\n## Facebook Accounts\n\n- [Data](https://www.facebook.com/data)\n- [Big Data Scientist](https://www.facebook.com/Bigdatascientist)\n- [Data Science 101](https://www.facebook.com/DataScience101)\n- [Data Science Day](https://www.facebook.com/DataScienceDay/)\n- [Data Science Academy](https://www.facebook.com/nycdatascience)\n- [Facebook Data Science Page](https://www.facebook.com/pages/Data-science/431299473579193?ref=br_rs)\n- [Data Science London](https://www.facebook.com/pages/Data-Science-London/226174337471513)\n- [Data Science Technology and Corporation](https://www.facebook.com/DataScienceTechnologyCorporation?ref=br_rs)\n- [Data Science - Closed Group](https://www.facebook.com/groups/1394010454157077/?ref=br_rs)\n- [Center for Data Science](https://www.facebook.com/centerdatasciences?ref=br_rs)\n- [Big data hadoop NOSQL Hive Hbase](https://www.facebook.com/groups/bigdatahadoop/)\n- [Analytics, Data Mining, Predictive Modeling, Artificial Intelligence](https://www.facebook.com/groups/data.analytics/)\n- [Big Data Analytics using R](https://www.facebook.com/groups/434352233255448/)\n- [Big Data Analytics with R and Hadoop](https://www.facebook.com/groups/rhadoop/)\n- [Big Data Learnings](https://www.facebook.com/groups/bigdatalearnings/)\n- [Big Data, Data Science, Data Mining & Statistics](https://www.facebook.com/groups/bigdatastatistics/)\n- [BigData/Hadoop Expert](https://www.facebook.com/groups/BigDataExpert/)\n- [Data Mining / Machine Learning / AI](https://www.facebook.com/groups/machinelearningforum/)\n- [Data Mining/Big Data - Social Network Ana](https://www.facebook.com/groups/dataminingsocialnetworks/)\n- [Vademecum of Practical Data Science](https://www.facebook.com/datasciencevademecum)\n- [Veri Bilimi Istanbul](https://www.facebook.com/groups/veribilimiistanbul/)\n- [The Data Science Blog](https://www.facebook.com/theDataScienceBlog/)\n\n## Twitter Accounts\n\n- [Big Data Combine](https://twitter.com/BigDataCombine) - Rapid-fire, live tryouts for data scientists seeking to monetize their models as trading strategies\n- [Big Data Mania](https://twitter.com/BigDataGal) - Data Viz Wiz | Data Journalist | Growth Hacker | Author of Data Science for Dummies (2015)\n- [Big Data Science](https://twitter.com/analyticbridge) - Big Data, Data Science, Predictive Modeling, Business Analytics, Hadoop, Decision and Operations Research.\n- [Charlie Greenbacker](https://twitter.com/greenbacker) - Director of Data Science at @ExploreAltamira\n- [Chris Said](https://twitter.com/Chris_Said) - Data scientist at Twitter\n- [Clare Corthell](https://twitter.com/clarecorthell) - Dev, Design, Data Science @mattermark #hackerei\n- [DADI Charles-Abner](https://twitter.com/DadiCharles) - #datascientist @Ekimetrics. , #machinelearning #dataviz #DynamicCharts #Hadoop #R #Python #NLP #Bitcoin #dataenthousiast\n- [Data Science Central](https://twitter.com/DataScienceCtrl) - Data Science Central is the industry's single resource for Big Data practitioners.\n- [Data Science London](https://twitter.com/ds_ldn) Data Science. Big Data. Data Hacks. Data Junkies. Data Startups. Open Data\n- [Data Science Renee](https://twitter.com/BecomingDataSci) - Documenting my path from SQL Data Analyst pursuing an Engineering Master's Degree to Data Scientist\n- [Data Science Report](https://twitter.com/TedOBrien93) - Mission is to help guide & advance careers in Data Science & Analytics\n- [Data Science Tips](https://twitter.com/datasciencetips) - Tips and Tricks for Data Scientists around the world! #datascience #bigdata\n- [Data Vizzard](https://twitter.com/DataVisualizati) - DataViz, Security, Military\n- [DataScienceX](https://twitter.com/DataScienceX)\n- [deeplearning4j](https://twitter.com/deeplearning4j) -\n- [DJ Patil](https://twitter.com/dpatil) - White House Data Chief, VP @ RelateIQ.\n- [Domino Data Lab](https://twitter.com/DominoDataLab)\n- [Drew Conway](https://twitter.com/drewconway) - Data nerd, hacker, student of conflict.\n- [Emilio Ferrara](https://twitter.com/jabawack) - #Networks, #MachineLearning and #DataScience. I work on #Social Media. Postdoc at @IndianaUniv\n- [Erin Bartolo](https://twitter.com/erinbartolo) - Running with #BigData--enjoying a love/hate relationship with its hype. @iSchoolSU #DataScience Program Mgr.\n- [Greg Reda](https://twitter.com/gjreda) Working @ _GrubHub_ about data and pandas\n- [Gregory Piatetsky](https://twitter.com/kdnuggets) - KDnuggets President, Analytics/Big Data/Data Mining/Data Science expert, KDD & SIGKDD co-founder, was Chief Scientist at 2 startups, part-time philosopher.\n- [Hakan Kardas](https://twitter.com/hakan_kardes) - Data Scientist\n- [Hilary Mason](https://twitter.com/hmason) - Data Scientist in Residence at @accel.\n- [Jeff Hammerbacher](https://twitter.com/hackingdata) ReTweeting about data science\n- [John Myles White](https://twitter.com/johnmyleswhite) Scientist at Facebook and Julia developer. Author of Machine Learning for Hackers and Bandit Algorithms for Website Optimization. Tweets reflect my views only.\n- [Juan Miguel Lavista](https://twitter.com/BDataScientist) - Principal Data Scientist @ Microsoft Data Science Team\n- [Julia Evans](https://twitter.com/b0rk) - Hacker - Pandas - Data Analyze\n- [Kenneth Cukier](https://twitter.com/kncukier) - The Economist's Data Editor and co-author of Big Data (http://big-data-book.com ).\n- [Kevin Davenport](https://twitter.com/KevinLDavenport) - Organizer of https://meetup.com/San-Diego-R-Users-Group/\n- [Kevin Markham](https://twitter.com/justmarkham) - Data science instructor, and founder of [Data School](http://www.dataschool.io/)\n- [Kim Rees](https://twitter.com/krees) - Interactive data visualization and tools. Data flaneur.\n- [Kirk Borne](https://twitter.com/KirkDBorne) - DataScientist, PhD Astrophysicist, Top #BigData Influencer.\n- [Linda Regber](https://twitter.com/LindaRegber) - Data story teller, visualizations.\n- [Luis Rei](https://twitter.com/lmrei) - PhD Student. Programming, Mobile, Web. Artificial Intelligence, Intelligent Robotics Machine Learning, Data Mining, Natural Language Processing, Data Science.\n- [Mark Stevenson](https://twitter.com/Agent_Analytics) - Data Analytics Recruitment Specialist at Salt (@SaltJobs) | Analytics - Insight - Big Data - Datascience\n- [Matt Harrison](https://twitter.com/__mharrison__) - Opinions of full-stack Python guy, author, instructor, currently playing Data Scientist. Occasional fathering, husbanding, ult|goalt-imate, organic gardening.\n- [Matthew Russell](https://twitter.com/ptwobrussell) - Mining the Social Web.\n- [Mert Nuhoğlu](https://twitter.com/mertnuhoglu) Data Scientist at BizQualify, Developer\n- [Monica Rogati](https://twitter.com/mrogati) - Data @ Jawbone. Turned data into stories & products at LinkedIn. Text mining, applied machine learning, recommender systems. Ex-gamer, ex-machine coder; namer.\n- [Noah Iliinsky](https://twitter.com/noahi) - Visualization & interaction designer. Practical cyclist. Author of vis books: http://www.oreilly.com/pub/au/4419\n- [Paul Miller](https://twitter.com/PaulMiller) - Cloud Computing/ Big Data/ Open Data Analyst & Consultant. Writer, Speaker & Moderator. Gigaom Research Analyst.\n- [Peter Skomoroch](https://twitter.com/peteskomoroch) - Creating intelligent systems to automate tasks & improve decisions. Entrepreneur, ex Principal Data Scientist @LinkedIn. Machine Learning, ProductRei, Networks\n- [Prash Chan](https://twitter.com/MDMGeek) - Solution Architect @ IBM, Master Data Management, Data Quality & Data Governance Blogger. Data Science, Hadoop, Big Data & Cloud.\n- [Quora Data Science](https://twitter.com/q_datascience) Quora's data science topic\n- [R-Bloggers](https://twitter.com/Rbloggers) - Tweet blog posts from the R blogosphere, data science conferences and (!) open jobs for data scientists.\n- [Rand Hindi](https://twitter.com/randhindi)\n- [Randy Olson](https://twitter.com/randal_olson) - Computer scientist researching artificial intelligence. Data tinkerer. Community leader for @DataIsBeautiful. #OpenScience advocate.\n- [Recep Erol](https://twitter.com/EROLRecep) - Data Science geek @ UALR\n- [Ryan Orban](https://twitter.com/ryanorban) - Data scientist, genetic origamist, hardware aficionado\n- [Sean J. Taylor](https://twitter.com/seanjtaylor) - Social Scientist. Hacker. Facebook Data Science Team. Keywords: Experiments, Causal Inference, Statistics, Machine Learning, Economics.\n- [Silvia K. Spiva](https://twitter.com/silviakspiva) - #DataScience at Cisco\n- [Spencer Nelson](https://twitter.com/spenczar_n) - Data nerd\n- [Talha Oz](https://twitter.com/tozCSS) - Enjoys ABM, SNA, DM, ML, NLP, HI, Python, Java. Top percentile kaggler/data scientist\n- [Tasos Skarlatidis](https://twitter.com/anskarl) - Complex Event Processing, Big Data, Artificial Intelligence and Machine Learning. Passionate about programming and open-source.\n- [Terry Timko](https://twitter.com/Terry_Timko) - InfoGov; Bigdata; Data as a Service; Data Science; Open, Social & Business Data Convergence\n- [Tony Baer](https://twitter.com/TonyBaer) - IT analyst with Ovum covering Big Data & data management with some systems engineering thrown in.\n- [Tony Ojeda](https://twitter.com/tonyojeda3) - Data Scientist | Author | Entrepreneur. Co-founder @DataCommunityDC. Founder @DistrictDataLab. #DataScience #BigData #DataDC\n- [Vamshi Ambati](https://twitter.com/vambati) - Data Science @ PayPal. #NLP, #machinelearning; PhD, Carnegie Mellon alumni (Blog: https://allthingsds.wordpress.com )\n- [Wes McKinney](https://twitter.com/wesmckinn) - Pandas (Python Data Analysis library).\n- [WileyEd](https://twitter.com/WileyEd) - Senior Manager - @Seagate Big Data Analytics | @McKinsey Alum | #BigData + #Analytics Evangelist | #Hadoop, #Cloud, #Digital, & #R Enthusiast\n- [WNYC Data News Team](https://twitter.com/datanews) - The data news crew at @WNYC. Practicing data-driven journalism, making it visual and showing our work.\n@SkymindIO's open-source deep learning for the JVM. Integrates with Hadoop, Spark. Distributed GPU/CPUs | http://nd4j.org | https://www.skymind.ai/\n\n## Youtube Videos & Channels\n\n - [What is machine learning?](https://www.youtube.com/watch?v=WXHM_i-fgGo)\n - [Andrew Ng: Deep Learning, Self-Taught Learning and Unsupervised Feature Learning](https://www.youtube.com/watch?v=n1ViNeWhC24)\n - [Deep Learning: Intelligence from Big Data](https://www.youtube.com/watch?v=czLI3oLDe8M)\n - [Interview with Google's AI and Deep Learning 'Godfather' Geoffrey Hinton](https://www.youtube.com/watch?v=1Wp3IIpssEc)\n - [Introduction to Deep Learning with Python](https://www.youtube.com/watch?v=S75EdAcXHKk)\n - [What is machine learning, and how does it work?](https://www.youtube.com/watch?v=elojMnjn4kk)\n - [Data School](https://www.youtube.com/channel/UCnVzApLJE2ljPZSeQylSEyg) - Data Science Education\n - [Neural Nets for Newbies by Melanie Warrick (May 2015)](https://www.youtube.com/watch?v=Cu6A96TUy_o)\n - [Neural Networks video series by Hugo Larochelle](https://www.youtube.com/playlist?list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH)\n - [Google DeepMind co-founder Shane Legg - Machine Super Intelligence](https://www.youtube.com/watch?v=evNCyRL3DOU)\n - [Data Science Primer](https://www.youtube.com/watch?v=cHzvYxBN9Ls&list=PLPqVjP3T4RIRsjaW07zoGzH-Z4dBACpxY)\n\n## Toolboxes - Environment\n\n * [neptune.ml](https://neptune.ml) -> Community-friendly platform supporting data scientists in creating and sharing machine learning models. Neptune facilitates teamwork, infrastructure management, models comparison and reproducibility.\n * [steppy](https://github.com/neptune-ml/steppy) -> Lightweight, Python library for fast and reproducible machine learning experimentation. Introduces very simple interface that enables clean machine learning pipeline design.\n* [steppy-toolkit](https://github.com/neptune-ml/steppy-toolkit) -> Curated collection of the neural networks, transformers and models that make your machine learning work faster and more effective.\n * [Datalab from Google](https://cloud.google.com/datalab/docs/) easily explore, visualize, analyze, and transform data using familiar languages, such as Python and SQL, interactively.\n * [Hortonworks Sandbox](http://hortonworks.com/products/sandbox/) is a personal, portable Hadoop environment that comes with a dozen interactive Hadoop tutorials.\n * [R](http://www.r-project.org/) is a free software environment for statistical computing and graphics.\n * [RStudio](https://www.rstudio.com) IDE – powerful user interface for R. It’s free and open source, works onWindows, Mac, and Linux.\n * [Python - Pandas - Anaconda](https://www.continuum.io/downloads) Completely free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing\n * [Scikit-Learn](http://scikit-learn.org/stable/) Machine Learning in Python\n * [NumPy](http://www.numpy.org/) NumPy is fundamental for scientific computing with Python. It supports large, multi-dimensional arrays and matrices and includes an assortment of high-level mathematical functions to operate on these arrays.\n * [SciPy](https://www.scipy.org/) SciPy works with NumPy arrays and provides efficient routines for numerical integration and optimization.\n * [Data Science Toolbox](https://www.coursera.org/learn/data-scientists-tools) - Coursera Course\n * [Data Science Toolbox](http://datasciencetoolbox.org/) - Blog\n * [Wolfram Data Science Platform](http://www.wolfram.com/data-science-platform/) Take numerical, textual, image, GIS or other data and give it the Wolfram treatment, carrying out a full spectrum of data science analysis and visualization and automatically generating rich interactive reports—all powered by the revolutionary knowledge-based Wolfram Language.\n * [Sense Data Science Development Platform](https://sense.io/) A New Cloud Platform for Data Science and Big Data Analytics\nCollaborate on, scale, and deploy data analysis and advanced analytics projects radically faster. Use the most powerful tools — R, Python, JavaScript, Redshift, Hive, Impala, Hadoop, and more — supercharged and integrated in the cloud.\n * [Datadog](https://www.datadoghq.com/) Solutions, code, and devops for high-scale data science.\n * [Variance](http://variancecharts.com/) Build powerful data visualizations for the web without writing JavaScript\n * [Kite Development Kit](http://kitesdk.org/docs/current/index.html) The Kite Software Development Kit (Apache License, Version 2.0), or Kite for short, is a set of libraries, tools, examples, and documentation focused on making it easier to build systems on top of the Hadoop ecosystem.\n * [Domino Data Labs](http://www.dominodatalab.com) Run, scale, share, and deploy your models — without any infrastructure or setup.\n * [Apache Flink](http://flink.apache.org/) A platform for efficient, distributed, general-purpose data processing.\n * [Apache Hama](http://hama.apache.org/) Apache Hama is an Apache Top-Level open source project, allowing you to do advanced analytics beyond MapReduce.\n * [Weka](http://www.cs.waikato.ac.nz/ml/weka/) Weka is a collection of machine learning algorithms for data mining tasks.\n * [Octave](https://www.gnu.org/software/octave/) GNU Octave is a high-level interpreted language, primarily intended for numerical computations.(Free Matlab)\n * [Apache Spark](https://spark.apache.org/) Lightning-fast cluster computing\n * [Hydrosphere Mist](https://github.com/Hydrospheredata/mist) - a service for exposing Apache Spark analytics jobs and machine learning models as realtime, batch or reactive web services.\n * [Caffe](http://caffe.berkeleyvision.org/) Deep Learning Framework\n * [Torch](http://torch.ch/) A SCIENTIFIC COMPUTING FRAMEWORK FOR LUAJIT\n * [Nervana's python based Deep Learning Framework](https://github.com/NervanaSystems/neon)\n * [Skale](https://github.com/skale-me/skale-engine) - High performance distributed data processing in NodeJS\n * [Aerosolve](http://airbnb.io/aerosolve/) - A machine learning package built for humans.\n * [Intel framework](https://github.com/01org/idlf) - Intel® Deep Learning Framework\n * [Datawrapper](https://www.datawrapper.de/) – An open source data visualization platform helping everyone to create simple, correct and embeddable charts. Also at [github.com](https://github.com/datawrapper/datawrapper)\n * [Tensor Flow](https://www.tensorflow.org/) - TensorFlow is an Open Source Software Library for Machine Intelligence\n * [Natural Language Toolkit](http://www.nltk.org/)\n * [nlp-toolkit for node.js](https://www.npmjs.com/package/nlp-toolkit)\n * [Julia](http://julialang.org) – high-level, high-performance dynamic programming language for technical computing\n * [IJulia](https://github.com/JuliaLang/IJulia.jl) – a Julia-language backend combined with the Jupyter interactive environment\n * [Apache Zeppelin](http://zeppelin.apache.org/) - Web-based notebook that enables data-driven, \ninteractive data analytics and collaborative documents with SQL, Scala and more\n* [Featuretools](https://github.com/featuretools/featuretools/) - An open source framework for automated feature engineering written in python\n* [Optimus](https://github.com/ironmussa/Optimus) - Cleansing, pre-processing, feature engineering, exploratory data analysis and easy ML with PySpark backend. \n* [Albumentations](https://github.com/albu/albumentations) - А fast and framework agnostic image augmentation library that implements a diverse set of augmentation techniques. Supports classification, segmentation, detection out of the box. Was used to win a number of Deep Learning competitions at Kaggle, Topcoder and those that were a part of the CVPR workshops.\n* [DVC](https://github.com/iterative/dvc) - An open-source data science version control system. It helps track, organize and make data science projects reproducible. In its very basic scenario it helps version control and share large data and model files.\n* [Lambdo](https://github.com/asavinov/lambdo) is a workflow engine which significantly simplifies data analysis by combining in one analysis pipeline (i) feature engineering and machine learning (ii) model training and prediction (iii) table population and column evaluation.\n\n## Visualization Tools - Environments\n \n * [addepar](http://opensource.addepar.com/ember-charts/#/overview)\n * [amcharts](https://www.amcharts.com/)\n * [anychart](http://www.anychart.com/)\n * [slemma](https://slemma.com/)\n * [cartodb](http://cartodb.github.io/odyssey.js/)\n * [Cube](http://square.github.io/cube/)\n * [d3plus](http://d3plus.org/)\n * [Data-Driven Documents(D3js)](https://d3js.org/)\n * [datahero](https://datahero.com/)\n * [dygraphs](http://dygraphs.com/)\n * [ECharts](http://echarts.baidu.com/index-en.html)\n * [exhibit](http://www.simile-widgets.org/exhibit/)\n * [Gatherplot](http://www.gatherplot.org/)\n * [gephi](https://gephi.org/)\n * [ggplot2](http://ggplot2.org/)\n * [Glue](http://www.glueviz.org/en/latest/)\n * [Google Chart Gallery](https://developers.google.com/chart/interactive/docs/gallery)\n * [highcarts](http://www.highcharts.com/)\n * [import.io](https://www.import.io/)\n * [jqplot](http://www.jqplot.com/)\n * [Matplotlib](http://matplotlib.org/)\n * [nvd3](http://nvd3.org/)\n * [Opendata-tools](http://opendata-tools.org/en/visualization/) - list of open source data visualization tools\n * [Openrefine](http://openrefine.org/)\n * [plot.ly](https://plot.ly/)\n * [raw](http://rawgraphs.io)\n * [rcharts](http://rcharts.io/)\n * [techanjs](http://techanjs.org/)\n * [tenxer](http://tenxer.github.io/xcharts/)\n * [Timeline](http://timeline.knightlab.com/)\n * [variancecharts](http://variancecharts.com/index.html)\n * [vida](https://vida.io/)\n * [Wrangler](http://vis.stanford.edu/wrangler/)\n * [r2d3](http://www.r2d3.us/visual-intro-to-machine-learning-part-1/)\n * [NetworkX](https://networkx.github.io/) - High-productivity software for complex networks\n * [Redash](https://redash.io/)\n * [C3](https://c3js.org/) - D3-based reusable chart library\n\n\n## Journals, Publications and Magazines\n\n * [ICML](http://icml.cc/2015/) - International Conference on Machine Learning\n * [epjdatascience](http://epjdatascience.springeropen.com/)\n * [Journal of Data Science](http://www.jds-online.com/) - an international journal devoted to applications of statistical methods at large\n * [Big Data Research](https://www.journals.elsevier.com/big-data-research)\n * [Journal of Big Data](http://journalofbigdata.springeropen.com/)\n * [Big Data & Society](http://journals.sagepub.com/home/bds)\n * [Data Science Journal](https://www.jstage.jst.go.jp/browse/dsj)\n * [datatau.com/news](http://www.datatau.com/news) - Like Hacker News, but for data\n * [Data Science Trello Board](https://trello.com/b/rbpEfMld/data-science)\n * [Medium Data Science Topic](https://medium.com/topic/data-science) - Data Science related publications on medium\n\n\n## Presentations\n\n* [How to Become a Data Scientist](http://www.slideshare.net/ryanorban/how-to-become-a-data-scientist)\n* [Introduction to Data Science](http://www.slideshare.net/NikoVuokko/introduction-to-data-science-25391618)\n* [Intro to Data Science for Enterprise Big Data](http://www.slideshare.net/pacoid/intro-to-data-science-for-enterprise-big-data)\n* [How to Interview a Data Scientist](http://www.slideshare.net/dtunkelang/how-to-interview-a-data-scientist)\n* [How to Share Data with a Statistician](https://github.com/jtleek/datasharing)\n* [The Science of a Great Career in Data Science](http://www.slideshare.net/katemats/the-science-of-a-great-career-in-data-science)\n* [What Does a Data Scientist Do?](http://www.slideshare.net/datasciencelondon/big-data-sorry-data-science-what-does-a-data-scientist-do)\n* [Building Data Start-Ups: Fast, Big, and Focused](http://www.slideshare.net/medriscoll/driscoll-strata-buildingdatastartups25may2011clean)\n* [How to win data science competitions with Deep Learning](http://www.slideshare.net/0xdata/how-to-win-data-science-competitions-with-deep-learning)\n\n## Competitions\n\n Some data mining competition platforms\n* [Kaggle](https://www.kaggle.com/)\n* [DrivenData](https://www.drivendata.org/)\n* [Analytics Vidhya](http://datahack.analyticsvidhya.com/)\n* [The Data Science Game](http://www.datasciencegame.com/)\n* [InnoCentive](https://www.innocentive.com/)\n* [TuneedIT](http://tunedit.org/challenges)\n\n## Comics\n![Digital Data](http://imgs.xkcd.com/comics/digital_data.png \"Digital Data\")\n\n## Tutorials\n* [Awesome Data Science with Python](https://www.kaggle.com/arunkumarramanan/data-science-with-python-awesome-tutorials)\n* [Awesome Data Science with R](https://www.kaggle.com/arunkumarramanan/data-science-with-r-awesome-tutorials)\n* [Machine Learning & Deep Learning Awesome Tutorials](https://www.kaggle.com/arunkumarramanan/ml-and-deep-learning-awesome-tutorials)\n* [Awesome Data Science for Beginners](https://kaggle.com/arunkumarramanan/awesome-data-science-for-beginners?scriptVersionId=8150727)\n\n## License\n\n[![CC0](http://mirrors.creativecommons.org/presskit/buttons/88x31/svg/cc-zero.svg)](http://creativecommons.org/publicdomain/zero/1.0)"},{"metadata":{"_uuid":"e2abbc0465c46cd42ba9cf2cd5e912927fd72d8c"},"cell_type":"markdown","source":"# NFL Punt Analytics\nAnalyze NFL game data and suggest rules to improve player safety during punt plays\n## Brief Exploratory Data Analysis EDA and Quick Plots\n### Overview\nHere I use some collection of code from the NFL Punt Analytics competition kernels as follows.\n"},{"metadata":{"trusted":true,"_uuid":"9c99e86a25feaa76eb874e49dd9b991109727341"},"cell_type":"code","source":"import pandas as pd\nimport glob\nfrom plotly import offline\nimport plotly.graph_objs as go","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9be82ce6a711339e8fad57edccf465fb29122732"},"cell_type":"code","source":"pd.set_option('max.columns', None)\noffline.init_notebook_mode()\nconfig = dict(showLink=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b83ead6e6c3dd85b5ebd88909e5b1c6e8fef530d"},"cell_type":"code","source":"# EDA of concussion plays\nvideo_review = pd.read_csv('../input/video_review.csv')\nvideo_review.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"20f9392df2fcea0d6216a9e28fd79929ecf8188d"},"cell_type":"code","source":"def plot_count_category(df, column):\n x = df[column].value_counts().index\n y = df[column].value_counts()\n trace = go.Bar(\n x=x,\n y=y\n )\n data = [trace]\n offline.iplot(data, config=config)\n\n# Players involved in Tackling seem to have the brunt of concussions. I'm shocked.\nplot_count_category(video_review, 'Player_Activity_Derived')\n\n# Helmet-to-helmet and heltmet-to-body impacts result in the most coccussions\nplot_count_category(video_review, 'Primary_Impact_Type')\n\n# Which games have multiple plays with concussions?\nvideo_review[video_review.duplicated(['GameKey'], keep=False)]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a5b09391b4fdf8362b1bbc79072f285a98be9002"},"cell_type":"code","source":"def load_layout():\n \"\"\"\n Returns a dict for a Football themed Plot.ly layout \n \"\"\"\n layout = dict(\n title = \"Player Activity\",\n plot_bgcolor='darkseagreen',\n showlegend=True,\n xaxis=dict(\n autorange=False,\n range=[0, 120],\n showgrid=False,\n zeroline=False,\n showline=True,\n linecolor='black',\n linewidth=1,\n mirror=True,\n ticks='',\n tickmode='array',\n tickvals=[10,20, 30, 40, 50, 60, 70, 80, 90, 100, 110],\n ticktext=['Goal', 10, 20, 30, 40, 50, 40, 30, 20, 10, 'Goal'],\n showticklabels=True\n ),\n yaxis=dict(\n title='',\n autorange=False,\n range=[-3.3,56.3],\n showgrid=False,\n zeroline=False,\n showline=True,\n linecolor='black',\n linewidth=1,\n mirror=True,\n ticks='',\n showticklabels=False\n ),\n shapes=[\n dict(\n type='line',\n layer='below',\n x0=0,\n y0=0,\n x1=120,\n y1=0,\n line=dict(\n color='white',\n width=2\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=0,\n y0=53.3,\n x1=120,\n y1=53.3,\n line=dict(\n color='white',\n width=2\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=10,\n y0=0,\n x1=10,\n y1=53.3,\n line=dict(\n color='white',\n width=10\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=20,\n y0=0,\n x1=20,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=30,\n y0=0,\n x1=30,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=40,\n y0=0,\n x1=40,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=50,\n y0=0,\n x1=50,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=60,\n y0=0,\n x1=60,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),dict(\n type='line',\n layer='below',\n x0=70,\n y0=0,\n x1=70,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),dict(\n type='line',\n layer='below',\n x0=80,\n y0=0,\n x1=80,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=90,\n y0=0,\n x1=90,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),dict(\n type='line',\n layer='below',\n x0=100,\n y0=0,\n x1=100,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=110,\n y0=0,\n x1=110,\n y1=53.3,\n line=dict(\n color='white',\n width=10\n )\n )\n ]\n )\n return layout\n\nlayout = load_layout()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2aa280c19a187e431dc75b20077a35b523e3885c"},"cell_type":"code","source":"# Loading and plotting functions\n\ndef load_plays_for_game(GameKey):\n \"\"\"\n Returns a dataframe of play data for a given game (GameKey)\n \"\"\"\n play_information = pd.read_csv('../input/play_information.csv')\n play_information = play_information[play_information['GameKey'] == GameKey]\n return play_information\n\n\ndef load_game_and_ngs(ngs_file=None, GameKey=None):\n \"\"\"\n Returns a dataframe of player movements (NGS data) for a given game\n \"\"\"\n if ngs_file is None:\n print(\"Specifiy an NGS file.\")\n return None\n if GameKey is None:\n print('Specify a GameKey')\n return None\n # Merge play data with NGS data \n plays = load_plays_for_game(GameKey)\n ngs = pd.read_csv(ngs_file, low_memory=False)\n merged = pd.merge(ngs, plays, how=\"inner\", on=[\"GameKey\", \"PlayID\", \"Season_Year\"])\n return merged\n\n\ndef plot_play(game_df, PlayID, player1=None, player2=None, custom_layout=False):\n \"\"\"\n Plots player movements on the field for a given game, play, and two players\n \"\"\"\n game_df = game_df[game_df.PlayID==PlayID]\n \n GameKey=str(pd.unique(game_df.GameKey)[0])\n HomeTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[0]\n VisitingTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[1]\n YardLine = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)]['YardLine'].iloc[0]\n \n traces=[] \n if (player1 is not None) & (player2 is not None):\n game_df = game_df[ (game_df['GSISID']==player1) | (game_df['GSISID']==player2)]\n for player in pd.unique(game_df.GSISID):\n player = int(player)\n trace = go.Scatter(\n x = game_df[game_df.GSISID==player].x,\n y = game_df[game_df.GSISID==player].y,\n name='GSISID '+str(player),\n mode='markers'\n )\n traces.append(trace)\n else:\n print(\"Specify GSISIDs for player1 and player2\")\n return None\n \n if custom_layout is not True:\n layout = load_layout()\n layout['title'] = HomeTeam + \\\n ' vs. ' + VisitingTeam + \\\n '
Possession: ' + \\\n YardLine.split(\" \")[0] +'@'+YardLine.split(\" \")[1]\n data = traces\n fig = dict(data=data, layout=layout)\n play_description = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)].iloc[0][\"PlayDescription\"]\n print(\"\\n\\n\\t\",play_description)\n offline.iplot(fig, config=config)\n \n# Load the movements of players in GameKey 280. \ngame280 = load_game_and_ngs('../input/NGS-2016-reg-wk13-17.csv',GameKey=280)\n\n# Plot a single play, with two players\nplot_play(game_df=game280, PlayID=2918, player1=32120, player2=32725)\n\nplot_play(game280,PlayID=3746, player1=27654, player2=33127)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bda4e2436189eb5072d62b7c67e018fc1d71b34b"},"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nfrom IPython.display import HTML\nimport seaborn as sns\nimport squarify\nimport matplotlib.pyplot as plt\nimport plotly.plotly as py\nfrom plotly.offline import download_plotlyjs, init_notebook_mode, plot,iplot\nimport plotly.figure_factory as ff\n#Always run this the command before at the start of notebook\ninit_notebook_mode(connected=True)\nimport plotly.graph_objs as go\nimport cufflinks as cf\ncf.set_config_file(offline=True, world_readable=True, theme='ggplot')\nimport warnings\nwarnings.filterwarnings('ignore')\nsns.set_style(\"white\")\nplt.style.use('seaborn')\n\npd.set_option('display.max_columns', 50)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4245b63939aff709e43bcba3b2c497362dff1ee0"},"cell_type":"code","source":"def game_data():\n dir_path = os.getcwd()\n game_data = pd.read_csv('../input/game_data.csv')\n \n return game_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3dbafdf2d1a12aba742082445d6939eeaf402f31"},"cell_type":"code","source":"def get_NGS_data():\n dir_path = os.getcwd()\n files = os.listdir('../input/')\n substring = 'NGS'\n \n NGS_files = []\n for item in files:\n if substring in item:\n NGS_files.append(item)\n \n data = pd.read_csv('../input/' + NGS_files[0]) \n for file in NGS_files[1:]:\n temp_data = pd.read_csv('../input/' + file)\n data.append(temp_data)\n \n return data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2c35b94f2755f15b05910d1430629cdac11aef24"},"cell_type":"code","source":"def vid_review_data():\n dir_path = os.getcwd()\n vid_data = pd.read_csv('../input/video_review.csv')\n \n return vid_data\n\n\ndef vid_injury_data():\n dir_path = os.getcwd()\n vid_data = pd.read_csv('../input/video_footage-injury.csv')\n \n return vid_data\n\n\ndef player_role_data():\n dir_path = os.getcwd()\n role_data = pd.read_csv('../input/play_player_role_data.csv')\n \n return role_data\n\n\ndef player_punt_data():\n dir_path = os.getcwd()\n punt_data = pd.read_csv('../input/player_punt_data.csv')\n \n return punt_data\n\n\ndef play_info_data():\n dir_path = os.getcwd()\n play_data = pd.read_csv('../input/play_information.csv')\n \n return play_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"53425f917e8920e30bc32674f393075a9f960627"},"cell_type":"code","source":"def generate_table(data):\n \n trace = go.Table(\n header=dict(values=[data.columns[0], data.columns[1]],\n fill = dict(color='#C2D4FF'),\n align = ['left'] * 5),\n cells=dict(values=[data[data.columns[0]], data[data.columns[1]]],\n fill = dict(color='#F5F8FF'),\n align = ['left'] * 5))\n layout = go.Layout(\n autosize=True)\n\n data = [trace]\n filename = 'pandas_table'\n \n fig = go.Figure(data=data, layout=layout)\n return [fig, filename]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"de86dcaba3a3a264184b84c90ace70863c7f3a6a"},"cell_type":"code","source":"def generate_table3(data):\n \n trace = go.Table(\n header=dict(values=[data.columns[0], data.columns[1], data.columns[2]],\n fill = dict(color='#C2D4FF'),\n align = ['left'] * 5),\n cells=dict(values=[data[data.columns[0]], data[data.columns[1]], data[data.columns[2]]],\n fill = dict(color='#F5F8FF'),\n align = ['left'] * 5))\n layout = go.Layout(\n autosize=True)\n\n data = [trace]\n filename = 'pandas_table'\n \n fig = go.Figure(data=data, layout=layout)\n return [fig, filename]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1e2865bb417af0031d67c0b63fa8425ada498ee6"},"cell_type":"code","source":"def generate_bar_plot(x_data, y_data, data, pal, xaxis_title, yaxis_title, title, hue):\n fig, axes = plt.subplots(1,1,figsize=(10,10))\n ax = sns.barplot(x = x_data, y=y_data, data=data, palette=pal, hue = hue)\n ax.set_ylabel(yaxis_title,fontsize=15)\n ax.set_xlabel(xaxis_title,fontsize=15)\n ax.set_title(title ,fontsize=15)\n ax.tick_params(labelsize=12.5)\n try:\n plt.setp(ax.get_legend().get_texts(), fontsize='15') # for legend text\n plt.setp(ax.get_legend().get_title(), fontsize='15') # for legend title\n \n plt.show()\n except:\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"36ffa717b47d37e745b3c754f8db52ad7e7ed083"},"cell_type":"code","source":"def generate_line_plot(x_data, y_data, data, xaxis_title, yaxis_title, title):\n fig, axes = plt.subplots(1,1,figsize=(10,10))\n ax = sns.lineplot(x = x_data, y=y_data, data=data)\n ax.set_ylabel(yaxis_title,fontsize=15)\n ax.set_xlabel(xaxis_title,fontsize=15)\n ax.set_title(title ,fontsize=15)\n ax.tick_params(labelsize=12.5)\n try:\n plt.setp(ax.get_legend().get_texts(), fontsize='15') # for legend text\n plt.setp(ax.get_legend().get_title(), fontsize='15') # for legend title\n \n plt.show()\n except:\n plt.show()\n\n\ndef pie_chart(data, x, y, colour, title): \n data.iplot(kind='pie', labels = y, values = x, pull=.1, hole=.1, \n colorscale = colour, textposition='outside', \n title = title) \n\n \ndef scatter_plot(x, y, label, colour, xaxis_title, yaxis_title, title):\n \n fig,ax = plt.subplots(1,1,figsize=(10, 10))\n plt.scatter(x, y, color= colour, alpha=1, label= label)\n ax.set_ylabel(yaxis_title,fontsize=15)\n ax.set_xlabel(xaxis_title,fontsize=15)\n ax.set_title(title ,fontsize=15)\n ax.tick_params(labelsize=10)\n plt.legend()\n plt.show()\n \n \ndef bubble_plot(data, x, y, colour, xaxis_title, yaxis_title, title, size, categories):\n data.iplot(kind ='bubble', colorscale = colour, categories= categories, x = x, y = y, size = size,\n xTitle = xaxis_title, yTitle = yaxis_title, title = title)\n \n \ndef plotly_stacked_bar(x_data, y_data, categories, x_title, title):\n \n colours = ['red', 'green', 'yellow', 'blue', 'orange', 'purple', 'grey', 'black', 'cyan', 'blue', 'orange', 'purple']\n temp_data = []\n for index, item in enumerate(categories):\n trace = go.Bar(\n x = x_data,\n y = y_data,\n name=item,\n marker=dict(color=colours[index]))\n temp_data.append(trace)\n\n\n data = temp_data\n layout = go.Layout(\n barmode='stack',\n title=title\n )\n\n fig = go.Figure(data=data, layout=layout)\n iplot(fig, filename='stacked-bar')\n \n\ndef pyramid_plot(data_1, data_2, name_1, name_2, x_title, title, category, category_2):\n\n values = data_2['Percentage'] * -1\n values = values.append(data_1['Percentage'])\n values = values.sort_values(ascending=True)\n\n min_v = int(values.min())\n max_v = int(values.max())\n values = [int(i) for i in values]\n\n labels = data_2['Percentage'] * -1\n labels = labels.append(data_1['Percentage'])\n labels = labels.sort_values(ascending=True)\n labels = [int(i) for i in labels]\n\n new_labels =[]\n for item in labels:\n if item < 0:\n item = item * -1\n new_labels.append(item)\n else:\n new_labels.append(item)\n\n data_2['Percentage'] = data_2['Percentage'] * -1\n\n\n layout = go.Layout(title=title,\n yaxis=go.layout.YAxis(tickangle=-15),\n xaxis=go.layout.XAxis(\n tickangle=-55,\n range=[min_v, max_v],\n tickvals= [int(i) for i in values],\n ticktext= new_labels,\n title=x_title),\n barmode='overlay',\n bargap=0.5,\n height=500,\n width=900, \n margin=go.layout.Margin(l=225, r=0))\n\n data = [go.Bar(y=data_1[category],\n x=data_1['Percentage'],\n orientation='h',\n name=name_1,\n marker=dict(color='green')\n ),\n go.Bar(y=data_1[category],\n x=data_2['Percentage'],\n orientation='h',\n name=name_2,\n marker=dict(color='orange')\n )]\n\n iplot(dict(data=data, layout=layout), filename='EXAMPLES/bar_pyramid')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4d1d9e92a02566a654c2c9a1d4a46817123e73dc"},"cell_type":"code","source":" \ndef draw_pitch(data, col1, col2, title, poss_team, oppostion):\n #layout sourced from https://fcpython.com/visualisation/drawing-pitchmap-adding-lines-circles-matplotlib\n #pitch is 53 yards by 100 yards excluding two 10 yard touchdown zones.\n labels = ['Goal','10','20','30','40','50','40','30','20','10','Goal']\n fig = plt.figure(facecolor='white', figsize=(12.5,5))\n ax = fig.add_subplot(1,1,1)\n ax.set_facecolor('green')\n plt.yticks([]) # disable yticks\n \n start_x = -10\n bottom_y = 0\n top_y = 53\n \n ticks = [item * 10 for item in range(0,11)]\n #(x1,x2) (y1,y2)\n \n plt.plot([-10, 110],[0, 0], color='white', linewidth=4)\n plt.plot([-10, 110],[53, 53], color='white', linewidth=4)\n\n \n for item in range(0,28):\n if item == 0:\n plt.plot([start_x, start_x],[bottom_y, top_y], color='white', linewidth=4)\n \n if item >=1 and item <= 28:\n if item % 2 == 1:\n if item == 0 or item == 27:\n plt.plot([start_x, start_x],[bottom_y, top_y], color='white', linewidth=4)\n start_x = start_x + 5\n else:\n plt.plot([start_x, start_x],[bottom_y, top_y], color='white', linestyle=\"dashed\")\n start_x = start_x + 5\n \n else:\n if start_x >=0 and start_x < 110:\n plt.plot([start_x, start_x],[bottom_y, top_y], color='white', linewidth=4)\n start_x = start_x + 5 \n \n y_value = []\n for i in range(len(data)):\n y_value.append(10 + i * 5)\n \n for item in range(len(data)):\n plt.scatter(data[col1][item], y_value[item], s=80, color=\"red\")\n plt.scatter(data[col2][item], y_value[item], s=80, color=\"yellow\")\n ax.text(data[col1][item], y_value[item], poss_team[item], ha='left', size=12.5, color='black')\n ax.text(data[col2][item], y_value[item], oppostion[item], ha='left', size=12.5, color='black')\n\n plt.xticks(ticks, labels, size=15)\n plt.title(title, fontsize=20)\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"116e6b1ea4d87c9b42da04134cf0c03135898b3f"},"cell_type":"code","source":"def plotly_scatter(x, y, mode, xaxis, yaxis, title ):\n data.iplot(kind='scatter', mode=mode, x=y_data, y=x_data, xTitle=xaxis_title,\n yTitle=yaxis_title, title=title, filename='cufflinks/simple-scatter')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"951e3b42cd1d6e4f70eb93136291c0fbfa49ae35"},"cell_type":"code","source":"def perform_merge(data1, data2, columns):\n merged_data = pd.merge(data1, data2, left_on=columns, right_on=columns, suffixes=['','_1'])\n return merged_data\n\n\ndef punt_received(data):\n yards_gained = []\n \n for row in data['PlayDescription']:\n temp = row.split('punts')[1].split(' ')[1]\n yards_gained.append(int(temp))\n \n data['kicked_to'] = yards_gained\n data['kicked_to'] = data['kicked_to'] + data['kicked_from']\n \n return data\n\n\ndef punt_from(data):\n yardline = []\n \n for row in data['YardLine']:\n temp = row.split(' ')[1]\n yardline.append(int(temp))\n \n data['kicked_from'] = yardline\n \n return data\n\n\ndef opposition_team(data): \n opposition = []\n \n for item in data.iterrows():\n teams = item[1]['Home_Team_Visit_Team'].split('-')\n poss_team = item[1]['Poss_Team']\n for element in teams:\n if poss_team != element:\n opposition.append(element)\n data['oppostion'] = opposition\n\n return data\n\n\ndef visiting_data(data):\n score_away = []\n away_team = []\n for item in data['Score_Home_Visiting']:\n scores = item.split('-')\n temp = int(scores[1])\n score_away.append(temp)\n \n for item in data['Home_Team_Visit_Team']:\n teams = item.split('-')\n temp = teams[1].strip()\n away_team.append(temp)\n \n data['visiting_team'] = away_team \n data['visit_score'] = score_away\n \n return data\n\n\ndef home_data(data):\n home_score = []\n home_team = []\n for item in data['Score_Home_Visiting']:\n scores = item.split('-')\n temp = int(scores[0])\n home_score.append(temp)\n \n for item in data['Home_Team_Visit_Team']:\n teams = item.split('-')\n temp = teams[0].strip()\n home_team.append(temp)\n\n data['home_team'] = home_team \n \n data['home_score'] = home_score\n \n return data\n\n\ndef score_difference(data):\n data['score_diff'] = abs(data['home_score'] - data['visit_score'])\n \n return data\n\n\ndef missing_data(data):\n missing = pd.DataFrame(data.isnull().sum()).reset_index()\n missing.columns = ['Column', 'Count']\n missing['Percentage_Observations_Missing'] = missing['Count'] / len(data) * 100\n \n return missing\n\n\ndef count_agg(group_columns, data):\n temp_data = data.groupby(group_columns, as_index=False).size()\n agg = pd.DataFrame(temp_data.reset_index())\n group_columns.append('Count')\n agg.columns = group_columns\n agg['Percentage'] = agg['Count'] / agg['Count'].sum() * 100\n \n return agg","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"95bcc94e8feebcab80c04fd1b7401ca927bc6c11"},"cell_type":"code","source":"NGS = get_NGS_data() #player position, direction etc.\ngame = game_data() #general game data, date, time, location etc.\nvideo_data = vid_review_data() #concussion data, how it happened etc.\nrole = player_role_data() #players role during the pun\nplay = play_info_data() #type of play, team in possesion etc.\npunt = player_punt_data() #player postion\ninjury = vid_injury_data()\n\ninjury = injury.rename(columns={'gamekey':'GameKey', 'playid':'PlayID'})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9edcd28335a03f50fb7ba7bb24d1cb1ac9afb71c"},"cell_type":"code","source":"print('The number of concussion observations is: ', len(video_data))\nvideo_data.head()\n\ncolumns = ['Player_Activity_Derived']\ndata = count_agg(columns, video_data)\n\nx_data = 'Player_Activity_Derived' \ny_data = 'Percentage'\nhue = 'Player_Activity_Derived'\n\npal = 'Greens_r'\nxaxis_title = 'Player Activity'\nyaxis_title = 'Percentage'\ntitle = 'Activities Causing Concussion'\n\n\ngenerate_bar_plot(x_data, y_data, data, pal, xaxis_title, yaxis_title, title, hue)\n\n\ncolumns = ['Primary_Impact_Type']\ndata = count_agg(columns, video_data)\n\nx_data = 'Primary_Impact_Type' \ny_data = 'Percentage'\nhue = 'Primary_Impact_Type'\n\npal = 'Oranges_r'\nxaxis_title = 'Type of Impact'\nyaxis_title = 'Percentage'\ntitle = 'Impacts Causing Concussion'\n\ngenerate_bar_plot(x_data, y_data, data, pal, xaxis_title, yaxis_title, title, hue)\n\n\ndata = video_data[['Player_Activity_Derived', 'Primary_Partner_Activity_Derived']]\nresult = generate_table(data)\n\niplot(result[0], result[1])\n\n\ntemp = video_data[video_data['Primary_Partner_Activity_Derived'].isnull()]\ntemp\n\n\ntemp = video_data[video_data['Primary_Impact_Type'] == 'Helmet-to-ground']\ntemp\n\n\ntemp = video_data[video_data['Player_Activity_Derived'] == video_data['Primary_Partner_Activity_Derived']]\nprint('The number of observations where player and primary partner activities are the same is: ', len(temp))\ntemp\n\n\ncolumns = ['Primary_Impact_Type', 'Player_Activity_Derived']\ndata = count_agg(columns, video_data)\n\nx_data = 'Player_Activity_Derived' \ny_data = 'Percentage'\nhue = 'Primary_Impact_Type'\n\npal = 'viridis'\nxaxis_title = 'Player Activity'\nyaxis_title = 'Percentage'\ntitle = 'Player Activity by Impact Type'\n\ngenerate_bar_plot(x_data, y_data, data, pal, xaxis_title, yaxis_title, title, hue)\n\ncolumns = ['Primary_Impact_Type', 'Primary_Partner_Activity_Derived']\nprimary = count_agg(columns, video_data)\n\nx_data = 'Primary_Partner_Activity_Derived' \ny_data = 'Percentage'\nhue = 'Primary_Impact_Type'\n\npal = 'viridis'\nxaxis_title = 'Primary Partner Activity'\nyaxis_title = 'Percentage'\ntitle = 'Primary Partner Activity by Impact Type'\n\ngenerate_bar_plot(x_data, y_data, primary, pal, xaxis_title, yaxis_title, title, hue)\n\ncolumns = ['GameKey']\nplay_counts = count_agg(columns, video_data)\nprint('Number of games with more than one concussion: ', len(play_counts[play_counts['Count'] >= 2]))\nplay_counts[play_counts['Count'] >= 2]\n\n\ncolumns = ['GameKey', 'GSISID' ,'PlayID']\nconcussion_role = perform_merge(video_data, role, columns)\n\ncolumns = ['Role', 'Player_Activity_Derived']\ndata = count_agg(columns, concussion_role)\ndata\nx = 'Role'\ny = 'Percentage'\nsize = 'Percentage'\ncategories = 'Role'\n\n\ncolour = None\nxaxis_title = 'Player Role'\nyaxis_title = 'Percentage of Concussions'\ntitle = 'Player Role vs. Percentage of Concussions'\n\nbubble_plot(data, x, y, colour, xaxis_title, yaxis_title, title, size, categories) \n\n\nmissing = missing_data(game)\ndata = missing[['Column', 'Percentage_Observations_Missing']]\nresult = generate_table(data)\n\niplot(result[0], result[1])\n\npunt_returners = concussion_role[concussion_role['Role'] == 'PR']\n\ncolumns = ['GameKey']\npunt_returners = perform_merge(punt_returners, game, columns)\n\ncolumns = ['Season_Type']\nPR_season = count_agg(columns, punt_returners)\n\ndata = PR_season[['Season_Type', 'Percentage']]\nresult = generate_table(data)\n\niplot(result[0], result[1])\n\ncolumns = ['Season_Year']\nPR_season = count_agg(columns, punt_returners)\n\ndata = PR_season[['Season_Year', 'Percentage']]\nresult = generate_table(data)\n\niplot(result[0], result[1])\n\ncolumns = ['Player_Activity_Derived', 'Primary_Impact_Type']\nPR_Concussions = count_agg(columns, punt_returners)\n\nx_data = 'Player_Activity_Derived' \ny_data = 'Percentage'\nhue = 'Primary_Impact_Type'\n\npal = 'viridis'\nxaxis_title = 'Player Activity'\nyaxis_title = 'Percentage'\ntitle = 'Punt Returner Activity and Impacts Causing Concussion'\n\ngenerate_bar_plot(x_data, y_data, PR_Concussions, pal, xaxis_title, yaxis_title, title, hue)\n\n\ndata = punt_returners[['Primary_Impact_Type', 'Week']]\nresult = generate_table(data)\n\niplot(result[0], result[1])\n\ncolumns = ['GameKey', 'PlayID']\nPR = perform_merge(punt_returners, play, columns)\n\nPR = visiting_data(PR)\nPR = home_data(PR)\nPR = score_difference(PR)\n\ndata = PR[['Quarter','score_diff' ]]\n\nx_data = 'Quarter'\ny_data = 'score_diff'\nmode='markers'\nxaxis_title = 'Score defecit'\nyaxis_title = 'Quarter'\ntitle = 'Score defecit vs Quarter'\n\nplotly_scatter(x_data, y_data, mode, xaxis_title, yaxis_title, title)\n\ncolumns = ['Quarter']\nPR_quarter = count_agg(columns, PR)\n\ny = 'Quarter'\nx = 'Count'\ncolour = 'Spectral'\ntitle = 'What Quarter Punt Returners Receive Concussions'\n\npie_chart(PR_quarter, x, y, colour, title)\n\n\nPR = opposition_team(PR)\nPR = punt_from(PR)\nPR = punt_received(PR)\n\ntitle = 'Yard line ball was kicked from (red) to yard line it reached (yellow)'\ndraw_pitch(PR, 'kicked_from', 'kicked_to', title, PR['Poss_Team'], PR['oppostion'])\n\nprlg = concussion_role[(concussion_role['Role'] == 'PRG') | (concussion_role['Role'] == 'PLG')]\n\ncolumns = ['GameKey']\nprlg = perform_merge(prlg, game, columns)\n\ncolumns = ['Role', 'Season_Type']\nPRLG_season = count_agg(columns, prlg)\n\ndata = PRLG_season[['Role', 'Season_Type', 'Count']]\ngenerate_bar_plot('Role', 'Count', data, 'viridis', 'Player Role', 'Count', 'Number of Concussions per Role by Season Type', 'Season_Type')\n\n\ncolumns = ['Role', 'Season_Year']\nPRLG_season = count_agg(columns, prlg)\n\ndata = PRLG_season[['Role', 'Season_Year', 'Count']]\ngenerate_bar_plot('Role', 'Count', data, 'viridis', 'Player Role', 'Count', 'Number of Concussions per Role by Year', 'Season_Year')\n\n\ndata = prlg[['Role','Primary_Impact_Type']]\ncolumns = ['Role','Primary_Impact_Type']\ndata = count_agg(columns, data)\n\ny = 'Primary_Impact_Type'\nx = 'Count'\ncolour = 'Spectral'\ntitle = \"What Impacts are causing PRG's and PLG's to get concussions?\"\n\npie_chart(data, x, y, colour, title)\n\ndata = prlg[['Role','Player_Activity_Derived', 'Primary_Impact_Type']]\ncolumns = ['Role','Player_Activity_Derived', 'Primary_Impact_Type']\ndata = count_agg(columns, data)\n\ngenerate_bar_plot('Player_Activity_Derived', 'Percentage', data, 'viridis', 'Player Activity', 'Percentage of Concussions', 'Percentage of Concussions per Player Activity coloured by Impact Type', 'Primary_Impact_Type')\n\n\ndata = prlg[['Role','Primary_Impact_Type']]\ncolumns = ['Role','Primary_Impact_Type']\ndata = count_agg(columns, data)\n\ndata = data[data['Role'] == 'PRG']\n\ny = 'Primary_Impact_Type'\nx = 'Count'\ncolour = 'PiYG'\ntitle = \"What Impacts are causing PRG's to get concussions?\"\n\npie_chart(data, x, y, colour, title)\n\n\ndata = prlg[['Role','Primary_Impact_Type']]\ncolumns = ['Role','Primary_Impact_Type']\ndata = count_agg(columns, data)\n\ndata = data[data['Role'] == 'PLG']\n\ny = 'Primary_Impact_Type'\nx = 'Count'\ncolour = 'Spectral'\ntitle = \"What Impacts are causing PLG's to get concussions?\"\n\npie_chart(data, x, y, colour, title)\n\n\ndata = prlg[['Role','Week','Season_Type']]\ndata = data[data['Season_Type'] == 'Reg']\n\nresult = generate_table3(data)\niplot(result[0], result[1])\n\n\ncolumns = ['GameKey', 'PlayID']\nprlg = perform_merge(prlg, play, columns)\n\nprlg = opposition_team(prlg)\nprlg = punt_from(prlg)\nprlg = punt_received(prlg)\n\ntitle = 'Yard line ball was kicked from (red) to yard line it reached (yellow)'\ndraw_pitch(prlg, 'kicked_from', 'kicked_to', title, prlg['Poss_Team'], prlg['oppostion'])\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"95ddbceeefb08c47ba49c3f653d8fd3eb036543c"},"cell_type":"code","source":"\n#import packages\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport plotly.plotly as py\nimport plotly.graph_objs as go\nfrom plotly import tools, offline\n%matplotlib inline \n\n#Read in data\n#Only reading in 1 week to save time/space\nNGS_df = pd.read_csv('../input/NGS-2017-reg-wk1-6.csv')\n#Creating a function to calculate the hang time\n\ndef get_hang_time(ngs_df, start_event='punt', *stop_events):\n punt_event = ngs_df.loc[ngs_df.Event==start_event] \\\n .groupby(['Season_Year', 'GameKey','PlayID'], as_index = False)['Time'].min()\n punt_event.rename(columns = {'Time':'punt_time'}, inplace=True)\n punt_event['punt_time'] = pd.to_datetime(punt_event['punt_time'],\\\n format='%Y-%m-%d %H:%M:%S.%f')\n \n receiving_event = ngs_df.loc[ngs_df.Event.isin(stop_events)] \\\n .groupby(['Season_Year', 'GameKey','PlayID'], as_index = False)['Time'].min()\n receiving_event.rename(columns = {'Time':'receiving_time'}, inplace=True)\n receiving_event['receiving_time'] = pd.to_datetime(receiving_event['receiving_time'],\\\n format='%Y-%m-%d %H:%M:%S.%f')\n \n punt_df = punt_event.merge(receiving_event, how='inner', on = ['Season_Year','GameKey','PlayID']) \\\n .reset_index(drop=True)\n \n punt_df['hang_time'] = (punt_df['receiving_time'] - punt_df['punt_time']).dt.total_seconds()\n \n return punt_df\n\npunt_df = get_hang_time(NGS_df, 'punt', 'punt_received', 'fair_catch')\n\n#Show histogram of the hang_time column\npunt_df.hang_time.hist();\n#Some general statistics for hang times\n\nprint('The average hang time of a punt is {} seconds' .format(round(punt_df['hang_time'].mean(), 1)))\n\nprint('The median hang time of a punt is {} seconds' .format(round(punt_df['hang_time'].median(), 1)))\n\nprint(str(round(len(punt_df.loc[punt_df.hang_time < 5.5]) / len(punt_df) * 100, 1)) \\\n + '% of hang times are less than 5 1/2 seconds')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2bd927d2aa33c2838fcd4d4a73c3a585f707c0ff"},"cell_type":"code","source":"\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\n\nimport bokeh\nfrom bokeh.io import show, output_notebook, push_notebook\nfrom bokeh.layouts import row, column, widgetbox\nfrom bokeh.plotting import figure\nfrom bokeh.models import CustomJS, ColumnDataSource, Slider\nfrom bokeh.models import ColumnDataSource, Range1d, LabelSet, Label\nfrom bokeh.models import BoxAnnotation\nfrom bokeh.models.glyphs import Text\nfrom bokeh.models.widgets import PreText, Div\n\noutput_notebook()\n\n# Read play data\nngs = pd.read_csv('../input/NGS-2016-pre.csv')\nngs.columns = [col.lower() for col in ngs.columns]\npprd = pd.read_csv('../input/play_player_role_data.csv')\npprd.columns = [col.lower() for col in pprd.columns]\nvr = pd.read_csv('../input/video_review.csv')\nvr.columns = [col.lower() for col in vr.columns]\n\n# Take a play as an example\nvr[['season_year','gamekey','playid']].sort_values(['season_year','gamekey','playid']).loc[1]\n\n# Filter a sepcific play\nseason_year=2016\ngamekey=21\nplayid=2587\n\nplay = ngs[(ngs['season_year'] == season_year) &\n (ngs['gamekey'] == gamekey) &\n (ngs['playid'] == playid)]\n\n# Pull just one play\nplay = pd.merge(play, pprd)\nplayxy = play.drop(['season_year','gamekey'], axis=1) \\\n .pivot(index='time',\n columns='role',\n values=['x','y'])\n\n# Create player colors\ncolors = ['red' if x in(vr['gsisid'].values) or x in(vr['primary_partner_gsisid'].values) else 'blue' for x in play.gsisid.unique()]\n\n# Setup Figure\n## Football Field Figure\nfig = figure(plot_width=800, plot_height=400, x_range=(0,120), y_range=(0, 53.3))\nfig.xgrid.grid_line_color = None\nfig.ygrid.grid_line_color = None\n# Green Field\nbox = BoxAnnotation(left=0, right=120, fill_color='green', fill_alpha=0.5)\nendzone1 = BoxAnnotation(left=0, right=10, fill_color='grey', fill_alpha=0.5)\nendzone2 = BoxAnnotation(left=110, right=120, fill_color='grey', fill_alpha=0.5)\nfig.add_layout(box)\nfig.add_layout(endzone1)\nfig.add_layout(endzone2)\n# Add lines|\nfig.line([10,10,20,20,30,30,40,40,50,50,60,60,70,70,80,80,90,90,\n 100,100,110,110,120,120],\n [55.5,0,0,55.5,55.5,0,0,55.5,55.5,0,0,55.5,55.5,0,0,55.5,\n 55.5,0,0,55.5,55.5,0,0,0], line_color='grey')\n# Line numbers\n\nline_nums = ColumnDataSource(dict(x=[20, 30, 40, 50, 60, 70, 80, 90, 100],\n y=[5, 5, 5, 5, 5, 5, 5, 5, 5],\n text=['10', '20', '30','40','50','40','30','20','10']))\nglyph = Text(x=\"x\", y=\"y\", text=\"text\", angle=0, text_color=\"black\", text_align='center')\nfig.add_glyph(line_nums, glyph)\n\n# Create values used in the initial states\nx_values = playxy.loc[playxy.index[1]]['x'].values\ny_values = playxy.loc[playxy.index[1]]['y'].values\ngsisid = playxy['x'].columns.values\nsource = ColumnDataSource(data=dict(x=x_values,\n y=y_values,\n gsisid=gsisid))\nlabels = LabelSet(x='x', y='y', text='gsisid', level='glyph',\n x_offset=5, y_offset=5, source=source, render_mode='canvas')\n\n# Add players as circles\nplt = fig.circle(x_values, y_values, size=20, alpha=0.5, color=colors, radius=1)\nfig.add_layout(labels)\n\ndiv = Div(text=\"\"\"....\"\"\",width=200, height=100)\n\ndef plot_time_pos(time_pos):\n x_values = playxy.loc[playxy.index[time_pos]]['x'].values\n y_values = playxy.loc[playxy.index[time_pos]]['y'].values\n gsisid = playxy['x'].columns.values\n # p = figure(plot_width=800, plot_height=400)\n # p.circle(x_values, y_values, size=20, color=\"navy\", alpha=0.5)\n source = ColumnDataSource(data=dict(x=x_values,\n y=y_values,\n gsisid=gsisid))\n labels = LabelSet(x='x', y='y', text='gsisid', level='glyph',\n x_offset=5, y_offset=5, source=source, render_mode='canvas')\n fig.renderers.pop()\n fig.add_layout(labels)\n plt.data_source.data['x'] = x_values\n plt.data_source.data['y'] = y_values\n div.text = playxy.loc[playxy.index[time_pos]].name\n\n push_notebook(handle=bokeh_handle)\n\n\nsource = ColumnDataSource(data=dict(x=x_values, y=y_values))\n\ncallback = CustomJS(code=\"\"\"\nif (IPython.notebook.kernel !== undefined) {\n var kernel = IPython.notebook.kernel;\n cmd = \"plot_time_pos(\" + cb_obj.value + \")\";\n kernel.execute(cmd, {}, {});\n}\n\"\"\")\n\n\nslider = Slider(start=1,\n end=len(playxy),\n value=1,\n step=1,\n title=\"position within play\",\n callback=callback)\n\nlayout = column(\n widgetbox(slider),\n fig,\n div\n)\n\nbokeh_handle = show(layout, notebook_handle=True)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d52839de9315d8c437244b6e3a7db514d3e0ca01"},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pylab as plt\nimport os\nplt.style.use('ggplot')\n\n# For interactive plots\nfrom plotly import offline\nimport plotly.graph_objs as go\n\n\npd.set_option('max.columns', None)\noffline.init_notebook_mode()\nconfig = dict(showLink=False)\n\n#Lets explore the smaller datasets\n\n# Read the input data\nppd = pd.read_csv('../input/player_punt_data.csv')\ngd = pd.read_csv('../input/game_data.csv')\npprd = pd.read_csv('../input/play_player_role_data.csv')\nvr = pd.read_csv('../input/video_review.csv')\nvfi = pd.read_csv('../input/video_footage-injury.csv')\npi = pd.read_csv('../input/play_information.csv')\n\ngd.head()\n\ngd.plot(kind='scatter', x='Week', y='Temperature', figsize=(15, 5), title='NFL Game Data Week vs. Temperature')\nplt.show()\n\ngd['count'] = 1\ngd.groupby('Turf') \\\n .count()[['count']] \\\n .sort_values('count', ascending=False) \\\n .plot(kind='bar', figsize=(15, 5), rot=85, title='Count of Games by Turf Type')\nplt.show()\n\ngd.groupby('Start_Time') \\\n .count()[['count']] \\\n .plot(kind='bar', figsize=(15, 5), rot=85, title='Count of Games by Start Time', color='g')\nplt.show()\n\nvr.shape\n\nvr.head()\n\nvr['count'] = 1\nvr.groupby('Player_Activity_Derived') \\\n .count()[['count']] \\\n .sort_values('count', ascending=False) \\\n .plot(kind='barh', figsize=(15, 5), title='Count of Player Activity Derived')\n\nplt.show()\n\nvr['count'] = 1\nvr.groupby('Primary_Partner_Activity_Derived') \\\n .count()[['count']] \\\n .sort_values('count', ascending=False) \\\n .plot(kind='barh', figsize=(15, 5), title='Count of Primary Partner Activity Derived', color='g')\n\nplt.show()\n\nvr['count'] = 1\nvr.groupby('Primary_Impact_Type') \\\n .count()[['count']] \\\n .sort_values('count', ascending=False) \\\n .plot(kind='barh', figsize=(15, 5), title='Count of Primary Impact Type', color='b')\nplt.show()\n\npi.head()\n\npi['count'] = 1\npi.groupby('Poss_Team').count()[['count']] \\\n .sort_values('count', ascending=False) \\\n .plot(kind='bar', figsize=(15, 5), title='Count of punts per team', color='k')\nplt.show()\n\n# They are all punts!!! :D\npi['Play_Type'].unique()\n\n# Loading and plotting functions\n\ndef load_plays_for_game(GameKey):\n \"\"\"\n Returns a dataframe of play data for a given game (GameKey)\n \"\"\"\n play_information = pd.read_csv('../input/play_information.csv')\n play_information = play_information[play_information['GameKey'] == GameKey]\n return play_information\n\n\ndef load_game_and_ngs(ngs_file=None, GameKey=None):\n \"\"\"\n Returns a dataframe of player movements (NGS data) for a given game\n \"\"\"\n if ngs_file is None:\n print(\"Specifiy an NGS file.\")\n return None\n if GameKey is None:\n print('Specify a GameKey')\n return None\n # Merge play data with NGS data \n plays = load_plays_for_game(GameKey)\n ngs = pd.read_csv(ngs_file, low_memory=False)\n merged = pd.merge(ngs, plays, how=\"inner\", on=[\"GameKey\", \"PlayID\", \"Season_Year\"])\n return merged\n\n\ndef plot_play(game_df, PlayID, player1=None, player2=None, custom_layout=False):\n \"\"\"\n Plots player movements on the field for a given game, play, and two players\n \"\"\"\n game_df = game_df[game_df.PlayID==PlayID]\n \n GameKey=str(pd.unique(game_df.GameKey)[0])\n HomeTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[0]\n VisitingTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[1]\n YardLine = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)]['YardLine'].iloc[0]\n \n traces=[] \n if (player1 is not None) & (player2 is not None):\n game_df = game_df[ (game_df['GSISID']==player1) | (game_df['GSISID']==player2)]\n for player in pd.unique(game_df.GSISID):\n player = int(player)\n trace = go.Scatter(\n x = game_df[game_df.GSISID==player].x,\n y = game_df[game_df.GSISID==player].y,\n name='GSISID '+str(player),\n mode='markers'\n )\n traces.append(trace)\n else:\n print(\"Specify GSISIDs for player1 and player2\")\n return None\n \n if custom_layout is not True:\n layout = load_layout()\n layout['title'] = HomeTeam + \\\n ' vs. ' + VisitingTeam + \\\n '
Possession: ' + \\\n YardLine.split(\" \")[0] +'@'+YardLine.split(\" \")[1]\n data = traces\n fig = dict(data=data, layout=layout)\n play_description = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)].iloc[0][\"PlayDescription\"]\n print(\"\\n\\n\\t\",play_description)\n offline.iplot(fig, config=config)\n \ndef load_layout():\n \"\"\"\n Returns a dict for a Football themed Plot.ly layout \n \"\"\"\n layout = dict(\n title = \"Player Activity\",\n plot_bgcolor='darkseagreen',\n showlegend=True,\n xaxis=dict(\n autorange=False,\n range=[0, 120],\n showgrid=False,\n zeroline=False,\n showline=True,\n linecolor='black',\n linewidth=1,\n mirror=True,\n ticks='',\n tickmode='array',\n tickvals=[10,20, 30, 40, 50, 60, 70, 80, 90, 100, 110],\n ticktext=['Goal', 10, 20, 30, 40, 50, 40, 30, 20, 10, 'Goal'],\n showticklabels=True\n ),\n yaxis=dict(\n title='',\n autorange=False,\n range=[-3.3,56.3],\n showgrid=False,\n zeroline=False,\n showline=True,\n linecolor='black',\n linewidth=1,\n mirror=True,\n ticks='',\n showticklabels=False\n ),\n shapes=[\n dict(\n type='line',\n layer='below',\n x0=0,\n y0=0,\n x1=120,\n y1=0,\n line=dict(\n color='white',\n width=2\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=0,\n y0=53.3,\n x1=120,\n y1=53.3,\n line=dict(\n color='white',\n width=2\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=10,\n y0=0,\n x1=10,\n y1=53.3,\n line=dict(\n color='white',\n width=10\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=20,\n y0=0,\n x1=20,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=30,\n y0=0,\n x1=30,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=40,\n y0=0,\n x1=40,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=50,\n y0=0,\n x1=50,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=60,\n y0=0,\n x1=60,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),dict(\n type='line',\n layer='below',\n x0=70,\n y0=0,\n x1=70,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),dict(\n type='line',\n layer='below',\n x0=80,\n y0=0,\n x1=80,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=90,\n y0=0,\n x1=90,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),dict(\n type='line',\n layer='below',\n x0=100,\n y0=0,\n x1=100,\n y1=53.3,\n line=dict(\n color='white'\n )\n ),\n dict(\n type='line',\n layer='below',\n x0=110,\n y0=0,\n x1=110,\n y1=53.3,\n line=dict(\n color='white',\n width=10\n )\n )\n ]\n )\n return layout\n\nlayout = load_layout()\n\n# Load the movements of players in GameKey 280. \ngame280 = load_game_and_ngs('../input/NGS-2016-reg-wk13-17.csv',GameKey=280)\n\n# Plot a single play, with two players\nplot_play(game_df=game280, PlayID=2918, player1=32864, player2=32725)\n\ndef plot_play_all_players(game_df, PlayID, custom_layout=False):\n \"\"\"\n Plots player movements on the field for a given game, play, and two players\n \"\"\"\n game_df = game_df[game_df.PlayID==PlayID]\n \n GameKey=str(pd.unique(game_df.GameKey)[0])\n HomeTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[0]\n VisitingTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[1]\n player1 = game_df[(game_df.PlayID==PlayID)]['GSISID'].values[0]\n YardLine = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)]['YardLine'].iloc[0]\n \n traces=[] \n for player in pd.unique(game_df.GSISID):\n player = int(player)\n trace = go.Scatter(\n x = game_df[game_df.GSISID==player].x,\n y = game_df[game_df.GSISID==player].y,\n name='GSISID '+str(player),\n mode='markers'\n )\n traces.append(trace)\n if custom_layout is not True:\n layout = load_layout()\n layout['title'] = HomeTeam + \\\n ' vs. ' + VisitingTeam + \\\n '
Possession: ' + \\\n YardLine.split(\" \")[0] +'@'+YardLine.split(\" \")[1]\n data = traces\n fig = dict(data=data, layout=layout)\n play_description = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)].iloc[0][\"PlayDescription\"]\n print(\"\\n\\n\\t\",play_description)\n offline.iplot(fig, config=config)\n\nplot_play_all_players(game_df=game280, PlayID=2918)\n#A few notes after plotting all players:\n\npprd.head()\n\n\n# inputs\ncustom_layout = False\ngame_df=game280\nPlayID=2918\n\n\n# Function code\ngame_df = game_df[game_df.PlayID==PlayID]\n\nGameKey=str(pd.unique(game_df.GameKey)[0])\nHomeTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[0]\nVisitingTeam = pd.unique(game_df.Home_Team_Visit_Team)[0].split(\"-\")[1]\nplayer1 = game_df[(game_df.PlayID==PlayID)]['GSISID'].values[0]\nYardLine = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)]['YardLine'].iloc[0]\n\ntraces=[] \nfor player in pd.unique(game_df.GSISID):\n player = int(player)\n trace = go.Scatter(\n x = game_df[game_df.GSISID==player].x,\n y = game_df[game_df.GSISID==player].y,\n name='GSISID '+str(player),\n mode='markers'\n )\n traces.append(trace)\nif custom_layout is not True:\n layout = load_layout()\n layout['title'] = HomeTeam + \\\n ' vs. ' + VisitingTeam + \\\n '
Possession: ' + \\\n YardLine.split(\" \")[0] +'@'+YardLine.split(\" \")[1]\ndata = traces\nfig = dict(data=data, layout=layout)\nplay_description = game_df[(game_df.PlayID==PlayID) & (game_df.GSISID==player1)].iloc[0][\"PlayDescription\"]\nprint(\"\\n\\n\\t\",play_description)\noffline.iplot(fig, config=config)\n\npprd.shape\n\npprd.head()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"39d612d240ae228b8b5386e1f931f3211faf1b95"},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport warnings\nwarnings.filterwarnings('ignore')\nfrom plotly.offline import download_plotlyjs,init_notebook_mode,plot, iplot\nimport cufflinks as cf\ninit_notebook_mode(connected = True)\ncf.go_offline()\n%matplotlib inline\n\nfrom plotly import tools\nimport plotly.plotly as py\nfrom plotly.offline import init_notebook_mode, iplot\ninit_notebook_mode(connected=True)\nimport plotly.graph_objs as go\nimport plotly.figure_factory as ff\nimport plotly.offline as offline\n# Squarify for treemaps\nimport squarify\n# Random for well, random stuff\nimport random\n# operator for sorting dictionaries\nimport operator\n# For ignoring warnings\nimport warnings\nwarnings.filterwarnings('ignore')\n\n\n\nvideo_review = pd.read_csv('../input/video_review.csv')\nvideo_review.head()\n\nvideo_review.info()\n\nvideo_review.describe()\n\n# Player activity during primary injury causing event\n\ntemp = video_review[\"Player_Activity_Derived\"].value_counts()\nfig = {\n \"data\": [\n {\n \"values\": temp.values,\n \"labels\": temp.index,\n \"domain\": {\"x\": [0, 1]},\n \"hole\": .6,\n \"type\": \"pie\"\n },\n \n ],\n \"layout\": {\n \"title\":\"Player activity during primary injury causing event\",\n \"annotations\": [\n {\n \"font\": {\n \"size\": 17\n },\n \"showarrow\": False,\n \"text\": \"Player activity\",\n \"x\": 0.5,\n \"y\": 0.5\n }\n \n ]\n }\n}\niplot(fig, filename='donut')\n\n# Primary_Impact_Type\n\n\ntemp = video_review[\"Primary_Impact_Type\"].value_counts()\nfig = {\n \"data\": [\n {\n \"values\": temp.values,\n \"labels\": temp.index,\n \"domain\": {\"x\": [0, 1]},\n \"hole\": .6,\n \"type\": \"pie\"\n },\n \n ],\n \"layout\": {\n \"title\":\"Impacting source that caused the concussion\",\n \"annotations\": [\n {\n \"font\": {\n \"size\": 17\n },\n \"showarrow\": False,\n \"text\": \"Primary Impact type\",\n \"x\": 0.5,\n \"y\": 0.5\n }\n \n ]\n }\n}\niplot(fig, filename='donut')\n\n# Game data\n\ngame_data = pd.read_csv('../input/game_data.csv')\ngame_data.head(2)\n\ncnt_srs = game_data['Turf'].value_counts()\n\ntrace = go.Bar(\n x=cnt_srs.index,\n y=cnt_srs.values,\n marker=dict(\n color=cnt_srs.values,\n ),\n)\n\nlayout = go.Layout(\n title='Turf type'\n)\n\ndata = [trace]\nfig = go.Figure(data=data, layout=layout)\noffline.iplot(fig, filename=\"Ratio\")\n\ncnt_srs = game_data['Start_Time'].value_counts()\n\ntrace = go.Bar(\n x=cnt_srs.index,\n y=cnt_srs.values,\n marker=dict(\n color=cnt_srs.values,\n ),\n)\n\nlayout = go.Layout(\n title='Start time'\n)\n\ndata = [trace]\nfig = go.Figure(data=data, layout=layout)\noffline.iplot(fig, filename=\"Ratio\")\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5aa060032273de0442ce76bb23140956b885f26e"},"cell_type":"code","source":"#import packages\nimport numpy as np\nimport pandas as pd\n\ndef convert_to_mph(dis_vector, converter):\n mph_vector = dis_vector * converter\n return mph_vector\n\ndef get_speed(ng_data, playId, gameKey, player, partner):\n ng_data = pd.read_csv(ng_data)\n ng_data['mph'] = convert_to_mph(ng_data['dis'], 20.455)\n player_data = ng_data.loc[(ng_data.GameKey == gameKey) & (ng_data.PlayID == playId) \n & (ng_data.GSISID == player)].sort_values('Time')\n partner_data = ng_data.loc[(ng_data.GameKey == gameKey) & (ng_data.PlayID == playId) \n & (ng_data.GSISID == partner)].sort_values('Time')\n player_grouped = player_data.groupby(['GameKey','PlayID','GSISID'], \n as_index = False)['mph'].agg({'max_mph': max,\n 'avg_mph': np.mean\n })\n player_grouped['involvement'] = 'player_injured'\n partner_grouped = partner_data.groupby(['GameKey','PlayID','GSISID'], \n as_index = False)['mph'].agg({'max_mph': max,\n 'avg_mph': np.mean\n })\n partner_grouped['involvement'] = 'primary_partner'\n return pd.concat([player_grouped, partner_grouped], axis = 0)[['involvement',\n 'max_mph',\n 'avg_mph']].reset_index(drop=True)\n\n#Run an example\nget_speed('../input/NGS-2016-pre.csv', 3129, 5, 31057, 32482)\n","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"### Please UPVOTE my kernel if you like it or wanna fork it.\n##### Hope you enjoyed this kernel!\n#### I am open to have your feedback for improving this kernel\n\n### Thanks for visiting my Kernel and please UPVOTE to stay connected and follow up the further updates!"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}