{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Business Analytics and Applications - Midterm Project\n**By Nam Jin Kang 2015310937**\n\nIn this notebook I will be taking a look at are \"🏀 2020 March Madness Data - First Look EDA\" and \"Redefining NCAA Basketball Positions using Data!\" by user 'Rob Mulla'. The first notebook a look at historical data accumulated throughout the years on everything from player by player statistics, team results. The second notebook uses unsupervised machine learning to cluster NCAA players into different player types. These notebook was a submission for a competition held by Google and the NCAA (National Collegiate Athletic Association).\n\nThe reason I've decided to combine the two notebooks is because they can work off of each other to provide us with very interesting analysis into the NCAA and it's players. \n\nI will be breaking down and paraphrasing the analysis he conducted by using what we learned in class, testing out his code and analysis, then expressing my thoughts on his method of analysis. \n\nI am in no way experienced with python and some of the more technical aspects are still out of my depth, but I will try my best to do an accurate, informative breakdown of this procedure and code.\n\n\n![](https://upload.wikimedia.org/wikipedia/en/thumb/2/28/March_Madness_logo.svg/440px-March_Madness_logo.svg.png) \n","metadata":{}},{"cell_type":"markdown","source":"## Importing Libraries \nFirst we start off by importing all the necessary libraries for the analysis. I will quickly breakdown these libraries and what each of them do. \n**Pandas** is a python library used for data manipulation and data analysis, and is especially useful for manipulating tables and managing time series. \n**Numpy** is a python library that adds a lot of functionality for managing arrays and matrices. It also enables the user to use very complex mathematical functions\n**Matplotlib** is used for visualizing data. It has many useful tools to make graphs and plots. \n**Seaborn** is also a library that focuses on data visualization, and is actually based on matplotlib. \n**os** and **gc**, which represents \"miscellaneous operating system interfaces\" and \"garbage collector\", is used for background tasks that is not relevant to the analysis itself. \n\n","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pylab as plt\nimport matplotlib as mpl\nfrom matplotlib.patches import Circle, Rectangle, Arc\nimport seaborn as sns\nplt.style.use('seaborn-dark-palette')\nmypal = plt.rcParams['axes.prop_cycle'].by_key()['color'] # Grab the color pal\nimport os\nimport gc\n\nMENS_DIR = '../input/google-cloud-ncaa-march-madness-2020-division-1-mens-tournament'\nWOMENS_DIR = '../input/google-cloud-ncaa-march-madness-2020-division-1-womens-tournament'","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Log Loss \nThis competition scores it's submissions using something called the **\"Log Loss Metric\".** \n\nThis is the equation that defines Log Loss, and is provided by Google for competition participants to use:\n\n![image.png](attachment:6d946129-6e98-47ee-bf32-eaad8b0e124a.png)\n\nwhere:\n\nn is the number of games played\n\n\\\\( \\hat{y}_i \\\\) is the predicted probability of team 1 beating team 2\n\n\\\\( y_i \\\\) is 1 if team 1 wins, 0 if team 2 wins\n\n\\\\( log() \\\\) is the natural (base e) logarithm\n\n-----------------------------------------------\n\nIn essence, this metric is used to punish overly confident and wrong predictions by more severly punishing your score based on how certain your prediction probability is. Meaning, assuming that you got the prediction wrong, you are more severly punished if you had predicted that there was a 0.8 chance vs. a 0.3 chance.\n","metadata":{},"attachments":{"6d946129-6e98-47ee-bf32-eaad8b0e124a.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAb4AAABFCAYAAAA1r4OHAAAUwUlEQVR4Ae2dB7AURRRFMWJCrVKrTJSWAYwoijljQkWMGL6imBVUDCjmiCJmUcSAWcyYwZxzTpgjZjHn7LNOl/1dx92dntnZNHtf1fzdv9sz3X3nbd/pF7o7mEQICAEhIASEQAsh0KGF+qquCgEhIASEgBAwEZ+UQAgIASEgBFoKARFfS91udVYICAEhIAREfNIBISAEhIAQaCkERHwtdbvVWSEgBISAEBDxSQeEgBAQAkKgpRAQ8bXU7VZnhYAQEAJCQMQnHRACQkAICIGWQkDE11K3W50VAkJACAgBEZ90IFMExo8fbwMGDLBevXrZIYccYsOGDbPevXu79998802mdeliQkAICIE0CIj40qCmc0oiMHbsWLv44ottjTXWsJEjR9rEiRPtwAMPtL59+9qkSZNKnqcvhIAQEAK1QkDEVyukW6QeyG3MmDHW1tZmL774ov344482aNAg69evn3311VctgoK6KQSEQCMjIOJr5LuTcds+//xzmzBhgt10001uBjZ8+HD77LPPMq3l999/t4EDB1r//v0d6b3zzjuO9EaMGGGffPJJpnXpYkJACAiBNAiI+NKg1qTn3H777TZ06FAbMmSIzTnnnHb44Yfbp59+mmlv8OMxuzv00EPt119/tQceeMD69Oljl156qV122WWZ1qWLCQEhIATSICDiS4Nak57DjO+tt96ye++912aeeWY76KCDMie+999/33bZZRcbN26c/fLLL3bffffZpptu6maBzDQlQkAICIF6IyDiq/cdqHH9mCJfeeUVm2WWWapCfDXujqoTAkJACCRGQMSXGLLmPkHE19z3T60XAkKgcgREfJVj2FRXEPE11e1SY4WAEKgCAiK+KoDayJeshPjwD6633no23XTT2fTTT2+zzz67LbXUUrb66qvbaqut9r/XVVZZxVZaaSVbeeWV3evSSy9t888/v/MvTjPNNDb11FO3H2uvvbY988wzjQyd2iYEhEBOEBDx5eRGhnajEuL7/vvv7eijj7Y55pjDJp98cuvUqZP7//7777fHH3/cHY899th/3vO/Px588EG788477frrr7dzzjnHBbz06NHDEWHnzp3t3HPPtb/++iu0KyonBISAEEiFgIgvFWzNe1IlxEevX3/9dZeczoxtyimntDXXXNOee+45+/PPPxOB8vPPP7u8vkcffdT22Wcf69atm2233XbK9UuEogoLASGQBgERXxrUmvicSokPgnvkkUdsoYUWso4dO7pZ3zbbbGPfffddYvIDRmZ45PtdffXVtuWWW9qNN97YxOiq6UJACDQDAiK+ZrhLGbWRZPXnn3/errvuOptxxhlt++23dzl9L730UqIavv32W7v22mtt7rnndrO+ueaay8466ywjTzCNQH4kvkN6rOtZTN544w3bYYcdbIMNNnALXxcr0+if0ceHHnrIjjvuuP81lVVtLrroIpf8D7ahwjVPOukkZzr++uuvQ0/LtFy5fmVaUY0u1giYFutqI+NcLf094YQTXB7woosuWgyS1J8FE98pp5xi66+/vs0333zOv0NDCEigYb/99lvqBoScWM+6Q9rXLGWuvPJK23jjja179+421VRTOeJaddVVbYsttkjUhT/++MMYZNl9gQAXzJ4LLrigI1FMmGkE8mP5NIihmEB8kCL+wXfffbdYkVSfnX/++W55NZLszzvvvFTXCDmJ3wh+0P3339+iDxo8kOAnPeqoo+zss8+2I444wu65556QyxozeNZE5bpc48svvww6L6tC5fqVVR21vk4STBtBf2qNT7S+aurvm2++aTfccIMjv2i9lfwfTHyvvfaaG9h23nlnN2gefPDBbjkqkqGT+neSNriedSdta6uUh/xeffVVp5CYPDt06OBMlXxWDX2A+AisefLJJzOF+O2333a7SBB5euKJJ2Z67cKLffDBB8YDHDtXRPF54YUX7IILLnDrqH7xxRd222232X777Vd4etn3PIRAlJiKiYytZYBQqX7VYl3YsqBU+GUopvXWn0bAudr6+/TTT9uuu+5a4R397+nBxMdpKAMzPJ7wL7/88qrP9AqbWs+6C9uh9/8iwAB7yy232LzzzmuTTTaZM3vuvffezmeX9eBbLeKjN3fddZdLxyhmgvy3t5W9w7xMEA8pIdWQDz/80EgfYbaIzzSpsKbqE088kfQ0ZzYv1q9arAubuLEJTwjFtJ76kwecuS3lsBbx1Yl0E/5eWqo4pjVmMaz9SZQnqQ4nn3yy25khSyCqSXzMlshFrBbxYQ7kgWCvvfaytKbgOCx5MMQUjEUGP25SwQzOIJpEyvWrFuvCJmlrmrKhmNZTf/KAM/emHNZNR3w4Y2+99VY3Td1zzz1duDozxY8//jixHiad8YXUjVmJQXunnXZyOWWbbLKJ266HYAEk7vvEncjhCczsuJ/c31lnndWZwbt27Wrk7JH3l5XUmvhC9Ce0b5jDdtxxR+e/y3om7NsAoZJWsu2229ro0aP9x8Gv7J6RNKI2rl+VRhAHN75KBUMxLUZ8tdSfZseZ21cO66YiPjpyxRVX2G677eY2JsXxTsj67rvvbqeddtp/IgBZ0Z8fK6v6U57tchhIl1lmGZffBTBJiC+0bto0YMAAl1SNrwKA9913Xzv++OPdTynu+yr93prusvj72GSWnD7M4KzqQhANZJWVRImPe3zmmWe6mVrPnj2dme6nn35y7bjqqqtcBGho3dGBK1R/uH6I7qJHpHywU0WhZNkH7gHRtkS+8htKKmmIr1S/fN31GpCzwjUU03rpT15wph/lsG4q4iMaB/MRRMIu3Ai+ByLXlltuOUc2fEb+1wEHHOCeVPEzYKYhZH2rrbZyIdrHHHOMOzcJ8YXWTfj4Rhtt5Hw82JiJKmTrHFYVQeK+d4X0xyHATAZT2bLLLuv8fSxHxka3KHQWEiU+AkAgPtIoll9+eRccgi5x7zH3EegRKtGBK1R/QnV37NixTp/JfyyULPtAwAwzbPreq1evwmqC3qchvlL98hXWi/iywjUU03rpT15wph/lsG544ps0aZI9/PDD7n5AHqQ8HHvsse1RbAyOhIwvtthibkZHQXboZj1HZl48sSOEZhNmT+g6NmwkjvjS1E1bqYd8NNaa5GmZ2YKvM+5717DIH9pBHwn19websvKeV/+e/5nZDhs2LHKF5v2XWR+Rl8z6CHYhxeGaa67JpENR4sNCQATjqFGjbIkllnAm9R9++MHuvvtul2YDtgjkxAMM6QqlJDpwZa27Y8aMcUTM0m2FEtqHwnNKvWfgoP9Ev6244oqlijmTEuHn7733nksLITWE95j3iSzlf38wm+WBsJSU6pcvXy/iS4JrOf0IxbRe+tNMOPu2lnoth3XDEx9hrQzoCOkODHzebOg7zJPlIoss4nbl5jOergklHzhwoPvhMitkBgghFfoc4ogvTd0QHDO8ddZZx1hAeYEFFrAVVlihfbCO+973qfCVzVcZLBio4w76jp+knFx44YUuzw6Myh0bbrihC56IXqt3796W5oheJ/R/Uk/wmZLiwOouRDNmIVHiI6jmo48+cmH/6BM+FWaXPHSgOzzAIOgTeXPMAkpJdODKWnfZeZ4ZaJT4Qvvg2x0ySONKQJdLCTp32GGHuUT5fv36tb9ClkSFFn6GX5LNiktJqX758kmJr1Jd9/UmwbWcfvjBOA7TeumP728z4Exb0+pvQxMf4BPWi3MdYSYD8THjK3To8zTGQLX11lu7ckSGMUtYa621nPMfcyczRcgPBfZSjvjS1s3ASWAGvhEGJVI1IBfID4n73ret8BXi4zwGmLiD0HZmvHkSBhJ2eJ9nnnkcCTF4ZCFR4uO6BHPgS2SA5n/0bPDgwc70yXf4epiFcvC+lEQHrqx1l4CuYsQX0ofCNocM0vj3MP2WEmaFPCSyRmrhMXToUDfrK/yMhHvKlpJS/fLlkw7I/rxKX0NxjdMPrgNecZjWS388To2Os29nWv1taOJjloM/jpkbgiOfLWsgMMgNQZEIGYfYWIkfAQzygAiB5zOe1FmyCfMLN9RLOeJLWzcmMMxaRG9CWMwa+L9Lly6u2rjvfdsKX/ETYjIin4rjyCOPdId/7z/nFcKHbPMkRHPi28UMxhNeVhIlPq7LLG7ddddt98liwmtra3M+Luom2f3UU0919+Gpp54q2RQ/cDH4I1nrLuZefNZRH19IH3yjQwZpfHwEiKXx8dHGcePG+eqCXsv1iwvUa0Cm7jjdoEycfjBehWBaT/1pBpxpYyX6W1fiw4TFDfYrt+CrYnbGQHfHHXc48wkmJgZzBF8XgSyYTm6++WZ79tlnXTnMYHvssYdb5d8DwmfMDBkY8Kvx1MkuADyp8xRfrboJjthss82cf4iZFz4jBj/yrZC4710h/WlHgPvG+p9Dhgxx946BIyspRnykyrCMHg9LPEChh8zY8REjPED5WQl6WkzwY5HSgp+QJ/sJEyZkqrvUye+GqE6iIKMS1wdfPm6QxswL2TP75feUVNIQX7l+8RCSxbqwSfvhy4fgGqcfIZjWW3+aAWfuSSX6G0d8I0aMcOZ9TPihErxyC85vv1bnFFNMYUsuuaQzTxLCTiTfTDPN5Mxbp59+envdmPr8eZhAeTrHf1Lo62A2iA2dxGfWAWWjUnxt5NTxY2Q25q/B91nWjT+IYADaxmCB2YwkYP9kHvd9e0db/A2zZQYASA9T48SJEzOL5vTQFiM+fHfoDnUyyKKLWBMgMgQS40fRv3//kqkVfI9pmwR8zLN9+/Z152alu1yMthM4xbqOUYnrgy8fN0jzRA0G6LK3pvhzQ17TEF+5fmW1LmxI24uVCcE1Tj9CMK23/jQDztyfSvQ3jviwFmLex9IUKsHEF3rBpOVQUAiVvdh4SsNEcckllzjiw1TKTFDSuAgwI4f0GNghmEp3US/0Bxf2Okp8lOOhiOAgnibx6WGNYId43wYsBhAjT4JpFk0orL/Y+1DdZQAlL3XQoEHOpO6vFdIHXzZukMYVQGAZD3C+//7ckFcGpvHjx4cUbS9Tql/tBer0JhTXOP2oFNO47leqP3HXr/b3oTjTjkr0N474cIuRG47FMVTqTnwjR4505kbMJphqsKkzoGGawkyKb0TSuAiQ+M/izszmWevR52ymaTHnosTFJEp8/BggujPOOMPpC/+TH4qfGf1BmLURRMXMHX9t1hKquwwQ1M+yZYX9C+mDb3PcIA2xk5KDC4IZeFIhdYjgtCRSql9JrlGNsqG4xulHpZjG9a1S/Ym7frW/D8WZdlSiv9RTbpFq3BxYEZMs3FB34uMHh3kG8xTbqzDA8XTA7I8ZRDVXzK+2YuT5+gx6pHtAKuTIMWgSHZtWID2iCPELF5Mo8WEZwMROYBQ/DHzL+NEIYvG+xZdfftnNgPDZok9ZSxLdZWZK3/A5gh0S0gff5nKDNA+MbOfEfWD266/vz63ma7F+VbO+kGuH4lpOP2qBaaX6E4JFNcuE4kwbKtHfOOIjnoRVwgioC5W6Ex/mEvKBmN350PTNN9/c+Vp4msd5K6kMAQiK6EbytPBnEWyEXZyBEvMA+/GV2gevWM0QC2He5OgRHITSebIpVr7cZwQQQHr4pzAH0s5iAvEV7sfHD4lFD/r06eMIDxMfAxlPf7WSJLpLWaJFIWq28kKS9KHUIE3kJFtBYeYked/PdmuJQbRftaq7VD1JcC12jVphWqn+FGt7LT9LgnNa/aWOuP34CKojBYmyoVJ34gttqMqlR4DZGETH4gGzzTabIwpMb8yw+IyEZ9IrQgVzNBGKkCb+WL/iTuj5vhwzE/woLHUGKeOcLpV2APHhR2zmHdghJfxoPm3C41DJK7NsFokgAAxzUj2kGv2qRz98nY2AqW9L4WvecKZvcViT7sUDOkFrWYqIL0s0G/RamIshP2YE0047rYuCxH/BEyczvW7durkgkJDmM9MjQZ1l5jDd4dcjipMAl3IHvi3MYqSmkBTNOo8ELrGqDCu8QMjM2jAxSYSAEBAC1URAxFdNdBvk2pAN6y9iRuzUqZMjHkyCkAwzNogvJJGe1BNmiUThzjDDDC6gBTMpASTlDgKUSE8h4pIAjB49erhFAvw2Rqzr2blzZ+f/ahDI1AwhIARyjICIL8c313cNPxqzMoKFMGuypBqC6YTAD7Z/wgRHuXLCEnKknbDtEGSV9CAHk81qix2sl4qfTyIEhIAQqDYCIr5qI9wg12flHHxykB9RUASjkIrAQgTMyHA+Ez0lEQJCQAjkHQERX97v8D/9I9SX2R4r67AmKf49cieZvZGSgK+PpeQkQkAICIG8IyDiy/sd/qd/5Llh0iTIhShMkpxJR2CpLiI82dCVbWEkQkAICIG8IyDiy/sd/qd/LJfVs2dPZ94kSIWDfLLu3bu7gBOCW5JukURwDKRJuHESIbCGnBuWyWJ9zaz27EvSBpUVAkKgdREQ8bXuva+45xAY6w2ygkMSISWCLXDYoqpr167tm8YmuYbKCgEhIATSIiDiS4tci5+Hj5CEaQ7eJxFmm/gZSYJfeOGF3covSc5XWSEgBIRAJQiI+CpBr4XPjdtfKwQaIkkXX3xxEV8IWCojBIRAZgiI+DKDsrUuFN1fi1Vb2AiWdfWKHawPyibDhSLiK0RD74WAEKgVAiK+WiGds3qi+2uRG8jqMATMFDtYazMaPCPiy5lSqDtCoEkQEPE1yY1qtGZG99cKmfH5ne19X0R8Hgm9CgEhUEsERHy1RDtHdZGOkHaTV7Z9IbiFxa67dOlio0aNal9GLUcQqStCQAg0KAIivga9MY3eLGZr7KaQZpNXtiIZPXq0tbW1uV0ZyC8cPHhwo3dZ7RMCQiAnCIj4cnIj1Q0hIASEgBAIQ0DEF4aTSgkBISAEhEBOEBDx5eRGqhtCQAgIASEQhoCILwwnlRICQkAICIGcICDiy8mNVDeEgBAQAkIgDAERXxhOKiUEhIAQEAI5QUDEl5MbqW4IASEgBIRAGAIivjCcVEoICAEhIARygoCILyc3Ut0QAkJACAiBMAREfGE4qZQQEAJCQAjkBAERX05upLohBISAEBACYQiI+MJwUikhIASEgBDICQJ/A2jeM0+kYnOXAAAAAElFTkSuQmCC"}}},{"cell_type":"code","source":"def logloss(true_label, predicted, eps=1e-15):\n    p = np.clip(predicted, eps, 1 - eps)\n    if true_label == 1:\n        return -np.log(p)\n    return -np.log(1 - p)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'Confident Wrong Prediction: \\t\\t {logloss(1, 0.01):0.4f}')\nprint(f'Confident Correct Prediction: \\t\\t {logloss(0, 0.01):0.4f}')\nprint(f'Non-Confident Wrong Prediction: \\t {logloss(1, 0.49):0.4f}')\nprint(f'Non-Confident Correct Prediction: \\t {logloss(0, 0.49):0.4f}')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Sample Submission\nThese are the samples that will be used by this notebook. \n\nAs you can see, there are 2 different samples, 1 for the mens league and another for the womens league.\n","metadata":{}},{"cell_type":"code","source":"Mss = pd.read_csv(f'{MENS_DIR}/MSampleSubmissionStage1_2020.csv')\nWss = pd.read_csv(f'{WOMENS_DIR}/WSampleSubmissionStage1_2020.csv')\nMss.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Team Data\nFrom the data sets, we can see the unique team ID, team name, 1st year they  participated in Division 1 League(D1), and last year they participated in D1 League. \n\nBecause there are 2 seperate datasets for male and female leagues, there are 2 tables, and as you can see, the newest team to start playing in Division 1 is Merrimack (Team ID: 1467). However the female league data does not contain information on when they first and last played in the D1 League.","metadata":{}},{"cell_type":"code","source":"MTeams = pd.read_csv(f'{MENS_DIR}/MDataFiles_Stage1/MTeams.csv')\nMTeams.sort_values('FirstD1Season', ascending=False).head(5)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"WTeams = pd.read_csv(f'{WOMENS_DIR}/WDataFiles_Stage1/WTeams.csv')\nWTeams.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Seasons Data\n### MSeasons.csv & WSeasons.csv\n\nThe M and F Seasons.csv includes historical data about the properties of a season. Meaning it has details on when the season started (year and date), and contains **nominal data** about which region is designated which alphabet (W, X, Y, Z). I believe it would have been less confusing down the road and a better form of data preparation if the regions were designated N, W, S, E labels instead. Later in this notebook, they use these labels in distinguishing different teams seeds. So for example, East region's number 1 seed team being W1. I feel it would have been less confusing if it was labeled E1 instead(E for east). \n\nI realize that each season has different values for W, X, Y and Z, but I believe that the modification of this data so that 1 alphabet was designated for 1 region should have been conducted as a part of the data preparation.","metadata":{}},{"cell_type":"code","source":"MSeasons = pd.read_csv(f'{MENS_DIR}/MDataFiles_Stage1/MSeasons.csv')\nWSeasons = pd.read_csv(f'{WOMENS_DIR}/WDataFiles_Stage1/WSeasons.csv')\nMSeasons.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Tourney Seed Data\n### MNCAATourneySeeds.csv & WNCAATourneySeeds.csv\nThe following 2 files contains historical data about all teams and their seed placement per each season they participated. \n\nFor your reference, a seed is basically the placement the team is in, in their respective regions, up until the start of playoffs. The seeds determines where in the playoff bracket the team will be playing in, with higher ranking seeds being placed in more favorable places in the playoff bracket. ","metadata":{}},{"cell_type":"code","source":"MNCAATourneySeeds = pd.read_csv(f'{MENS_DIR}/MDataFiles_Stage1/MNCAATourneySeeds.csv')\nWNCAATourneySeeds = pd.read_csv(f'{WOMENS_DIR}/WDataFiles_Stage1/WNCAATourneySeeds.csv')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This code is used to join the team data with the seed data in order to identify which teams competed with each other (according to their seeds) ","metadata":{}},{"cell_type":"code","source":"WNCAATourneySeeds.merge(WTeams, validate='many_to_one').head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Regular Season Results\n### MRegularSeasonCompactResults.csv & WRegularSeasonCompactResults.csv\n\nThe following 2 files contains historical data on every game result for all seasons.","metadata":{}},{"cell_type":"code","source":"MRegularSeasonCompactResults = pd.read_csv(f'{MENS_DIR}/MDataFiles_Stage1/MRegularSeasonCompactResults.csv')\nWRegularSeasonCompactResults = pd.read_csv(f'{WOMENS_DIR}/WDataFiles_Stage1/WRegularSeasonCompactResults.csv')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The code below shows  the winning team, losing team, and the score ","metadata":{}},{"cell_type":"code","source":"MRegularSeasonCompactResults.head(5)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The code below is used to join regular season results on the team names in order to identify the games more easily. ","metadata":{}},{"cell_type":"code","source":"# Add the winning and losing team names to the results\nMRegularSeasonCompactResults = \\\n    MRegularSeasonCompactResults \\\n    .merge(MTeams[['TeamName', 'TeamID']],\n           left_on='WTeamID',\n           right_on='TeamID',\n           validate='many_to_one') \\\n    .drop('TeamID', axis=1) \\\n    .rename(columns={'TeamName': 'WTeamName'}) \\\n    .merge(MTeams[['TeamName', 'TeamID']],\n           left_on='LTeamID',\n           right_on='TeamID') \\\n    .drop('TeamID', axis=1) \\\n    .rename(columns={'TeamName': 'LTeamName'})\n\nWRegularSeasonCompactResults = \\\n    WRegularSeasonCompactResults \\\n    .merge(WTeams[['TeamName', 'TeamID']],\n           left_on='WTeamID',\n           right_on='TeamID',\n           validate='many_to_one') \\\n    .drop('TeamID', axis=1) \\\n    .rename(columns={'TeamName': 'WTeamName'}) \\\n    .merge(WTeams[['TeamName', 'TeamID']],\n           left_on='LTeamID',\n           right_on='TeamID') \\\n    .drop('TeamID', axis=1) \\\n    .rename(columns={'TeamName': 'LTeamName'})","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"WRegularSeasonCompactResults.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the following code, the score differential is calculated for all games. \nThe score differential is the difference between the winning team's score and the losing team's score. ","metadata":{}},{"cell_type":"code","source":"WRegularSeasonCompactResults['Score_Diff'] = WRegularSeasonCompactResults['WScore'] - WRegularSeasonCompactResults['LScore']\nMRegularSeasonCompactResults['Score_Diff'] = MRegularSeasonCompactResults['WScore'] - MRegularSeasonCompactResults['LScore']","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The calculated score differential is plotted in a graph using matplotlib to show the spread of games in terms of score differential.","metadata":{}},{"cell_type":"code","source":"plt.style.use('fivethirtyeight')\nMRegularSeasonCompactResults['Score_Diff'] \\\n    .plot(kind='hist',\n          bins=90,\n          figsize=(15, 5),\n          label='Mens',\n          alpha=0.5)\nWRegularSeasonCompactResults['Score_Diff'] \\\n    .plot(kind='hist',\n          bins=105,\n          figsize=(15, 5),\n          label='Womens',\n          alpha=0.5)\nplt.title('Score Differential')\nplt.xlim(0,60)\nplt.legend()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Matplotlib is used once again to show the teams with the most season wins and the least wins for both men and women.\n\nIt's interesting to note that there are certain, more dominant teams in the womens league, while the mens league has a more continuous downward slope as the number of wins decreases. It is also interesting that this trend also applies to the teams with the least wins.","metadata":{}},{"cell_type":"code","source":"plt.style.use('fivethirtyeight')\nfig, axs = plt.subplots(1, 2, figsize=(15, 5))\nWRegularSeasonCompactResults['counter'] = 1\nWRegularSeasonCompactResults.groupby('WTeamName')['counter'] \\\n    .count() \\\n    .sort_values() \\\n    .tail(20) \\\n    .plot(kind='barh',\n          title='⬆️ Most Regular Season Wins (Womens)',\n          figsize=(15, 8),\n          xlim=(400, 680),\n          color=mypal[0],\n          ax=axs[0])\nWRegularSeasonCompactResults.groupby('WTeamName')['counter'] \\\n    .count() \\\n    .sort_values(ascending=False) \\\n    .tail(20) \\\n    .plot(kind='barh',\n          title='⬇️ Least Regular Season Wins (Womens)',\n          figsize=(15, 8),\n          xlim=(0, 150),\n          color=mypal[1],\n          ax=axs[1])\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.style.use('fivethirtyeight')\nfig, axs = plt.subplots(1, 2, figsize=(15, 5))\nMRegularSeasonCompactResults['counter'] = 1\nMRegularSeasonCompactResults.groupby('WTeamName')['counter'] \\\n    .count() \\\n    .sort_values() \\\n    .tail(20) \\\n    .plot(kind='barh',\n          title='⬆️ Most Regular Season Wins (Mens)',\n          figsize=(15, 8),\n          xlim=(600, 920),\n          color=mypal[2],\n         ax=axs[0])\nMRegularSeasonCompactResults.groupby('WTeamName')['counter'] \\\n    .count() \\\n    .sort_values(ascending=False) \\\n    .tail(20) \\\n    .plot(kind='barh',\n          title='⬇️ Least Regular Season Wins (Mens)',\n          figsize=(15, 8),\n          xlim=(0, 150),\n          color=mypal[3],\n          ax=axs[1])\naxs[1].set_ylabel('')\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Event Data\n\nFrom here, we now turn our attention towards **events**. \"Events\" refers to all of the events that occurs during a basketball game. Anything from baskets made to rebounds, assists, steals and turnovers are all considered events. It is also interesting to note that the data also makes distinctions on how the basket was made. Meaning free throws, 2 pointers and 3 pointers are all logged differently. \n\nBelow are the Male and Female event files that list play-by-play events that happened in the 99.5% of games throughout the season. \n\nThe event is logged with uniquely every time that a unique team or player triggers an event (for example, by scoring). The event is logged with team ID, player ID, along with time, final score, current score, court area, and many other game/event related information. It should be noted that each event is logged seperately, even if they came from a single situation. So for example, if there was a basket made and a player had an assist, they will be logged as 2 seperate events. One for the basket made and another for the assist. \n\nThe event files are seperated per year and by gender. \n\nMens Event Files:\n- MEvents2015.csv, MEvents2016.csv, MEvent2017.csv, MEvents2018.csv, MEvents2019.csv\n\nWomens Event Files:\n- WEvents2015.csv, WEvents2016.csv, WEvents2017.csv, WEvents2018.csv, WEvents2019.csv\n\nAll of the data are put into 2 massive dataframes, seperated by gender. ","metadata":{}},{"cell_type":"code","source":"mens_events = []\nfor year in [2015, 2016, 2017, 2018, 2019]:\n    mens_events.append(pd.read_csv(f'{MENS_DIR}/MEvents{year}.csv'))\nMEvents = pd.concat(mens_events)\nprint(MEvents.shape)\nMEvents.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"womens_events = []\nfor year in [2015, 2016, 2017, 2018, 2019]:\n    womens_events.append(pd.read_csv(f'{WOMENS_DIR}/WEvents{year}.csv'))\nWEvents = pd.concat(womens_events)\nprint(WEvents.shape)\nWEvents.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del mens_events\ndel womens_events\ngc.collect()","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Common Event Types\nBelow we can see a breakdown of events and their frequencies per gender league. Note how there are multiple 'made' and 'miss' event types. As mentioned before, the type of shot (made 1 = freethrows, made 2 = 2 pointers, made 3 = 3 pointers) are logged seperately.  \n\n","metadata":{}},{"cell_type":"code","source":"# Event Types\nplt.style.use('fivethirtyeight')\nMEvents['counter'] = 1\nMEvents.groupby('EventType')['counter'] \\\n    .sum() \\\n    .sort_values(ascending=False) \\\n    .plot(kind='bar',\n          figsize=(15, 5),\n         color=mypal[2],\n         title='Event Type Frequency (Mens)')\nplt.xticks(rotation=0)\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Event Types\nplt.style.use('fivethirtyeight')\nWEvents['counter'] = 1\nWEvents.groupby('EventType')['counter'] \\\n    .sum() \\\n    .sort_values(ascending=False) \\\n    .plot(kind='bar',\n          figsize=(15, 5),\n         color=mypal[3],\n         title='Event Type Frequency (Womens)')\nplt.xticks(rotation=0)\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Area of Event\nThere are 13 predetermined \"areas\" on a basketball court. This data can be used to map out where each of these events occured. The areas are assigned a number. I will refer to them as \"area numbers\" from now on. \n\n**These are the 13 areas on the court**: \n\n1=under basket\n\n2=in the paint\n\n3=inside right wing\n\n4=inside right\n\n5=inside center\n\n6=inside left\n\n7=inside left wing\n\n8=outside right wing\n\n9=outside right\n\n10=outside center\n\n11=outside left\n\n12=outside left wing\n\n13=backcourt.\n\nNotice how the first bit of code is used to map the number to their corresponding names so that it is easier to analyze and visualilze the data later on. \n","metadata":{}},{"cell_type":"code","source":"area_mapping = {0: np.nan,\n                1: 'under basket',\n                2: 'in the paint',\n                3: 'inside right wing',\n                4: 'inside right',\n                5: 'inside center',\n                6: 'inside left',\n                7: 'inside left wing',\n                8: 'outside right wing',\n                9: 'outside right',\n                10: 'outside center',\n                11: 'outside left',\n                12: 'outside left wing',\n                13: 'backcourt'}\n\nMEvents['Area_Name'] = MEvents['Area'].map(area_mapping)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"MEvents.groupby('Area_Name')['counter'].sum() \\\n    .sort_values() \\\n    .plot(kind='barh',\n          figsize=(15, 8),\n          title='Frequency of Event Area')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Below, the event data is plotted and visualized on the court. Each of the \"areas\" are color coded and the each event fills up 1 dot in it's respective area.\n\nSo for example: area number 13, which is the \"backcourt\" can be assigned the color blue. Then it creates a dot in the \"backcourt area\" and is colored blue. ","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(15, 8))\nfor i, d in MEvents.loc[~MEvents['Area_Name'].isna()].groupby('Area_Name'):\n    d.plot(x='X', y='Y', style='.', label=i, ax=ax, title='Visualizing Event Areas')\n    ax.legend()\nplt.legend(bbox_to_anchor=(1.04,1), loc=\"upper left\")\nax.set_xticks([])\nax.set_yticks([])\nax.set_xlabel('')\nax.set_xlim(0, 100)\nax.set_ylim(0, 100)\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Plotting X, Y Data\nThe data also includes the X and Y coordinates for where a certain event took place. \n\nI think this is some of the most interesting data points in this entire notebook, and although I don't know how exactly they were able to get X and Y coordiantes for such a large data set, it made me have a lot of respect for the datascientists that were able to gather this data. \n\nHowever, the X and Y data is: \n1. Limited to the following 4 types of events: Field-goal Attempts (2 point or 3 pointers), fouls, and turnovers. \n2. Missing some X and Y coordinates for some of the games, meaning this is not a complete sample. \n","metadata":{}},{"cell_type":"code","source":"# Normalize X, Y positions for court dimentions\n# Court is 50 feet wide and 94 feet end to end.\nMEvents['X_'] = (MEvents['X'] * (94/100))\nMEvents['Y_'] = (MEvents['Y'] * (50/100))\n\nWEvents['X_'] = (WEvents['X'] * (94/100))\nWEvents['Y_'] = (WEvents['Y'] * (50/100))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# NCAA Court Plot Function\nThis section of the code is used to essentially draw the court so that the X and Y data can be placed on it, for a better visual representation and display of the data. \n\nAlthough the code is very long, most of it is generating shapes using coordinates to make an accurately proportioned court. At the very end, you can see the resulting court diagram, which will be used to plot the events onto the court. ","metadata":{}},{"cell_type":"code","source":"def create_ncaa_full_court(ax=None, three_line='mens', court_color='#dfbb85',\n                           lw=3, lines_color='black', lines_alpha=0.5,\n                           paint_fill='blue', paint_alpha=0.4,\n                           inner_arc=False):\n    \"\"\"\n    Version 2020.2.19\n    Creates NCAA Basketball Court\n    Dimensions are in feet (Court is 97x50 ft)\n    Created by: Rob Mulla / https://github.com/RobMulla\n\n    * Note that this function uses \"feet\" as the unit of measure.\n    * NCAA Data is provided on a x range: 0, 100 and y-range 0 to 100\n    * To plot X/Y positions first convert to feet like this:\n    ```\n    Events['X_'] = (Events['X'] * (94/100))\n    Events['Y_'] = (Events['Y'] * (50/100))\n    ```\n    \n    ax: matplotlib axes if None gets current axes using `plt.gca`\n\n\n    three_line: 'mens', 'womens' or 'both' defines 3 point line plotted\n    court_color : (hex) Color of the court\n    lw : line width\n    lines_color : Color of the lines\n    lines_alpha : transparency of lines\n    paint_fill : Color inside the paint\n    paint_alpha : transparency of the \"paint\"\n    inner_arc : paint the dotted inner arc\n    \"\"\"\n    if ax is None:\n        ax = plt.gca()\n\n    # Create Pathes for Court Lines\n    center_circle = Circle((94/2, 50/2), 6,\n                           linewidth=lw, color=lines_color, lw=lw,\n                           fill=False, alpha=lines_alpha)\n    hoop_left = Circle((5.25, 50/2), 1.5 / 2,\n                       linewidth=lw, color=lines_color, lw=lw,\n                       fill=False, alpha=lines_alpha)\n    hoop_right = Circle((94-5.25, 50/2), 1.5 / 2,\n                        linewidth=lw, color=lines_color, lw=lw,\n                        fill=False, alpha=lines_alpha)\n\n    # Paint - 18 Feet 10 inches which converts to 18.833333 feet - gross!\n    left_paint = Rectangle((0, (50/2)-6), 18.833333, 12,\n                           fill=paint_fill, alpha=paint_alpha,\n                           lw=lw, edgecolor=None)\n    right_paint = Rectangle((94-18.83333, (50/2)-6), 18.833333,\n                            12, fill=paint_fill, alpha=paint_alpha,\n                            lw=lw, edgecolor=None)\n    \n    left_paint_boarder = Rectangle((0, (50/2)-6), 18.833333, 12,\n                           fill=False, alpha=lines_alpha,\n                           lw=lw, edgecolor=lines_color)\n    right_paint_boarder = Rectangle((94-18.83333, (50/2)-6), 18.833333,\n                            12, fill=False, alpha=lines_alpha,\n                            lw=lw, edgecolor=lines_color)\n\n    left_arc = Arc((18.833333, 50/2), 12, 12, theta1=-\n                   90, theta2=90, color=lines_color, lw=lw,\n                   alpha=lines_alpha)\n    right_arc = Arc((94-18.833333, 50/2), 12, 12, theta1=90,\n                    theta2=-90, color=lines_color, lw=lw,\n                    alpha=lines_alpha)\n    \n    leftblock1 = Rectangle((7, (50/2)-6-0.666), 1, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    leftblock2 = Rectangle((7, (50/2)+6), 1, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    ax.add_patch(leftblock1)\n    ax.add_patch(leftblock2)\n    \n    left_l1 = Rectangle((11, (50/2)-6-0.666), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    left_l2 = Rectangle((14, (50/2)-6-0.666), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    left_l3 = Rectangle((17, (50/2)-6-0.666), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    ax.add_patch(left_l1)\n    ax.add_patch(left_l2)\n    ax.add_patch(left_l3)\n    left_l4 = Rectangle((11, (50/2)+6), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    left_l5 = Rectangle((14, (50/2)+6), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    left_l6 = Rectangle((17, (50/2)+6), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    ax.add_patch(left_l4)\n    ax.add_patch(left_l5)\n    ax.add_patch(left_l6)\n    \n    rightblock1 = Rectangle((94-7-1, (50/2)-6-0.666), 1, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    rightblock2 = Rectangle((94-7-1, (50/2)+6), 1, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    ax.add_patch(rightblock1)\n    ax.add_patch(rightblock2)\n\n    right_l1 = Rectangle((94-11, (50/2)-6-0.666), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    right_l2 = Rectangle((94-14, (50/2)-6-0.666), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    right_l3 = Rectangle((94-17, (50/2)-6-0.666), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    ax.add_patch(right_l1)\n    ax.add_patch(right_l2)\n    ax.add_patch(right_l3)\n    right_l4 = Rectangle((94-11, (50/2)+6), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    right_l5 = Rectangle((94-14, (50/2)+6), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    right_l6 = Rectangle((94-17, (50/2)+6), 0.166, 0.666,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    ax.add_patch(right_l4)\n    ax.add_patch(right_l5)\n    ax.add_patch(right_l6)\n    \n    # 3 Point Line\n    if (three_line == 'mens') | (three_line == 'both'):\n        # 22' 1.75\" distance to center of hoop\n        three_pt_left = Arc((6.25, 50/2), 44.291, 44.291, theta1=-78,\n                            theta2=78, color=lines_color, lw=lw,\n                            alpha=lines_alpha)\n        three_pt_right = Arc((94-6.25, 50/2), 44.291, 44.291,\n                             theta1=180-78, theta2=180+78,\n                             color=lines_color, lw=lw, alpha=lines_alpha)\n\n        # 4.25 feet max to sideline for mens\n        ax.plot((0, 11.25), (3.34, 3.34),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n        ax.plot((0, 11.25), (50-3.34, 50-3.34),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n        ax.plot((94-11.25, 94), (3.34, 3.34),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n        ax.plot((94-11.25, 94), (50-3.34, 50-3.34),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n        ax.add_patch(three_pt_left)\n        ax.add_patch(three_pt_right)\n\n    if (three_line == 'womens') | (three_line == 'both'):\n        # womens 3\n        three_pt_left_w = Arc((6.25, 50/2), 20.75 * 2, 20.75 * 2, theta1=-85,\n                              theta2=85, color=lines_color, lw=lw, alpha=lines_alpha)\n        three_pt_right_w = Arc((94-6.25, 50/2), 20.75 * 2, 20.75 * 2,\n                               theta1=180-85, theta2=180+85,\n                               color=lines_color, lw=lw, alpha=lines_alpha)\n\n        # 4.25 inches max to sideline for mens\n        ax.plot((0, 8.3), (4.25, 4.25), color=lines_color,\n                lw=lw, alpha=lines_alpha)\n        ax.plot((0, 8.3), (50-4.25, 50-4.25),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n        ax.plot((94-8.3, 94), (4.25, 4.25),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n        ax.plot((94-8.3, 94), (50-4.25, 50-4.25),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n\n        ax.add_patch(three_pt_left_w)\n        ax.add_patch(three_pt_right_w)\n\n    # Add Patches\n    ax.add_patch(left_paint)\n    ax.add_patch(left_paint_boarder)\n    ax.add_patch(right_paint)\n    ax.add_patch(right_paint_boarder)\n    ax.add_patch(center_circle)\n    ax.add_patch(hoop_left)\n    ax.add_patch(hoop_right)\n    ax.add_patch(left_arc)\n    ax.add_patch(right_arc)\n    \n    if inner_arc:\n        left_inner_arc = Arc((18.833333, 50/2), 12, 12, theta1=90,\n                             theta2=-90, color=lines_color, lw=lw,\n                       alpha=lines_alpha, ls='--')\n        right_inner_arc = Arc((94-18.833333, 50/2), 12, 12, theta1=-90,\n                        theta2=90, color=lines_color, lw=lw,\n                        alpha=lines_alpha, ls='--')\n        ax.add_patch(left_inner_arc)\n        ax.add_patch(right_inner_arc)\n\n    # Restricted Area Marker\n    restricted_left = Arc((6.25, 50/2), 8, 8, theta1=-90,\n                        theta2=90, color=lines_color, lw=lw,\n                        alpha=lines_alpha)\n    restricted_right = Arc((94-6.25, 50/2), 8, 8,\n                         theta1=180-90, theta2=180+90,\n                         color=lines_color, lw=lw, alpha=lines_alpha)\n    ax.add_patch(restricted_left)\n    ax.add_patch(restricted_right)\n    \n    # Backboards\n    ax.plot((4, 4), ((50/2) - 3, (50/2) + 3),\n            color=lines_color, lw=lw*1.5, alpha=lines_alpha)\n    ax.plot((94-4, 94-4), ((50/2) - 3, (50/2) + 3),\n            color=lines_color, lw=lw*1.5, alpha=lines_alpha)\n    ax.plot((4, 4.6), (50/2, 50/2), color=lines_color,\n            lw=lw, alpha=lines_alpha)\n    ax.plot((94-4, 94-4.6), (50/2, 50/2),\n            color=lines_color, lw=lw, alpha=lines_alpha)\n\n    # Half Court Line\n    ax.axvline(94/2, color=lines_color, lw=lw, alpha=lines_alpha)\n\n    # Boarder\n    boarder = Rectangle((0.3,0.3), 94-0.4, 50-0.4, fill=False, lw=3, color='black', alpha=lines_alpha)\n    ax.add_patch(boarder)\n    \n    # Plot Limit\n    ax.set_xlim(0, 94)\n    ax.set_ylim(0, 50)\n    ax.set_facecolor(court_color)\n    ax.set_xticks([])\n    ax.set_yticks([])\n    ax.set_xlabel('')\n    return ax\n\n\nfig, ax = plt.subplots(figsize=(15, 8.5))\ncreate_ncaa_full_court(ax, three_line='both', paint_alpha=0.4)\nplt.show()\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"With the accurately proportioned drawing of the court, along with normalized X and Y coordinates for the event data, we can start plotting them on the court. \n\nThe example below is used to display all of the turnover locations from all games in the mens season. ","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(15, 7.8))\nms = 10\nax = create_ncaa_full_court(ax, paint_alpha=0.1)\nMEvents.query('EventType == \"turnover\"') \\\n    .plot(x='X_', y='Y_', style='X',\n          title='Turnover Locations (Mens)',\n          c='red',\n          alpha=0.3,\n         figsize=(15, 9),\n         label='Steals',\n         ms=ms,\n         ax=ax)\nax.set_xlabel('')\nax.get_legend().remove()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And here we plot down all of the 2 and 3 pointers made and missed for the womens season. Notice how with the 3 pointers, there are definitely spots like the top of the key (middle area) where there's a higher concentration of made shots. There are similar concentrations on the two ends, as well as the 45 degree angle between each end and the middle. Similarly, with the 2 pointers made, there's most definitely an almost exponential decrease between the relationship between distance and shots made. ","metadata":{}},{"cell_type":"code","source":"COURT_COLOR = '#dfbb85'\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 4))\n# Where are 3 pointers made from? (This is really cool)\nWEvents.query('EventType == \"made3\"') \\\n    .plot(x='X_', y='Y_', style='.',\n          color='blue',\n          title='3 Pointers Made (Womens)',\n          alpha=0.01, ax=ax1)\nax1 = create_ncaa_full_court(ax1, lw=0.5, three_line='womens', paint_alpha=0.1)\nax1.set_facecolor(COURT_COLOR)\nWEvents.query('EventType == \"miss3\"') \\\n    .plot(x='X_', y='Y_', style='.',\n          title='3 Pointers Missed (Womens)',\n          color='red',\n          alpha=0.01, ax=ax2)\nax2.set_facecolor(COURT_COLOR)\nax2 = create_ncaa_full_court(ax2, lw=0.5, three_line='womens', paint_alpha=0.1)\nax1.get_legend().remove()\nax2.get_legend().remove()\nax1.set_xticks([])\nax1.set_yticks([])\nax2.set_xticks([])\nax2.set_yticks([])\nax1.set_xlabel('')\nax2.set_xlabel('')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"COURT_COLOR = '#dfbb85'\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 4))\n# Where are 3 pointers made from? (This is really cool)\nWEvents.query('EventType == \"made2\"') \\\n    .plot(x='X_', y='Y_', style='.',\n          color='blue',\n          title='2 Pointers Made (Womens)',\n          alpha=0.01, ax=ax1)\nax1.set_facecolor(COURT_COLOR)\nax1 = create_ncaa_full_court(ax1, lw=0.5, three_line='womens', paint_alpha=0.1)\nWEvents.query('EventType == \"miss2\"') \\\n    .plot(x='X_', y='Y_', style='.',\n          title='2 Pointers Missed (Womens)',\n          color='red',\n          alpha=0.01, ax=ax2)\nax2.set_facecolor(COURT_COLOR)\nax2 = create_ncaa_full_court(ax2, lw=0.5, three_line='womens', paint_alpha=0.1)\nax1.get_legend().remove()\nax2.get_legend().remove()\nax1.set_xticks([])\nax1.set_yticks([])\nax2.set_xticks([])\nax2.set_yticks([])\nax1.set_xlabel('')\nax2.set_xlabel('')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## PlayerIDs\nApparently, there is an issue that occurs with the notebook when reading a line where the player name has a comma in it. It is solved by using 'error_bad_lines=false', however, I believe this data should have been cleaned when we first imported and organized the player data.","metadata":{}},{"cell_type":"code","source":"MPlayers = pd.read_csv(f'{MENS_DIR}/MPlayers.csv', error_bad_lines=False)\nWPlayers = pd.read_csv(f'{WOMENS_DIR}/WPlayers.csv')","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"MPlayers.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Merge Player name onto events\nMEvents = MEvents.merge(MPlayers,\n              how='left',\n              left_on='EventPlayerID',\n              right_on='PlayerID')\n\nWEvents = WEvents.merge(WPlayers,\n              how='left',\n              left_on='EventPlayerID',\n              right_on='PlayerID')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Common Events by Player","metadata":{}},{"cell_type":"markdown","source":"With what we've done so far, we can now look at specific players and their statistics. For example, if we want to graph out player number 2825's event types and their frequencies we can do so like below.\n\nIt looks like player number 2825 is called Zion Williamson. \n\nWe can use this data to do further analysis by comparing it with the entire season and all of it's players average to see what the potentia strengths of this player might be. As an example, I added some of my own code to see if I could compare and make my own observations based on the data. ","metadata":{}},{"cell_type":"code","source":"MEvents.loc[MEvents['PlayerID'] == 2825].groupby('EventType')['EventID'].count() \\\n    .sort_values() \\\n    .plot(kind='barh',\n          figsize=(15, 5),\n          title='Zion Williamson event type count',\n          color=mypal[1])\nplt.show()\n\n##The code below is one I wrote myself to see if I could compare Zion's stats to the average of the entire player data set. \n\nMEvents.groupby('EventType')['counter'] \\\n    .sum() \\\n    .sort_values(ascending=True) \\\n    .plot(kind='barh',\n          figsize=(15, 5),\n         title='Event Type Frequency (Mens)')\nplt.xticks(rotation=0)\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### From this comparison I have come up with a couple observations. \n\nWith this comparison and data, we can make observations about this player's play style and possition. \n\nI do not know who Zion Williamson is but just by looking at this data, I could infer that this player is most likely a power foward or center (taller players that play more in the back court), because he has such a high rebound, made2, and fouled rate while having such a small made3 ratio compared to the player average. This doesn't necessarily tell us how good this player is, but it can tell us what type of play style this player has because we are able to compare it with the average player's ratios.","metadata":{}},{"cell_type":"markdown","source":"# Plotting Specific Players' Made/Missed Shots\nNow lets take a look at Zion Williamson's shots made and missed and plot it onto the court diagram we previously generated. ","metadata":{"trusted":true}},{"cell_type":"code","source":"ms = 10 # Marker Size\nFirstName = 'Zion'\nLastName = 'Williamson'\nfig, ax = plt.subplots(figsize=(15, 8))\nax = create_ncaa_full_court(ax)\nMEvents.query('FirstName == @FirstName and LastName == @LastName and EventType == \"made2\"') \\\n    .plot(x='X_', y='Y_', style='o',\n          title='Shots (Zion Williamson)',\n          alpha=0.5,\n         figsize=(15, 8),\n         label='Made 2',\n         ms=ms,\n         ax=ax)\nplt.legend()\nMEvents.query('FirstName == @FirstName and LastName == @LastName and EventType == \"miss2\"') \\\n    .plot(x='X_', y='Y_', style='X',\n          alpha=0.5, ax=ax,\n         label='Missed 2',\n         ms=ms)\nplt.legend()\nMEvents.query('FirstName == @FirstName and LastName == @LastName and EventType == \"made3\"') \\\n    .plot(x='X_', y='Y_', style='o',\n          c='brown',\n          alpha=0.5,\n         figsize=(15, 8),\n         label='Made 3', ax=ax,\n         ms=ms)\nplt.legend()\nMEvents.query('FirstName == @FirstName and LastName == @LastName and EventType == \"miss3\"') \\\n    .plot(x='X_', y='Y_', style='X',\n          c='green',\n          alpha=0.5, ax=ax,\n         label='Missed 3',\n         ms=ms)\nax.set_xlabel('')\nplt.legend()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Notice once again how most shot attempts are made under the basket, which helps validate my deduction about this player's position as a power forward in the team. ","metadata":{}},{"cell_type":"markdown","source":"Next lets take a look at Katie Low Samuelson from the womens league. ","metadata":{}},{"cell_type":"code","source":"ms = 10 # Marker Size\nFirstName = 'Katie Lou'\nLastName = 'Samuelson'\nfig, ax = plt.subplots(figsize=(15, 8))\nax = create_ncaa_full_court(ax, three_line='womens')\nWEvents.query('FirstName == @FirstName and LastName == @LastName and EventType == \"made2\"') \\\n    .plot(x='X_', y='Y_', style='o',\n          title='Shots (Katie Lou Samuelson)',\n          alpha=0.5,\n         figsize=(15, 8),\n         label='Made 2',\n         ms=ms,\n         ax=ax)\nplt.legend()\nWEvents.query('FirstName == @FirstName and LastName == @LastName and EventType == \"miss2\"') \\\n    .plot(x='X_', y='Y_', style='X',\n          alpha=0.5, ax=ax,\n         label='Missed 2',\n         ms=ms)\nplt.legend()\nWEvents.query('FirstName == @FirstName and LastName == @LastName and EventType == \"made3\"') \\\n    .plot(x='X_', y='Y_', style='o',\n          c='brown',\n          alpha=0.5,\n         figsize=(15, 8),\n         label='Made 3', ax=ax,\n         ms=ms)\nplt.legend()\nWEvents.query('FirstName == @FirstName and LastName == @LastName and EventType == \"miss3\"') \\\n    .plot(x='X_', y='Y_', style='X',\n          c='green',\n          alpha=0.5, ax=ax,\n         label='Missed 3',\n         ms=ms)\nax.set_xlabel('')\nplt.legend()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"With Katie, there are a much smaller amount of shot attempts under the basket, while there are a lot of shot attempts made outside the 3 point line. This suggest that this player is most likely a point guard or shooting guard. We can also notice that she has a tendency to shoot from far behind the 3 point line, which suggests she is a confident and capable 3 point shooter. ","metadata":{}},{"cell_type":"markdown","source":"# Shot Heatmap\nSimliar to plotting the shots on the court, we can also make a heat of of where the shots occured. I've customized the original code so we can try comparing the heatmaps of the mens court versus the womens court.","metadata":{}},{"cell_type":"code","source":"# Mens heat map \nN_bins = 100\nshot_events = MEvents.loc[MEvents['EventType'].isin(['miss3','made3','miss2','made2']) & (MEvents['X_'] != 0)]\nfig, ax = plt.subplots(figsize=(15, 7))\nax = create_ncaa_full_court(ax,\n                            paint_alpha=0.0,\n                            three_line='mens',\n                            court_color='black',\n                            lines_color='white')\n_ = plt.hist2d(shot_events['X_'].values + np.random.normal(0, 0.1, shot_events['X_'].shape), # Add Jitter to values for plotting\n           shot_events['Y_'].values + np.random.normal(0, 0.1, shot_events['Y_'].shape),\n           bins=N_bins, norm=mpl.colors.LogNorm(),\n               cmap='plasma')\n\n# Plot a colorbar with label.\ncb = plt.colorbar()\ncb.set_label('Number of shots')\n\nax.set_title('Shot Heatmap (Mens)')\nplt.show()\n\n# Womens heat map \nN_bins = 100\nshot_events = WEvents.loc[WEvents['EventType'].isin(['miss3','made3','miss2','made2']) & (WEvents['X_'] != 0)]\nfig, ax = plt.subplots(figsize=(15, 7))\nax = create_ncaa_full_court(ax, three_line='womens', paint_alpha=0.0,\n                            court_color='black',\n                            lines_color='white')\n_ = plt.hist2d(shot_events['X_'].values + np.random.normal(0, 0.2, shot_events['X_'].shape),\n           shot_events['Y_'].values + np.random.normal(0, 0.2, shot_events['Y_'].shape),\n           bins=N_bins, norm=mpl.colors.LogNorm(),\n               cmap='plasma')\n\n# Plot a colorbar with label.\ncb = plt.colorbar()\ncb.set_label('Number of shots')\n\nax.set_title('Shot Heatmap (Womens)')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As you can see the heatmap shows which areas of the court has a higher concentration of shots. Essentially, the brighter the color, the higher the concentration of shots. \n\nAn interesting observation when comparing the heat map of the mens and womens data is the difference in the concentration of shots right below the basket. Notice how the mens shots are more concentrated right below the basket, while the womens is concentrated more to either side of the basket and less right below. ","metadata":{}},{"cell_type":"markdown","source":"# Notebook 2: Redefining the Roster with Unsupervised Machine Learning\nNow we will move on to the second notebook, where we will use **unsupervised machine learning to cluster the previously discussed data** in order to come up with a deeper analysis of the players. \n\nTraditionally, there are 5 positions in a basketball team. Those are Point Guards, Shooting Guards, Small Forwards, Power Forwards, and Centers. However, these positions were defined a very long time ago, and as the game of basketball has evolved, the lines between these positions have been blurred and have become, in some cases, meaningless. \n\nBy using unsupervised learning we will cluster the players into 5 entirely new categories/positions that are solely based on their statistics and data. \n\nThese new positions are: \n\n- Ball Distributors\n- Elite Bigs\n- Shot Takers\n- 3 Point Specialists\n- Paint Dominators\n\n\n*This notebook only makes use of the mens league data. \n","metadata":{}},{"cell_type":"markdown","source":"## Import Libraries\nFirst we need import more libraries. Of these, the most important ones for our use will be the KMeans, TSNE and StandardScaler from the sklearn library. The sklearn library, also known as the scikit-learn library is a free sfotware machine learning library for python. ","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pylab as plt\nimport seaborn as sns\nfrom tqdm.notebook import tqdm\n\n\nfrom sklearn.cluster import KMeans\nfrom sklearn.manifold import TSNE\nfrom sklearn.preprocessing import StandardScaler\nimport umap\n\nimport lightgbm as lgb\n\nfrom scipy.spatial.distance import cdist \n\nimport time\n\nimport matplotlib.pylab as plt\nimport matplotlib as mpl\nfrom matplotlib.patches import Circle, Rectangle, Arc\n\nimport plotly.express as px\n\nfrom IPython.display import display, HTML\n\nimport warnings\nwarnings.filterwarnings('ignore')\n\nsns.set(rc={'figure.figsize':(15, 5)})\npalette = sns.color_palette(\"bright\", 10)\nsns.set_style(\"whitegrid\")\n\npd.set_option('max_columns', 500)\npd.set_option('max_rows', 500)\npd.options.display.float_format = '{:.2f}'.format\n# plt.rcParams['figure.figsize'] = (15.0, 5.0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## NCAA Court Plot Function\nThis is a similar function to the one used previously to draw the entire court. However, this one will only draw half of the court. ","metadata":{}},{"cell_type":"code","source":"def create_ncaa_half_court(ax=None, three_line='mens', court_color='#dfbb85',\n                           lw=3, lines_color='black', lines_alpha=0.5,\n                           paint_fill='blue', paint_alpha=0.4,\n                          inner_arc=False):\n    \"\"\"\n    Version 2020.2.19\n\n    Creates NCAA Basketball Half Court\n    Dimensions are in feet (Court is 97x50 ft)\n    Created by: Rob Mulla / https://github.com/RobMulla\n\n    * Note that this function uses \"feet\" as the unit of measure.\n    * NCAA Data is provided on a x range: 0, 100 and y-range 0 to 100\n    * To plot X/Y positions first convert to feet like this:\n    ```\n    Events['X_'] = (Events['X'] * (94/100))\n    Events['Y_'] = (Events['Y'] * (50/100))\n    ```\n    ax: matplotlib axes if None gets current axes using `plt.gca`\n    \n    three_line: 'mens', 'womens' or 'both' defines 3 point line plotted\n    court_color : (hex) Color of the court\n    lw : line width\n    lines_color : Color of the lines\n    lines_alpha : transparency of lines\n    paint_fill : Color inside the paint\n    paint_alpha : transparency of the \"paint\"\n    inner_arc : paint the dotted inner arc\n    \"\"\"\n    if ax is None:\n        ax = plt.gca()\n\n    # Create Pathes for Court Lines\n    center_circle = Circle((50/2, 94/2), 6,\n                           linewidth=lw, color=lines_color, lw=lw,\n                           fill=False, alpha=lines_alpha)\n    hoop = Circle((50/2, 5.25), 1.5 / 2,\n                       linewidth=lw, color=lines_color, lw=lw,\n                       fill=False, alpha=lines_alpha)\n\n    # Paint - 18 Feet 10 inches which converts to 18.833333 feet - gross!\n    paint = Rectangle(((50/2)-6, 0), 12, 18.833333,\n                           fill=paint_fill, alpha=paint_alpha,\n                           lw=lw, edgecolor=None)\n    \n    paint_boarder = Rectangle(((50/2)-6, 0), 12, 18.833333,\n                           fill=False, alpha=lines_alpha,\n                           lw=lw, edgecolor=lines_color)\n    \n    arc = Arc((50/2, 18.833333), 12, 12, theta1=-\n                   0, theta2=180, color=lines_color, lw=lw,\n                   alpha=lines_alpha)\n    \n    block1 = Rectangle(((50/2)-6-0.666, 7), 0.666, 1, \n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    block2 = Rectangle(((50/2)+6, 7), 0.666, 1, \n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    ax.add_patch(block1)\n    ax.add_patch(block2)\n    \n    l1 = Rectangle(((50/2)-6-0.666, 11), 0.666, 0.166,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    l2 = Rectangle(((50/2)-6-0.666, 14), 0.666, 0.166,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    l3 = Rectangle(((50/2)-6-0.666, 17), 0.666, 0.166,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    ax.add_patch(l1)\n    ax.add_patch(l2)\n    ax.add_patch(l3)\n    l4 = Rectangle(((50/2)+6, 11), 0.666, 0.166,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    l5 = Rectangle(((50/2)+6, 14), 0.666, 0.166,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    l6 = Rectangle(((50/2)+6, 17), 0.666, 0.166,\n                           fill=True, alpha=lines_alpha,\n                           lw=0, edgecolor=lines_color,\n                           facecolor=lines_color)\n    ax.add_patch(l4)\n    ax.add_patch(l5)\n    ax.add_patch(l6)\n    \n    # 3 Point Line\n    if (three_line == 'mens') | (three_line == 'both'):\n        # 22' 1.75\" distance to center of hoop\n        three_pt = Arc((50/2, 6.25), 44.291, 44.291, theta1=12,\n                            theta2=168, color=lines_color, lw=lw,\n                            alpha=lines_alpha)\n\n        # 4.25 feet max to sideline for mens\n        ax.plot((3.34, 3.34), (0, 11.20),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n        ax.plot((50-3.34, 50-3.34), (0, 11.20),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n        ax.add_patch(three_pt)\n\n    if (three_line == 'womens') | (three_line == 'both'):\n        # womens 3\n        three_pt_w = Arc((50/2, 6.25), 20.75 * 2, 20.75 * 2, theta1=5,\n                              theta2=175, color=lines_color, lw=lw, alpha=lines_alpha)\n        # 4.25 inches max to sideline for mens\n        ax.plot( (4.25, 4.25), (0, 8), color=lines_color,\n                lw=lw, alpha=lines_alpha)\n        ax.plot((50-4.25, 50-4.25), (0, 8.1),\n                color=lines_color, lw=lw, alpha=lines_alpha)\n\n        ax.add_patch(three_pt_w)\n\n    # Add Patches\n    ax.add_patch(paint)\n    ax.add_patch(paint_boarder)\n    ax.add_patch(center_circle)\n    ax.add_patch(hoop)\n    ax.add_patch(arc)\n    \n    if inner_arc:\n        inner_arc = Arc((50/2, 18.833333), 12, 12, theta1=180,\n                             theta2=0, color=lines_color, lw=lw,\n                       alpha=lines_alpha, ls='--')\n        ax.add_patch(inner_arc)\n\n    # Restricted Area Marker\n    restricted_area = Arc((50/2, 6.25), 8, 8, theta1=0,\n                        theta2=180, color=lines_color, lw=lw,\n                        alpha=lines_alpha)\n    ax.add_patch(restricted_area)\n    \n    # Backboard\n    ax.plot(((50/2) - 3, (50/2) + 3), (4, 4),\n            color=lines_color, lw=lw*1.5, alpha=lines_alpha)\n    ax.plot( (50/2, 50/2), (4.3, 4), color=lines_color,\n            lw=lw, alpha=lines_alpha)\n\n    # Half Court Line\n    ax.axhline(94/2, color=lines_color, lw=lw, alpha=lines_alpha)\n\n    \n    # Plot Limit\n    ax.set_xlim(0, 50)\n    ax.set_ylim(0, 94/2 + 2)\n    ax.set_facecolor(court_color)\n    ax.set_xticks([])\n    ax.set_yticks([])\n    ax.set_xlabel('')\n    return ax","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Preparations\nMany of the steps in this code is the same as what we discussed previously. We are essentially **preparing the data** for the clustering process.","metadata":{}},{"cell_type":"code","source":"MENS_PBP_DIR = '../input/march-madness-analytics-2020/MPlayByPlay_Stage2'\n\nMPlayers = pd.read_csv(f'{MENS_PBP_DIR}/MPlayers.csv', error_bad_lines=False)\nMTeamSpelling = pd.read_csv('../input/march-madness-analytics-2020/MDataFiles_Stage2/MTeamSpellings.csv',\n                            engine='python')\nmens_events = []\nfor year in [2015, 2016, 2017, 2018, 2019]:\n    mens_events.append(pd.read_csv(f'{MENS_PBP_DIR}/MEvents{year}.csv'))\nMEvents = pd.concat(mens_events)\n\nMTeams = pd.read_csv('../input/march-madness-analytics-2020/MDataFiles_Stage2/MTeams.csv')\nMPlayers = MPlayers.merge(MTeams[['TeamID','TeamName']], on='TeamID', how='left')\n\narea_mapping = {0: np.nan,\n                1: 'under basket',\n                2: 'in the paint',\n                3: 'inside right wing',\n                4: 'inside right',\n                5: 'inside center',\n                6: 'inside left',\n                7: 'inside left wing',\n                8: 'outside right wing',\n                9: 'outside right',\n                10: 'outside center',\n                11: 'outside left',\n                12: 'outside left wing',\n                13: 'backcourt'}\n\nMEvents['Area_Name'] = MEvents['Area'].map(area_mapping)\n\n# Normalize X, Y positions for court dimentions\n# Court is 50 feet wide and 94 feet end to end.\nMEvents['X_'] = (MEvents['X'] * (94/100))\nMEvents['Y_'] = (MEvents['Y'] * (50/100))\n\nMEvents['X_half'] = MEvents['X']\nMEvents.loc[MEvents['X'] > 50, 'X_half'] = (100 - MEvents['X'].loc[MEvents['X'] > 50])\nMEvents['Y_half'] = MEvents['Y']\nMEvents.loc[MEvents['X'] > 50, 'Y_half'] = (100 - MEvents['Y'].loc[MEvents['X'] > 50])\n\nMEvents['X_half_'] = (MEvents['X_half'] * (94/100))\nMEvents['Y_half_'] = (MEvents['Y_half'] * (50/100))\n\n# Merge Player name onto events\nMEvents = MEvents.merge(MPlayers,\n              how='left',\n              left_on='EventPlayerID',\n              right_on='PlayerID')\n\n# Only Look at Events with Player assoicated and X, Y position\nMPlayerEvents = MEvents.query('EventPlayerID != 0 and X > 0')\n\n# Create GameId\nMPlayerEvents['GameId'] = \\\n    MPlayerEvents['Season'].astype('str') + '_' + \\\n    MPlayerEvents['DayNum'].astype('str') + '_' + \\\n    MPlayerEvents['WTeamID'].astype('str') + '_' + \\\n    MPlayerEvents['LTeamID'].astype('str')\n\nEventPlayerSeasonCombo = MPlayerEvents[['EventPlayerID','Season']].drop_duplicates().reset_index(drop=True)\n\n# Expand MPlayers to have a row for each player\nMPlayerSeason = MPlayers.merge(EventPlayerSeasonCombo,\n               left_on=['PlayerID'],\n               right_on=['EventPlayerID'],\n              validate='1:m') \\\n    .drop('EventPlayerID', axis=1)\n\nMPlayerSeason['PlayerID_Season'] = MPlayerSeason['PlayerID'].astype('int').astype('str') + '_' + MPlayerSeason['Season'].astype('str')\nMPlayerEvents['PlayerID_Season'] = MPlayerEvents['PlayerID'].astype('int').astype('str') + '_' + MPlayerEvents['Season'].astype('str')\n\nMPlayerSeasonStats = pd.read_csv('../input/ncaa-mplayerseasonstats/MPlayerSeasonStats.csv')\n\nMPlayerSeasonStats.loc[MPlayerSeasonStats['Weight'] < 150, 'Weight'] = 150 # Fix for zero weights\n\nMPlayerSeasonStats['Position'] = MPlayerSeasonStats['Pos'].map({'F':'Forward',\n                                                      'G':'Guard',\n                                                      'C':'Center'})\nMPlayerSeasonStats['RSCI Top 100 Number'] = MPlayerSeasonStats['RSCI Top 100'].str.split(' ', expand=True)[0].fillna(1000).astype('int')\n\nKAGGLE_PER_GAME_FEATURES = [c for c in MPlayerSeasonStats if 'per_game' in c and 'ShotCount' in c]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Defining Features\nHere we are defining features to be used for the unsupervised machine learning. ","metadata":{}},{"cell_type":"code","source":"f_name_dict = { 'MP' : 'Minutes Played',\n                 'FG' : 'Field Goals',\n                 'FGA':'Field Goal Attempts',\n                 'FG%' : 'Field Goal Percentage',\n                 '2P' : '2-Point Field Goals',\n                 '2PA': '2-Point Field Goal Attempts',\n                 '2P%': '2-Point Field Goal Percentage',\n                 '3P' : '3-Point Field Goals',\n                 '3PA' : '3-Point Field Goal Attempts',\n                 '3P%' : '3-Point Field Goal Percentage',\n                 'FT' : 'Free Throws',\n                 'FTA' : 'Free Throw Attempts',\n                 'FT%' : 'Free Throw Percentage',\n                 'ORB' : 'Offensive Rebounds',\n                 'DRB':'Defensive Rebounds',\n                 'TRB' : 'Total Rebounds',\n                 'AST' : 'Assists',\n                 'STL' : 'Steals',\n                 'BLK' : 'Blocks',\n                 'TOV' : 'Turnovers',\n                 'PF' : 'Personal Fouls',\n                 'PTS' : 'Points'}\n\n# Make Features Per Minute Played\nfor f in ['FG', 'FGA', '2P', '2PA', '3P', '3PA', 'FTA',\n          'ORB', 'DRB',  'TRB', 'AST', 'STL', 'BLK', 'TOV', 'PF', 'PTS'] + KAGGLE_PER_GAME_FEATURES:\n    if f in f_name_dict:\n        MPlayerSeasonStats[f_name_dict[f] + ' Per Minute'] = MPlayerSeasonStats[f] / MPlayerSeasonStats['MP']\n    else:\n        MPlayerSeasonStats[f] = MPlayerSeasonStats[f] / MPlayerSeasonStats['MP']","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Features\nKAGGLE_PER_GAME_FEATURES = [c for c in MPlayerSeasonStats if 'per_game' in c and 'ShotCount' in c]\nSR_PER_GAME_FEATURES = ['MP', 'FG', 'FGA', 'FG%', '2P', '2PA', '2P%', '3P',\n                        '3PA', '3P%', 'FT', 'FTA',\n                        'FT%', 'ORB', 'DRB', 'TRB', 'AST',\n                        'STL', 'BLK', 'TOV', 'PF',\n                        'PTS', 'HeightFeet','Weight']\nPLAYER_FEATS = ['Position', 'PlayerID_Season','FirstName',\n                'LastName','HeightFeet','TeamName','Season',\n                'G', 'GS','PTS','TRB','HeightFeet','FTA','STL']\n\nFEATURES = ['Field Goal Attempts Per Minute',\n            '2-Point Field Goals Per Minute',\n            '3-Point Field Goals Per Minute',\n            'Free Throw Attempts Per Minute',\n            'Offensive Rebounds Per Minute',\n            'Defensive Rebounds Per Minute',\n            'Assists Per Minute',\n            'Blocks Per Minute',\n            'Turnovers Per Minute',\n            'Personal Fouls Per Minute',\n           ]\n\nMPlayerSeasonStats_ = \\\n    MPlayerSeasonStats \\\n    .query('GS > 20 and EventCount >= 0') \\\n    [FEATURES + PLAYER_FEATS] \\\n    .dropna() \\\n    .copy()\n\nX = MPlayerSeasonStats_[FEATURES].values\nss = StandardScaler()\nX = ss.fit_transform(X)\ny = MPlayerSeasonStats_['PlayerID_Season'].values\n\n# Create 5 Player Clusters\nmodel = KMeans(n_clusters=5, random_state=529)\nkmeans_pred = model.fit_predict(X)\nMPlayerSeasonStats_['kmeans_Cluster'] = kmeans_pred\nMPlayerSeasonStats_['Data_Position'] = MPlayerSeasonStats_['kmeans_Cluster'] + 1\n\ncluster_names_dict = {1 : 'Shot Taker',\n                      2 : 'Skilled Big',\n                      3 : '3 Point Specialist',\n                      4 : 'Ball Distributor',\n                      5 : 'Paint Dominator'}\n\nMPlayerSeasonStats_['Cluster_Name'] = MPlayerSeasonStats_['Data_Position'].map(cluster_names_dict)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The plot below shows a graph with 2 axies: Points and Rebounds, according to the newly created 5 positions mentioned earlier. Although it is a little bit messy, it is color coded for easier visualization. You can already see that there are clusters forming just by looking at the colors!","metadata":{}},{"cell_type":"code","source":"data = MPlayerSeasonStats_.merge(MPlayerSeasonStats[['PlayerID_Season','TRB','FGA']])\nfrom matplotlib.patches import Ellipse\n\nx_feat = 'TRB'\ny_feat = 'PTS'\n\nfig, ax = plt.subplots(figsize=(15, 15))\ncolor_pal = [x['color'] for x in plt.rcParams['axes.prop_cycle']]\n\nidx = 0\nfor i, d in data.query('GS > 20').groupby('Cluster_Name'):\n    d.plot(x='PTS', y='TRB', kind='scatter', ax=ax,\n           color=color_pal[idx], alpha=0.5, label=f'{i}', s=25)\n    idx += 1\nidx = 0\n\nax.annotate('Zion Williamson 2019 (Skilled Big)',\n            xy=(22.60, 8.90), xycoords='data',\n            xytext=(0.99, 0.65), textcoords='axes fraction',\n            arrowprops=dict(facecolor='black', shrink=0.05),\n            horizontalalignment='right', verticalalignment='top')\n\nax.annotate('Lonzo Ball 2017 (Ball Distributor)',\n            xy=(14.60, 6), xycoords='data',\n            xytext=(0.98, 0.51), textcoords='axes fraction',\n            arrowprops=dict(facecolor='black', shrink=0.05),\n            horizontalalignment='right', verticalalignment='top')\n\nax.annotate('Aaron Harrison 2015 (3Pt Specialist)',\n            xy=(11.00, 2.6), xycoords='data',\n            xytext=(0.5, 0.05), textcoords='axes fraction',\n            arrowprops=dict(facecolor='black', shrink=0.05),\n            horizontalalignment='right', verticalalignment='top')\n\nax.annotate('Kaleb Tarczewski 2016 (Paint Dominator)',\n            xy=(9.40, 9.30), xycoords='data',\n            xytext=(0.32, 0.72), textcoords='axes fraction',\n            arrowprops=dict(facecolor='black', shrink=0.05),\n            horizontalalignment='right', verticalalignment='top')\n\nax.annotate('Romeo Langford 2019 (Shot Taker)',\n            xy=(16.50, 5.40), xycoords='data',\n            xytext=(0.95, 0.10), textcoords='axes fraction',\n            arrowprops=dict(facecolor='black', shrink=0.05),\n            horizontalalignment='right', verticalalignment='top')\n\nplt.title('Data Driven Positions: Rebounds vs Points', fontsize=25)\nplt.xlabel('Points / Game')\nplt.ylabel('Total Rebounds / Game')\nplt.legend()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Features and Approach\n\nAs previously mentioned, we will only be using data from the mens league. The data was normalized according to the number of minutes played by each player. This was done because we don't want the data to be skewed based on the number of minutes the players play. Instead the data should reflect the events that occured with each minute given to the player. It should also be noted that the data down filtered down so that it only included players who have played at least 20 games per season.\n\nThe following are features that will be used for clustering: \n- Field Goal Attempts\n- 2-Point Field Goals\n- 3-Point Field Goals\n- Free Throw Attempts\n- Offensive Rebounds\n- Defensive Rebounds\n- Assists\n- Blocks\n- Turnovers\n- Personal Fouls\n\nWe will be using **KMeans Clustering**. After many tests to determine the appropriate number of clusters, this notebook concluded that 5 would be the optimal number of clusters. ","metadata":{}},{"cell_type":"markdown","source":"## Comparing the Groups in Detail\nThe graphs below shows the feature stats by the newly defined positions. As yoiu can see, some positions are more suited for defence(rebounds, blocks), and other are more suited for offense(field goals, assists). ","metadata":{}},{"cell_type":"code","source":"MPlayerSeasonStats_[FEATURES + ['Cluster_Name']].groupby('Cluster_Name') \\\n    .mean().sort_index(ascending=False).T \\\n    .plot(kind='barh', figsize=(15, 12),\n          title='Average Stats by Group')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Below we visualize the same data, but this time using box plots. By using box plots it is more **easy to identify outliers** in the data. ","metadata":{}},{"cell_type":"code","source":"clus_name_short ={'3 Point Specialist': '3PT Spec',\n                  'Shot Taker':'Shot Taker',\n                  'Ball Distributor' : 'Distrib',\n                  'Skilled Big':'SBig',\n                  'Paint Dominator': 'PaintDom'}\n\nMPlayerSeasonStats_['Cluster_Name_Short'] = MPlayerSeasonStats_['Cluster_Name'].map(clus_name_short)\n\nfig, axs = plt.subplots(3, 3, figsize=(15, 10))\naxs = axs.flatten()\nidx = 0\nfor f in FEATURES[:9]:\n    sns.boxplot(x='Cluster_Name_Short', y=f, data=MPlayerSeasonStats_, ax=axs[idx])\n    axs[idx].set_title(f)\n    axs[idx].set_xlabel('')\n    axs[idx].set_ylabel('')\n    idx += 1\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# The New Roster\nBased on the clusters that were made, we will now define and identify characteristics of the 5 newly defined position.\n\nAs previously mentioned, these new positions are: \n\n- Ball Distributors\n- Elite Bigs\n- Shot Takers\n- 3 Point Specialists\n- Paint Dominators","metadata":{}},{"cell_type":"code","source":"fig, axs = plt.subplots(1, 5, figsize=(18, 3.6))\naxs = axs.flatten()\nidx = 0\nfor cluster in MPlayerSeasonStats_['Cluster_Name'].unique():\n    playerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\n    Made = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n        .query('EventType == \"made2\" or EventType == \"made3\" or EventType == \"miss2\" or EventType == \"miss3\"')\n    axs[idx] = create_ncaa_half_court(ax=axs[idx],\n                                 three_line='mens',\n                                 court_color='black',\n                                      lines_color='white',\n                                 paint_alpha=0,\n                                 inner_arc=True)\n    #     hb1 = axs[idx].plot(x=Turnover.query('Y_ != 0')['Y_half_'],\n    #                      y=Turnover.query('Y_ != 0')['X_half_'],\n    #                      gridsize=10, bins='log', cmap='inferno')\n    Made.query('Y_ != 0').plot(x='Y_half_',\n                               y='X_half_',\n                               kind='scatter',\n                               ax=axs[idx],\n                               alpha=0.1)\n    axs[idx].set_title(f'{cluster}', size=15)\n    axs[idx].set_xlabel('')\n    axs[idx].set_ylabel('')\n    axs[idx].set_ylim(0, 38)\n    idx+= 1\nfig.suptitle('Shot Selection by Group', y=1.05, fontsize=15)\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# The Skilled Big\nThe following are the characteristics for this position based on the cluster's data:\n- Tallest players in the team\n- Takes the 2nd most number of shots\n- Have a high average foul stat and also are more often fouled\n- Take more 2 point shots than 3 point shots \n- Makes the most free throws","metadata":{}},{"cell_type":"code","source":"MPlayerSeasonStats_ = MPlayerSeasonStats_.loc[:,~MPlayerSeasonStats_.columns.duplicated()]\nax = MPlayerSeasonStats_.groupby('Cluster_Name')['HeightFeet'].mean().sort_values() \\\n    .plot(kind='barh',\n          xlim=(6, 6.8),\n          title='Skilled bigs are the tallest of all groups')\nax.patches[4].set_color('orange')\nax.set_xlabel('Player Height (Feet)')\nax.set_ylabel('')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cluster = 'Skilled Big'\n\nfig, axs = plt.subplots(1, 2, figsize=(14, 7))\naxs = axs.flatten()\n\nidx = 0\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nShots = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\" or EventType == \"miss2\" or EventType == \"miss3\"')\naxs[idx] = create_ncaa_half_court(ax=axs[idx],\n                             three_line='mens',\n                             court_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\nhb1 = axs[idx].hexbin(x=Shots.query('Y_ != 0')['Y_half_'],\n                 y=Shots.query('Y_ != 0')['X_half_'],\n                 gridsize=20, bins='log', cmap='inferno')\naxs[idx].set_title(f'Shot Attempts: {cluster}', size=15)\n\naxs[idx].set_ylim(0, 40)\naxs[idx].set_xlim(1, 49)\nplt.tight_layout()\n\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nMade = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\"')\naxs[idx+1] = create_ncaa_half_court(ax=axs[idx+1],\n                             three_line='mens',\n                             court_color='black',\n                                  lines_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\n#     hb1 = axs[idx].plot(x=Turnover.query('Y_ != 0')['Y_half_'],\n#                      y=Turnover.query('Y_ != 0')['X_half_'],\n#                      gridsize=10, bins='log', cmap='inferno')\nMade.query('Y_ != 0').plot(x='Y_half_',\n                 y='X_half_',\n              kind='scatter', ax=axs[idx+1], alpha=0.3)\naxs[idx+1].set_title(f'Made Shots: {cluster}', size=15)\n\naxs[idx+1].set_ylim(0, 40)\naxs[idx+1].set_xlim(1, 49)\naxs[idx+1].set_xlabel('')\naxs[idx+1].set_ylabel('')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# The Shot Taker\nThe following are the characteristics for this position based on the cluster's data:\n- Take more shots than any other position\n- They have a good balance of 2 point shots and 3 point shots\n- Although this position takes the most unber of shots, their shooting percentage(how many shots go in) is not that different from the other positions. ","metadata":{}},{"cell_type":"code","source":"fig, (ax, ax1) = plt.subplots(1, 2, figsize=(15, 4))\nfig.suptitle('High shot count but average shooting %', fontsize=15, y=1.03)\nMPlayerSeasonStats[['PlayerID_Season','FGA']].merge(MPlayerSeasonStats_[['PlayerID_Season','Cluster_Name']]) \\\n    .groupby('Cluster_Name')['FGA'].mean() \\\n    .sort_values() \\\n    .plot(kind='barh', rot=0, ax=ax)\n# ax.set_title('Almost 2x more assists than the next group', fontsize=15)\nax.set_xlabel('Field Goal Attempts Per Game')\nax.set_ylabel('')\nax.patches[4].set_color('orange')\n\n\nMPlayerSeasonStats[['PlayerID_Season','FG%']].merge(MPlayerSeasonStats_[['PlayerID_Season','Cluster_Name']]) \\\n    .groupby('Cluster_Name')['FG%'].mean() \\\n    .sort_values() \\\n    .plot(kind='barh', rot=0, ax=ax1)\n# ax1.set_title('Almost 2x more assists than the next group', fontsize=15)\nax1.set_xlabel('Field Goal %')\nax1.set_ylabel('')\nax1.patches[2].set_color('orange')\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cluster = 'Shot Taker'\n\nfig, axs = plt.subplots(1, 2, figsize=(14, 7))\naxs = axs.flatten()\n\nidx = 0\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nShots = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\" or EventType == \"miss2\" or EventType == \"miss3\"')\naxs[idx] = create_ncaa_half_court(ax=axs[idx],\n                             three_line='mens',\n                             court_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\nhb1 = axs[idx].hexbin(x=Shots.query('Y_ != 0')['Y_half_'],\n                 y=Shots.query('Y_ != 0')['X_half_'],\n                 gridsize=20, bins='log', cmap='inferno')\naxs[idx].set_title(f'Shot Attempts: {cluster}', size=15)\n\naxs[idx].set_ylim(0, 40)\naxs[idx].set_xlim(1, 49)\nplt.tight_layout()\n\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nMade = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\"')\naxs[idx+1] = create_ncaa_half_court(ax=axs[idx+1],\n                             three_line='mens',\n                             court_color='black',\n                                  lines_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\n#     hb1 = axs[idx].plot(x=Turnover.query('Y_ != 0')['Y_half_'],\n#                      y=Turnover.query('Y_ != 0')['X_half_'],\n#                      gridsize=10, bins='log', cmap='inferno')\nMade.query('Y_ != 0').plot(x='Y_half_',\n                 y='X_half_',\n              kind='scatter', ax=axs[idx+1], alpha=0.3)\naxs[idx+1].set_title(f'Made Shots: {cluster}', size=15)\n\naxs[idx+1].set_ylim(0, 40)\naxs[idx+1].set_xlim(1, 49)\naxs[idx+1].set_xlabel('')\naxs[idx+1].set_ylabel('')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# The Ball Distributor\nThe following are the characteristics for this position based on the cluster's data:\n- They have the most amount of assists out of all other positions\n- Have very low average defensive stats (rebounds, blocks)","metadata":{}},{"cell_type":"code","source":"ax = MPlayerSeasonStats[['PlayerID_Season','AST']] \\\n    .merge(MPlayerSeasonStats_[['PlayerID_Season','Cluster_Name']]) \\\n    .groupby('Cluster_Name')['AST'].mean() \\\n    .sort_values() \\\n    .plot(kind='bar', rot=0)\nax.set_title('1.5x more assists than the next group', fontsize=15)\nax.set_ylabel('Assists Per Game')\nax.set_xlabel('')\nax.patches[4].set_color('orange')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cluster = 'Ball Distributor'\n\nfig, axs = plt.subplots(1, 2, figsize=(14, 7))\naxs = axs.flatten()\n\nidx = 0\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nShots = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\" or EventType == \"miss2\" or EventType == \"miss3\"')\naxs[idx] = create_ncaa_half_court(ax=axs[idx],\n                             three_line='mens',\n                             court_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\nhb1 = axs[idx].hexbin(x=Shots.query('Y_ != 0')['Y_half_'],\n                 y=Shots.query('Y_ != 0')['X_half_'],\n                 gridsize=20, bins='log', cmap='inferno')\naxs[idx].set_title(f'Shot Attempts: {cluster}', size=15)\n\naxs[idx].set_ylim(0, 40)\naxs[idx].set_xlim(1, 49)\nplt.tight_layout()\n\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nMade = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\"')\naxs[idx+1] = create_ncaa_half_court(ax=axs[idx+1],\n                             three_line='mens',\n                             court_color='black',\n                                  lines_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\n#     hb1 = axs[idx].plot(x=Turnover.query('Y_ != 0')['Y_half_'],\n#                      y=Turnover.query('Y_ != 0')['X_half_'],\n#                      gridsize=10, bins='log', cmap='inferno')\nMade.query('Y_ != 0').plot(x='Y_half_',\n                 y='X_half_',\n              kind='scatter', ax=axs[idx+1], alpha=0.3)\naxs[idx+1].set_title(f'Made Shots: {cluster}', size=15)\n\naxs[idx+1].set_ylim(0, 40)\naxs[idx+1].set_xlim(1, 49)\naxs[idx+1].set_xlabel('')\naxs[idx+1].set_ylabel('')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# The Three Point Specialist\nThe following are the characteristics for this position based on the cluster's data:\n- Takes the most 3 point shots \n- Takes more 3 point shot attempts than they do 2 point shots","metadata":{}},{"cell_type":"code","source":"fig, (ax, ax1) = plt.subplots(1, 2, figsize=(15, 4))\nfig.suptitle('Large disparity between 3 and 2 point shots', fontsize=15, y=1.03)\nMPlayerSeasonStats[['PlayerID_Season','3P']].merge(MPlayerSeasonStats_[['PlayerID_Season','Cluster_Name']]) \\\n    .groupby('Cluster_Name')['3P'].mean() \\\n    .sort_values() \\\n    .plot(kind='barh', rot=0, ax=ax)\n# ax.set_title('Almost 2x more assists than the next group', fontsize=15)\nax.set_xlabel('3 Point Shots Per Game')\nax.set_ylabel('')\nax.patches[4].set_color('orange')\n\n\nMPlayerSeasonStats[['PlayerID_Season','2P']].merge(MPlayerSeasonStats_[['PlayerID_Season','Cluster_Name']]) \\\n    .groupby('Cluster_Name')['2P'].mean() \\\n    .sort_values() \\\n    .plot(kind='barh', rot=0, ax=ax1)\n# ax1.set_title('Almost 2x more assists than the next group', fontsize=15)\nax1.set_xlabel('2 Point Shots Per Game')\nax1.set_ylabel('')\nax1.patches[0].set_color('orange')\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cluster = '3 Point Specialist'\n\nfig, axs = plt.subplots(1, 2, figsize=(14, 7))\naxs = axs.flatten()\n\nidx = 0\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nShots = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\" or EventType == \"miss2\" or EventType == \"miss3\"')\naxs[idx] = create_ncaa_half_court(ax=axs[idx],\n                             three_line='mens',\n                             court_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\nhb1 = axs[idx].hexbin(x=Shots.query('Y_ != 0')['Y_half_'],\n                 y=Shots.query('Y_ != 0')['X_half_'],\n                 gridsize=20, bins='log', cmap='inferno')\naxs[idx].set_title(f'Shot Attempts: {cluster}', size=15)\n\naxs[idx].set_ylim(0, 40)\naxs[idx].set_xlim(1, 49)\nplt.tight_layout()\n\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nMade = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\"')\naxs[idx+1] = create_ncaa_half_court(ax=axs[idx+1],\n                             three_line='mens',\n                             court_color='black',\n                                  lines_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\n#     hb1 = axs[idx].plot(x=Turnover.query('Y_ != 0')['Y_half_'],\n#                      y=Turnover.query('Y_ != 0')['X_half_'],\n#                      gridsize=10, bins='log', cmap='inferno')\nMade.query('Y_ != 0').plot(x='Y_half_',\n                 y='X_half_',\n              kind='scatter', ax=axs[idx+1], alpha=0.3)\naxs[idx+1].set_title(f'Made Shots: {cluster}', size=15)\n\naxs[idx+1].set_ylim(0, 40)\naxs[idx+1].set_xlim(1, 49)\naxs[idx+1].set_xlabel('')\naxs[idx+1].set_ylabel('')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# The Paint Dominator\nThe following are the characteristics for this position based on the cluster's data:\n- Remains in the \"paint\": The paint is essentially the area closest to the basketball hoop. \n- They have the 2nd largest amount of rebounds and blocks \n- More 2 point shots than 3 point shots.","metadata":{}},{"cell_type":"code","source":"fig, (ax, ax1) = plt.subplots(1, 2, figsize=(15, 4))\nfig.suptitle('Similar to the Skilled Big - with less shooting', fontsize=15, y=1.03)\nMPlayerSeasonStats[['PlayerID_Season','BLK']].merge(MPlayerSeasonStats_[['PlayerID_Season','Cluster_Name']]) \\\n    .groupby('Cluster_Name')['BLK'].mean() \\\n    .sort_values() \\\n    .plot(kind='barh', rot=0, ax=ax)\nax.set_xlabel('Blocks Per Game')\nax.set_ylabel('')\nax.patches[3].set_color('orange')\n\n\nMPlayerSeasonStats[['PlayerID_Season','2P']].merge(MPlayerSeasonStats_[['PlayerID_Season','Cluster_Name']]) \\\n    .groupby('Cluster_Name')['2P'].mean() \\\n    .sort_values() \\\n    .plot(kind='barh', rot=0, ax=ax1)\nax1.set_xlabel('2 Point Shots Per Game')\nax1.set_ylabel('')\nax1.patches[2].set_color('orange')\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cluster = 'Paint Dominator'\n\nfig, axs = plt.subplots(1, 2, figsize=(14, 7))\naxs = axs.flatten()\n\nidx = 0\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nShots = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\" or EventType == \"miss2\" or EventType == \"miss3\"')\naxs[idx] = create_ncaa_half_court(ax=axs[idx],\n                             three_line='mens',\n                             court_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\nhb1 = axs[idx].hexbin(x=Shots.query('Y_ != 0')['Y_half_'],\n                 y=Shots.query('Y_ != 0')['X_half_'],\n                 gridsize=20, bins='log', cmap='inferno')\naxs[idx].set_title(f'Shot Attempts: {cluster}', size=15)\n\naxs[idx].set_ylim(0, 40)\naxs[idx].set_xlim(1, 49)\nplt.tight_layout()\n\nplayerid_seasons = MPlayerSeasonStats_.loc[MPlayerSeasonStats_['Cluster_Name'] == cluster]['PlayerID_Season'].unique()\nMade = MPlayerEvents.loc[MPlayerEvents['PlayerID_Season'].isin(playerid_seasons)] \\\n    .query('EventType == \"made2\" or EventType == \"made3\"')\naxs[idx+1] = create_ncaa_half_court(ax=axs[idx+1],\n                             three_line='mens',\n                             court_color='black',\n                                  lines_color='white',\n                             paint_alpha=0,\n                             inner_arc=True)\nMade.query('Y_ != 0').plot(x='Y_half_',\n                 y='X_half_',\n              kind='scatter', ax=axs[idx+1], alpha=0.3)\naxs[idx+1].set_title(f'Made Shots: {cluster}', size=15)\n\naxs[idx+1].set_ylim(0, 40)\naxs[idx+1].set_xlim(1, 49)\naxs[idx+1].set_xlabel('')\naxs[idx+1].set_ylabel('')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Schools and Player Types\nNow that we have clustered the players into 5 new positions, we will take a look at the player rosters of 11 teams and see what newly defined positions their player roster consists of. \n\nAs you can see schools have different but clearly defined preferences for what type of player they want in their roster. ","metadata":{}},{"cell_type":"code","source":"cmap=sns.light_palette((210, 90, 60), input=\"husl\", as_cmap=True)\nTOP_BBALL_SCHOOLS = ['Kentucky','North Carolina','Duke','Villanova','Kansas','UCLA','Connecticut','Indiana',\n               'Louisville','Virginia','Florida']\nd = MPlayerSeasonStats_.query('TeamName in @TOP_BBALL_SCHOOLS') \\\n    .groupby(['TeamName','Cluster_Name'])['Position'].count() \\\n    .unstack('TeamName')\\\n    .T.sort_values(['3 Point Specialist','Ball Distributor','Paint Dominator','Shot Taker','Skilled Big']).T\nd.index.name = ''\nd.style.background_gradient(cmap='BuGn', axis=0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## What Types of Rosters Fare well in March Madness?\nNow lets take a look at the tournament winning and losing teams and see if there are any corrolations between results and the types of players they have. As you can see winning teams tend to have more \"Ball Distributors\" and \"Skilled Bigs\" in their player roster while the losing teams tend to have more of the other three (especially \"Shot Taker\"). Shot Takers shot a lot but don't have a high shooting percentage, which could explain why teams with more Shot Takers are in the losing side. ","metadata":{}},{"cell_type":"code","source":"# Add Team ID\nMTourneyResults = pd.read_csv('../input/march-madness-analytics-2020/MDataFiles_Stage2/MNCAATourneyCompactResults.csv')\nMPlayerSeasonStats_ = MPlayerSeasonStats_.merge(MTeams[['TeamID','TeamName']], how='left')\nTourneyWinTeams = MPlayerSeasonStats_[['TeamID','Season','Cluster_Name']] \\\n    .merge(MTourneyResults, left_on=['Season','TeamID'], right_on=['Season','WTeamID'])\nTourneyLossTeams = MPlayerSeasonStats_[['TeamID','Season','Cluster_Name']] \\\n    .merge(MTourneyResults, left_on=['Season','TeamID'], right_on=['Season','LTeamID'])\nTourneyTeams = TourneyWinTeams.groupby('Cluster_Name')[['TeamID']].count() \\\n    .rename(columns={'TeamID':'Winning Team Rosters'})\nTourneyTeams['Losing Team Rosters'] = TourneyLossTeams.groupby('Cluster_Name')[['TeamID']].count().values\nTourneyTeams['Diff'] = TourneyTeams['Winning Team Rosters'] - TourneyTeams['Losing Team Rosters']\nTourneyTeams['Winning Team Makeup'] = TourneyTeams['Winning Team Rosters'] / TourneyTeams['Winning Team Rosters'].sum()\nTourneyTeams['Losing Team Makeup'] = TourneyTeams['Losing Team Rosters'] / TourneyTeams['Losing Team Rosters'].sum()\nTourneyTeams['Makeup Diff'] = TourneyTeams['Winning Team Makeup'] - TourneyTeams['Losing Team Makeup']\nax = TourneyTeams['Makeup Diff'].plot(figsize=(15, 5), kind='barh', rot=0, title='Tourney Winning vs. Losing Team Roster Differential')\nax.axvline(0, color='black')\nax.grid(False)\nax.set_xlabel('<----- Losing Tourney Teams have more | Winning Tourney Teams have more ---->')\nax.set_ylabel('')\nax.patches[0].set_color('lightcoral')\nax.patches[1].set_color('seagreen')\nax.patches[2].set_color('lightcoral')\nax.patches[3].set_color('lightcoral')\nax.patches[4].set_color('seagreen')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Interactive 3D Visualization of Clusters\nThis is the last visualization in the notebook. It plots out the clusters in 3-Dimensional space. Each of the dots on the graph represents a player and the season they played in. \n\nThis graph was made using an algorithm called tSNE, which stands for t-distributed Stochastic Neighbor Embedding, and was used to reduce all of the data into 3 dimensions. \n\nThe graph is also interactive, so you can use your mouse to rotate and zoom in. ","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"whitegrid\")\n\ntsne = TSNE(n_components=3, verbose=0, perplexity=2, n_iter=250, random_state=529)\nX_tsne = MPlayerSeasonStats_[FEATURES + ['Data_Position']].values\n\ntsne_results = tsne.fit_transform(X_tsne)\nMPlayerSeasonStats_['tsne-3d-one'] = tsne_results[:,0]\nMPlayerSeasonStats_['tsne-3d-two'] = tsne_results[:,1]\nMPlayerSeasonStats_['tsne-3d-three'] = tsne_results[:,2]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.scatter_3d(MPlayerSeasonStats_.merge(MPlayerSeasonStats[['PlayerID_Season','Height']]),\n                    x='tsne-3d-one',\n                    y='tsne-3d-two',\n                    z='tsne-3d-three',\n                    size_max=5,\n                    color='Cluster_Name',\n                    hover_data=[\"FirstName\", \"LastName\",\"TeamName\",\"Height\"],\n                    opacity=0.7)\n# fig.update_layout(hovermode=\"FirstName\")\nfig.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusion\nThrough these 2 notebooks, we were able to go from initial data preparations, to visualizing simple patterns, and lastly into using unsupervised machine learning to cluster the data into clusters that have not been made before. The use of machine learning to identify new positions based solely on player statistics is a very interesting topic and I believe may be the future of sports. Much like \"Money Ball\" in baseball, where Billy Beane and Peter Brandt used statistics in order to build a World Series contender team, I believe analysis like this could be implemented into sports teams so they can optimize their player rosters and give themselves a better chance at winning tournaments. Data analytics and applications have entered all facets of our society, and just because sports are games in the physical realm doesn't mean it can't take advantage of this as well. \n\nThis has been an extremely informative excercise in going through the process of sorting, organizing, preparing data and using it to come up with meaningful observations, and the real life usage of KMeans Clustering to come up with new insights was especially eye opening for me. \n\nThis has been my analysis of \"Rob Mulla\"s \"🏀 2020 March Madness Data - First Look EDA\" and \"Redefining NCAA Basketball Positions using Data!\". \n\nThank you for reading.","metadata":{}},{"cell_type":"markdown","source":"# Reference\n1. Court Lines code inspired by code made for plotting the NBA court. http://savvastjortjoglou.com/nba-shot-sharts.html\n2. Official NCAA Basketball Court Dimensions:\n\n3. https://www.espn.com/mens-college-basketball/story/_/id/9799167/traditional-positions-being-eliminated-college-basketball\n\nData Prep Notebooks:\n- https://www.kaggle.com/robikscube/march-madness-2020-sports-reference-com-scrape\n- https://www.kaggle.com/robikscube/rob-s-march-madness-2020-data-prep","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}}]}