{"cells":[{"metadata":{},"cell_type":"markdown","source":"**Disclaimer**\n\n*I am a published author of suspense and young adult novels as well as political scientist, not a coder or data analyst. This notebook is a part of the storytelling challenge included in the NCAA 2020 March Madness contest.*\n\n*My original plan was to supplement this notebook with the 2020 March Madness brackets and make some mascot-based picks. However, as we know, both men's and women's tournaments were cancelled because of the recent corona virus outbreak.*\n\n*It is also a fact that this is the third Jupyter Notebook I've done in my life.*"},{"metadata":{},"cell_type":"markdown","source":"# OK Boomer and Sooner: college mascots in March Madness "},{"metadata":{},"cell_type":"markdown","source":"**TABLE OF CONTENTS**\n1. Introduction <br>\n2. Mascot Data <br>\n3. Mascot Exploration <br>\n4. Mascot Success in March Madness <br>\n5. Team Selection and Data Leak <br>\n6. Mascot Types in March Madness <br>\n7. Further Mascot Classifying <br>\n8. Mascots and Upsets <br>\n9. Elite Eight Mascots <br>"},{"metadata":{},"cell_type":"markdown","source":"## 1. Introduction"},{"metadata":{},"cell_type":"markdown","source":"When the students of Yale university adopted a bulldog as a mascot back in 1889, they probably didn't know they would start a whole new tradition. The original bulldog **Handsome Dan** has since had sixteen successors, but the idea of college sports teams having their own distinctive mascot is these days more of a norm than an exception all around the world. \n\nTo be fair, not all mascots have been as successful as Handsome Dan. During the years many of them have been retired for different reasons related to religious, ethnic and cultural sensitivity. To avoid any unwanted controversy, nowadays professional services for creating mascots are widely available. However there are still iconic old-school mascots around from the early days, including **Boomer** and **Sooner**, the two horses pulling the **Sooner Schooner** wagon in the opening ceremony of University of Oklahoma football home games. \n\nIn this notebook, the March Madness dataset will be analyzed from the viewpoint of college mascots. *For example, are teams with an eagle mascot more likely to become cinderella stories? Or what mascot types represent teams failing their fans with an upset first-round exit despite their high seeding?*"},{"metadata":{},"cell_type":"markdown","source":"## 2. Mascot data"},{"metadata":{},"cell_type":"markdown","source":"The AllMascots dataframe used in the analysis is based on scraping data from Wikipedia page listing U.S. college mascots ([link](https://en.wikipedia.org/wiki/List_of_college_mascots_in_the_United_States)). That dataset was then extended to cover all teams listed in the provided March Madness dataset. In total, the AllMascots dataframe consists of nearly 700 individual mascots. Despite its name the *AllMascots dataframe is not a comprehensive take on all college mascots in the U.S.* However it does provide a guideline on what sort of scenario we are about to dwell in. Below is a small sample from AllMascots dataframe (feel free to explore more):"},{"metadata":{"trusted":true},"cell_type":"code","source":"#import modules\nimport pandas as pd\nimport numpy as np\n\n#load dataframe\ndf_2 = pd.read_csv(\"../input/march-madness-mascots/AllMascots.csv\", low_memory = False)\n\n#drop unnecessary id column\ndf_2.drop('rowid', axis=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- **MascotID** column lists individually all mascots (for example Boomer, Sooner and the Sooner Schooner wagon all have their own MascotID. \n- **MascotName** column includes the individual names of mascots (some of them only have generic names like Tiger or Wildcat).\n- **MascotType** column divides the mascots in different subcategories (see more below).\n- **College** column includes the name of the college using the mascot in question."},{"metadata":{},"cell_type":"markdown","source":"When we plot the dataframe and divide it by mascot type, we get the following result (use the scroll bar to see more):"},{"metadata":{"trusted":true},"cell_type":"code","source":"# import module\nimport plotly.express as px\n\n# set plot parameters\nfig = px.histogram(df_2, y=\"MascotType\", histnorm = 'percent', orientation = 'h', height = 2000)\nfig.data[0].marker.color = \"orange\"\nfig.data[0].marker.line.width = 1\nfig.data[0].marker.line.color = \"black\"\n\n# set plot title\nfig.update_layout(\n    title={\n        'text': \"U.S. college mascot types divided by percentage\",\n        'y':0.99,\n        'x':0.5,\n        'xanchor': 'center',\n        'yanchor': 'top'})\n\n# show plot\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As one can see, some third of all mascots are labeled under the 'mascot' type. This type basically covers all mascots that don't fit any of the other categories. Mostly they are abstract (for example Big Red of Western Kentucky University\n([link](https://en.wikipedia.org/wiki/Big_Red_%28Western_Kentucky_University%29)) or unique human-like figures described best by their name (for example Lumberjack or Pirate). \n\nThe most important thing to take notice on is the large variety of mascots. If we print out the number of unique mascot types - including the internally diverse 'mascot' subcategory - we get a respectable number 116:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# print value count of MascotType column\ndf_2['MascotType'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To effectively combine mascot data with the provided March Madness dataset, a new dataframe called MarchMascots is created next. MarchMascots includes only teams and their mascots that have made it to the March Madness tournament prior to year 2020. Also, to refine results, the 'mascot' subcategory was browsed through manually to further label different mascot types."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load dataframe\ndf = pd.read_csv(\"../input/march-madness-mascots/MarchMascots.csv\", low_memory = False)\n\n# drop unnecessary id column\ndf = df.drop('rowid', axis=1)\n\n# printout\ndf.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"MarchMascots has some additional columns compared to AllMascots:\n\n- **MTeamID** column has the team ID number used in men's March Madness dataset.\n- **WTeamID** column includes the team ID number used in women's March Madness dataset.\n- **TeamName** column refers to abbreviated college names used both in men's and women's March Madness dataset.\n\nIt's worth noting that the ID numbers in men's and women's dataset are the same, the only difference being the format (men are numbered in 1xxx format, women 3xxx). However since we use both men's and women's March Madness datasets, these two ID numbers are kept separate. Also, as the University of Oklahoma example showed us, teams sometimes have more than one mascot. Thus a team ID may show up in its respective column more than once."},{"metadata":{},"cell_type":"markdown","source":"The MarchMascots dataframe has roughly half of all teams from AllMascots dataframe. This is as such understandable, because there are over four thousand institutions offering degrees in the U.S. and only some of them have dedicated college sports programs. Narrowing the selection even further, there are specific colleges concentrating primarily on basketball, meaning they are more likely to show up at the March Madness tournament year after year."},{"metadata":{},"cell_type":"markdown","source":"Before retiring the AllMascots dataframe altogether, it would be a fun warm-up task to compare if it differs from the MarchMascots dataframe in any way other than size. Below is the MarchMascots dataframe divided by mascot type:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# set plot parameters\nfig = px.histogram(df, y=\"MascotType\", histnorm = 'percent', orientation = 'h', height = 2000)\nfig.data[0].marker.color = \"orange\"\nfig.data[0].marker.line.width = 1\nfig.data[0].marker.line.color = \"black\"\n\n# set plot title\nfig.update_layout(\n    title={\n        'text': \"March Madness mascots 2015-2019 divided by mascot type\",\n        'y':0.99,\n        'x':0.5,\n        'xanchor': 'center',\n        'yanchor': 'top'})\n\n# show plot\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The printout of MarchMascot types looks like following:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# print value count of MascotType column\ndf['MascotType'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It looks like we can retire the AllMascots dataframe for now, since MarchMascots gives us a more detailed view on different mascot types.\n\nThis is also a good time to bring in the March Madness teams, but this was not without problems at least in my case. \n\n***As I write this, I am for some reason unable to browse or link any of the dataframes included in the NCAA dataset to this notebook. Therefore I will use duplicates of those dataframes I uploaded as my own dataset. If and when the original dataframes become available, I will remove the duplicates and use the original dataframes.***"},{"metadata":{},"cell_type":"markdown","source":"## 3. Mascot exploration"},{"metadata":{},"cell_type":"markdown","source":"All seeded March Madness teams back from 1985 to 2019 are included in the MNCAATourneySeeds (men) and WNCAATourneySeeds (women) dataframes. Since there are more extensive data on years 2015-2019 included in the dataset, we will concentrate on that specific time period in this analysis.\n\nIt is a well-known fact that for example Duke and Villanova both have superb men's basketball programs, meaning their respective mascots - **Blue Devil** and **Will D. Cat** - are often around when men's March Madness tournament starts. However, based on the five-year period available, we will ask our first actual research question as follows:\n\n***Are there differences between men's and women's team mascot types in years 2015-2019? For example, is it more likely for a wildcat (or a devil) to make the men's tournament than women's March Madness?***\n\nFirst we will select years 2015-2019 from men's March Madness seeding dataframe, resulting in men's March Madness teams from that specific time period."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load dataframe\ndf_mseed = pd.read_csv('../input/ncaa-data/MNCAATourneySeeds.csv', low_memory = False)\n\n# select rows with season value 2015 or larger (dataframe ends in 2019)\ndf_m1519 = df_mseed.loc[df_mseed['Season'] >= 2015]\n\n# rename TeamID column to fit the mascots MTeamID column\ndf_m1519b = df_m1519.rename(columns = {'TeamID':'MTeamID'}) \n\n# printout\ndf_m1519b.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next we repeat the same with women's tournament seeding:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# load dataframe\ndf_wseed = pd.read_csv('../input/ncaa-data/WNCAATourneySeeds.csv', low_memory = False)\n\n# select rows with season value 2015 or larger (dataframe ends in 2019)\ndf_w1519 = df_wseed.loc[df_wseed['Season'] >= 2015]\n\n# rename TeamID column to fit the mascots MTeamID column\ndf_w1519b = df_w1519.rename(columns = {'TeamID':'WTeamID'}) \n\n# printout\ndf_w1519b.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Below the men's March Madness teams 2015-2019 are merged with the respective team mascots. It is notable that in case of exact matching names in the MascotName category, there is a college indicator after the mascot name. For example, the mascot name **Boomer (ok)** in the dataset specifically refers to the wagon-pulling Boomer at University of Oklahoma, excluding other mascots named Boomer. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# merge two dataframes on common rows\n# in SQL the same would be done by using left join\ndf_Mseeded = pd.merge(df_m1519b, df, on='MTeamID', how='left')\n\n# drop unnecessary columns\nunused_cols = ['WTeamID']\ndf_Mseeded = df_Mseeded.drop(unused_cols, axis=1)\n\n# printout\ndf_Mseeded.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As we see below, men's tournament seeds - the 'mascot' category excluded - seem to be most often having either a wildcat or a cartoon-ish human cheering for them. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# set plot parameters\nfig = px.histogram(df_Mseeded, y=\"MascotType\", histnorm = 'percent', orientation = 'h', height = 2000)\nfig.data[0].marker.color = \"orange\"\nfig.data[0].marker.line.width = 1\nfig.data[0].marker.line.color = \"black\"\n\n# set plot title\nfig.update_layout(\n    title={\n        'text': \"Men's March Madness mascots 2015-2019 divided by mascot type\",\n        'y':0.99,\n        'x':0.5,\n        'xanchor': 'center',\n        'yanchor': 'top'})\n\n# show plot\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see quickly if the same applies to women's tournament."},{"metadata":{"trusted":true},"cell_type":"code","source":"#merge two dataframes on common rows\ndf_Wseeded = pd.merge(df_w1519b, df, on='WTeamID', how='left')\n\n# drop unnecessary columns\nunused_cols2 = ['MTeamID']\ndf_Wseeded = df_Wseeded.drop(unused_cols2, axis=1)\n\n# set plot parameters\nfig = px.histogram(df_Wseeded, y=\"MascotType\", histnorm = 'percent', orientation = 'h', height = 2000)\nfig.data[0].marker.color = \"orange\"\nfig.data[0].marker.line.width = 1\nfig.data[0].marker.line.color = \"black\"\n\n# set plot title etc.\nfig.update_layout(\n    title={\n        'text': \"Women's March Madness mascots 2015-2019 divided by mascot type\",\n        'y':0.99,\n        'x':0.5,\n        'xanchor': 'center',\n        'yanchor': 'top'})\n\n# show plot\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In women's March Madness tournament, the wildcat has been replaced by bulldog when overall tournament appearances are concerned. As noted before, in relatively small datasets like this one consistently successful team like Villanova can make a difference: without Villanova wildcat would a minor representative in the mascot popularity column. Let's see if this is a factor here by listing all women's games Villanova played in 2015-2019 March Madness tournaments:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# select all rows with value 'Villanova' in TeamName column\ndf_Wseeded.loc[df_Wseeded['TeamName'] == 'Villanova']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As we can see, in the last five years Villanova mascot has made only one appearance in women's March Madness. The team was a nine-seed in their bracket in 2018, but in the end they were one and done. Just for curiosity, we will load the 2018 women's tournament dataset and  see who was the winner of that particular game. Since we know now that Villanova only appeared in the 2018 tournament once, this should be possible even without specific Game ID number - something that is distinctively missing from NCAA dataset.\n\nInstead of Game ID, we search the 2018 women's March Madness tournament results by using the Villanova team ID. *It is notable that **in the results dataset the WTeamID column includes winning team's ID, not women's team ID as in MarchMascot dataframe**. Since we are not about to merge these dataframes, changing the ID numbers is not necessary.*\n\nAlso, since we know Villanova lost the one game they played, we search only the column LTeamID, which consists of losing team ID numbers. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# load dataframe\ndf_Wresults = pd.read_csv('../input/ncaa-data/WNCAATourneyCompactResults.csv', low_memory = False)\n\n# select rows with season value 2018\ndf_Wresults2018 = df_Wresults.loc[df_Wresults['Season'] == 2018]\n\n# select all rows with Villanova women's team ID (3437) in the LTeamID column\ndf_Wresults2018.loc[df_Wresults2018['LTeamID'] == 3437]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we know that in 2018 March Madness tournament, Villanova lost to a team with team ID number 3323. Finally, let's go back to our MarchMascots dataframe to unveil the winning team and its mascot."},{"metadata":{"trusted":true},"cell_type":"code","source":"# from MarchMascots dataframe select row or rows with women's team ID 3323\ndf.loc[df['WTeamID'] == 3323]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It appears that the Fighting Irish **Leprechaun** of Notre Dame was too much for Villanova's wildcat mascot that year.\n\n***This also gives us a new research question.***\n\n***Are there any differences in how specific mascots and mascot types have fared in March Madness tournaments in the last five years?***  "},{"metadata":{},"cell_type":"markdown","source":"## 4. Mascot Success in March Madness"},{"metadata":{},"cell_type":"markdown","source":"Since the Notre Dame mascot came up, let's first run a quick search on how many times the Leprechaun has appeared in women's March Madness tournament in the last five years:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# select rows with season value 2015 up to latest year in dataset (2019)\ndf_Wresults2015_ = df_Wresults.loc[df_Wresults['Season'] >= 2015]\n\n# select all rows where either WTeamID or LTeamID is Notre Dame women's team ID (3323) \ndf_Wresults2015_ =  df_Wresults2015_[np.logical_or(df_Wresults2015_['WTeamID'] == 3323, df_Wresults2015_['LTeamID'] == 3323)]\n\n# show results\ndf_Wresults2015_","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Women's March Madness tournament consists of 64 teams and six rounds:\n\n- **Round of 64** with all teams playing.\n- **Round of 32** with winning teams from the first round.\n- **Sweet Sixteen** has sixteen teams remaining.\n- **Elite Eight** decides the semi-finalists.\n- **Final Four** rewards the winners with a place in the final.\n- **March Madness Final** crowns the tournament champions.\n\nNot only we see that Notre Dame women's team is indeed a regular in March Madness, we also find out that in 2018 Notre Dame played six games and their last game ID is in the WTeamID column. *This means that in 2018 Notre Dame won women's March Madness tournament.* We can also see that in 2019, Notre Dame reached the final again but lost in a heartbreaker by one point. Going back to 2015, we find there's another final loss.\n\nAll this is next transferred into more visual form below."},{"metadata":{"trusted":true},"cell_type":"code","source":"# import modules\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\n\n# set plot size etc.\nsns.set(rc={'figure.figsize':(9.7,8.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(x = 'Season',\n              data = df_Wresults2015_,\n              order = df_Wresults2015_['Season'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Notre Dame women's team in March Madness 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Year\",fontsize=18)\nplot.set_ylabel(\"Number of games played per year\",fontsize=18)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since we have March Madness reaults from longer time period, we can quickly check if the Notre Dame success has come of late or whether it is a longstanding phenomenon. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# select all rows where either WTeamID or LTeamID is Notre Dame women's team ID (3323) \ndf_Wresults =  df_Wresults[np.logical_or(df_Wresults['WTeamID'] == 3323, df_Wresults['LTeamID'] == 3323)]\n\n# set plot size etc.\nsns.set(rc={'figure.figsize':(16.7,8.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(x = 'Season',\n              data = df_Wresults,\n              order = df_Wresults['Season'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Notre Dame women's team in March Madness 1998-2019\",fontsize=20)\nplot.set_xlabel(\"Year\",fontsize=18)\nplot.set_ylabel(\"Number of games played per year\",fontsize=18)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see that six of seven Notre Dame final spots (i.e. the team played in all six rounds) in March Madness tournament have taken place in 2010s. Thus we can conclude that there has indeed been some positive development going on in Notre Dame in the last decade or so.\n\nNow let's see who else has played well in women's March Madness tournament in the last five years. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# load data\ndf_Wfinalists = pd.read_csv('../input/ncaa-data/WNCAATourneyCompactResults.csv', low_memory = False)\n\n#select rows with season value 2015 up to latest year in dataset (2019)\ndf_Wfinalists2015_ = df_Wfinalists.loc[df_Wfinalists['Season'] >= 2015]\n\n# list teams with most frequently show up in WTeamID column\ndf_Wfinalists2015_['WTeamID'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Keeping in that the women's team ID 3323 means Notre Dame, we can see that they are the second most successful March Madness team in the last five years, when game appearances are concerned. Let's have the same information in visual form. "},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\n# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,20.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'WTeamID',\n              data = df_Wfinalists2015_,\n              order = df_Wfinalists2015_['WTeamID'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Number of March Madness games played by women's teams 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of games\",fontsize=18)\nplot.set_ylabel(\"Team ID\",fontsize=16)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"What we can see is that only eight teams have played a total of ten or more March Madness games in the last five years. We already know that Notre Dame is one of them, so let's find out the whole bunch with their respective mascots. \n\nNext the data above will be merged with the mascots dataframe. *As one might recall, **both datasets have a column called WTeamID, but it means two different things**. In March Madness dataset WTeamID refers to winning teams, but in our MarchMascots dataframe the same column name refers to women's team ID.*\n\nHowever, since we are about to compare only winning women's teams, we can actually merge these two datasets successfully using the WTeamID column name as such. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# merge dataframes\ndf_BestWomen = pd.merge(df_Wfinalists2015_, df, on='WTeamID', how='left')\n\n# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,22.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'MascotName',\n              data = df_BestWomen,\n              order = df_BestWomen['MascotName'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Mascots with most appearances in women's March Madness tournament 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of appearances\",fontsize=18)\nplot.set_ylabel(\"Mascot name\",fontsize=16)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As noted, some colleges have more than one mascot in games. Therefore - before merging tournament data with mascot data - we will pick only the first mascot instance of each college. This can be done because we are looking for unique colleges - not unique mascots - here."},{"metadata":{"trusted":true},"cell_type":"code","source":"# get only the first count of a row value in a column\ndf_Wuniques = df.drop_duplicates(subset='College', keep=\"first\")\n\n# merge dataframes\ndf_BestWomen = pd.merge(df_Wfinalists2015_, df_Wuniques, on='WTeamID', how='left')\n\n# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,22.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'College',\n              data = df_BestWomen,\n              order = df_BestWomen['College'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Games per college in women's March Madness tournament 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of games\",fontsize=18)\nplot.set_ylabel(\"College\",fontsize=16)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 5. Team Selection and Data Leak"},{"metadata":{},"cell_type":"markdown","source":"Getting into March Madness tournament is not quite straightforward. More information about the actual selection process can be found on the official NCAA March Madness website ([link](https://www.ncaa.com/news/basketball-men/article/2018-10-19/how-field-68-teams-picked-march-madness))\n\n**There has been some discussion on Kaggle forums concerning data leak** within the March Madness dataset. I am far from being an expert in this subject, but one thing that came to my mind was the way March Madness is put together as an event. The key here is that **regular season success does not directly correlate with March Madness seeding.** After regular season ends, there is the conference championship tournament phase followed by Selection Sunday (a committee meeting). As such, **Selection Sunday is subjective process**, although based on team success and performance data. Therefore this whole scheme may create issues when data science and deep learning algorithms are concerned, because the subjective seeding process lies outside of the data used by ML models.\n\nFor fans, the most obvious difference between women's and men's March Madness is the number of teams. The basic format of men's tourmament is same as in women's March Madness:\n\n- **Round of 64** with all teams playing.\n- **Round of 32** with winning teams from the first round.\n- **Sweet Sixteen** has sixteen teams remaining.\n- **Elite Eight** decides the semi-finalists.\n- **Final Four** rewards the winners with a place in the final.\n- **March Madness Final** crowns the tournament champions.\n\nIn addition to this, in men's tournament the Round of 64 is preceded by **First Four**. As Wikipedia ([link](https://en.wikipedia.org/wiki/First_Four)) has it, **First Four games are \"contested between teams holding the four lowest seeded automatic bids and the four lowest seeded at-large bids\"**. In practice this means that **there are eight teams (four 11th seed teams as well as four 16th seed teams) playing in the First Four**.\n\n**As the First Four games are - at least in some cases - included in the NCAA dataset results dataframes, they may actually become another potential data leak source**. For example, if a team wins its First Four game, in merged tournament datasets - or in model train data - this would look like there were a 16th seed team with one win. This would falsely suggest that the 16th seed team actually won its match against the 1st seeded team on the Round of 64.\n\nAlso - because of different conferences and their automatic qualifiers - **in the March Madness Round of 64 seeding dataframe there are always duplicates holding the same seed (11th and 16th seed)**. This issue will be further discussed later, when we search for previous upset teams in March Madness tournament. \n\nAs we did earlier with women's tournament, let's first take a closer look at men's March Madness data from the last five years."},{"metadata":{},"cell_type":"markdown","source":"## 6. Mascot Types in March Madness"},{"metadata":{"trusted":true},"cell_type":"code","source":"# load dataframe\ndf_Mresults = pd.read_csv('../input/ncaa-data/MNCAATourneyCompactResults.csv', low_memory = False)\n\n# select rows with season value 2015 or larger\ndf_m1519 = df_mseed.loc[df_mseed['Season'] >= 2015]\n\n# select rows with season value 2015 up to latest year\ndf_Mresults2015_ = df_Mresults.loc[df_Mresults['Season'] >= 2015]\n\n# list teams with most frequently show up in WTeamID column\ndf_Mresults2015_['WTeamID'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"At first sight, the group consisting of teams with most March Madness appearances seems to be a bit more even than in women's tournament. Let's see how the visuals turn out."},{"metadata":{"trusted":true},"cell_type":"code","source":"# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,26.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'WTeamID',\n              data = df_Mresults2015_,\n              order = df_Mresults2015_['WTeamID'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Number of March Madness games played by men's teams 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of games\",fontsize=18)\nplot.set_ylabel(\"Team ID\",fontsize=16)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Before merging the men's data with mascot data, we must adjust the column names to fit each other. We do that by changing the WTeamID (winning team ID) in men's tournament data to MTeamID, which means men's team ID in the mascot dataframe."},{"metadata":{"trusted":true},"cell_type":"code","source":"# rename TeamID column to fit the mascots dataframe MTeamID column\ndf_Mwinners = df_Mresults2015_.rename(columns = {'WTeamID':'MTeamID'}) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can now bring in the mascot data without problems."},{"metadata":{"trusted":true},"cell_type":"code","source":"# merge dataframes\ndf_BestMen = pd.merge(df_Mwinners, df, on='MTeamID', how='left')\n\n# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,32.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'MascotName',\n              data = df_BestMen,\n              order = df_BestMen['MascotName'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Mascots with most appearances in men's March Madness tournament 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of appearances\",fontsize=18)\nplot.set_ylabel(\"Mascot name\",fontsize=16)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Below we can see the mascot home colleges. Again one must keep in mind that for example Oklahoma has three different mascots, which would raise their relative ranking with a sum of those three appearances. Therefore, before merging dataframes, we will pick only the first instances of each college in the MarchMascot dataframe."},{"metadata":{"trusted":true},"cell_type":"code","source":"# get only the first count of a row value in a column\ndf_Muniques = df.drop_duplicates(subset='College', keep='first')\n\n# merge dataframes\ndf_BestMen_ = pd.merge(df_Mwinners, df_Muniques, on='MTeamID', how='left')\n\n# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,22.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'College',\n              data = df_BestMen_,\n              order = df_BestMen_['College'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Games per college in men's March Madness tournament 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of games\",fontsize=18)\nplot.set_ylabel(\"College\",fontsize=16)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,20.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'MascotType',\n              data = df_BestMen_,\n              order = df_BestMen_['MascotType'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Games per mascot type in men's March Madness tournament 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of games\",fontsize=18)\nplot.set_ylabel(\"Mascot type\",fontsize=16)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When we take a look at the percentages, we see that human mascots form about 17 percent of all men's March Madness mascots in years 2015-2019."},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the values in MascotType column by percentage\n(df_BestMen_['MascotType'].value_counts()/df_BestMen_['MascotType'].count())*100","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,20.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'MascotType',\n              data = df_BestWomen,\n              order = df_BestWomen['MascotType'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Games per mascot type in women's March Madness tournament 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of games\",fontsize=18)\nplot.set_ylabel(\"Mascot type\",fontsize=16)\nplot.tick_params(labelsize=14)\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In percentages the result on women's March Madness mascot types looks the following:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the values in MascotType column by percentage\n(df_BestWomen['MascotType'].value_counts()/df_BestWomen['MascotType'].count())*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Human mascots still rule also in women's tournament, but husky - practically invisible in men's March Madness - is gaining ground compared to men's mascots. This is of course almost solely because of **Jonathan**, the husky mascot of University of Connecticut and the success of their team in women's March Madness over the last five years.\n\nOne further research question becomes obvious when looking at the mascot type graphs.\n\n***Could it be possible to for example hold human mascots as one class along with some other mascot classes (bird, dog etc.) so that the outcome would cover most mascot types? In other words, can we create even more specific and descriptive subcategories for mascots than the ones we use now?***\n\nLet's print out all unique mascot types included in MarchMascot dataframe."},{"metadata":{},"cell_type":"markdown","source":"## 7. Further Mascot Classifying"},{"metadata":{"trusted":true},"cell_type":"code","source":"# printout\nprint (df.MascotType.unique())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- **eagle**\n- **cardinal**\n- **falcon** \n- **bird** as generic mascot type\n- **rooster** although some animal scientist might disagree\n- **burrowing owl** \n- **owl**\n- **bald eagle**\n- **red bird** \n- **mountain hawk**\n- **hawk**\n- **thunderbird** \n- **pelican** \n- **mockingbird**\n- **blue hen**\n- **osprey** \n- **duck** \n- **commodore**\n- **emperor penguin**, again a subject of debate class-wise\n\nNext we will create a new column ClassBird in the MarchMascots dataframe. The new column will have a value 0, if the mascot does not belong to ClassBird category. Conversely, the column value is 1 if the mascot is included in the list we just made above."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load dataframe\ndf_classes = pd.read_csv(\"../input/march-madness-mascots/MarchMascots.csv\", low_memory = False)\n\n# drop unnecessary id columna (not necessary but clarifies the dataset)\nidcolumns = ['rowid']\ndf_classes = df_classes.drop(idcolumns, axis=1)\n\n# create a new column with value 0 or 1 depending on whether mascot is a bird\ndf_classes['ClassBird'] = df_classes.MascotType.apply(lambda x: 1 if x in [\n    'eagle',\n    'cardinal',\n    'falcon',\n    'bird',\n    'rooster',\n    'burrowing owl',\n    'owl',\n    'bald eagle',\n    'red bird',\n    'mountain hawk',\n    'hawk',\n    'thunderbird',\n    'pelican',\n    'mockingbird',\n    'blue hen',\n    'osprey',\n    'duck',\n    'commodore',\n    'emperor penguin'\n] else 0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If we print a couple of first rows, we can see the new column in the right."},{"metadata":{"trusted":true},"cell_type":"code","source":"# printout\ndf_classes.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's find out how many mascots in our dataframe fits into ClassBirds category. The line of code below show the percentage of ones and zeros in ClassBirds column."},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the values in ClassBird column by percentage\n(df_classes['ClassBird'].value_counts()/df_classes['ClassBird'].count())*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Exactly 20 percent of all mascots are birds, as we can see. Let's use the same process for human mascots, already identified in the dataframe."},{"metadata":{"trusted":true},"cell_type":"code","source":"# create a new column with value 0 or 1 depending on whether mascot is a human\ndf_classes['ClassHuman'] = df_classes.MascotType.apply(lambda x: 2 if x in [\n    'human',\n] else 0)\n\n# count the values in ClassHuman column by percentage\n(df_classes['ClassHuman'].value_counts()/df_classes['ClassHuman'].count())*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"A bit under 17 percent of all mascots are humans. Next we list all mascot types related to wildcats."},{"metadata":{"trusted":true},"cell_type":"code","source":"# create a new column with value 0 or 1 depending on whether mascot is a wildcat\ndf_classes['ClassWildcat'] = df_classes.MascotType.apply(lambda x: 3 if x in [\n    'wildcat',\n    'tiger',\n    'cougar',\n    'lion',\n    'jaguar',\n    'bobcat',\n    'tiger mascot'  \n] else 0)\n\n# count the values in ClassWildcat column by percentage\n(df_classes['ClassWildcat'].value_counts()/df_classes['ClassWildcat'].count())*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Wildcats cover some 14 percent of all mascots. Altogether birds, human mascots and wildcats cover about half of March Madness mascots. We will now add a class for dogs."},{"metadata":{"trusted":true},"cell_type":"code","source":"# create a new column with value 0 or 1 depending on whether mascot is a bird\ndf_classes['ClassDog'] = df_classes.MascotType.apply(lambda x: 4 if x in [\n    'dog',\n    'terrier',\n    'bulldog',\n    'husky',\n    'greyhound',\n    'Saint Bernard dog',\n    'English bulldog',\n    'hound'\n] else 0)\n\n# count the values in ClassHuman column by percentage\n(df_classes['ClassDog'].value_counts()/df_classes['ClassDog'].count())*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Our four categories cover roughly two thirds of all mascots. In lambda functions it is common to use values 0 and 1, but in this case there was a point for applying values from 1-4. If the new classes are merged into one column AllClasses, we can do that just by adding the class columns together. In the new AllClasses column, the value 0 means any other mascot types than the four subtypes we just defined. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# create column AllClasses from mascot class columns\ndf_classes[\"AllClasses\"] = (df_classes[\"ClassBird\"] + df_classes[\"ClassHuman\"] + df_classes[\"ClassDog\"] + df_classes[\"ClassWildcat\"]).astype(\"int\")\n\n# print first rows\ndf_classes.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's check once more the mascot type percentages, this time using the AllClasses column."},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the values in AllClasses column by percentage\n(df_classes['AllClasses'].value_counts()/df_classes['AllClasses'].count())*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Earlier we compiled a dataframe of women's March Madness tournament in years 2015-2019. Let's use that data and merge it with the mascot type dataframe we just created."},{"metadata":{"trusted":true},"cell_type":"code","source":"# merge dataframes\ndf_ClassWomen = pd.merge(df_Wfinalists2015_, df_classes, on='WTeamID', how='left')\n\n# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,7.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'AllClasses',\n              data = df_ClassWomen,\n              order = df_ClassWomen['AllClasses'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Games per mascot type in women's March Madness tournament 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of games\",fontsize=18)\nplot.set_ylabel(\"Mascot type\",fontsize=16)\nplot.tick_params(labelsize=14)\nplot.legend (loc=4, fontsize = 16, fancybox=True, framealpha=1, shadow=True, borderpad=1, \n             title = '0=other, 1=bird, 2=human, 3=wildcat, 4=dog')\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see the percentages next:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the values in AllClasses column by percentage\n(df_ClassWomen['AllClasses'].value_counts()/df_ClassWomen['AllClasses'].count())*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In March Madness mascot dataframe, dog mascots (class 4) formed eleven percent of all mascots. In women's tournament data however, 20 percent of mascots in the tournament are dogs. Let's see if men's tournament looks similar."},{"metadata":{"trusted":true},"cell_type":"code","source":"# merge dataframes\ndf_ClassMen = pd.merge(df_Mwinners, df_classes, on='MTeamID', how='left')\n\n# set plot size etc.\nsns.set(rc={'figure.figsize':(14.7,9.27)})\nsns.set(font='sans-serif', palette='colorblind')\n\n# set plot parameters\nplot = sns.countplot(y = 'AllClasses',\n              data = df_ClassMen,\n              order = df_ClassMen['AllClasses'].value_counts().index)\n\n# set plot title etc.\nplot.axes.set_title(\"Games per mascot type in men's March Madness tournament 2015-2019\",fontsize=20)\nplot.set_xlabel(\"Number of games\",fontsize=18)\nplot.set_ylabel(\"Mascot type\",fontsize=16)\nplot.tick_params(labelsize=14)\nplot.legend (loc=4, fontsize = 16, fancybox=True, framealpha=1, shadow=True, borderpad=1, \n             title = '0=other, 1=bird, 2=human, 3=wildcat, 4=dog')\n\n# show plot\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# count the values in AllClasses column by percentage\n(df_ClassMen['AllClasses'].value_counts()/df_ClassMen['AllClasses'].count())*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Men's March Madness mascots fall pretty well into the overall division. One small difference is that the wildcat category (class 3) now has a bit more values compared to women's tournament.\n\nAt this point, we already know quite a bit about the big hitters. Next we will take a quick peek at the teams beating the odds in March Madness tournament. \n\n"},{"metadata":{},"cell_type":"markdown","source":"## 8. Mascots and Upsets"},{"metadata":{},"cell_type":"markdown","source":"Earlier we already created a dataframe including all men's March Madness seedings from 2015-2019. Below is a small printout of the dataframe as a refresher. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# printout\ndf_m1519b.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"March Madness tournament consists of 4x16 teams each seeded by number 1-16 (*men's First Four games are played prior to Round of 64*). This means that sixteen of the teams considered weakest in the bracket of 64 have seedings 13-16.\n\nWe can find those teams by searching the Seed column for values 13-16. We also know that the four seed groups in bracket are separated from each other by letters W,X,Y and Z. \n\nHere we are only interested in weaker teams playing more than once in March Madness tournament. There are also lower caps letters 'a' and 'b' included in the Seed column, so we are to remove them too. Next we eliminate all letters in the Seed column. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# remove letters from Seeds column\ndf_m1519c = df_m1519b.replace(regex=['W','X','Y','Z','a','b'], value=' ')\n\n# make Seeds column numeric\ndf_m1519c['Seed'] = pd.to_numeric(df_m1519c['Seed'])\n\n# check column datatypes\ndf_m1519c.dtypes","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's check the first few rows:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# rename MTeamID column\ndf_m1519c = df_m1519c.rename(columns = {'MTeamID':'TeamID'}) \n\n# printout\ndf_m1519c.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next we will load the MTeams dataframe including all the names of Men's March Madness teams."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load men's teams dataframe\ndf_MTeams = pd.read_csv('../input/ncaa-data/MTeams.csv', low_memory = False)\n\n# drop unnecessary columna\ndropcolumns = ['FirstD1Season', 'LastD1Season']\ndf_MTeams = df_MTeams.drop(dropcolumns, axis=1)\n\n# printout\ndf_MTeams.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We combine the team names with seeds into a new dataframe."},{"metadata":{"trusted":true},"cell_type":"code","source":"# merge dataframes\ndf_SeedNames = pd.merge(df_m1519c, df_MTeams, on='TeamID', how='left')\n\n# printout\ndf_SeedNames.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To make things more clear, we first select seeds 13-16 as well as only the March Madness tournament 2019."},{"metadata":{"trusted":true},"cell_type":"code","source":"# select only seeds 13-16\ndf_LowMen = df_SeedNames[(df_SeedNames['Seed']>=13)]\n\n# select rows with season value 2019\ndf_LowMen19 = df_LowMen.loc[df_LowMen['Season'] == 2019]\n\n# printout\ndf_LowMen19.head(8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next we will load the results dataframe. In this case we will only use the 2019 results. Also it is good to keep in mind that at least some of the results from the First Four round are included in the original dataset.\n\nWhen the losing team ID (LTeamID) in the dataset is renamed as TeamID, we will next have the option to combine results dataframe with the seeding dataframe."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load dataframe\ndf_Mfinalists = pd.read_csv('../input/ncaa-data/MNCAATourneyCompactResults.csv', low_memory = False)\n\n# select rows with season value 2019\ndf_Mfinalists2019 = df_Mfinalists.loc[df_Mfinalists['Season'] == 2019]\n\n# drop unnecessary columna\ndropcolumns = ['WTeamID','WLoc', 'NumOT']\ndf_Mfinalists2019 = df_Mfinalists2019.drop(dropcolumns, axis=1)\n\n# rename LTeamID column\ndf_Mfinalists2019 = df_Mfinalists2019.rename(columns = {'LTeamID':'TeamID'}) \n\n# printout\ndf_Mfinalists2019.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We will now merge the seeding dataframe with results dataframe. The required labels (TeamID) are now teams that appear in the losing team column (the one that was named LTeamID). "},{"metadata":{"trusted":true},"cell_type":"code","source":"# merge the dataframes\ndf_LowLosses = pd.merge(df_LowMen19, df_Mfinalists2019, on='TeamID', how='left')\n\n# drop unnecessary columna\ndropcolumns = ['Season_y', 'DayNum']\ndf_LowLosses = df_LowLosses.drop(dropcolumns, axis=1)\n\n# rename Season_x column\ndf_LowLosses =  df_LowLosses.rename(columns = {'Season_x':'Season'}) \n\n# drop columns with no value\ndf_LowLosses = df_LowLosses.dropna()\n\n# printout\ndf_LowLosses","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next will we do the opposite. We will look for 13-16 seeds that won a game in 2019 March Madness."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load dataframe\ndf_Mfinalists = pd.read_csv('../input/ncaa-data/MNCAATourneyCompactResults.csv', low_memory = False)\n\n#select rows with season value 2019\ndf_Mfinalists2019 = df_Mfinalists.loc[df_Mfinalists['Season'] == 2019]\n\n# drop unnecessary columna\ndropcolumns = ['LTeamID','WLoc', 'NumOT']\ndf_Mfinalists2019 = df_Mfinalists2019.drop(dropcolumns, axis=1)\n\n# rename WTeamID column\ndf_Mfinalists2019 =  df_Mfinalists2019.rename(columns = {'WTeamID':'TeamID'}) \n\n# merge the dataframes\ndf_LowWins = pd.merge(df_LowMen19, df_Mfinalists2019, on='TeamID', how='left')\n\n# drop unnecessary columna\ndropcolumns = ['Season_y', 'DayNum']\ndf_LowWins = df_LowWins.drop(dropcolumns, axis=1)\n\n# rename Season_x column\ndf_LowWins =  df_LowWins.rename(columns = {'Season_x':'Season'}) \n\n# drop columns with no value\ndf_LowWins = df_LowWins.dropna()\n\n# printout\ndf_LowWins","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Because the First Four round games are - at least partially - included in the original dataset, they show up also in our results.**\n\nWe could prove this by for example searching for the number of games by each 1st seeded team in 2019. None of the teams played only one game, so it is not possible for both 1st and 16th seed teams to appear as winners in the same game. Here the obvious flaw in the original NCAA March Madness dataset - not having unique ID tags for each game included in the dataset - makes things a bit hard to comprehend at first sight.\n\nFor example, **in the above table North Dakota State is listed as a 16th seed winner. This is because the team won its First Four game against North Carolina Central 78-74**. In the Round of 64 however, North Dakota State lost to 1st seeded Duke 85-62. The same applies to Farleigh Dickinson showing up as a winning 16th seeded team (they won First Four game against Prairie View A&M 82-76 but lost to 1st seeded Gonzaga 87-49 in the Round of 64).\n\nThus, **when the March Madness games from Round of 64 onwards are concerned, the only 13-16 seed team to win a game in 2019 men's March Madness was the 13th seed UC Irvine (they upset the 4th seeded Kansas State 70-64)**."},{"metadata":{},"cell_type":"markdown","source":"Let's use the same method for tournaments played in 2015-2018. We saw that the winning teams' column is much shorter and easier to read, so this time we are only searching for winners."},{"metadata":{"trusted":true},"cell_type":"code","source":"# select only seeds 13-16\ndf_LowMen2 = df_SeedNames[(df_SeedNames['Seed']>=13)]\n\n# select seasons 2015-2018\ndf_Low1518 = df_LowMen2[(df_LowMen2['Season']>=2015) & (df_LowMen2['Season']<=2018)]\n\n# printout\ndf_Low1518.tail(8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Above we have the last rows of teams seeded 13-16 from years 2015-2018.\n\nBelow we will create a dataframe of 2015-2018 men's teams with winning team's ID converted as TeamID."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load dataframe\ndf_Mfinalists = pd.read_csv('../input/ncaa-data/MNCAATourneyCompactResults.csv', low_memory = False)\n\n# select seasons 2015-2019\ndf_Mfinalists1518 = df_Mfinalists[(df_Mfinalists['Season']>=2015) & (df_Mfinalists['Season']<=2018)]\n\n# drop unnecessary columna\ndropcolumns = ['LTeamID','WLoc', 'NumOT']\ndf_Mfinalists1518 = df_Mfinalists1518.drop(dropcolumns, axis=1)\n\n# rename WTeamID column\ndf_Mfinalists1518 =  df_Mfinalists1518.rename(columns = {'WTeamID':'TeamID'}) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Below we merge the 2015-2018 results with respective seeds. \n\n*It is notable that - because of multiple seasons - we now use two different parameters (Season, TeamID) in merging the two dataframes.*\n\nOtherwise we might end up with the same result score of a particular game appearing in two different seasons in cases where the same team would have made it to March Madness in different years."},{"metadata":{"trusted":true},"cell_type":"code","source":"# merge the dataframes\ndf_LowWins2 = pd.merge(df_Low1518, df_Mfinalists1518,  how='left', left_on=['Season','TeamID'], right_on = ['Season','TeamID'])\n\n# drop unnecessary columna\ndropcolumns = ['DayNum']\ndf_LowWins2 = df_LowWins2.drop(dropcolumns, axis=1)\n\n# drop columns with no value\ndf_LowWins2 = df_LowWins2.dropna()\n\n# make column values integers\ndf_LowWins2[['WScore', 'LScore']] = df_LowWins2[['WScore', 'LScore']].astype(int)\n\n# printout\ndf_LowWins2","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Again, a quick glance shows that no teams with seed 16 have two wins in the same year in our results. This means the 16th seed wins we see here all come from the First Four games.\n\nFor clarity, next the same dataframe will be printed without the 16th seed teams."},{"metadata":{"trusted":true},"cell_type":"code","source":"# select seeds 13-15\ndf_LowWins2 = df_LowWins2[(df_LowWins2['Seed']>=13) & (df_LowWins2['Seed']<=15)]\n\n# printout\ndf_LowWins2","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As seen above, **in four seasons (2015-2018) seeds 13-15 have won a total of seven games in men's Round of 64.** For someone filling a March Madness bracket, predicting such upset would definitely be betting against the odds.\n\nLet's finally check out what kind of mascots were included in these few upsets. This time we throw in the 2019 season too, since we found out earlier that UC Irvine was among the winning low-seeded teams with their 2019 March Madness performance."},{"metadata":{"trusted":true},"cell_type":"code","source":"# select seasons 2015-2019\ndf_Low1519 = df_LowMen2[(df_LowMen2['Season']>=2015) & (df_LowMen2['Season']<=2019)]\n\n# load dataframe\ndf_Mfinalists = pd.read_csv('../input/ncaa-data/MNCAATourneyCompactResults.csv', low_memory = False)\n\n# select seasons 2015-2019\ndf_Mfinalists1519 = df_Mfinalists[(df_Mfinalists['Season']>=2015) & (df_Mfinalists['Season']<=2019)]\n\n# drop unnecessary columna\ndropcolumns = ['LTeamID','WLoc', 'NumOT']\ndf_Mfinalists1519 = df_Mfinalists1519.drop(dropcolumns, axis=1)\n\n# rename WTeamID column\ndf_Mfinalists1519 =  df_Mfinalists1519.rename(columns = {'WTeamID':'TeamID'}) \n\n# merge the dataframes\ndf_LowWins2 = pd.merge(df_Low1519, df_Mfinalists1519,  how='left', left_on=['Season','TeamID'], right_on = ['Season','TeamID'])\n\n# drop unnecessary columna\ndropcolumns = ['DayNum']\ndf_LowWins2 = df_LowWins2.drop(dropcolumns, axis=1)\n\n# drop columns with no value\ndf_LowWins2 = df_LowWins2.dropna()\n\n# make column values integers\ndf_LowWins2[['WScore', 'LScore']] = df_LowWins2[['WScore', 'LScore']].astype(int)\n\n# select seeds 13-15\ndf_LowWins2 = df_LowWins2[(df_LowWins2['Seed']>=13) & (df_LowWins2['Seed']<=15)]\n\n# rename MTeamID column\ndf_classes = df_classes.rename(columns = {'MTeamID':'TeamID'}) \n\n# merge the dataframes\ndf_UpsetMascots = pd.merge(df_LowWins2, df_classes, on='TeamID', how='left')\n\n# rename TeamName_x column\ndf_UpsetMascots = df_UpsetMascots.rename(columns = {'TeamName_x':'TeamName'}) \n\n# drop unnecessary columna\ndropcolumns = ['WTeamID','TeamName_y']\ndf_UpsetMascots = df_UpsetMascots.drop(dropcolumns, axis=1)\n\n# printout\ndf_UpsetMascots","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"None of the upset teams seeded 13-15 have birds, wildcats or dogs as mascots, and only one of them came in with a human-figure mascot. Therefore - if looking for an upset in men's March Madness bracket - the following statement can be made based on mascots:\n\n- ***Forget the 16th seed teams. They haven't won a Round of 64 game in five years.***\n- ***Only one upset team in five years has been a 15th seed.***\n- ***As for 13th and 14th seed, look for teams with other mascots than human figures, birds, wildcats and dogs.*** \n- ***Since 2016, the successful upset team has always been the 13th seed.*** \n"},{"metadata":{},"cell_type":"markdown","source":"Let's see quickly what is the situation with low seed upsets in women's March Madness."},{"metadata":{"trusted":true},"cell_type":"code","source":"# remove letters from Seeds column\ndf_w1519c = df_w1519b.replace(regex=['W','X','Y','Z','a','b'], value=' ')\n\n# make Seeds column numeric\ndf_w1519c['Seed'] = pd.to_numeric(df_w1519c['Seed'])\n\n# rename TeamID column\ndf_w1519c = df_w1519c.rename(columns = {'WTeamID':'TeamID'}) \n\n# load women's teams dataset\ndf_WTeams = pd.read_csv('../input/ncaa-data/WTeams.csv', low_memory = False)\n\n# merge dataframes\ndf_WSeedNames = pd.merge(df_w1519c, df_WTeams, on='TeamID', how='left')\n\n# select seeds 13-16\ndf_LowWomen = df_WSeedNames[(df_WSeedNames['Seed']>=13)]\n\n#load dataframe\ndf_Wfinalists = pd.read_csv('../input/ncaa-data/WNCAATourneyCompactResults.csv', low_memory = False)\n\n# select seasons 2015-2019\ndf_Wfinalists1519_ = df_Wfinalists[(df_Wfinalists['Season']>=2015) & (df_Wfinalists['Season']<=2019)]\n\n# drop unnecessary columna\ndropcolumns = ['DayNum','LTeamID','WLoc','NumOT']\ndf_Wfinalists1519_ = df_Wfinalists1519_.drop(dropcolumns, axis=1)\n\n# rename TeamID column\ndf_Wfinalists1519_ =  df_Wfinalists1519_.rename(columns = {'WTeamID':'TeamID'}) \n\n# merge the dataframes\ndf_WLowWins2 = pd.merge(df_LowWomen, df_Wfinalists1519_,  how='left', left_on=['Season','TeamID'], right_on = ['Season','TeamID'])\n\n# drop columns with no value\ndf_WLowWins2 = df_WLowWins2.dropna()\n\n# printout\ndf_WLowWins2.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"***The result should be an empty table with column names only.*** \n\n***I first checked the code several times, after which I went online and manually checked all the women's March Madness results from years 2015-2019.***\n\n***The result above is correct. No women's team seeded 13-16 playing in Round of 64 in March Madness has won a single game in years 2015-2019.***"},{"metadata":{},"cell_type":"markdown","source":"Thus our focus must turn to seeds 9-12 in women's March Madness tournaments 2015-2019. However, as the 9th seed plays against the 8th seed and the 10th seed faces the 7th, the winners of those relatively even games cannot really be considered as providing an upset.\n\nFor minor upsets, we look for wins when teams seeded 11th and 12th are considered. The previous code cell above is pretty much applicable as such in this. We only need to change the scope of seeds we are looking for."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load women's teams dataframe\ndf_WTeams = pd.read_csv('../input/ncaa-data/WTeams.csv', low_memory = False)\n\n# merge dataframes\ndf_WSeedNames = pd.merge(df_w1519c, df_WTeams, on='TeamID', how='left')\n\n# select seeds 11-12\ndf_LowWomen2 = df_WSeedNames[(df_WSeedNames['Seed']>=11) & (df_WSeedNames['Seed']<=12)]\n\n# select seasons 2015-2019 for 11th and 12th seeded teams\ndf_WSeeds1519 = df_LowWomen2[(df_LowWomen2['Season']>=2015) & (df_LowWomen2['Season']<=2019)]\n\n# load results dataframe\ndf_Wfinalists = pd.read_csv('../input/ncaa-data/WNCAATourneyCompactResults.csv', low_memory = False)\n\n# select results from seasons 2015-2019\ndf_Wfinalists1519_ = df_Wfinalists[(df_Wfinalists['Season']>=2015) & (df_Wfinalists['Season']<=2019)]\n\n# drop unnecessary columna\ndropcolumns = ['DayNum','LTeamID','WLoc','NumOT']\ndf_Wfinalists1519_ = df_Wfinalists1519_.drop(dropcolumns, axis=1)\n\n# rename TeamID column\ndf_Wfinalists1519_ =  df_Wfinalists1519_.rename(columns = {'WTeamID':'TeamID'}) \n\n# merge the dataframes\ndf_WLowWins3 = pd.merge(df_WSeeds1519, df_Wfinalists1519_,  how='left', left_on=['Season','TeamID'], right_on = ['Season','TeamID'])\n\n# drop columns with no value\ndf_WLowWins3 = df_WLowWins3.dropna()\n\n# printout\ndf_WLowWins3.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see that in 2015 Gonzaga went right up to Sweet Sixteen by winning two games along with Ark Little Rock and Miami winning their Round of 64 games. Next year South Dakota State won their first-rounder as 12th seed, but after that there are no winners for 11th and 12th seed teams. So, if you are building a successful women's March Madness bracket, there is only one advice that can be extracted from the dataset.\n\n- ***When betting on winners, forget all 11th-16th seed teams in women's March Madness bracket. They haven't won a Round of 64 game in three years.***"},{"metadata":{},"cell_type":"markdown","source":"Finally we take a quick look at games where the going gets tough and the tough turn semi-pro: the Elite Eight. We already know that no team taking part in First Four - played prior to Round of 64 - has ever made it to Elite Eight. Therefore teams that do make it start their March Madness campaign in Round of 64, after which they continue to Round of 32 and Sweet Sixteen.\n\nAs college life is with young athletes graduating, March Madness team rosters tend to change every three or four years. Therefore predicting current success from earlier results is not sensible, since basically none of the players taking part in March Madness are the same as only a couple of years ago. \n\n***Next we will find out all Elite Eight teams from men's March Madness 2019 tournament.***"},{"metadata":{},"cell_type":"markdown","source":"## 9. Elite Eight Mascots"},{"metadata":{"trusted":true},"cell_type":"code","source":"# load results dataframe\ndf_Mfinalists = pd.read_csv('../input/ncaa-data/MNCAATourneyCompactResults.csv', low_memory = False)\n\n# select seasons 2015-2019\ndf_Mfinalists2019 = df_Mfinalists[(df_Mfinalists['Season']==2019)]\n\n# drop unnecessary columna\ndropcolumns = ['DayNum', 'WLoc', 'NumOT']\ndf_Mfinalists2019 = df_Mfinalists2019.drop(dropcolumns, axis=1)\n\n# printout of last eight rows\ndf_Mfinalists2019.tail(8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The table above lists the last eight games of the 2019 men's March Madness tournament. This is because the easiest way to comprehend the data we are after is to think March Madness from end to beginning. Then the final is the last game of the tournament preceded by two semi-finals and four teams.\n\nBefore that, the Elite Eight have played four games to decide the semifinalists. Therefore, as the games are arranged in chronological order, all the Elite Eight teams appear in the last seven rows of the table above. The one row above them - in this case row number 2243 - was only printed to show this in practice. The losing team ID on that row (1261) does not appear on any other rows after that, because that game was the last Sweet Sixteen game: the team losing that game was out of 2019 Elite Eight.\n\nIn our case, the Elite Eight teams are on rows 2244-2247, so we select only those rows."},{"metadata":{"trusted":true},"cell_type":"code","source":"# select the four rows mentioned\ndf_ElEight19 = df_Mfinalists2019.loc[2244:2247]\n\n# printout\ndf_ElEight19","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From here we can make lists of winning and losing teams and go on with that."},{"metadata":{"trusted":true},"cell_type":"code","source":"# make lists of team IDs\nEliteTeamsW = df_ElEight19.WTeamID.tolist()\nEliteTeamsL = df_ElEight19.LTeamID.tolist()\n\n# combine the two lists\nEliteTeams19 = EliteTeamsW + EliteTeamsL\n\n# convert list to dataframe\ndf_EliteMen = pd.DataFrame(EliteTeams19,columns=['TeamID'])\n\n# load men's teams dataframe\ndf_MTeams = pd.read_csv('../input/ncaa-data/MTeams.csv', low_memory = False)\n\n# merge dataframes\ndf_EliteMen2019 = pd.merge(df_EliteMen, df_MTeams, on='TeamID', how='left')\n\n# drop unnecessary columna\ndropcolumns = ['FirstD1Season','LastD1Season']\ndf_EliteMen2019 = df_EliteMen2019.drop(dropcolumns, axis=1)\n\n# merge the dataframes\ndf_EliteMascots = pd.merge(df_EliteMen2019, df_classes, on='TeamID', how='left')\n\n# rename TeamName_x column\ndf_EliteMascots = df_EliteMascots.rename(columns = {'TeamName_x':'TeamName'}) \n\n# drop unnecessary columna\ndropcolumns = ['WTeamID','TeamName_y']\ndf_EliteMascots = df_EliteMascots.drop(dropcolumns, axis=1)\n\n# printout\ndf_EliteMascots","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Mascot-wise we see that Purdue has two mascots - a human figure and a locomotive - but otherwise the Elite Eight mascots from 2019 are pretty evenly split across the mascot types (as mentioned, the zero category includes mascot types outside categories 1-4). \n\nLet's quickly see how women's March Madness 2019 looked like."},{"metadata":{"trusted":true},"cell_type":"code","source":"# load results dataframe\ndf_Wfinalists = pd.read_csv('../input/ncaa-data/WNCAATourneyCompactResults.csv', low_memory = False)\n\n# select seasons 2015-2019\ndf_Wfinalists2019 = df_Wfinalists[(df_Wfinalists['Season']==2019)]\n\n# drop unnecessary columna\ndropcolumns = ['DayNum', 'WLoc', 'NumOT']\ndf_Wfinalists2019 = df_Wfinalists2019.drop(dropcolumns, axis=1)\n\n# show last eight rows\ndf_Wfinalists2019.tail(8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The Elite Eight teams are on rows 1379-1382, so we use only those next."},{"metadata":{"trusted":true},"cell_type":"code","source":"# select the four rows mentioned\ndf_WElEight19 = df_Wfinalists2019.loc[1379:1382]\n\n# make lists of team IDs\nWEliteTeamsW = df_WElEight19.WTeamID.tolist()\nWEliteTeamsL = df_WElEight19.LTeamID.tolist()\n\n# combine the two lists\nWEliteTeams19 = WEliteTeamsW + WEliteTeamsL\n\n# convert list to dataframe\ndf_EliteWomen = pd.DataFrame(WEliteTeams19,columns=['TeamID'])\n\n# load men's teams dataframe\ndf_WTeams = pd.read_csv('../input/ncaa-data/WTeams.csv', low_memory = False)\n\n# merge dataframes\ndf_EliteWomen2019 = pd.merge(df_EliteWomen, df_WTeams, on='TeamID', how='left')\n\n# rename team ID column\ndf_EliteWomen2019 = df_EliteWomen2019.rename(columns = {'TeamID':'WTeamID'}) \n\n# merge the dataframes\ndf_WEliteMascots = pd.merge(df_EliteWomen2019, df_classes, on='WTeamID', how='left')\n\n# drop unnecessary columna\ndropcolumns = ['TeamID','TeamName_y']\ndf_WEliteMascots = df_WEliteMascots.drop(dropcolumns, axis=1)\n\n# rename TeamName_x and WTeamID columns\ndf_WEliteMascots = df_WEliteMascots.rename(columns = {'TeamName_x':'TeamName', 'WTeamID':'TeamID'}) \n\n# printout\ndf_WEliteMascots","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In women's 2019 March Madness Elite Eight, Baylor came in with a pair of bear mascots. Wildcats were not included in the best eight teams, and only one team had a human figure mascot. The three bird mascots were definitely an increase compared to one eagle in men's 2019 Elite Eight round. \n\nThe most obvious difference in Elite Eight teams is however the teams themselves, and this could actually be one basic rule when making brackets.\n\n***In 2019, no college made it to Elite Eight in both men's and women's March Madness tournament.***"},{"metadata":{},"cell_type":"markdown","source":"***This concludes the analysis of previous seasons. Next we would have created our predictions for both men's and women's March Madness 2020 tournament based on team mascots. However, as both tournaments were cancelled, this is not possible.***"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.4"}},"nbformat":4,"nbformat_minor":4}