{"cells":[{"metadata":{},"cell_type":"markdown","source":"First of all, I want to apologize for my English, and you will see the reason."},{"metadata":{},"cell_type":"markdown","source":"# EDA(数据探索性分析)\n\nThis notebook is mainly used to make a overview of the data in this analysis competition. If you like this notebook, please upvote it and make more kaggler see this, and if you have any idea, please tell me, I will update here as soon as possible.(本notebook主要用于对该次分析竞赛的大量数据进行简单概览，以及普及一些关于NCAA相关的知识，毕竟对于国内的同学来说，看NCAA的人应该是少数吧，提前声明，本人只看NBA，不看NCAA，所以可能有些信息有误的地方，望大家评论区指出，感谢)；"},{"metadata":{},"cell_type":"markdown","source":"## What is NCAA March Madness(什么是NCAA疯狂三月)\n\nFirst of all, NCAA is the National University Sports Association of the United States. It is an association of hundreds of universities and colleges in the United States. Its main activities are various sports leagues held every year, among which the basketball leagues in the first half of the year and rugby leagues in the second half of the year are the most concerned. March Madness is actually the NCAA tournament. It was founded in 1939 and is one of the largest national sports events in the United States. Because most of the NCAA tournament match are held in March, so we call it March Madness.(首先，NCAA是美国的全国大学体育协会，它是由美国千百所大学院校所参与结盟的一个协会。其主要活动是每年举办的各种体育项目联赛，其中最受关注的是上半年的篮球联赛和下半年的橄榄球联赛。而疯狂三月其实是通称的NCAA锦标赛，即美国大学体育总会(NCAA)一级联赛男篮锦标赛，成立于1939年，是美国最大的全国性体育赛事之一。因为NCAA锦标赛的大部分比赛都在3月进行，俗称疯狂三月)\n\nIn short, March Madness is the time when NCAA Basketball tournament in March every year.(简单说，疯狂三月就是每年三月份，NCAA篮球锦标赛集中进行比赛的这段时间)\n\n### Tournament Rule(赛制规则)\n\n1. The first is the \"First Four\", which consists of the four lowest ranked champion teams in 32 division areas and the four lowest ranked teams in 36 teams selected by NCAA. The winning four teams and the other 60 teams make up the top 64 teams.(首先进行的是“最先四场（First Four）”，由32个分赛区里排名最低的四支冠军队伍和NCAA选拔的36支队伍中排名最低的四支捉对厮杀，胜出的四队和另外的60支队伍组成64强)\n2. 64->32->16->8->4->2->Champion\n\nIn addition, it should be noted that the tournament is **one-round**, so compared with the seven games in the NBA playoffs, it is more likely to have black horses, which also brings More surprises.(另外要注意，比赛是**一轮制**，因此相比较NBA季后赛的7场，更加容易出现黑马，这也带来了更大的观赏性)"},{"metadata":{},"cell_type":"markdown","source":"## Data overview and visualization(数据简单概览及可视化)\n\n### Load lib and set path(加载相关库，设置路径)"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\npd.set_option('display.max_columns', 100)\n\nfrom matplotlib import pyplot as plt\nimport seaborn as sns\n%matplotlib inline\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\nmen_folder_path = '/kaggle/input/march-madness-analytics-2020/2020DataFiles/2020-Mens-Data/'\nwomen_folder_path = '/kaggle/input/march-madness-analytics-2020/2020DataFiles/2020-Womens-Data/'\nstage1_folder_path = 'MDataFiles_Stage1/'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Load section 1 data and merge(加载数据并merge到一个DataFrame)"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def load_section1():\n    folder_path = men_folder_path\n    prefix = 'M'\n    team_section1 = pd.read_csv(folder_path+stage1_folder_path+prefix+'Teams.csv')\n    season_section1 = pd.read_csv(folder_path+stage1_folder_path+prefix+'Seasons.csv')\n    seed_section1 = pd.read_csv(folder_path+stage1_folder_path+prefix+'NCAATourneySeeds.csv')\n    regular_section1 = pd.read_csv(folder_path+stage1_folder_path+prefix+'RegularSeasonCompactResults.csv')\n    nacc_section1 = pd.read_csv(folder_path+stage1_folder_path+prefix+'NCAATourneyCompactResults.csv')\n    regular_section1['is_regular'] = 1\n    nacc_section1['is_regular'] = 0\n    regular_detail_section1 = pd.read_csv(folder_path+stage1_folder_path+prefix+'RegularSeasonDetailedResults.csv')\n    nacc_detail_section1 = pd.read_csv(folder_path+stage1_folder_path+prefix+'NCAATourneyDetailedResults.csv')\n    regular_section1 = regular_section1.merge(regular_detail_section1)\n    nacc_section1 = nacc_section1.merge(nacc_detail_section1)\n    \n    section1 = regular_section1.append(nacc_section1, ignore_index=True).sort_values(by=['Season','DayNum'])\n    section1 = section1.merge(seed_section1,left_on=['Season','WTeamID'],right_on=['Season','TeamID']).drop('TeamID',axis=1)\n    section1 = section1.merge(seed_section1,left_on=['Season','LTeamID'],right_on=['Season','TeamID'],suffixes=('_W', '_L')).drop('TeamID',axis=1)\n    section1 = section1.merge(season_section1,left_on=['Season'],right_on=['Season'])\n    section1 = section1.merge(team_section1,left_on=['WTeamID'],right_on=['TeamID']).drop('TeamID',axis=1)\n    section1 = section1.merge(team_section1,left_on=['LTeamID'],right_on=['TeamID'],suffixes=('_W', '_L')).drop('TeamID',axis=1)\n    section1 = section1.sort_values(by=['Season','DayNum'])\n    \n    return section1\n\nmSection1 = load_section1()\nmSection1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Count of Champion and second(冠军、亚军的次数可视化)"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"final_round = mSection1.query('DayNum == 154')\n\nplt.subplots(figsize=(20, 8))\nplt.subplots_adjust(wspace=0, hspace=.6)\n\nplt.subplot(2,1,1)\nplt.xticks(rotation=30)\nsns.countplot(x=\"TeamName_W\", data=final_round)\n\nplt.subplot(2,1,2)\nplt.xticks(rotation=30)\nsns.countplot(x=\"TeamName_L\", data=final_round)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see(可以看到)：\n- Duke won the most Championships, 5 times in total, go Zion. Followed by North Carolina and Connecticut, both 4 times.(冠军数最多的是杜克大学，共获得5次锦标赛冠军，其次是北卡和康涅狄格州，分别获得4次冠军)\n- Duke and Michigan were the top runners up, four times, followed by Kansas.(亚军数最多的是杜克和密歇根大学，均为4次，其次是堪萨斯的3次)"},{"metadata":{},"cell_type":"markdown","source":"### Top 10 in 32,16,8,4,2(进入32,16,8,4,2强次数前10的球队)"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"team32 = mSection1.query('DayNum == 136 or DayNum == 137').groupby('TeamName_W')['Season'].count().rename('Count32').reset_index().sort_values(by='Count32',ascending=False).iloc[:10]\nteam16 = mSection1.query('DayNum == 138 or DayNum == 139').groupby('TeamName_W')['Season'].count().rename('Count16').reset_index().sort_values(by='Count16',ascending=False).iloc[:10]\nteam8 = mSection1.query('DayNum == 143 or DayNum == 144').groupby('TeamName_W')['Season'].count().rename('Count8').reset_index().sort_values(by='Count8',ascending=False).iloc[:10]\nteam4 = mSection1.query('DayNum == 145 or DayNum == 146').groupby('TeamName_W')['Season'].count().rename('Count4').reset_index().sort_values(by='Count4',ascending=False).iloc[:10]\nteam2 = mSection1.query('DayNum == 152').groupby('TeamName_W')['Season'].count().rename('Count2').reset_index().sort_values(by='Count2',ascending=False).iloc[:10]\n\nplt.subplots(figsize=(20, 10))\n\nplt.subplot(2,3,1)\nplt.xticks(rotation=30)\nsns.barplot(x=\"TeamName_W\", y='Count32', data=team32)\nplt.subplot(2,3,2)\nplt.xticks(rotation=30)\nsns.barplot(x=\"TeamName_W\", y='Count16', data=team16)\nplt.subplot(2,3,3)\nplt.xticks(rotation=30)\nsns.barplot(x=\"TeamName_W\", y='Count8', data=team8)\nplt.subplot(2,3,4)\nplt.xticks(rotation=30)\nsns.barplot(x=\"TeamName_W\", y='Count4', data=team4)\nplt.subplot(2,3,5)\nplt.xticks(rotation=30)\nsns.barplot(x=\"TeamName_W\", y='Count2', data=team2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"we see(可以看到)：\n- Kansas had 32 times in the top 32.(进入32强最多的球队是堪萨斯的32次)\n- Duke had 25 times in the top 16.(进入16强最多的球队是杜克的25次)\n- Kentucky had 17 times in the top 8.进入8强最多的球队是肯塔基的17次；\n- Duke had 12 times in the top 4.进入4强最多的球队是杜克的12次；\n- Duke had 9 times in the top 2.进入2强最多的球队是杜克的9次；\n\nCombined with the previous analysis, Duke entered the final nine times, and finally won five championships and four runner-up, WOW.(其中结合之前的分析，杜克大学9次进入决赛，最终获得5次冠军，4次亚军的傲人成绩)"},{"metadata":{},"cell_type":"markdown","source":"### The score gap in every season(各个赛季随时间推进胜负分差的趋势)"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"tournament = mSection1.query('DayNum >= 136 and DayNum <= 154')\ntournament['ScoreDiv'] = tournament['WScore'] - tournament['LScore']\ntournament = tournament.groupby('Season')[['WScore','LScore','ScoreDiv']].mean().reset_index()\n\nplt.subplots(figsize=(20, 5))\nsns.barplot(x=\"Season\", y='ScoreDiv', data=tournament)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"we see(可以看到)：\n- Max gap in 1993 season.(锦标赛平均分差最大的是1993年)\n- Min gap in 1985 season.(锦标赛平均分差最小的是1985年)\n- Mean gap is almost 11 in last 5 years.(近5年平均分差在11分左右)\n- There is little difference between seasons.(各个赛季之间差异不大)"},{"metadata":{},"cell_type":"markdown","source":"### Seed number and region with champion(冠军对应的种子号统计、区域统计)"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"final_round = mSection1.query('DayNum == 154')\nfinal_round['Seed_W_num'] = final_round.Seed_W.apply(lambda seed:int(seed[1:]) if len(seed)==3 else int(seed[1:-1]))\nfinal_round['Seed_L_num'] = final_round.Seed_L.apply(lambda seed:int(seed[1:]) if len(seed)==3 else int(seed[1:-1]))\nfinal_round['Seed_W_code'] = final_round.Seed_W.apply(lambda seed:'Region'+seed[0])\nfinal_round['Seed_L_code'] = final_round.Seed_L.apply(lambda seed:'Region'+seed[0])\nfinal_round['Seed_W_region'] = final_round.apply(lambda row:row[row['Seed_W_code']], axis=1)\nfinal_round['Seed_L_region'] = final_round.apply(lambda row:row[row['Seed_L_code']], axis=1)\n\nplt.subplots(figsize=(20, 10))\n\nplt.subplot(3,1,1)\nsns.countplot(x=\"Seed_W_num\", data=final_round)\nplt.subplot(3,1,2)\nsns.countplot(x=\"Seed_W_region\", data=final_round)\nplt.subplot(3,1,3)\nsns.lineplot(x=\"Season\", y=\"Seed_W_num\", estimator=None, data=final_round)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"we see(可以看到)：\n- The No.1 seed of each division is still the hot spot of the champion, No.1 seed had 22 times winning the champion, and the count of Champions is basically linearly negatively related to the seed number, which shows that the seed designation before the competition is quite accurate.(各个分区的1号种子依然是冠军的大热门，共22次获得冠军，且冠军数量与种子号基本线性负相关，这说明赛前的种子指定的还是相当准确的)\n- n terms of competition area, Midwest won the most Championships, followed by south.(赛区上看，Midwest获得冠军最多，其次是South，这说明在赛区这一个维度上，实例是不太平衡的，这点跟NBA不太一样，NBA由于存在这工资帽这种机制，使得其各队实力相对会被限制到一个范围内，而大学则是有篮球名校的存在的)\n- From the trend chart of the number of seeds of the team that won the championship, it is more and more difficult for the champion to have a complete black horse, which basically appears in the number 1, 2 and 3 seeds.(从获得冠军的球队种子号的趋势图看，冠军越来越难以出现完全的黑马，基本上以1,2,3号种子中出现，当然了，并不是说3号种子就不是黑马)"},{"metadata":{},"cell_type":"markdown","source":"### Game detail infomation visualization(比赛详细统计数据可视化)"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"cols = ['TeamID','Score','FGM','FGA','FGM3','FGA3','FTM','FTA','OR','DR','Ast','TO','Stl','Blk','PF']\ncols = ['W'+col for col in cols] + ['L'+col for col in cols]\n\nregular_data = mSection1.query('is_regular == 1')[cols].mean().rename('value')\ntournament_data = mSection1.query('is_regular == 0 and DayNum < 154')[cols].mean().rename('value')\nfinal_data = mSection1.query('DayNum == 154')[cols].mean().rename('value')\n\ntmp = pd.DataFrame({})\ntmp = tmp.append([['regular']+list(regular_data.values)],ignore_index=True)\ntmp = tmp.append([['tournament']+list(tournament_data.values)],ignore_index=True)\ntmp = tmp.append([['final']+list(final_data.values)],ignore_index=True)\ntmp.columns = ['match_type']+list(regular_data.index)\ntmp","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.subplots(figsize=(20, 3*5))\n\nfor i,col in enumerate(['FGM','FGA','FGM3','FGA3','FTM','FTA','OR','DR','Ast','TO','Stl','Blk','PF']):\n#     plt.subplot(5,3,i*3+1)\n#     sns.barplot(x=\"match_type\", y='W'+col, data=tmp)\n#     plt.subplot(5,3,i*3+2)\n#     sns.barplot(x=\"match_type\", y='L'+col, data=tmp)\n#     plt.subplot(5,3,i*3+3)\n#     tmp['W'+col+'_L'+col] = tmp['W'+col] - tmp['L'+col]\n#     sns.barplot(x=\"match_type\", y='W'+col+'_L'+col, data=tmp)\n    plt.subplot(5,3,i+1)\n    tmp['W'+col+'_L'+col] = tmp['W'+col] - tmp['L'+col]\n    sns.barplot(x=\"match_type\", y='W'+col+'_L'+col, data=tmp)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"we see(可以看到)：\n- FGM&FGA: There are similar numbers in FGM with both side, but winning team has less FGA.(命中数上看，决赛中球队的差异最小，未命中上看，决赛中，胜利的一方似乎出手更稳重，也就是投丢了更少的球)\n- FGM3&FGA3: The winning (三分球上看，在决赛中，胜利的一方甚至进的三分球要少于失败的一方，这也说明一个现象，通常来说决赛因为防守方式、吹罚方式、球员心态等各种因素，一般球队会采取更保守的得分手段，而较少采用不稳定的三分球)\n- FTM&FTA: There is little difference in free throws. The winning team has a higher percentage of free throws.(罚球上看差异不大，胜利方的罚球命中率更高)\n- Rebounds: On the offensive rebounds, there are always fewer winning teams, especially in the final. Generally we can said, less offensive rebounds can show that the tactics of the team are conservative, that is, to actively retreat from defense, so as to avoid the opportunity of fast attack caused by grabbing offensive rebounds. On the defensive rebounds, the winning side obviously performed better.(篮板球：灌篮高手里面说，赢得篮板的人赢得比赛，看看是不是这样。可以看到对于进攻篮板，胜利的球队总是更少的，决赛尤其如此，通常来说进攻篮板少可以说明球队的战术偏保守，也就是积极退防，以避免因为抢进攻篮板导致对方出现快攻的机会。而防守篮板上看，胜利方明显表现更好)\n- Assists and mistakes: in the final, these two are relatively less, which is because the team usually needs to take more personal attacks due to the problems of defensive strength, penalty scale and so on.(助攻和失误：在决赛，这两个相对都更少，这是因为通常决赛由于防守强度、判罚尺度等问题，球队通常需要采取更多的个人进攻，这也是明星球员的作用，而这种方式通常对应的助攻和失误都会少一些)\n- Steals and blocks: These are two different defense methods. Generally, steals have certain risk-taking elements. Because the failure of steals usually means loss the position, so there are fewer steals in the final, and more blocks means more attacks to the paint area, which is also caused by more attack methods in the final.(抢断和盖帽：这是两种不同的防守方式，通常抢断是有一定冒险成分的，因为抢断失败通常意味着失位，因此决赛中抢断更少出现，而盖帽多则表示到油漆区的进攻更多，这也是决赛中更多的进攻方式选择导致的)\n- In personal fouls, the difference between the winner and the loser in the final is smaller. Of course, it's not very friendly for some players who like to make fouls.(个人犯规上，决赛中胜利方和失败方相比差异更小，一定程度上说也是因为不想把比赛的结果交给裁判，而是更多的给球员发挥，当然这对一些喜欢造犯规的球员来说就不是很友好)"},{"metadata":{},"cell_type":"markdown","source":"## 数据文件描述(This part only for chinese guy)\n\n数据集中的名称、日期等都做了标准化的处理，对于男女比赛，分别用`M`开头和`W`开头来区分。如果需要引入外部数据的话，可以通过`MTeamSpellings.csv`和`MTeamSpellings.csv`进行名称的映射。\n\n### 数据第一部分 - 基础数据\n- 基础信息：\n    - 球队ID和球队名。\n    - 历史锦标赛种子。\n    - 常规赛、竞标赛、NCAA锦标赛的最终分数。\n    - 包括日期和地区名称的赛季级的详细信息(关于赛季，比如2020赛季，通常指的是结束于2020年，而不是开始于2020年，因此如果是2019年开始，2020年结束，依然称之为2020赛季)。\n- MTeams.csv and WTeams.csv\n    - 球队信息，注意不是每个球队在每个赛季都有比赛，因为比赛数据只包含甲级部分。\n    - 球队ID：标识一个大学的一只球队，4位数字，1000~1999表示男队，3000~3999表示女队，该ID是不会变的。\n    - 球队大学名称的简洁写法。\n    - 球队为1级联赛的第一年，最早是1985年，注意这部分只有男队。\n    - 球队为1级联赛的最近的一年，如果目前依然是一级，那么该值为2020，同样也只有男队。\n- MSeasons.csv and WSeasons.csv\n    - 不同赛季的历史数据，包含一些赛季属性。\n    - 赛季表示比赛的年份，注意当前是2020赛季。\n    - DayZero表示赛季开始的那一天的日期，集合DayNum可以计算当前日期。\n    - RegionW,RegionX,RegionY,RegionZ表示四个分组区域，最后四强就是这4个区域中各自的最后冠军。\n- MNCAATourneySeeds.csv and WNCAATourneySeeds.csv\n    - 锦标赛的种子信息。\n    - 赛季年份。\n    - 种子：格式为3/4字符组成，前3个如下，例如W01，表示W分区的1号种子，如果有第4位，可能是a/b，进一步区分前3个字符相同的球队。\n    - 球队Id。\n- MRegularSeasonCompactResults.csv\n    - 除NCAA锦标赛外，或者说之前所有比赛信息。\n    - 赛季年份。\n    - DayNum。\n    - 胜利方的Id、分数，失败方的Id、分数。\n    - 胜利方是主队、客队，还是在第三方球场。\n    - 加时次数，大于等于0.\n- MNCAATourneyCompactResults.csv\n    - NCAA锦标赛的历史信息，数据与上面一致，注意WLoc全部为N，表示比赛总是在第三方进行。\n    - 通过DayNum判断目前锦标赛处于哪一轮：\n        1. DayNum=134或135（星期二/星期三）- 最先四场，确定最终的64支球队。\n        2. DayNum=136或137（星期四/星期五）- 第一轮，64进32。\n        3. DayNum=138或139（周六/周日）- 第二轮，32进16。\n        4. DayNum=143或144（星期四/星期五）- 第3轮，也被称为“甜蜜16强”，16进8。\n        5. DayNum=145或146（周六/周日）- 第4轮，也被称为“精英八强”或“区域总决赛”，8进4。\n        6. DayNum=152（Sat）- 第5轮，也被称为“最终四强”或“全国半决赛”，4进2。\n        7. DayNum=154（周一）- 第6轮，也被称为“全国总决赛”或“全国锦标赛”，决出最终的锦标赛冠军。\n- MSampleSubmissionStage1.csv\n    - 提交文件，第一阶段是过去5年的NCAA每一场比赛做胜负预测，第二阶段则是对今年的锦标赛做预测。\n    - ID由SSSS_XXXX_YYYY组成，XXXX和YYYY为两支球队的ID，低位的在前，SSSS为赛季号，这里也看出赛制是一轮制。\n    - Pred为XXXX赢得比赛的概率，如果你认为YYYY赢为60%，那这里就应该是40%。\n    \n### 数据第二部分 - 球队统计得分数据\n- 基本信息：\n    - 自02-03赛季以来的各种比赛的罚球、防守篮板、失误等数据。\n    - 与第1部分的CompactResult相比，前8列是一致的，从WTeamId到NumOT，但是这里会有其他很多数据，比如二分球个数、三分球个数、发球个数、犯规次数等等。\n    - 最终分数可以由以下方式表示：2×FGM + FGM3 + FTM，即2×进球数+三分球数+罚球数=总分。\n    - 这里的信息与第1部分中的比赛信息是严格对应的。\n- MRegularSeasonDetailedResults.csv\n    - 03赛季以来所有非NCAA锦标赛以外的比赛的详细信息。\n- MNCAATourneyDetailedResults.csv\n    - 03赛季以来所有NCAA锦标赛以外的比赛的详细信息。\n- 详细指标\n    - WFGM：投篮命中数。\n    - WFGA：未命中数。\n    - WFGM3：3分命中数，注意这部分数据是包含在WFGM中的。\n    - WFGA3：3分未命中数。\n    - WFTM：罚球命中数。\n    - WFTA：罚球未命中数。\n    - WOR：进攻篮板。\n    - WDR：防守篮板。\n    - WAst：助攻数。\n    - WTO：失误数。\n    - WStl：抢断数。\n    - WBlk：盖帽数。\n    - WPF：个人犯规数。\n    - 失败方为用L替换W，意思一致。"},{"metadata":{},"cell_type":"markdown","source":"## The end."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}