{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n\n

October 2022 TPS EDA

","metadata":{}},{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as patches\nimport os\nimport seaborn as sb\nimport imageio.v2 as imageio\n\nfrom tqdm import tqdm","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loading the data and checking the values","metadata":{}},{"cell_type":"code","source":"DIR_NAME = '.'\n\ntrain = pd.DataFrame()\nfor i in tqdm(range(9)):\n train = pd.concat([train, pd.read_csv(os.path.join(DIR_NAME, f'train_{i}.csv'))])\n\ntypes = pd.read_csv(os.path.join(DIR_NAME, 'train_dtypes.csv'))\ntypes_dict = {i['column']: i['dtype'] for i in types.to_dict('records')}\n\ntrain = train.astype(types_dict)\ntrain.info()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.shape","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Firstly, we can check out how much data is missing.","metadata":{}},{"cell_type":"code","source":"df_missing = train.isna().sum()[train.isna().sum() != 0]\ndf_missing ","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Insight:**\n\nAlmost all of these features are numerics so we can use some kind of imputing technique to estimate the values.\n``team_scoring_next`` is a categorical feature and has a lot of NA values. Although, these values might actually make sense eg. there might be no goals towards the end of the match.\n\nLet's check for monotonous and constant features.","metadata":{}},{"cell_type":"code","source":"constant = train.nunique()[train.nunique() == 1]\nconstant","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"monotonous = train.nunique()[train.nunique() > 0.8*train.shape[0]]\nmonotonous","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Brief data visualizations","metadata":{}},{"cell_type":"code","source":"train.hist(figsize=(30, 30), bins=20)\nplt.show()","metadata":{"scrolled":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Most of these distributions are what I expected for this task. \n\n**Insights:**\n\n* ``ball_pos_x``: Interesting mode at x=0. This might be due to goals happening after which the ball is transferred to the center of the field. The players then race towards it in order to get control of it. Same goes for ``y``.\n* ``ball_pos_z``: Heavily right skewed histogram makes sense because of the effect of gravity towards the ball.\n* ``p{0, 1, 2}_pos_y``: Most of the time the players will be closer to their goal so that's why there's a right skew in the distribution. An interesting thing to notice is that this is a mirror image of the ``p{3, 4, 5}_pos_y`` dsitributions. In fact, this is also true for the ``x`` and ``z`` variables.\n* ``p{0, 1, 2, 3, 4, 5}_pos_z``: Also heavily right skewed histogram because of gravity.\n* ``p{0, 1, 2, 3, 4, 5}_boost``: Interesting that both tails are heavy.\n\n**Things to try out:**\n* Transforming the right skewed variables using cube root, square root or logarithm transform\n* Transforming the left skewed variables using cube or square transform","metadata":{}},{"cell_type":"code","source":"fig = plt.figure(figsize=(30, 30))\nsb.heatmap(\n train.corr(),\n cmap='viridis'\n)\nplt.show()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Most of the variables aren't heavily correlated so this shouldn't cause any problems.","metadata":{}},{"cell_type":"markdown","source":"# Visualizing a match in 2D\n\nLet's try to visualize some games using matplotlib.","metadata":{}},{"cell_type":"code","source":"game_one = train[train['game_num'] == 1]\ngame_one.head()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"max_x, min_x = train.ball_pos_x.max(), train.ball_pos_x.min()\nmax_y, min_y = train.ball_pos_y.max(), train.ball_pos_y.min()\n\ndef draw_football_field():\n ax = plt.axes()\n ax.set_facecolor('green')\n \n # draw center circles\n ax.axhline(0, color='white')\n ax.add_patch(patches.Circle((0, 0), 1, color='white'))\n ax.add_patch(patches.Circle((0, 0), 20, color='white', fill=False))\n \n # draw circles around goals\n ax.add_patch(patches.Circle((0, min_y), 30, color='lightblue'))\n ax.add_patch(patches.Circle((0, min_y), 40, color='lightblue', fill=False))\n ax.add_patch(patches.Circle((0, max_y), 30, color='orange'))\n ax.add_patch(patches.Circle((0, max_y), 40, color='orange', fill=False))\n \n ax.set_xlim((max_x, min_x))\n ax.set_ylim((max_y, min_y))\n return ax\n\ndef draw_ball(ax, i, game, annotate=False):\n x, y = game[['ball_pos_x', 'ball_pos_y']].iloc[i]\n ellipse = patches.Circle((x, y), 5, facecolor='white', edgecolor='black')\n ax.add_patch(ellipse)\n if annotate:\n label = ax.annotate(\n 'B', xy=(x, y), \n fontsize=15, \n verticalalignment='center', \n horizontalalignment='center'\n )\n return ax\n \ndef draw_players(ax, i, game):\n for j in range(3):\n x, y = game[[f'p{j}_pos_x', f'p{j}_pos_y']].iloc[i]\n circle = patches.Circle((x, y), 5, facecolor='lightblue', edgecolor='black')\n ax.add_patch(circle)\n label = ax.annotate(\n j, xy=(x, y), \n fontsize=15, \n verticalalignment='center', \n horizontalalignment='center'\n )\n \n for j in range(3, 6):\n x, y = game[[f'p{j}_pos_x', f'p{j}_pos_y']].iloc[i]\n circle = patches.Circle((x, y), 5, facecolor='orange', edgecolor='black')\n ax.add_patch(circle)\n label = ax.annotate(\n j, xy=(x, y), \n fontsize=15, \n verticalalignment='center', \n horizontalalignment='center'\n )\n return ax\n\ndef save_images_for_gif(game, path='./imgs/gif1'):\n for i in tqdm(range(game.shape[0])):\n fig = plt.figure(figsize=(8, 10))\n ax = draw_football_field()\n ax = draw_ball(ax, i, game, True)\n ax = draw_players(ax, i, game)\n ax.set_xticks([])\n ax.set_yticks([])\n final_path = os.path.join(path, f'{i}.png')\n plt.tight_layout()\n plt.savefig(final_path)\n plt.close() \n\ndef build_gif(img_dir, gif_path):\n img_paths = [f for f in os.listdir(img_dir) if os.path.isfile(os.path.join(img_dir, f))]\n with imageio.get_writer(gif_path, mode='I') as writer:\n for _, filename in tqdm(enumerate(sorted(img_paths, key=lambda x: int(x.split('.')[0])))):\n image = imageio.imread(os.path.join(img_dir, filename))\n writer.append_data(image)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_images_for_gif(game_one, './imgs/gif1')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"build_gif('./imgs/gif1', './imgs/game1_gif.gif')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Thanks to [Maher](https://www.kaggle.com/maherelouahabi) for noticing my mistake!\n\n2D animation of the first game:\n\n","metadata":{}},{"cell_type":"code","source":"game_two = train[train['game_num'] == 2]\nsave_images_for_gif(game_two, './imgs/gif2')\nbuild_gif('./imgs/gif2', './imgs/game2_gif.gif')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"2D animation of the second game:\n\n","metadata":{}}]}