{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# NFL Player Contact Detection - Ultralytics YOLOv8 for object detection and image segmentation","metadata":{}},{"cell_type":"markdown","source":"# Problem Statement","metadata":{}},{"cell_type":"markdown","source":"## Goal of the Competition\nThe goal of this competition is to detect external contact experienced by players during an NFL football game. You will use video and player tracking data to identify moments with contact to help improve player safety.\n\n## Context\nThe National Football League (NFL) has teamed up with Amazon Web Services (AWS) to strengthen its commitment to predict player injuries. The NFL aspires to have the best injury surveillance and mitigation program in any sport. With your machine learning and computer vision skills, you can help the NFL accurately identify when players experience contact throughout a football play.\n\nIn prior years, the NFL challenged the Kaggle community to create helmet impact detection and identification algorithms. This year the NFL looks to automatically identify all moments when players experience contact. This competition will be successful if we can reliably detect moments when players are in contact with one another and when a player’s body is in contact with the ground.\n\nCurrently, the NFL uses its tracking system to monitor a large number of statistics about players’ load during the season. The league has a solution that predicts contact between players, but it only leverages the player tracking data. This competition hopes to improve the predictive power by including video in addition to tracking data. Categorizing ground contact will also provide a more comprehensive view of impacts, improving analysis for player health and safety.\n\nMore accurate data is an important step toward the NFL’s injury surveillance and mitigation goals. With complete contact detection, the league can identify correlations between certain types of contact and injury, a contributor to future prevention. Your efforts could help mitigate unsafe situations to reduce injury to all players.\n\nThe National Football League is America's most popular sports league. Founded in 1920, the NFL developed the model for the successful modern sports league and is committed to advancing progress in the diagnosis, prevention, and treatment of sports-related injuries. This competition is part of the Digital Athlete, a joint effort between the NFL and AWS to build a virtual, 360-degree representation of an NFL player’s experience. The Digital Athlete hopes to generate a precise picture of what they need when it comes to preventing and recovering from injuries while performing at their best. Health and safety efforts include support for independent medical research and engineering advancements as well as a commitment to work to better protect players and make the game safer, including enhancements to medical protocols and improvements to how our game is taught and played. For more information about the NFL's health and safety efforts, please visit the [NFL Player Health and Safety website](https://www.nfl.com/playerhealthandsafety/).\n\n## Evaluation\nSubmissions are evaluated on [Matthews Correlation Coefficient](https://en.wikipedia.org/wiki/Phi_coefficient) between the predicted and actual contact events.\n\n\n$$MCC = \\frac{TP * TN - FP * FN}{\\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}$$\n\nFor every allowable `1contact_id` (formed by concatenating the `game_play_step_player1_player2`), you must predict whether the involved players are in contact at that moment in time. sample_submission.csv provides the exhaustive list of `contact_ids`. Note that the ground, denoted as player G, is included as a possible contact in place of player2. The player with the lower id is always listed first in the `contact_id`.\n\nThe file should contain a header and have the following format:\n\n```\ncontact_id,contact\n58168_003392_0_38590_43854,0\n58168_003392_0_38590_41257,1\n58168_003392_0_38590_41944,0\netc.\n```","metadata":{}},{"cell_type":"markdown","source":"# Import dependencies","metadata":{}},{"cell_type":"code","source":"!pip install ultralytics","metadata":{"execution":{"iopub.status.busy":"2023-01-25T09:27:36.861726Z","iopub.execute_input":"2023-01-25T09:27:36.862463Z","iopub.status.idle":"2023-01-25T09:27:53.917381Z","shell.execute_reply.started":"2023-01-25T09:27:36.862356Z","shell.execute_reply":"2023-01-25T09:27:53.915997Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import ultralytics\nultralytics.checks()","metadata":{"execution":{"iopub.status.busy":"2023-01-25T09:27:53.920175Z","iopub.execute_input":"2023-01-25T09:27:53.920588Z","iopub.status.idle":"2023-01-25T09:27:57.003272Z","shell.execute_reply.started":"2023-01-25T09:27:53.920545Z","shell.execute_reply":"2023-01-25T09:27:57.002346Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import dependencies\nimport os\nimport subprocess\nimport IPython\nfrom IPython.display import Video, display","metadata":{"execution":{"iopub.status.busy":"2023-01-25T09:27:57.006472Z","iopub.execute_input":"2023-01-25T09:27:57.006819Z","iopub.status.idle":"2023-01-25T09:27:57.014455Z","shell.execute_reply.started":"2023-01-25T09:27:57.006782Z","shell.execute_reply":"2023-01-25T09:27:57.013234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Ultralytics YOLOv8","metadata":{}},{"cell_type":"code","source":"def play_video(video_path: str):\n    frac = 0.65 # scaling factor for display \n    display(\n        Video(data=video_path, embed=True, height=int(720*frac), width=int(1280*frac))\n    )","metadata":{"execution":{"iopub.status.busy":"2023-01-25T09:27:57.017192Z","iopub.execute_input":"2023-01-25T09:27:57.018010Z","iopub.status.idle":"2023-01-25T09:27:57.024431Z","shell.execute_reply.started":"2023-01-25T09:27:57.017972Z","shell.execute_reply":"2023-01-25T09:27:57.023659Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def detect_and_segment_video(video_path:str) -> str:\n    \"\"\"\n    Annotates video with object detection bounding boxes and\n    segmentation masks using YOLOv8 model.\n    \"\"\"\n    \n    video_name = video_path.split('/')[-1]\n    folder_name = 'predict'\n    \n    # YOLOv8 Object Detection and Segmentation\n    # you can edit the model if you want to use a different yolo8 model\n    os.system(f\"yolo segment predict model=yolov8n-seg.pt source={video_path}\")\n    \n    # path to latest output \n    max = 0\n    for dir_name in os.listdir('/kaggle/working/runs/segment'):\n        if dir_name!='predict':\n            if int(dir_name[-1])>max:\n                max = int(dir_name[-1])\n    if max>0:\n        folder_name += str(max)\n    tmp_output_path = f'/kaggle/working/runs/segment/{folder_name}/{video_name}'\n    output_path = f'/kaggle/working/runs/segment/{folder_name}/final-{video_name}'\n    \n    \n    # Not all browsers support the codec, we will re-load the file at tmp_output_path\n    # and convert to a codec that is more broadly readable using ffmpeg\n    if os.path.exists(output_path):\n            os.remove(output_path)\n    subprocess.run(\n            [\n                \"ffmpeg\",\n                \"-i\",\n                tmp_output_path,\n                \"-crf\",\n                \"18\",\n                \"-preset\",\n                \"veryfast\",\n                \"-hide_banner\",\n                \"-loglevel\",\n                \"error\",\n                \"-vcodec\",\n                \"libx264\",\n                output_path,\n            ]\n        )\n    os.remove(tmp_output_path)\n    \n    return output_path","metadata":{"execution":{"iopub.status.busy":"2023-01-25T09:27:57.025593Z","iopub.execute_input":"2023-01-25T09:27:57.026154Z","iopub.status.idle":"2023-01-25T09:27:57.036763Z","shell.execute_reply.started":"2023-01-25T09:27:57.026117Z","shell.execute_reply":"2023-01-25T09:27:57.035990Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# change the video path to predict on a different video\nvideo_path = '/kaggle/input/nfl-player-contact-detection/train/58173_003606_Endzone.mp4'\n# prediction\noutput_path = detect_and_segment_video(video_path)","metadata":{"execution":{"iopub.status.busy":"2023-01-25T09:29:28.081468Z","iopub.execute_input":"2023-01-25T09:29:28.081861Z","iopub.status.idle":"2023-01-25T09:30:37.884247Z","shell.execute_reply.started":"2023-01-25T09:29:28.081831Z","shell.execute_reply":"2023-01-25T09:30:37.882457Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"play_video(output_path)","metadata":{"execution":{"iopub.status.busy":"2023-01-25T09:30:37.886628Z","iopub.execute_input":"2023-01-25T09:30:37.887087Z","iopub.status.idle":"2023-01-25T09:30:38.415956Z","shell.execute_reply.started":"2023-01-25T09:30:37.887024Z","shell.execute_reply":"2023-01-25T09:30:38.414368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# How can this be used?\n- Off the top my head, I think this can be paired with helmets to identify the masks for each of the players.\n- This can be used to identify the pair of players whose contact is in question in a single cropped frame.\n- It could also be used for background removal and isolation of the pair of players.\n- It can also remove the need of cropping at all, since we can feed the data to a model and output the contact between every possible pair of players.","metadata":{}},{"cell_type":"markdown","source":"# References\n- https://github.com/ultralytics/ultralytics\n- https://www.kaggle.com/code/robikscube/nfl-player-contact-detection-getting-started\n- https://www.kaggle.com/code/samir95/nfl-players-instance-segmentation-yolov7","metadata":{}}]}