{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\n# for dirname, _, filenames in os.walk('/kaggle/input'):\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-02-21T10:22:32.860880Z","iopub.execute_input":"2023-02-21T10:22:32.861249Z","iopub.status.idle":"2023-02-21T10:22:32.867438Z","shell.execute_reply.started":"2023-02-21T10:22:32.861209Z","shell.execute_reply":"2023-02-21T10:22:32.866229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip uninstall opencv-python-headless -y \n\n!pip install opencv-python --upgrade","metadata":{"execution":{"iopub.status.busy":"2023-02-21T14:37:29.386034Z","iopub.execute_input":"2023-02-21T14:37:29.386520Z","iopub.status.idle":"2023-02-21T14:37:51.267210Z","shell.execute_reply.started":"2023-02-21T14:37:29.386425Z","shell.execute_reply":"2023-02-21T14:37:51.266003Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Goal of the Competition\n - The goal of this competition is to detect external contact experienced by players during an NFL football game. You will use video and player tracking data to identify moments with contact to help improve player safety.\n \n -  NFL looks to automatically identify all moments when players experience contact. This competition will be successful if we can reliably detect moments when players are in contact with one another and when a player’s body is in contact with the ground.\n \n - This competition is part of the Digital Athlete, a joint effort between the NFL and AWS to build a virtual, 360-degree representation of an NFL player’s experience. The Digital Athlete hopes to generate a precise picture of what they need when it comes to preventing and recovering from injuries while performing at their best. \n \n - You are tasked with predicting moments of contact between player pairs, as well as when players make non-foot contact with the ground using game footage and tracking data.\n \n - Each play has four associated videos. Two videos, showing a sideline and endzone view, are time synced and aligned with each other. Additionally, an All29 view is provided but not guaranteed to be time synced. \n\n- Training set videos are in train/ with corresponding labels in train_labels.csv, while the videos for which you must predict are in the test/ folder.\n\n- This year we are also providing baseline helmet detection and assignment boxes for the training and test set. train_baseline_helmets.csv is the output from last year's winning player assignment model.\n\n- train_player_tracking.csv provides 10 Hz tracking data for each player on the field during the provided plays.\n\n- train_video_metadata.csv contains timestamps associated with each Sideline and Endzone view for syncing with the player tracking data.\n","metadata":{}},{"cell_type":"markdown","source":"# Files\n- [train/test] mp4 videos of each play. Each play has three videos. The two main view are shot from the endzone and sideline. Sideline and Endzone video pairs are matched frame for frame in time, but different players may be visible in each view.     These videos all contain a frame rate of 59.94 HZ. The moment of snap occurs 5 seconds into the video.\n\n- train_labels.csv Contains a row for every combination of players, and players with the ground for each 0.1 second timestamp in the play : \n\n    - contact_id: A combination of the game_play, player_ids and step columns.\n    -  game_play: the unique ID for the game and play.\n    - nfl_player_id_1 The lower numbered player id in the contact pair. If contact with ground then this is just the player id.\n    - nfl_player_id_2: The larger number player id in the contact pair. If for contact with the ground, this will contain an uppercase \"G\"\n    - step: A number representing each each timestep for each play, starting at 0 at the moment of the play starting, and incrementing by 1 every 0.1 seconds.\n    - datetime: The timetamp of the contact, at 10Hz\n    - contact: Whether contact occurred\n    \n\n- [train/test]_baseline_helmets.csv contains imperfect baseline predictions for helmet boxes and player assignments for the Sideline and Endzone video view. The model used to create these predictions are from the winning solution from last year's competition and can be used to leverage your predictions.\n\n       - game_play: Unique game key and play id combination for the play.\n       - game_key: the ID code for the game.\n       - play_id: the ID code for the play.\n       - view: The video view, either Sideline or Endzone\n       - video: The filename of the associated video.\n       - frame: The associated frame within the video.\n       - nfl_player_id: The imperfect predicted player id.\n       - player_label: The player label. A combination of V/H (home or visiting team) and the player jersey number.\n       - [left/width/top/height]: the specification of the bounding box of the prediction.\n\n\n- [train/test]_player_tracking.csv Each player wears a sensor that allows us to locate them on the field; that information is reported in these two files.\n\n          - game_play: Unique game key and play id combination for the play.\n          - game_key: the ID code for the game.\n          - play_id: the ID code for the play.\n          - nfl_player_id: the player's ID code.\n          - datetime: timestamp at 10 Hz.\n          - step: timestep within play relative to the play start.\n          - position: the football position of the player.\n          - team: team of the player, either home or away.\n          - jersey_number: Player jersey number\n          - x_position: player position along the long axis of the field. See figure below.\n          - y_position: player position along the short axis of the field. See figure below.\n          - speed: speed in yards/second.\n          - distance: distance traveled from prior time point, in yards.\n          - orientation: orientation of player (deg).\n          - direction: angle of player motion (deg).\n          - acceleration: magnitiude of the total acceleration in yards/second^2.\n          - sa: Signed acceleration yards/second^2 in the direction the player is moving.\n          \n          \n          \n \n - [train/test]_video_metadata.csv Metadata for each sideline and endzone video file including the timestamp information to be used to sync with player tracking data.\n\n         - game_play: Unique game key and play id combination for the play.\n         - game_key: the ID code for the game.\n         - play_id: the ID code for the play.\n         - view: The video view, either Sideline or Endzone\n         - start_time: The timestamp of the video start.\n         - end_time: The timestamp when the video ends.\n         - snap_time: The timestamp when the play starts within the video. This is 5 seconds (300 frames) into the video.","metadata":{}},{"cell_type":"code","source":"import cv2\nimport numpy as np\nimport matplotlib.pyplot as plt\nFILE = \"/kaggle/input/nfl-player-contact-detection/test/58168_003392_All29.mp4\"","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:22:50.317466Z","iopub.execute_input":"2023-02-21T10:22:50.318157Z","iopub.status.idle":"2023-02-21T10:22:50.323073Z","shell.execute_reply.started":"2023-02-21T10:22:50.318122Z","shell.execute_reply":"2023-02-21T10:22:50.321862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Read and display video frame-by-frame\ndef get_frames(filename):\n    video=cv2.VideoCapture(filename)\n    while video.isOpened():\n        rete,frame=video.read()\n        if rete:\n            yield frame\n        else:\n            break\n        video.release()\n        yield None","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:22:50.360586Z","iopub.execute_input":"2023-02-21T10:22:50.360856Z","iopub.status.idle":"2023-02-21T10:22:50.365975Z","shell.execute_reply.started":"2023-02-21T10:22:50.360826Z","shell.execute_reply":"2023-02-21T10:22:50.365058Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"get_frames(FILE)","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:22:50.368006Z","iopub.execute_input":"2023-02-21T10:22:50.368628Z","iopub.status.idle":"2023-02-21T10:22:50.379478Z","shell.execute_reply.started":"2023-02-21T10:22:50.368591Z","shell.execute_reply":"2023-02-21T10:22:50.378536Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get a single video frame\n\ndef get_frame(filename,index):\n    counter=0\n    video=cv2.VideoCapture(filename)\n    while video.isOpened():\n        rete,frame=video.read()\n        if rete:\n            if counter==index:\n                return frame\n            counter +=1\n        else:\n            break\n    video.release()\n    return None","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:22:54.370884Z","iopub.execute_input":"2023-02-21T10:22:54.371254Z","iopub.status.idle":"2023-02-21T10:22:54.378452Z","shell.execute_reply.started":"2023-02-21T10:22:54.371224Z","shell.execute_reply":"2023-02-21T10:22:54.377250Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"frame = get_frame(FILE,45)\nprint('shape is', frame.shape)\nprint('pixel at (60,21)',frame[60,21,:])\nprint('pixel at (120,10)',frame[120,10,:])","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:22:54.841118Z","iopub.execute_input":"2023-02-21T10:22:54.842175Z","iopub.status.idle":"2023-02-21T10:22:55.190542Z","shell.execute_reply.started":"2023-02-21T10:22:54.842129Z","shell.execute_reply":"2023-02-21T10:22:55.189616Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(frame)\n","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:22:56.450553Z","iopub.execute_input":"2023-02-21T10:22:56.451146Z","iopub.status.idle":"2023-02-21T10:22:57.074546Z","shell.execute_reply.started":"2023-02-21T10:22:56.451111Z","shell.execute_reply":"2023-02-21T10:22:57.073718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fix_frame=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\nprint('pixel at (120,10)',fix_frame[120,10,:])\nplt.imshow(fix_frame)","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:22:57.901066Z","iopub.execute_input":"2023-02-21T10:22:57.901761Z","iopub.status.idle":"2023-02-21T10:22:58.648530Z","shell.execute_reply.started":"2023-02-21T10:22:57.901721Z","shell.execute_reply":"2023-02-21T10:22:58.647737Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(fix_frame[220:430,300:600])\n","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:22:59.310844Z","iopub.execute_input":"2023-02-21T10:22:59.311213Z","iopub.status.idle":"2023-02-21T10:22:59.603333Z","shell.execute_reply.started":"2023-02-21T10:22:59.311182Z","shell.execute_reply":"2023-02-21T10:22:59.602457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# pinpoint to a particular point on our selected frame using cv2 rectangle function.\n\nframe=get_frame(FILE,200)\ncv2.rectangle(frame,(500,700),(1200,400),color=(0,0,255),thickness=30)\nfixed_frame=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\nplt.imshow(fixed_frame)","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:01.960851Z","iopub.execute_input":"2023-02-21T10:23:01.961213Z","iopub.status.idle":"2023-02-21T10:23:03.599516Z","shell.execute_reply.started":"2023-02-21T10:23:01.961183Z","shell.execute_reply":"2023-02-21T10:23:03.598678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ENV_DIR = '../input'\nDATA_DIR = f'{ENV_DIR}/nfl-player-contact-detection'","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:07.765982Z","iopub.execute_input":"2023-02-21T10:23:07.766351Z","iopub.status.idle":"2023-02-21T10:23:07.771367Z","shell.execute_reply.started":"2023-02-21T10:23:07.766320Z","shell.execute_reply":"2023-02-21T10:23:07.770283Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.listdir(DATA_DIR)","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:08.068547Z","iopub.execute_input":"2023-02-21T10:23:08.069427Z","iopub.status.idle":"2023-02-21T10:23:08.077877Z","shell.execute_reply.started":"2023-02-21T10:23:08.069381Z","shell.execute_reply":"2023-02-21T10:23:08.076623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\ntrain_df = pd.read_csv(f'{DATA_DIR}/train_labels.csv')\n\n# Tracking Information using Sensor\ntrain_tracking_df = pd.read_csv(f'{DATA_DIR}/train_player_tracking.csv')\ntest_tracking_df = pd.read_csv(f'{DATA_DIR}/test_player_tracking.csv')\n\ntrain_predict_df = pd.read_csv(f'{DATA_DIR}/train_baseline_helmets.csv')\n\ntest_predict_df = pd.read_csv(f'{DATA_DIR}/test_baseline_helmets.csv')","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:08.080266Z","iopub.execute_input":"2023-02-21T10:23:08.080771Z","iopub.status.idle":"2023-02-21T10:23:34.177612Z","shell.execute_reply.started":"2023-02-21T10:23:08.080734Z","shell.execute_reply":"2023-02-21T10:23:34.176628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image, ImageDraw\n\ndef get_frame_from_video(video_path, frame):\n    video_path = f\"{DATA_DIR}/train/{video_path}\"\n    frame = frame - 1\n    \n    !ffmpeg \\\n        -hide_banner \\\n        -loglevel fatal \\\n        -nostats \\\n        -i $video_path -vf \"select=eq(n\\,$frame)\" -vframes 1 frame.png\n    \n    img = Image.open('frame.png')\n    os.remove('frame.png')\n    return img","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:34.179074Z","iopub.execute_input":"2023-02-21T10:23:34.179429Z","iopub.status.idle":"2023-02-21T10:23:34.189160Z","shell.execute_reply.started":"2023-02-21T10:23:34.179394Z","shell.execute_reply":"2023-02-21T10:23:34.188128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"get_frame('/kaggle/input/nfl-player-contact-detection/train/58168_003392_Endzone.mp4', 1)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-02-21T10:23:34.192660Z","iopub.execute_input":"2023-02-21T10:23:34.193645Z","iopub.status.idle":"2023-02-21T10:23:34.250444Z","shell.execute_reply.started":"2023-02-21T10:23:34.193609Z","shell.execute_reply":"2023-02-21T10:23:34.249547Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"get_frame_from_video('58168_003392_All29.mp4', 1)","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:41.687353Z","iopub.execute_input":"2023-02-21T10:23:41.687731Z","iopub.status.idle":"2023-02-21T10:23:43.795926Z","shell.execute_reply.started":"2023-02-21T10:23:41.687700Z","shell.execute_reply":"2023-02-21T10:23:43.787668Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def draw_rect(image, bbox_df):\n    new_image = image.copy()\n    draw = ImageDraw.Draw(new_image)\n    for _, (left, width, top, height) in bbox_df[['left', 'width', 'top', 'height']].iterrows():\n        draw.rectangle(((left, top), (left + width, top + height)), outline=(255, 0, 0), width=2)\n    \n    return new_image","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:48.236364Z","iopub.execute_input":"2023-02-21T10:23:48.236748Z","iopub.status.idle":"2023-02-21T10:23:48.243612Z","shell.execute_reply.started":"2023-02-21T10:23:48.236715Z","shell.execute_reply":"2023-02-21T10:23:48.242418Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from IPython.display import Video, display\n\ndef video(video_path, ratio=0.7):\n    nfl_video = Video(f\"{DATA_DIR}/train/{video_path}\",\n                      embed=True,\n                      height=int(720 * ratio),\n                      width=int(1280 * ratio))\n    return nfl_video\n    \nvideo('58168_003392_Endzone.mp4')","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:48.801074Z","iopub.execute_input":"2023-02-21T10:23:48.801510Z","iopub.status.idle":"2023-02-21T10:23:49.009152Z","shell.execute_reply.started":"2023-02-21T10:23:48.801472Z","shell.execute_reply":"2023-02-21T10:23:49.007760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip install -U openmim\n# !mim install mmcv-full","metadata":{"execution":{"iopub.status.busy":"2023-02-21T03:40:08.600555Z","iopub.execute_input":"2023-02-21T03:40:08.600936Z","iopub.status.idle":"2023-02-21T03:40:08.608283Z","shell.execute_reply.started":"2023-02-21T03:40:08.600901Z","shell.execute_reply":"2023-02-21T03:40:08.607507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_dir = '/kaggle/input/nfl-player-contact-detection/'\nexample_video = f'{data_dir}/train/58168_003392_All29.mp4'\n\nfrac = 0.65\n\ndisplay(Video(example_video, embed=True, height=int(720*frac), width=int(1280*frac)))","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:53.351101Z","iopub.execute_input":"2023-02-21T10:23:53.351472Z","iopub.status.idle":"2023-02-21T10:23:53.484268Z","shell.execute_reply.started":"2023-02-21T10:23:53.351440Z","shell.execute_reply":"2023-02-21T10:23:53.482989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert video to images\n# import dependencies\nimport os\nimport subprocess\nimport shutil\n\ndef convert_to_image(source_dir_path, dst_dir_path, types, img_ext, maxSize=1024):\n    if not os.path.exists(dst_dir_path):\n        os.makedirs(dst_dir_path)\n\n    # check that video formats equal types \n    for file_name in os.listdir(source_dir_path):\n        for type in types:\n            if type not in file_name:\n                continue\n\n        name, ext = os.path.splitext(file_name)\n    \n        video_file_path = os.path.join(source_dir_path, file_name)\n\n        # skip large files\n        # if os.path.getsize(video_file_path) > maxSize * 1000:\n        #\tcontinue\n        \n        # get frames\n        cmd = 'ffmpeg -i \\\"{}\\\" -qscale:v 2 \\\"{}/{}_%d.{}\\\"'.format(video_file_path, dst_dir_path, name, img_ext)\n\n        print(cmd)\n        subprocess.call(cmd, shell=True)\n        print('\\n')","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:23:59.352372Z","iopub.execute_input":"2023-02-21T10:23:59.352762Z","iopub.status.idle":"2023-02-21T10:23:59.360784Z","shell.execute_reply.started":"2023-02-21T10:23:59.352728Z","shell.execute_reply":"2023-02-21T10:23:59.359533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# create destination folder for images\nimg_ext = 'png'\ndst_dir_path = f'/kaggle/working/{img_ext}_from_mp4/'\nos.makedirs(dst_dir_path, exist_ok=True)\n\n# source folder with videos\n# source_dir_path = '../input/nfl-player-contact-detection/test'\nsource_dir_path = '/kaggle/input/nfl-player-contact-detection/test'\n\n# /kaggle/input/nfl-player-contact-detection/test\n# /kaggle/input/nfl-player-contact-detection\n# /kaggle/input/nfl-player-contact-detection/train\n# allowed video types \ntypes = ['.mp4']\n\n# convert\nconvert_to_image(source_dir_path, dst_dir_path, types, img_ext)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-02-21T10:31:01.520949Z","iopub.execute_input":"2023-02-21T10:31:01.521341Z","iopub.status.idle":"2023-02-21T10:31:01.526531Z","shell.execute_reply.started":"2023-02-21T10:31:01.521306Z","shell.execute_reply":"2023-02-21T10:31:01.525242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert one video into image(jpg)\nimport cv2\nvidcap = cv2.VideoCapture('/kaggle/input/nfl-player-contact-detection/train/58168_003392_All29.mp4')\ndef getFrame(sec):\n    vidcap.set(cv2.CAP_PROP_POS_MSEC,sec*1000)\n    hasFrames,image = vidcap.read()\n    if hasFrames:\n        cv2.imwrite(\"image\"+str(count)+\".jpg\", image)     # save frame as JPG file\n    return hasFrames\nsec = 0\nframeRate = 0.5 #//it will capture image in each 0.5 second\ncount=1\nsuccess = getFrame(sec)\nwhile success:\n    count = count + 1\n    sec = sec + frameRate\n    sec = round(sec, 2)\n    success = getFrame(sec)","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:32:35.051473Z","iopub.execute_input":"2023-02-21T10:32:35.052180Z","iopub.status.idle":"2023-02-21T10:32:41.755271Z","shell.execute_reply.started":"2023-02-21T10:32:35.052140Z","shell.execute_reply":"2023-02-21T10:32:41.754262Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# conver video into jpg\n# Importing all necessary libraries \nimport cv2 \nimport os \n\n# Read the video from specified path \ncam = cv2.VideoCapture(\"/kaggle/input/nfl-player-contact-detection/test/58168_003392_Endzone.mp4\") \n\ntry: \n  # creating a folder named data \n   if not os.path.exists('data'): \n        os.makedirs('data') \n\n# if not created then raise error \nexcept OSError: \n    print ('Error: Creating directory of data') \n\n# frame \ncurrentframe = 0\n\nwhile(True): \n    \n     # reading from frame \n    ret,frame = cam.read() \n\n    if ret: \n      # if video is still left continue creating images \n        name = './data/frame' + str(currentframe) + '.jpg'\n        print ('Creating...' + name) \n\n        # writing the extracted images \n        cv2.imwrite(name, frame) \n\n        # increasing counter so that it will \n        # show how many frames are created \n        currentframe += 1\n    else: \n        break\n\n# Release all space and windows once done \ncam.release() \ncv2.destroyAllWindows()","metadata":{"execution":{"iopub.status.busy":"2023-02-21T10:42:49.149760Z","iopub.execute_input":"2023-02-21T10:42:49.150130Z","iopub.status.idle":"2023-02-21T10:43:01.606698Z","shell.execute_reply.started":"2023-02-21T10:42:49.150099Z","shell.execute_reply":"2023-02-21T10:43:01.605243Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# MMDetection\n- MMDetection is an open source object detection toolbox based on PyTorch\n- MMDetection is a toolbox containing many pre-built models and each model has its own architecture, this toolbox defines a general architecture that can adapt to any model.\n\n- source : https://debuggercafe.com/getting-started-with-mmdetection-training-for-object-detection/\n\n\n# MMDetection Architecture\n  - MMDetection is a toolbox containing many pre-built models and each model has its own architecture, this toolbox defines a general architecture that can adapt to any model. This general architecture comprises the following parts:\n           - Backbone\n           - Neck\n           - DenseHead (AnchorHead/AnchorFreeHead)\n           - RoIExtractor\n           - RoIHead (BBoxHead/MaskHead)\n \n \n - Backbone that transforms the input images into raw feature maps.\n - Neck connects the Backbone with heads and performs reconfigurations and refinements on the raw feature maps so that heads can further process them. \n - DenseHead is a part that processes the dense locations of the feature maps fed by Neck.   \n \n - RoIExtractor identifies the region of interest (RoI) and extracts RoI features from the feature maps. \n","metadata":{}},{"cell_type":"code","source":"# !pip uninstall mmcv-full -y\n# !pip uninstall mmde -y","metadata":{"execution":{"iopub.status.busy":"2023-02-21T13:20:00.962914Z","iopub.execute_input":"2023-02-21T13:20:00.963505Z","iopub.status.idle":"2023-02-21T13:20:00.972775Z","shell.execute_reply.started":"2023-02-21T13:20:00.963464Z","shell.execute_reply":"2023-02-21T13:20:00.968501Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip uninstall torch -y\n# !pip uninstall torch -y","metadata":{"execution":{"iopub.status.busy":"2023-02-21T12:35:36.998298Z","iopub.execute_input":"2023-02-21T12:35:36.998717Z","iopub.status.idle":"2023-02-21T12:35:41.018517Z","shell.execute_reply.started":"2023-02-21T12:35:36.998680Z","shell.execute_reply":"2023-02-21T12:35:41.017317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check nvcc version\n!nvcc -V\n# Check GCC version\n!gcc --version","metadata":{"execution":{"iopub.status.busy":"2023-02-21T12:47:55.242908Z","iopub.execute_input":"2023-02-21T12:47:55.243333Z","iopub.status.idle":"2023-02-21T12:47:57.164571Z","shell.execute_reply.started":"2023-02-21T12:47:55.243298Z","shell.execute_reply":"2023-02-21T12:47:57.163348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip install -U openmim\n# !mim install mmcv-full","metadata":{"execution":{"iopub.status.busy":"2023-02-21T12:46:10.578182Z","iopub.execute_input":"2023-02-21T12:46:10.578614Z","iopub.status.idle":"2023-02-21T12:46:37.700320Z","shell.execute_reply.started":"2023-02-21T12:46:10.578570Z","shell.execute_reply":"2023-02-21T12:46:37.699074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install /kaggle/input/mmsegmentation/mmcv-full/addict-2.4.0-py3-none-any.whl\n!pip install /kaggle/input/mmsegmentation/mmcv-full/mmcv_full-1.5.3-cp37-cp37m-linux_x86_64.whl","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# install dependencies: (use cu111 because colab has CUDA 11.1)\n!pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html\n\n# install mmcv-full thus we could use CUDA operators\n!pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html\n\n# Install mmdetection\n!rm -rf mmdetection\n!git clone https://github.com/open-mmlab/mmdetection.git\n%cd mmdetection\n\n!pip install -e .","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-02-21T12:36:10.975239Z","iopub.execute_input":"2023-02-21T12:36:10.975873Z","iopub.status.idle":"2023-02-21T12:39:43.502125Z","shell.execute_reply.started":"2023-02-21T12:36:10.975822Z","shell.execute_reply":"2023-02-21T12:39:43.500866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# mmcv-build-1.4.0\n!pip wheel mmcv-full==1.4.0\n","metadata":{"execution":{"iopub.status.busy":"2023-02-21T12:59:23.284924Z","iopub.execute_input":"2023-02-21T12:59:23.285388Z","iopub.status.idle":"2023-02-21T13:20:00.959860Z","shell.execute_reply.started":"2023-02-21T12:59:23.285350Z","shell.execute_reply":"2023-02-21T13:20:00.958633Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# from mmcv import collect_env\n# collect_env()\n","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-02-21T13:32:41.022493Z","iopub.execute_input":"2023-02-21T13:32:41.023245Z","iopub.status.idle":"2023-02-21T13:32:41.143132Z","shell.execute_reply.started":"2023-02-21T13:32:41.023204Z","shell.execute_reply":"2023-02-21T13:32:41.141516Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # Check Pytorch installation\n# import torch\n# torch.cuda.is_available()\n# import torch, torchvision\n# print(torch.__version__, torch.cuda.is_available())\n\n# # Check MMDetection installation\n# import mmdet\n# print(mmdet.__version__)\n\n# # Check mmcv installation\n# from mmcv.ops import get_compiling_cuda_version, get_compiler_version\n# print(get_compiling_cuda_version())\n# print(get_compiler_version())","metadata":{"execution":{"iopub.status.busy":"2023-02-21T12:54:16.534919Z","iopub.execute_input":"2023-02-21T12:54:16.535550Z","iopub.status.idle":"2023-02-21T12:54:16.586543Z","shell.execute_reply.started":"2023-02-21T12:54:16.535507Z","shell.execute_reply":"2023-02-21T12:54:16.584036Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !find / -iname libc10.so","metadata":{"execution":{"iopub.status.busy":"2023-02-21T12:52:44.921914Z","iopub.execute_input":"2023-02-21T12:52:44.922308Z","iopub.status.idle":"2023-02-21T12:53:01.138309Z","shell.execute_reply.started":"2023-02-21T12:52:44.922276Z","shell.execute_reply":"2023-02-21T12:53:01.136979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Convolutional neural network (CNN) as backbone to extract features from an image","metadata":{}},{"cell_type":"code","source":"# # Download the pre-trained checkpoints for inference and finetuning.\n# !mkdir checkpoints\n# !wget -c https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_caffe_fpn_mstrain_3x_coco/faster_rcnn_r50_caffe_fpn_mstrain_3x_coco_20210526_095054-1f77628b.pth \\\n#       -O checkpoints/faster_rcnn_r50_caffe_fpn_mstrain_3x_coco_20210526_095054-1f77628b.pth","metadata":{"execution":{"iopub.status.busy":"2023-02-21T03:47:38.742745Z","iopub.execute_input":"2023-02-21T03:47:38.743668Z","iopub.status.idle":"2023-02-21T03:47:48.419258Z","shell.execute_reply.started":"2023-02-21T03:47:38.743629Z","shell.execute_reply":"2023-02-21T03:47:48.417861Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import mmcv\n# from mmcv.runner import load_checkpoint\n\n# from mmdet.apis import inference_detector, show_result_pyplot\n# from mmdet.models import build_detector\n\n# # Choose to use a config and initialize the detector\n# config = 'configs/faster_rcnn/faster_rcnn_r50_caffe_fpn_mstrain_3x_coco.py'\n# # Setup a checkpoint file to load\n# checkpoint = 'checkpoints/faster_rcnn_r50_caffe_fpn_mstrain_3x_coco_20210526_095054-1f77628b.pth'","metadata":{"execution":{"iopub.status.busy":"2023-02-21T03:48:15.552776Z","iopub.execute_input":"2023-02-21T03:48:15.553156Z","iopub.status.idle":"2023-02-21T03:48:17.803360Z","shell.execute_reply.started":"2023-02-21T03:48:15.553123Z","shell.execute_reply":"2023-02-21T03:48:17.802404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# # Set the device to be used for evaluation\n# device='cuda:0'\n\n# # Load the config\n# config = mmcv.Config.fromfile(config)\n# # Set pretrained to be None since we do not need pretrained model here\n# config.model.pretrained = None\n\n# # Initialize the detector\n# model = build_detector(config.model)\n\n# # Load checkpoint\n# checkpoint = load_checkpoint(model, checkpoint, map_location=device)\n\n# # Set the classes of models for inference\n# model.CLASSES = checkpoint['meta']['CLASSES']\n\n# # We need to set the model's cfg for inference\n# model.cfg = config\n\n# # Convert the model to GPU\n# model.to(device)\n# # Convert the model into evaluation mode\n# model.eval()","metadata":{"execution":{"iopub.status.busy":"2023-02-21T03:48:42.197775Z","iopub.execute_input":"2023-02-21T03:48:42.198142Z","iopub.status.idle":"2023-02-21T03:48:46.341656Z","shell.execute_reply.started":"2023-02-21T03:48:42.198111Z","shell.execute_reply":"2023-02-21T03:48:46.340550Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}