![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F448347%2F39f624f6ffab585d65adb434ddb604df%2FScreen%20Shot%202023-03-02%20at%2010.04.04%20PM.png?generation=1677762260999335&alt=media\")
Thanks to the organizers and congrats to all the winners and my wonderful teammates @boliu0 @tanakar @ryotayoshinobu
\nIt is an interesting competition with very rich data types, we can train the model on video, or on table data, or a combination of both.
\nSo I believe it's very likely that each team uses a different approach. That's the biggest thing I'm looking forward to about this competition - because I'll be able to learn a lot of different approachs.
\nThis was also confirmed after we merge the team. When we teamed up with another team ( @tanakar @ryotayoshinobu ) we found that they and I surprisingly used a completely different training pipeline. After spending some time we found that our two pipelines even can hardly borrow ideas from each other. So in the end our choice was to optimize the pipelines individually and finally do a simple average ensemble.
\n@tanakar @ryotayoshinobu part have already been post here: https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391620
\nNext I'm going to describe our part of solution.
\nWe designed a 2-stage pipeline.
\nFor a single player in one play, crop his boxes from all frames for both 2 views, using helmet bb extend height and width to 5x
\n
This is an example where the top 6 imgs are Endzone and the bottom 6 imgs are Sideline, of the same player.
\nFrom left to right, each img is one step (6 frames exactly, we don't do 59.94hz…).
\nFor a single crop, it crop through 5 frames so it's 2.5D CNN
\nFor training 2.5 CNN w/ LSTM, we use seq_len=128.
\nWhich means if a video is more than 128 steps, the rest are dropped. (around 0.2% pos samples in training data are dropped here.)
\nThe CNN w/ LSTM model take 2 view sequences as input at the same time.
\nWe concat 2 view pairs features after backbone, then feed into LSTM.
\nThe output of this model is:
\nprob_p : whether the player contact with anyoneprob_g : whether the player on the ground After stage 1 models (5folds) training done, we predict OOF for all training data. Here we get contact with ground prediction which directly used for submission.
\nAnd contact with people prediction for stage 2.
\nWe selected the samples with prob_p > 0.145, and matched the pairs of players whose distance between players was less than 1.6 as stage2 training data.
For a single pair, we crop through 9 frames like this ↓
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F448347%2F8d760c85635cac713e19729e5852495b%2FScreen%20Shot%202023-03-02%20at%2010.10.38%20PM.png?generation=1677762664204997&alt=media\")
Take two helmet bboxes and find the center point of them, expand height and width then crop out.
\nWe also draw the two bboxes of the player pair on the picture, so that the model knows which player pair is needed when there are many people in the images.
\nIn this way we are able to train 2.5D CNN to obtain the contact prediction between player pairs extracted from stage 1.
\nAs for GBDT, we simple extract some features from tracking.csv, nothing special there.
The final CV score of our part is 0.796
After ensemble with @tanakar @ryotayoshinobu we got CV 0.804
I trained many models in this competition using Z by HP Z8G4 Workstation with dual A6000 GPU as usual, it's stable and never had a problem with the training process. Thanks to Z by HP for sponsoring!
", "messages": [ { "id": "2165784", "postDate": "03/02/2023 13:05:04", "content": "Thanks to the organizers and congrats to all the winners and my wonderful teammates @boliu0 @tanakar @ryotayoshinobu
\nIt is an interesting competition with very rich data types, we can train the model on video, or on table data, or a combination of both.
\nSo I believe it's very likely that each team uses a different approach. That's the biggest thing I'm looking forward to about this competition - because I'll be able to learn a lot of different approachs.
\nThis was also confirmed after we merge the team. When we teamed up with another team ( @tanakar @ryotayoshinobu ) we found that they and I surprisingly used a completely different training pipeline. After spending some time we found that our two pipelines even can hardly borrow ideas from each other. So in the end our choice was to optimize the pipelines individually and finally do a simple average ensemble.
\n@tanakar @ryotayoshinobu part have already been post here: https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391620
\nNext I'm going to describe our part of solution.
\nWe designed a 2-stage pipeline.
\nFor a single player in one play, crop his boxes from all frames for both 2 views, using helmet bb extend height and width to 5x
\n
This is an example where the top 6 imgs are Endzone and the bottom 6 imgs are Sideline, of the same player.
\nFrom left to right, each img is one step (6 frames exactly, we don't do 59.94hz…).
\nFor a single crop, it crop through 5 frames so it's 2.5D CNN
\nFor training 2.5 CNN w/ LSTM, we use seq_len=128.
\nWhich means if a video is more than 128 steps, the rest are dropped. (around 0.2% pos samples in training data are dropped here.)
\nThe CNN w/ LSTM model take 2 view sequences as input at the same time.
\nWe concat 2 view pairs features after backbone, then feed into LSTM.
\nThe output of this model is:
\nprob_p : whether the player contact with anyoneprob_g : whether the player on the ground After stage 1 models (5folds) training done, we predict OOF for all training data. Here we get contact with ground prediction which directly used for submission.
\nAnd contact with people prediction for stage 2.
\nWe selected the samples with prob_p > 0.145, and matched the pairs of players whose distance between players was less than 1.6 as stage2 training data.
For a single pair, we crop through 9 frames like this ↓
\n
Take two helmet bboxes and find the center point of them, expand height and width then crop out.
\nWe also draw the two bboxes of the player pair on the picture, so that the model knows which player pair is needed when there are many people in the images.
\nIn this way we are able to train 2.5D CNN to obtain the contact prediction between player pairs extracted from stage 1.
\nAs for GBDT, we simple extract some features from tracking.csv, nothing special there.
The final CV score of our part is 0.796
After ensemble with @tanakar @ryotayoshinobu we got CV 0.804
I trained many models in this competition using Z by HP Z8G4 Workstation with dual A6000 GPU as usual, it's stable and never had a problem with the training process. Thanks to Z by HP for sponsoring!
", "rawMarkdown": "Thanks to the organizers and congrats to all the winners and my wonderful teammates @boliu0 @tanakar @ryotayoshinobu \n\nIt is an interesting competition with very rich data types, we can train the model on video, or on table data, or a combination of both.\n\nSo I believe it's very likely that each team uses a different approach. That's the biggest thing I'm looking forward to about this competition - because I'll be able to learn a lot of different approachs.\n\nThis was also confirmed after we merge the team. When we teamed up with another team ( @tanakar @ryotayoshinobu ) we found that they and I surprisingly used a completely different training pipeline. After spending some time we found that our two pipelines even can hardly borrow ideas from each other. So in the end our choice was to optimize the pipelines individually and finally do a simple average ensemble.\n\n@tanakar @ryotayoshinobu part have already been post here: https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391620\n\nNext I'm going to describe our part of solution.\n\n# Summary\n\nWe designed a 2-stage pipeline. \n\n* Stage 1: Single player sequence 2.5D CNN w/ LSTM classification model.\n* Stage 2: Player pairs classification model (2.5D CNN and GBDT).\n\n# Stage 1: Single player CNN w/ LSTM\n\nFor a single player in one play, crop his boxes from all frames for both 2 views, using helmet bb extend height and width to 5x\n\n\n\n\n\nThis is an example where the top 6 imgs are Endzone and the bottom 6 imgs are Sideline, of the same player.\n\nFrom left to right, each img is one step (6 frames exactly, we don't do 59.94hz...).\n\nFor a single crop, it crop through 5 frames so it's 2.5D CNN\n\nFor training 2.5 CNN w/ LSTM, we use seq_len=128.\n\nWhich means if a video is more than 128 steps, the rest are dropped. (around 0.2% pos samples in training data are dropped here.)\n\nThe CNN w/ LSTM model take 2 view sequences as input at the same time.\n\nWe concat 2 view pairs features after backbone, then feed into LSTM.\n\nThe output of this model is:\n\n* `prob_p` : whether the player contact with anyone\n* `prob_g` : whether the player on the ground \n \n\nAfter stage 1 models (5folds) training done, we predict OOF for all training data. Here we get contact with ground prediction which directly used for submission.\n\nAnd contact with people prediction for stage 2.\n\n# Stage 2: Player pairs classification\n\nWe selected the samples with `prob_p > 0.145`, and matched the pairs of players whose distance between players was less than `1.6` as stage2 training data.\n\nFor a single pair, we crop through 9 frames like this ↓\n\n\n\nTake two helmet bboxes and find the center point of them, expand height and width then crop out.\n\nWe also draw the two bboxes of the player pair on the picture, so that the model knows which player pair is needed when there are many people in the images.\n\nIn this way we are able to train 2.5D CNN to obtain the contact prediction between player pairs extracted from stage 1.\n\nAs for GBDT, we simple extract some features from `tracking.csv`, nothing special there.\n\nThe final CV score of our part is `0.796`\n\nAfter ensemble with @tanakar @ryotayoshinobu we got CV `0.804`\n\n\n\n\n# Acknowledge\n\nI trained many models in this competition using Z by HP Z8G4 Workstation with dual A6000 GPU as usual, it's stable and never had a problem with the training process. Thanks to Z by HP for sponsoring!", "votes": null }, { "id": "2168955", "postDate": "03/04/2023 16:56:25", "content": "Thanks for posting your solution. I 100% agree about what you said \"So I believe it's very likely that each team uses a different approach. That's the biggest thing I'm looking forward to about this competition - because I'll be able to learn a lot of different approachs.\"
\nCongrats on your finish!
", "rawMarkdown": "Thanks for posting your solution. I 100% agree about what you said \"So I believe it's very likely that each team uses a different approach. That's the biggest thing I'm looking forward to about this competition - because I'll be able to learn a lot of different approachs.\"\n\nCongrats on your finish!", "votes": null }, { "id": "2173649", "postDate": "03/08/2023 14:35:17", "content": "@haqishen
\nCongrats on your gold medal!
\nOne question, is the shape of the input to the 1st stage model, for example, as follows?
(batch_size, 2, 128, 5, h, w)\nI mean,
\n2: 2views \n128: 128steps \n5: 5frames(+-2frames?)\nAnd did you input (bs*2*128, 5, h, w) into the backbone?
", "rawMarkdown": "haqishen \nCongrats on your gold medal!\nOne question, is the shape of the input to the 1st stage model, for example, as follows?\n```\n(batch_size, 2, 128, 5, h, w)\n```\nI mean,\n```\n2: 2views \n128: 128steps \n5: 5frames(+-2frames?)\n```\nAnd did you input (bs\\*2\\*128, 5, h, w) into the backbone?", "votes": null }, { "id": "2174517", "postDate": "03/09/2023 07:22:38", "content": "yes that's correct!
", "rawMarkdown": "yes that's correct!", "votes": null } ], "comments": [ { "id": 2168955, "author_name": "robikscube", "author_url": "", "post_date": "03/04/2023 16:56:25", "content": "Thanks for posting your solution. I 100% agree about what you said \"So I believe it's very likely that each team uses a different approach. That's the biggest thing I'm looking forward to about this competition - because I'll be able to learn a lot of different approachs.\"
\nCongrats on your finish!
", "votes": null, "replies": [] }, { "id": 2173649, "author_name": "yujiariyasu", "author_url": "", "post_date": "03/08/2023 14:35:17", "content": "@haqishen
\nCongrats on your gold medal!
\nOne question, is the shape of the input to the 1st stage model, for example, as follows?
(batch_size, 2, 128, 5, h, w)\nI mean,
\n2: 2views \n128: 128steps \n5: 5frames(+-2frames?)\nAnd did you input (bs*2*128, 5, h, w) into the backbone?
", "votes": null, "replies": [ { "id": 2174517, "author_name": "haqishen", "author_url": "", "post_date": "03/09/2023 07:22:38", "content": "yes that's correct!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2165784": "Thanks to the organizers and congrats to all the winners and my wonderful teammates @boliu0 @tanakar @ryotayoshinobu \n\nIt is an interesting competition with very rich data types, we can train the model on video, or on table data, or a combination of both.\n\nSo I believe it's very likely that each team uses a different approach. That's the biggest thing I'm looking forward to about this competition - because I'll be able to learn a lot of different approachs.\n\nThis was also confirmed after we merge the team. When we teamed up with another team ( @tanakar @ryotayoshinobu ) we found that they and I surprisingly used a completely different training pipeline. After spending some time we found that our two pipelines even can hardly borrow ideas from each other. So in the end our choice was to optimize the pipelines individually and finally do a simple average ensemble.\n\n@tanakar @ryotayoshinobu part have already been post here: https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391620\n\nNext I'm going to describe our part of solution.\n\n# Summary\n\nWe designed a 2-stage pipeline. \n\n* Stage 1: Single player sequence 2.5D CNN w/ LSTM classification model.\n* Stage 2: Player pairs classification model (2.5D CNN and GBDT).\n\n# Stage 1: Single player CNN w/ LSTM\n\nFor a single player in one play, crop his boxes from all frames for both 2 views, using helmet bb extend height and width to 5x\n\n\n\n\n\nThis is an example where the top 6 imgs are Endzone and the bottom 6 imgs are Sideline, of the same player.\n\nFrom left to right, each img is one step (6 frames exactly, we don't do 59.94hz...).\n\nFor a single crop, it crop through 5 frames so it's 2.5D CNN\n\nFor training 2.5 CNN w/ LSTM, we use seq_len=128.\n\nWhich means if a video is more than 128 steps, the rest are dropped. (around 0.2% pos samples in training data are dropped here.)\n\nThe CNN w/ LSTM model take 2 view sequences as input at the same time.\n\nWe concat 2 view pairs features after backbone, then feed into LSTM.\n\nThe output of this model is:\n\n* `prob_p` : whether the player contact with anyone\n* `prob_g` : whether the player on the ground \n \n\nAfter stage 1 models (5folds) training done, we predict OOF for all training data. Here we get contact with ground prediction which directly used for submission.\n\nAnd contact with people prediction for stage 2.\n\n# Stage 2: Player pairs classification\n\nWe selected the samples with `prob_p > 0.145`, and matched the pairs of players whose distance between players was less than `1.6` as stage2 training data.\n\nFor a single pair, we crop through 9 frames like this ↓\n\n\n\nTake two helmet bboxes and find the center point of them, expand height and width then crop out.\n\nWe also draw the two bboxes of the player pair on the picture, so that the model knows which player pair is needed when there are many people in the images.\n\nIn this way we are able to train 2.5D CNN to obtain the contact prediction between player pairs extracted from stage 1.\n\nAs for GBDT, we simple extract some features from `tracking.csv`, nothing special there.\n\nThe final CV score of our part is `0.796`\n\nAfter ensemble with @tanakar @ryotayoshinobu we got CV `0.804`\n\n\n\n\n# Acknowledge\n\nI trained many models in this competition using Z by HP Z8G4 Workstation with dual A6000 GPU as usual, it's stable and never had a problem with the training process. Thanks to Z by HP for sponsoring!", "2168955": "Thanks for posting your solution. I 100% agree about what you said \"So I believe it's very likely that each team uses a different approach. That's the biggest thing I'm looking forward to about this competition - because I'll be able to learn a lot of different approachs.\"\n\nCongrats on your finish!", "2173649": "haqishen \nCongrats on your gold medal!\nOne question, is the shape of the input to the 1st stage model, for example, as follows?\n```\n(batch_size, 2, 128, 5, h, w)\n```\nI mean,\n```\n2: 2views \n128: 128steps \n5: 5frames(+-2frames?)\n```\nAnd did you input (bs\\*2\\*128, 5, h, w) into the backbone?", "2174517": "yes that's correct!" }, "source": "meta" }