Thank you Kaggle and Pop sign for hosting this competition. I enjoyed working on this problem and learned quite a bit from really good discussion posts and code shared - Thank you @hengck23
\nThings that didn’t work
\nBased on private LB scores it seems that data in private LB is much less noisy than training set.
\nMy code here - https://github.com/rashmibanthia/ASL
", "messages": [ { "id": "2242453", "postDate": "05/02/2023 09:45:01", "content": "Thank you Kaggle and Pop sign for hosting this competition. I enjoyed working on this problem and learned quite a bit from really good discussion posts and code shared - Thank you @hengck23
\nThings that didn’t work
\nBased on private LB scores it seems that data in private LB is much less noisy than training set.
\nMy code here - https://github.com/rashmibanthia/ASL
", "rawMarkdown": "Thank you Kaggle and Pop sign for hosting this competition. I enjoyed working on this problem and learned quite a bit from really good [discussion posts](https://www.kaggle.com/competitions/asl-signs/discussion/391265) and [code](https://www.kaggle.com/datasets/hengck23/asl-demo) shared - Thank you @hengck23 \n\n\n- Pytorch 2.0 (pure pytorch no training library)→ ONNX → TFLITE\n- Max length i.e. number of frames per video = 256 (Center crop)\n- Normalization by LIP, SPOSE, Lhand and Rhand (total 90 kepyoints)\n- Flip augmentation (p=0.5)\n- Distance, motion and acceleration features (Total 1050)\n- Model almost same as shared [here](https://www.kaggle.com/code/hengck23/lb-0-73-single-fold-transformer-architecture]) - with positional encoding 16, added layernorm in FeedForwardNetwork and 2 fully connected layers with batchnorm, CLS token + mean pooling for final fully connected layer\n- Loss function cross entropy with label smoothing = 0.75\n- Epochs 50 - dropout 0.0 for 15 epochs, dropout 0.4 from 15-35 epochs, dropout 0.2 for the remaining\n- FP16 quantization reduced model size significantly - final size ~26MB and ~60 mins for 5 folds\n\nThings that didn’t work \n\n- External data - It improved by 3-4 ranks but decided to not use it , too messy data and no commercial license\n- Adding more features, reducing max length, removing data with too high/low number of frames\n- SWA - couldn’t get it to work\n- Augment Affine improved CV and private LB but not public LB.\n- Various hyper parameters like # of encoder blocks, embedding dimension\n- Arcface Loss - Based on this [post](https://www.kaggle.com/competitions/asl-signs/discussion/406301) it seems Arcface loss + label smoothing didn’t work . Should have tried just Arcface and no label smoothing.\n- Also tried to convert TF model [here](https://www.kaggle.com/code/markwijkhuizen/gislr-tf-data-processing-transformer-training) to PyTorch\n\nBased on private LB scores it seems that data in private LB is much less noisy than training set. \n\n----\n\nMy code here - https://github.com/rashmibanthia/ASL", "votes": null }, { "id": "2242597", "postDate": "05/02/2023 11:44:43", "content": "Superb Finish ☺️
", "rawMarkdown": "Superb Finish ☺️", "votes": null }, { "id": "2242700", "postDate": "05/02/2023 13:07:56", "content": "Thank you :)
", "rawMarkdown": "Thank you :)", "votes": null }, { "id": "2242927", "postDate": "05/02/2023 15:25:31", "content": "@rashmibanthia congrats with silver!🎉 Thank you for sharing your code and explanation! 👍 It helps a lot🙂
", "rawMarkdown": "rashmibanthia congrats with silver!🎉 Thank you for sharing your code and explanation! 👍 It helps a lot🙂", "votes": null } ], "comments": [ { "id": 2242597, "author_name": "aman1391", "author_url": "", "post_date": "05/02/2023 11:44:43", "content": "Superb Finish ☺️
", "votes": null, "replies": [ { "id": 2242700, "author_name": "rashmibanthia", "author_url": "", "post_date": "05/02/2023 13:07:56", "content": "Thank you :)
", "votes": null, "replies": [] } ] }, { "id": 2242927, "author_name": "ivanisaev", "author_url": "", "post_date": "05/02/2023 15:25:31", "content": "@rashmibanthia congrats with silver!🎉 Thank you for sharing your code and explanation! 👍 It helps a lot🙂
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2242453": "Thank you Kaggle and Pop sign for hosting this competition. I enjoyed working on this problem and learned quite a bit from really good [discussion posts](https://www.kaggle.com/competitions/asl-signs/discussion/391265) and [code](https://www.kaggle.com/datasets/hengck23/asl-demo) shared - Thank you @hengck23 \n\n\n- Pytorch 2.0 (pure pytorch no training library)→ ONNX → TFLITE\n- Max length i.e. number of frames per video = 256 (Center crop)\n- Normalization by LIP, SPOSE, Lhand and Rhand (total 90 kepyoints)\n- Flip augmentation (p=0.5)\n- Distance, motion and acceleration features (Total 1050)\n- Model almost same as shared [here](https://www.kaggle.com/code/hengck23/lb-0-73-single-fold-transformer-architecture]) - with positional encoding 16, added layernorm in FeedForwardNetwork and 2 fully connected layers with batchnorm, CLS token + mean pooling for final fully connected layer\n- Loss function cross entropy with label smoothing = 0.75\n- Epochs 50 - dropout 0.0 for 15 epochs, dropout 0.4 from 15-35 epochs, dropout 0.2 for the remaining\n- FP16 quantization reduced model size significantly - final size ~26MB and ~60 mins for 5 folds\n\nThings that didn’t work \n\n- External data - It improved by 3-4 ranks but decided to not use it , too messy data and no commercial license\n- Adding more features, reducing max length, removing data with too high/low number of frames\n- SWA - couldn’t get it to work\n- Augment Affine improved CV and private LB but not public LB.\n- Various hyper parameters like # of encoder blocks, embedding dimension\n- Arcface Loss - Based on this [post](https://www.kaggle.com/competitions/asl-signs/discussion/406301) it seems Arcface loss + label smoothing didn’t work . Should have tried just Arcface and no label smoothing.\n- Also tried to convert TF model [here](https://www.kaggle.com/code/markwijkhuizen/gislr-tf-data-processing-transformer-training) to PyTorch\n\nBased on private LB scores it seems that data in private LB is much less noisy than training set. \n\n----\n\nMy code here - https://github.com/rashmibanthia/ASL", "2242597": "Superb Finish ☺️", "2242700": "Thank you :)", "2242927": "rashmibanthia congrats with silver!🎉 Thank you for sharing your code and explanation! 👍 It helps a lot🙂" }, "source": "meta" }