![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3543139%2Fdf0bb9e27b1053f0638883ba4d0422a5%2Fmodel.png?generation=1587741613868657&alt=media\")
This is a brief write-up, if you want to you can read my full ramblings here along with my code: https://github.com/GreatGameDota/Deepfake-Detection
\n\nOnce again an amazing Kaggle competition that I had a ton of pleasure participating in! I will like to thank Kaggle and the hosts for putting together this competition and working hard to make sure it went as smooth as it did!
\n\nI will also like to shout out some great people who helped me a lot in the competition:
\n\nThanks to @phunghieu Hieu Phung for his amazing dataset that I used which can be found here: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/128954#736780
\n\nThanks to @unkownhihi Shangqiu Li and team for all of the things they shared over the course of the competition. Most importantly the Mobilenet face extractor which can be found here: https://www.kaggle.com/unkownhihi/mobilenet-face-extractor-helper-code
\n\nThanks to @humananalog for his starter code notebook: https://www.kaggle.com/humananalog/binary-image-classifier-training-demo
\n\n
Image resolution: 150x150\nMobilenet was used to extract faces for training and inference\nThe dataset I used is public and can be found here: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/134420
\n\nI ensembled all the models using the geometric mean of their probabilities.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3543139%2F63e59bfe9c96f5014bd4c42bab52849c%2Fscores.png?generation=1587741641643531&alt=media\")
This was another great Kaggle competition! I'm so happy I achieved such a great performance on a competition that a lot of people just skipped due to the amount of data. I'm also surprised I performed so well with only Kaggle kernals!
\n\nI'm so excited to get my second ever competition medal and I'm even more excited because it is my first silver medal! I now graduate to Kaggle competition expert!
\n\nMy baseline kernal: https://www.kaggle.com/greatgamedota/xception-classifier-w-ffhq-training-lb-537\nMy submission kernal: https://www.kaggle.com/greatgamedota/69th-place-solution-mobilenet-inference/\nImage dataset kernal: https://www.kaggle.com/greatgamedota/deepfake-detection-full-data
", "messages": [ { "id": 818452, "postDate": "2020-04-23T22:37:34.633Z", "content": "This is a brief write-up, if you want to you can read my full ramblings here along with my code: https://github.com/GreatGameDota/Deepfake-Detection
\n\nOnce again an amazing Kaggle competition that I had a ton of pleasure participating in! I will like to thank Kaggle and the hosts for putting together this competition and working hard to make sure it went as smooth as it did!
\n\nI will also like to shout out some great people who helped me a lot in the competition:
\n\nThanks to @phunghieu Hieu Phung for his amazing dataset that I used which can be found here: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/128954#736780
\n\nThanks to @unkownhihi Shangqiu Li and team for all of the things they shared over the course of the competition. Most importantly the Mobilenet face extractor which can be found here: https://www.kaggle.com/unkownhihi/mobilenet-face-extractor-helper-code
\n\nThanks to @humananalog for his starter code notebook: https://www.kaggle.com/humananalog/binary-image-classifier-training-demo
\n\n
Image resolution: 150x150\nMobilenet was used to extract faces for training and inference\nThe dataset I used is public and can be found here: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/134420
\n\nI ensembled all the models using the geometric mean of their probabilities.
\n\n
This was another great Kaggle competition! I'm so happy I achieved such a great performance on a competition that a lot of people just skipped due to the amount of data. I'm also surprised I performed so well with only Kaggle kernals!
\n\nI'm so excited to get my second ever competition medal and I'm even more excited because it is my first silver medal! I now graduate to Kaggle competition expert!
\n\nMy baseline kernal: https://www.kaggle.com/greatgamedota/xception-classifier-w-ffhq-training-lb-537\nMy submission kernal: https://www.kaggle.com/greatgamedota/69th-place-solution-mobilenet-inference/\nImage dataset kernal: https://www.kaggle.com/greatgamedota/deepfake-detection-full-data
", "rawMarkdown": "This is a brief write-up, if you want to you can read my full ramblings here along with my code: https://github.com/GreatGameDota/Deepfake-Detection\n\n# TLDR: Ensemble for the win!\n\nOnce again an amazing Kaggle competition that I had a ton of pleasure participating in! I will like to thank Kaggle and the hosts for putting together this competition and working hard to make sure it went as smooth as it did!\n\nI will also like to shout out some great people who helped me a lot in the competition:\n\nThanks to @phunghieu Hieu Phung for his amazing dataset that I used which can be found here: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/128954#736780\n\nThanks to @unkownhihi Shangqiu Li and team for all of the things they shared over the course of the competition. Most importantly the Mobilenet face extractor which can be found here: https://www.kaggle.com/unkownhihi/mobilenet-face-extractor-helper-code\n\nThanks to @humananalog for his starter code notebook: https://www.kaggle.com/humananalog/binary-image-classifier-training-demo\n\n## Models\n\n- Xception\n- EfficientNet B1\n\n\n\n## Dataset\n\nImage resolution: 150x150\nMobilenet was used to extract faces for training and inference\nThe dataset I used is public and can be found here: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/134420\n\n## Augmentation\n\n- Horizontal flip\n- Jpeg Compression\n- Downscale\n\n## Training\n\n- Adam optimizer\n- 20 epochs\n- Binary CrossEntropy Loss function\n- ReduceLROnPlateau\n- Save checkpoint based on validation LogLoss\n- Folder-wise Validation split: 0-40 train, 41-49 validation\n- Simple undersampling was used to balance the data\n- 1 frame was randomly picked from every video while training\n\n## Ensembling/Blending\n\nI ensembled all the models using the geometric mean of their probabilities.\n\n## Final Submission\n\n\n\n## What didn't work\n\n- LRCN (CNN + RNN)\n- External data\n- Post-processing/Clipping\n\n## Final Thoughts\n\nThis was another great Kaggle competition! I'm so happy I achieved such a great performance on a competition that a lot of people just skipped due to the amount of data. I'm also surprised I performed so well with only Kaggle kernals!\n\nI'm so excited to get my second ever competition medal and I'm even more excited because it is my first silver medal! I now graduate to Kaggle competition expert!\n\n## Some helpful Links\n\nMy baseline kernal: https://www.kaggle.com/greatgamedota/xception-classifier-w-ffhq-training-lb-537\nMy submission kernal: https://www.kaggle.com/greatgamedota/69th-place-solution-mobilenet-inference/\nImage dataset kernal: https://www.kaggle.com/greatgamedota/deepfake-detection-full-data", "votes": 18 }, { "id": 1108709, "postDate": "2020-12-10T23:40:06.700Z", "content": "@greatgamedota Hi, Can you help me with understanding your model architecture please? https://github.com/GreatGameDota/Deepfake-Detection/blob/master/training.ipynb
\nI can see that you initialise two BatchNorm1d:
\nself.b1 = nn.BatchNorm1d(in_f)
\nself.b2 = nn.BatchNorm1d(512)
but in forward function you do not use them, could you explain why?
", "rawMarkdown": "@greatgamedota Hi, Can you help me with understanding your model architecture please? https://github.com/GreatGameDota/Deepfake-Detection/blob/master/training.ipynb\n\nI can see that you initialise two BatchNorm1d:\nself.b1 = nn.BatchNorm1d(in_f)\nself.b2 = nn.BatchNorm1d(512)\n\nbut in forward function you do not use them, could you explain why?", "votes": 1, "replies": [ { "id": 1108728, "postDate": "2020-12-11T00:23:36.127Z", "content": "As said in my write-up, each model varies in the architecture. My finished code I cleaned up and deleted the commented out batchnorm calls because my last experiments didn't use them. A version that still has the batch norms is my baseline notebook model.
", "rawMarkdown": "As said in my write-up, each model varies in the architecture. My finished code I cleaned up and deleted the commented out batchnorm calls because my last experiments didn't use them. A version that still has the batch norms is my baseline notebook model." } ] }, { "id": 819446, "postDate": "2020-04-24T15:43:05.103Z", "content": "Such a great solution, thanks for sharing! And ... Congratulation!!!
", "rawMarkdown": "Such a great solution, thanks for sharing! And ... Congratulation!!!", "votes": 1 }, { "id": 818710, "postDate": "2020-04-24T04:17:20.433Z", "content": "Wow, congratulation, excellent. Thanks for sharing.
", "rawMarkdown": "Wow, congratulation, excellent. Thanks for sharing.", "votes": 1 }, { "id": 818688, "postDate": "2020-04-24T03:55:04.463Z", "content": "Congratulations!
", "rawMarkdown": "Congratulations!", "votes": 1 }, { "id": 819441, "postDate": "2020-04-24T15:36:34.973Z", "content": "I have just updated this post with a brief write up! Thanks for the kind words everyone!
", "rawMarkdown": "I have just updated this post with a brief write up! Thanks for the kind words everyone!" }, { "id": 819015, "postDate": "2020-04-24T09:25:33.213Z", "content": "congratulation! Wish someday I can get a silver too.
", "rawMarkdown": "congratulation! Wish someday I can get a silver too.", "votes": 1, "isDeleted": true } ], "comments": [ { "id": 1108709, "author_name": "AugustasM", "author_url": "", "post_date": "2020-12-10T23:40:06.700000", "content": "@greatgamedota Hi, Can you help me with understanding your model architecture please? https://github.com/GreatGameDota/Deepfake-Detection/blob/master/training.ipynb
\nI can see that you initialise two BatchNorm1d:
\nself.b1 = nn.BatchNorm1d(in_f)
\nself.b2 = nn.BatchNorm1d(512)
but in forward function you do not use them, could you explain why?
", "votes": 1, "replies": [ { "id": 1108728, "author_name": "GreatGameDota", "author_url": "", "post_date": "2020-12-11T00:23:36.127000", "content": "As said in my write-up, each model varies in the architecture. My finished code I cleaned up and deleted the commented out batchnorm calls because my last experiments didn't use them. A version that still has the batch norms is my baseline notebook model.
", "votes": 0, "replies": [] } ] }, { "id": 819446, "author_name": "Hieu Phung", "author_url": "", "post_date": "2020-04-24T15:43:05.103000", "content": "Such a great solution, thanks for sharing! And ... Congratulation!!!
", "votes": 1, "replies": [] }, { "id": 818710, "author_name": "Zungmann", "author_url": "", "post_date": "2020-04-24T04:17:20.433000", "content": "Wow, congratulation, excellent. Thanks for sharing.
", "votes": 1, "replies": [] }, { "id": 818688, "author_name": "Hilal Shaath", "author_url": "", "post_date": "2020-04-24T03:55:04.463000", "content": "Congratulations!
", "votes": 1, "replies": [] }, { "id": 819441, "author_name": "GreatGameDota", "author_url": "", "post_date": "2020-04-24T15:36:34.973000", "content": "I have just updated this post with a brief write up! Thanks for the kind words everyone!
", "votes": 0, "replies": [] }, { "id": 819015, "author_name": "", "author_url": "", "post_date": "2020-04-24T09:25:33.213000", "content": "congratulation! Wish someday I can get a silver too.
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "818452": "This is a brief write-up, if you want to you can read my full ramblings here along with my code: https://github.com/GreatGameDota/Deepfake-Detection\n\n# TLDR: Ensemble for the win!\n\nOnce again an amazing Kaggle competition that I had a ton of pleasure participating in! I will like to thank Kaggle and the hosts for putting together this competition and working hard to make sure it went as smooth as it did!\n\nI will also like to shout out some great people who helped me a lot in the competition:\n\nThanks to @phunghieu Hieu Phung for his amazing dataset that I used which can be found here: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/128954#736780\n\nThanks to @unkownhihi Shangqiu Li and team for all of the things they shared over the course of the competition. Most importantly the Mobilenet face extractor which can be found here: https://www.kaggle.com/unkownhihi/mobilenet-face-extractor-helper-code\n\nThanks to @humananalog for his starter code notebook: https://www.kaggle.com/humananalog/binary-image-classifier-training-demo\n\n## Models\n\n- Xception\n- EfficientNet B1\n\n\n\n## Dataset\n\nImage resolution: 150x150\nMobilenet was used to extract faces for training and inference\nThe dataset I used is public and can be found here: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/134420\n\n## Augmentation\n\n- Horizontal flip\n- Jpeg Compression\n- Downscale\n\n## Training\n\n- Adam optimizer\n- 20 epochs\n- Binary CrossEntropy Loss function\n- ReduceLROnPlateau\n- Save checkpoint based on validation LogLoss\n- Folder-wise Validation split: 0-40 train, 41-49 validation\n- Simple undersampling was used to balance the data\n- 1 frame was randomly picked from every video while training\n\n## Ensembling/Blending\n\nI ensembled all the models using the geometric mean of their probabilities.\n\n## Final Submission\n\n\n\n## What didn't work\n\n- LRCN (CNN + RNN)\n- External data\n- Post-processing/Clipping\n\n## Final Thoughts\n\nThis was another great Kaggle competition! I'm so happy I achieved such a great performance on a competition that a lot of people just skipped due to the amount of data. I'm also surprised I performed so well with only Kaggle kernals!\n\nI'm so excited to get my second ever competition medal and I'm even more excited because it is my first silver medal! I now graduate to Kaggle competition expert!\n\n## Some helpful Links\n\nMy baseline kernal: https://www.kaggle.com/greatgamedota/xception-classifier-w-ffhq-training-lb-537\nMy submission kernal: https://www.kaggle.com/greatgamedota/69th-place-solution-mobilenet-inference/\nImage dataset kernal: https://www.kaggle.com/greatgamedota/deepfake-detection-full-data", "1108709": "@greatgamedota Hi, Can you help me with understanding your model architecture please? https://github.com/GreatGameDota/Deepfake-Detection/blob/master/training.ipynb\n\nI can see that you initialise two BatchNorm1d:\nself.b1 = nn.BatchNorm1d(in_f)\nself.b2 = nn.BatchNorm1d(512)\n\nbut in forward function you do not use them, could you explain why?", "819446": "Such a great solution, thanks for sharing! And ... Congratulation!!!", "818710": "Wow, congratulation, excellent. Thanks for sharing.", "818688": "Congratulations!", "819441": "I have just updated this post with a brief write up! Thanks for the kind words everyone!", "819015": "congratulation! Wish someday I can get a silver too." } }