Hello DFDC competitors,
\n\nWe’re happy to announce the winners of the Deepfake Detection Challenge. Over the course of 3.5 months, there were 2,114 participants who submitted over 35,000 models to the DFDC.
\n\nWe want to extend a huge congratulations to the following teams who placed in the top five and will receive prizes:
\n\nWe took extreme care when running the final submissions against the private test set. We invested all necessary resources to make sure we were scoring everyone fairly, ruling out any execution errors related to environment issues. Faulty submissions stemmed mostly from coding bugs and timeouts, among other factors. Unfortunately, we can’t provide a detailed log for every participant.
\n\nYou may notice that the top rankings have changed. Unfortunately, the top two teams in the preliminary standings used external data sources in their winning submissions that were not allowed under the rules of this competition.
\n\nWhile the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique. As a result of an extensive review of the context and licensing of the prior first-place team’s external data sources, it was determined that team removal was not warranted and their second selected submission should stand. This has resulted in their final standing being seventh place.
\n\nWe’re grateful for the contributions of all participants in this challenge. We know that many of you have dedicated a lot of time to develop high quality and unique approaches.
\n\nWe recognize that no single organization or individual can solve the issue of deepfakes on their own. This open and collaborative effort has enabled us to work together, accelerate progress, and ultimately we hope will help prevent people from being deceived by the images and videos they see online.
\n\nThanks,\nCristian Canton Ferrer, Research Manager, Facebook
", "messages": [ { "id": 883409, "postDate": "2020-06-12T16:00:00.253Z", "content": "congratulation @selimsef, @jamesphoward and all the prize winners, we will release a statement later to further explain the situation regarding our team
", "rawMarkdown": "congratulation @selimsef, @jamesphoward and all the prize winners, we will release a statement later to further explain the situation regarding our team", "votes": 30, "replies": [ { "id": 883692, "postDate": "2020-06-12T20:44:00.613Z", "content": "We've now posted our statement: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/157983
", "rawMarkdown": "We've now posted our statement: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/157983", "votes": 19 }, { "id": 886337, "postDate": "2020-06-15T00:11:02.660Z", "content": "@anokas @titericz @yifanxie , you guys will always be super stars!. I am happy that you will remain in the community, and I totally support Kaggle's decision to keep you. We need you for important problems, and to continue and learn from your work...
\n\nPlease continue to contribute to this community, many of us learn so much of you.
", "rawMarkdown": "@anokas @titericz @yifanxie , you guys will always be super stars!. I am happy that you will remain in the community, and I totally support Kaggle's decision to keep you. We need you for important problems, and to continue and learn from your work...\n\nPlease continue to contribute to this community, many of us learn so much of you. ", "votes": 1 } ] }, { "id": 885420, "postDate": "2020-06-14T07:39:01.873Z", "content": "It is quite unusual at kaggle that the winner announcement post has more downvotes than upvotes.
\n\n> We invested all necessary resources to make sure we were scoring everyone fairly
\n\nFairness is the last thing that comes to my mind... @cristiancanton as you have PhD in Computer Vision probably you see how absurd/questionable is the reasoning behind disqualifying the best submission.\n I don't expect that facebook changes the decision because of anything written in the forum but\n I still recommend you to read the other thread and try to understand the frustration of competitors.\n You spent two months but you decided this way anyway (which has probably more to Legal/PR/Marketing departments).
", "rawMarkdown": "It is quite unusual at kaggle that the winner announcement post has more downvotes than upvotes.\n\n> We invested all necessary resources to make sure we were scoring everyone fairly\n\n Fairness is the last thing that comes to my mind... @cristiancanton as you have PhD in Computer Vision probably you see how absurd/questionable is the reasoning behind disqualifying the best submission.\n I don't expect that facebook changes the decision because of anything written in the forum but\n I still recommend you to read the other thread and try to understand the frustration of competitors.\n You spent two months but you decided this way anyway (which has probably more to Legal/PR/Marketing departments).", "votes": 25 }, { "id": 907369, "postDate": "2020-06-29T22:40:28.683Z", "content": "The public and half private LB datasets are now available for download.
\n\nUnfortunately, after analysing this data and reran our best submission, I had even more questions to the organizer @cristiancanton and authors of the paper, Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, Cristian Canton Ferrer:
\n\nHow did you collect the real ”in-the-wild” Deepfake videos and, more importantly, how did you label it?
Why didn't you warn the participants that voice manipulation will not be evaluated in this competition? Instead, you made it very clear that we should detect both audio and video modifications.
If you silently decide not to include voice manipulations into DFDC-like half of private dataset, how can you guarantee that the real \"in-the-wild\" videos do not have voice manipulation?
Hello, have the organizers ever given you any explanation or reply ~ ?
", "rawMarkdown": "Hello, have the organizers ever given you any explanation or reply ~ ?" } ] }, { "id": 883368, "postDate": "2020-06-12T15:11:47.587Z", "content": "So pleased with our final placing, and we've uploaded our full source code to GitHub here.
\n\nMany thanks to the team at Kaggle and Facebook!
", "rawMarkdown": "So pleased with our final placing, and we've uploaded our full source code to GitHub [here](https://github.com/jphdotam/DFDC).\n\nMany thanks to the team at Kaggle and Facebook!", "votes": 19, "replies": [ { "id": 886334, "postDate": "2020-06-15T00:06:03.057Z", "content": "Congrate @jamesphoward . Thank you for sharing the code
", "rawMarkdown": "Congrate @jamesphoward . Thank you for sharing the code" }, { "id": 893107, "postDate": "2020-06-19T11:52:23.733Z", "content": "Thanks for sharing the code! Very helpful.
", "rawMarkdown": "Thanks for sharing the code! Very helpful." }, { "id": 919216, "postDate": "2020-07-07T19:02:10.973Z", "content": "Thanks
", "rawMarkdown": "Thanks \n" } ] }, { "id": 883521, "postDate": "2020-06-12T17:58:29.290Z", "content": "Congrats to the winners!
\n\nBUT\nThe semi-disqualification is fishy... I am very disappointed about the decision and it undermines trust from competitor point of view. Waiting for further details from All Faces Are Real before drawing too harsh conclusions.
This competition was honestly a mess especially considering the importance of this problem and the large prize. Hopefully the organizers got something somewhat helpful and I hope they'll release the full data set for future research.\nCongrats to the winners!
", "rawMarkdown": "This competition was honestly a mess especially considering the importance of this problem and the large prize. Hopefully the organizers got something somewhat helpful and I hope they'll release the full data set for future research.\nCongrats to the winners!", "votes": 15 }, { "id": 886431, "postDate": "2020-06-15T03:25:55.543Z", "content": "we are 2nd removed Team. if anyone is interested in our solution, welcome to this link
\n\nThis competition draws huge attention from both industry and academia, it will definitely boost the development of deepfake detection techniques to fight against fake videos.
\n\nBut the rules are complicated to new Kagglers and we believe it would be much clearer to state that no external data is allowed for this competition at the very beginning. We are deeply disappointed by the final decision that both our scores are removed from the leaderboards. This simply erased all our efforts devoted to this competition.
\n\nAnyway, we still like to thank Facebook and Kaggle for organizing this event
", "rawMarkdown": "we are 2nd removed Team. if anyone is interested in our solution, welcome to this [link](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/158506) \n\nThis competition draws huge attention from both industry and academia, it will definitely boost the development of deepfake detection techniques to fight against fake videos. \n\nBut the rules are complicated to new Kagglers and we believe **it would be much clearer to state that no external data is allowed for this competition at the very beginning**. We are deeply disappointed by the final decision that both our scores are removed from the leaderboards. This simply erased all our efforts devoted to this competition.\n\nAnyway, we still like to thank Facebook and Kaggle for organizing this event", "votes": 13 }, { "id": 889534, "postDate": "2020-06-17T02:03:14.680Z", "content": "please release the full dataset (public/private test set) for future deepfake research 🙏
", "rawMarkdown": "please release the full dataset (public/private test set) for future deepfake research 🙏 ", "votes": 10 }, { "id": 883509, "postDate": "2020-06-12T17:40:49.177Z", "content": "I don't know what just happened. Congrats @selimsef. You must be super happy. 2nd GM after b. e. s. from Belarus :)
", "rawMarkdown": "I don't know what just happened. Congrats @selimsef. You must be super happy. 2nd GM after b. e. s. from Belarus :)", "votes": 7, "replies": [ { "id": 883573, "postDate": "2020-06-12T18:32:55.683Z", "content": "Thank you! Yes, I'm very happy with the results ✌️
", "rawMarkdown": "Thank you! Yes, I'm very happy with the results ✌️ ", "votes": 7 }, { "id": 886333, "postDate": "2020-06-15T00:05:39.510Z", "content": "Congrats @selimsef
", "rawMarkdown": "Congrats @selimsef ", "votes": 1 }, { "id": 1014700, "postDate": "2020-09-17T16:42:25.893Z", "content": "Congrats @selimsef for winning the competition! Is your code available publicly somewhere? I am doing some research on deepfakes and would like to test some videos against it?
", "rawMarkdown": "Congrats @selimsef for winning the competition! Is your code available publicly somewhere? I am doing some research on deepfakes and would like to test some videos against it?", "votes": 1 } ] }, { "id": 883470, "postDate": "2020-06-12T16:52:31.690Z", "content": "\n\n\nWhile the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique. As a result of an extensive review of the context and licensing of the prior first-place team’s external data sources, it was determined that team removal was not warranted and their second selected submission should stand. This has resulted in their final standing being seventh place.
\n
\"different and unique\" Translation: they actually didn't violate any official competition rules, but we didn't like their submission.
", "rawMarkdown": "> While the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique. As a result of an extensive review of the context and licensing of the prior first-place team’s external data sources, it was determined that team removal was not warranted and their second selected submission should stand. This has resulted in their final standing being seventh place.\n\n\"different and unique\" Translation: they actually **didn't** violate any official competition rules, but we didn't like their submission.\n\n", "votes": 9, "replies": [ { "id": 883500, "postDate": "2020-06-12T17:32:00.123Z", "content": "please, show some respect to hosts, thanks
", "rawMarkdown": "please, show some respect to hosts, thanks", "votes": -21 }, { "id": 893108, "postDate": "2020-06-19T11:53:31.893Z", "content": "Ohhhhh dear
", "rawMarkdown": "Ohhhhh dear " } ] }, { "id": 883535, "postDate": "2020-06-12T18:04:58.320Z", "content": "@cristiancanton did you deliberately mislead the participants with voice manipulations? After reading the paper I still do not understand why you did not clarify this in the competition description:
\n\n\n", "rawMarkdown": "@cristiancanton did you deliberately mislead the participants with voice manipulations? After reading the [paper](https://scontent.fhen1-1.fna.fbcdn.net/v/t39.8562-6/10000000_2949149428508489_1422548062431945722_n.pdf?_nc_cat=111&_nc_sid=ae5e01&_nc_ohc=B5PFMIJniQkAX9Uee--&_nc_ht=scontent.fhen1-1.fna&oh=73fe20850318a06f3e7309cd611fdbfd&oe=5F07F184) I still do not understand why you did not clarify this in the competition description:\n> Although these audio manipulations were not considered ‘Deepfakes’ for the competition, we included them in the set to provide more breadth of manipulation type, and they may be of use in further research.\n\n", "votes": 6, "replies": [ { "id": 887413, "postDate": "2020-06-15T16:59:15.577Z", "content": "Although these audio manipulations were not considered ‘Deepfakes’ for the competition, we included them in the set to provide more breadth of manipulation type, and they may be of use in further research.
\n
It is very misleading if audio manipulations were not labeled as Deepfake in the private dataset considering the subtitle of the competition states \"Identify videos with facial or voice manipulations\". It is even more misleading considering that audio analysis significantly helps in the provided dataset and also in public LB, implying that videos with audio alterations were labeled as FAKE in both datasets. Whereas in the private dataset, this is clearly not the case, as people who have reported including audio analysis in the private solution had a massive performance hit (for instance, I went from 1st public to 905th private) which statistically could not be explained by generalization error (at least in my case).
", "rawMarkdown": "It is very misleading if audio manipulations were not labeled as Deepfake in the private dataset considering the subtitle of the competition states \"Identify videos with facial or voice manipulations\". It is even more misleading considering that audio analysis significantly helps in the provided dataset and also in public LB, implying that videos with audio alterations were labeled as FAKE in both datasets. Whereas in the private dataset, this is clearly not the case, as people who have reported including audio analysis in the private solution had a massive performance hit (for instance, I went from 1st public to 905th private) which statistically could not be explained by generalization error (at least in my case).", "votes": 1 }, { "id": 887669, "postDate": "2020-06-15T19:48:22.503Z", "content": "Few hours ago we emailed to the authors and @cristiancanton answered that audio manipulations are not considered as DeepFakes, but they were labelled as FAKE in the dataset. He also mentioned that the data will be published this week. I hope it will shed some light on the audio analysis, and we will be able to reproduce the public/private LB scores.
", "rawMarkdown": "Few hours ago we emailed to the authors and @cristiancanton answered that audio manipulations are not considered as DeepFakes, but they were labelled as FAKE in the dataset. He also mentioned that the data will be published this week. I hope it will shed some light on the audio analysis, and we will be able to reproduce the public/private LB scores.", "votes": 3 } ] }, { "id": 883411, "postDate": "2020-06-12T16:03:47.257Z", "content": "Out of curiosity: Would it be possible to know what external datasets were used that are not allowed?
", "rawMarkdown": "Out of curiosity: Would it be possible to know what external datasets were used that are not allowed?", "votes": 4, "replies": [ { "id": 883415, "postDate": "2020-06-12T16:07:16.207Z", "content": "\n\n\nJan Bre wrote:
\n \nOut of curiosity: Would it be possible to know what external datasets were used that are not allowed?
\n
from our point of view, it is quite a story, we will share when we are ready
", "rawMarkdown": "> **Jan Bre wrote:**\n> \n> Out of curiosity: Would it be possible to know what external datasets were used that are not allowed?\n\nfrom our point of view, it is quite a story, we will share when we are ready", "votes": 14 }, { "id": 883418, "postDate": "2020-06-12T16:08:34.723Z", "content": "Thank you 🙌
", "rawMarkdown": "Thank you 🙌 ", "votes": 1 } ] }, { "id": 883340, "postDate": "2020-06-12T14:47:19.187Z", "content": "Hello DFDC competitors,
\n\nWe’re happy to announce the winners of the Deepfake Detection Challenge. Over the course of 3.5 months, there were 2,114 participants who submitted over 35,000 models to the DFDC.
\n\nWe want to extend a huge congratulations to the following teams who placed in the top five and will receive prizes:
\n\nWe took extreme care when running the final submissions against the private test set. We invested all necessary resources to make sure we were scoring everyone fairly, ruling out any execution errors related to environment issues. Faulty submissions stemmed mostly from coding bugs and timeouts, among other factors. Unfortunately, we can’t provide a detailed log for every participant.
\n\nYou may notice that the top rankings have changed. Unfortunately, the top two teams in the preliminary standings used external data sources in their winning submissions that were not allowed under the rules of this competition.
\n\nWhile the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique. As a result of an extensive review of the context and licensing of the prior first-place team’s external data sources, it was determined that team removal was not warranted and their second selected submission should stand. This has resulted in their final standing being seventh place.
\n\nWe’re grateful for the contributions of all participants in this challenge. We know that many of you have dedicated a lot of time to develop high quality and unique approaches.
\n\nWe recognize that no single organization or individual can solve the issue of deepfakes on their own. This open and collaborative effort has enabled us to work together, accelerate progress, and ultimately we hope will help prevent people from being deceived by the images and videos they see online.
\n\nThanks,\nCristian Canton Ferrer, Research Manager, Facebook
", "rawMarkdown": "Hello DFDC competitors,\n\nWe’re happy to announce the winners of the Deepfake Detection Challenge. Over the course of 3.5 months, there were 2,114 participants who submitted over 35,000 models to the DFDC.\n\nWe want to extend a huge congratulations to the following teams who placed in the top five and will receive prizes:\n\n- Selim Seferbekov\n- \\WM/\n- NtechLab\n- Eighteen years old\n- The Medics\n\nWe took extreme care when running the final submissions against the private test set. We invested all necessary resources to make sure we were scoring everyone fairly, ruling out any execution errors related to environment issues. Faulty submissions stemmed mostly from coding bugs and timeouts, among other factors. Unfortunately, we can’t provide a detailed log for every participant.\n\nYou may notice that the top rankings have changed. Unfortunately, the top two teams in the preliminary standings used external data sources in their winning submissions that were not allowed under the rules of this competition. \n\nWhile the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique. As a result of an extensive review of the context and licensing of the prior first-place team’s external data sources, it was determined that team removal was not warranted and their second selected submission should stand. This has resulted in their final standing being seventh place.\n\nWe’re grateful for the contributions of all participants in this challenge. We know that many of you have dedicated a lot of time to develop high quality and unique approaches.\n\nWe recognize that no single organization or individual can solve the issue of deepfakes on their own. This open and collaborative effort has enabled us to work together, accelerate progress, and ultimately we hope will help prevent people from being deceived by the images and videos they see online.\n\nThanks,\nCristian Canton Ferrer, Research Manager, Facebook \n", "votes": -48 }, { "id": 883518, "postDate": "2020-06-12T17:54:23.270Z", "content": "Congratulations to the winners!
\n\n@cristiancanton are you going to open late submissions and/or give and access to the test data? I wan't to research the problem further. I already got fairly good result on validation set and wish I can try it on test set.
", "rawMarkdown": "Congratulations to the winners! \n\n@cristiancanton are you going to open late submissions and/or give and access to the test data? I wan't to research the problem further. I already got fairly good result on validation set and wish I can try it on test set.", "votes": -1 }, { "id": 886395, "postDate": "2020-06-15T02:42:19.630Z", "content": "I wonder if there will be a certificate for the medal winners(Gold, Silver, Bronze), not for money, but to commemorate their work in the past few months. Most people have taken months of times in this challenge, but gain nothing after that. @cristiancanton
", "rawMarkdown": "I wonder if there will be a certificate for the medal winners(Gold, Silver, Bronze), not for money, but to commemorate their work in the past few months. Most people have taken months of times in this challenge, but gain nothing after that. @cristiancanton", "votes": -1, "replies": [ { "id": 896248, "postDate": "2020-06-22T02:42:06.803Z", "content": "mate you earned a gold medal already :) ... check your profile ...
", "rawMarkdown": "mate you earned a gold medal already :) ... check your profile ...", "votes": 1 } ] }, { "id": 3463875, "postDate": "2026-05-28T06:48:53.287Z", "content": "Please, I want to download videos for deepfake detection. Can you help me
", "rawMarkdown": "Please, I want to download videos for deepfake detection. Can you help me\n" }, { "id": 3463871, "postDate": "2026-05-28T06:46:39.737Z", "content": "Please, I want to download the dataset deepfake detection video
", "rawMarkdown": "Please, I want to download the dataset deepfake detection video " }, { "id": 971443, "postDate": "2020-08-15T14:23:47.657Z", "content": "*Nice one *
", "rawMarkdown": "*Nice one *" }, { "id": 916012, "postDate": "2020-07-05T09:25:03.690Z", "content": "Congrats to the winner
", "rawMarkdown": "Congrats to the winner" }, { "id": 891288, "postDate": "2020-06-18T04:33:58.713Z", "content": "Congratulation to the winners. It was indeed a touch challenge and required lot of time and resources. But I learnt a lot while working on this challenge.
", "rawMarkdown": "Congratulation to the winners. It was indeed a touch challenge and required lot of time and resources. But I learnt a lot while working on this challenge.", "replies": [ { "id": 891290, "postDate": "2020-06-18T04:38:42.473Z", "content": "Congrats @selimsef
", "rawMarkdown": "Congrats @selimsef" } ] }, { "id": 886908, "postDate": "2020-06-15T11:03:17.773Z", "content": "Yesssss..... I am so happy to see Selim won this competition eventually. Congratulations bro👍. Well done....
", "rawMarkdown": "Yesssss..... I am so happy to see Selim won this competition eventually. Congratulations bro👍. Well done...." }, { "id": 886331, "postDate": "2020-06-15T00:03:09.970Z", "content": "It's my hope that with post, that we move on. First, I think there were many issues in this competition, especially at the evaluation, but also in earlier stages (height and width leak ..etc). I think thought that this was not the intention of the organizers or Kaggle. Second, I think that this could have been resolved in a better way, there were many suggestions. Ultimately the goal should have always been to find the best model for this important competition, given the circumstances.
\n\nI think the organizers, in all due fairness should have applied the clause below to everyone in the competition:
\n\n\"While the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique.\"
\n\nThat is something which I hope they consider in the future, I think many teams deserve a similar consideration (I don't think I am one of those though, since I don't think my model is good enough), and the hope is that this was the case . Or I hope they have already done that, and the winning model is indeed the best one.
\n\nI am also happy to learn that the all start teams initially revealed were not disqualified. Indeed those are some of the best data scientists on the planet, and we need them to continue to be in this community, and help solve interesting problems in the future.
\n\nCongratulations for the winners, and I also congratulate every one who participated in this competition. For being so civil and considerate for every one. Indeed Its a honor to be a member of this community. I have learned so much.
\n\nI do hope that Kaggle never looses it's initial spirit :), and I hope that they have learned a lot from this competition, and I am sure in the future we will have better organization.
\n\nThank you, and I am sorry for the long post.
", "rawMarkdown": "It's my hope that with post, that we move on. First, I think there were many issues in this competition, especially at the evaluation, but also in earlier stages (height and width leak ..etc). I think thought that this was not the intention of the organizers or Kaggle. Second, I think that this could have been resolved in a better way, there were many suggestions. Ultimately the goal should have always been to find the best model for this important competition, given the circumstances.\n\nI think the organizers, in all due fairness should have applied the clause below to everyone in the competition:\n\n\"While the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique.\"\n\nThat is something which I hope they consider in the future, I think many teams deserve a similar consideration (I don't think I am one of those though, since I don't think my model is good enough), and the hope is that this was the case . Or I hope they have already done that, and the winning model is indeed the best one.\n\nI am also happy to learn that the all start teams initially revealed were not disqualified. Indeed those are some of the best data scientists on the planet, and we need them to continue to be in this community, and help solve interesting problems in the future.\n\nCongratulations for the winners, and I also congratulate every one who participated in this competition. For being so civil and considerate for every one. Indeed Its a honor to be a member of this community. I have learned so much.\n\nI do hope that Kaggle never looses it's initial spirit :), and I hope that they have learned a lot from this competition, and I am sure in the future we will have better organization.\n\nThank you, and I am sorry for the long post." }, { "id": 2995839, "postDate": "2024-09-22T16:40:13.833Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 883409, "author_name": "Yifan Xie", "author_url": "", "post_date": "2020-06-12T16:00:00.253000", "content": "congratulation @selimsef, @jamesphoward and all the prize winners, we will release a statement later to further explain the situation regarding our team
", "votes": 30, "replies": [ { "id": 883692, "author_name": "anokas", "author_url": "", "post_date": "2020-06-12T20:44:00.613000", "content": "We've now posted our statement: https://www.kaggle.com/c/deepfake-detection-challenge/discussion/157983
", "votes": 19, "replies": [] }, { "id": 886337, "author_name": "Hilal Shaath", "author_url": "", "post_date": "2020-06-15T00:11:02.660000", "content": "@anokas @titericz @yifanxie , you guys will always be super stars!. I am happy that you will remain in the community, and I totally support Kaggle's decision to keep you. We need you for important problems, and to continue and learn from your work...
\n\nPlease continue to contribute to this community, many of us learn so much of you.
", "votes": 1, "replies": [] } ] }, { "id": 885420, "author_name": "beluga", "author_url": "", "post_date": "2020-06-14T07:39:01.873000", "content": "It is quite unusual at kaggle that the winner announcement post has more downvotes than upvotes.
\n\n> We invested all necessary resources to make sure we were scoring everyone fairly
\n\nFairness is the last thing that comes to my mind... @cristiancanton as you have PhD in Computer Vision probably you see how absurd/questionable is the reasoning behind disqualifying the best submission.\n I don't expect that facebook changes the decision because of anything written in the forum but\n I still recommend you to read the other thread and try to understand the frustration of competitors.\n You spent two months but you decided this way anyway (which has probably more to Legal/PR/Marketing departments).
", "votes": 25, "replies": [] }, { "id": 907369, "author_name": "ivan", "author_url": "", "post_date": "2020-06-29T22:40:28.683000", "content": "The public and half private LB datasets are now available for download.
\n\nUnfortunately, after analysing this data and reran our best submission, I had even more questions to the organizer @cristiancanton and authors of the paper, Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, Cristian Canton Ferrer:
\n\nHow did you collect the real ”in-the-wild” Deepfake videos and, more importantly, how did you label it?
Why didn't you warn the participants that voice manipulation will not be evaluated in this competition? Instead, you made it very clear that we should detect both audio and video modifications.
If you silently decide not to include voice manipulations into DFDC-like half of private dataset, how can you guarantee that the real \"in-the-wild\" videos do not have voice manipulation?
Hello, have the organizers ever given you any explanation or reply ~ ?
", "votes": 0, "replies": [] } ] }, { "id": 883368, "author_name": "James Howard", "author_url": "", "post_date": "2020-06-12T15:11:47.587000", "content": "So pleased with our final placing, and we've uploaded our full source code to GitHub here.
\n\nMany thanks to the team at Kaggle and Facebook!
", "votes": 19, "replies": [ { "id": 886334, "author_name": "Hilal Shaath", "author_url": "", "post_date": "2020-06-15T00:06:03.057000", "content": "Congrate @jamesphoward . Thank you for sharing the code
", "votes": 0, "replies": [] }, { "id": 893107, "author_name": "Hector", "author_url": "", "post_date": "2020-06-19T11:52:23.733000", "content": "Thanks for sharing the code! Very helpful.
", "votes": 0, "replies": [] }, { "id": 919216, "author_name": "Sovil", "author_url": "", "post_date": "2020-07-07T19:02:10.973000", "content": "Thanks
", "votes": 0, "replies": [] } ] }, { "id": 883521, "author_name": "beluga", "author_url": "", "post_date": "2020-06-12T17:58:29.290000", "content": "Congrats to the winners!
\n\nBUT\nThe semi-disqualification is fishy... I am very disappointed about the decision and it undermines trust from competitor point of view. Waiting for further details from All Faces Are Real before drawing too harsh conclusions.
This competition was honestly a mess especially considering the importance of this problem and the large prize. Hopefully the organizers got something somewhat helpful and I hope they'll release the full data set for future research.\nCongrats to the winners!
", "votes": 15, "replies": [] }, { "id": 886431, "author_name": "walton", "author_url": "", "post_date": "2020-06-15T03:25:55.543000", "content": "we are 2nd removed Team. if anyone is interested in our solution, welcome to this link
\n\nThis competition draws huge attention from both industry and academia, it will definitely boost the development of deepfake detection techniques to fight against fake videos.
\n\nBut the rules are complicated to new Kagglers and we believe it would be much clearer to state that no external data is allowed for this competition at the very beginning. We are deeply disappointed by the final decision that both our scores are removed from the leaderboards. This simply erased all our efforts devoted to this competition.
\n\nAnyway, we still like to thank Facebook and Kaggle for organizing this event
", "votes": 13, "replies": [] }, { "id": 889534, "author_name": "apache2046", "author_url": "", "post_date": "2020-06-17T02:03:14.680000", "content": "please release the full dataset (public/private test set) for future deepfake research 🙏
", "votes": 10, "replies": [] }, { "id": 883509, "author_name": "Manoj", "author_url": "", "post_date": "2020-06-12T17:40:49.177000", "content": "I don't know what just happened. Congrats @selimsef. You must be super happy. 2nd GM after b. e. s. from Belarus :)
", "votes": 7, "replies": [ { "id": 883573, "author_name": "Selim Seferbekov", "author_url": "", "post_date": "2020-06-12T18:32:55.683000", "content": "Thank you! Yes, I'm very happy with the results ✌️
", "votes": 7, "replies": [] }, { "id": 886333, "author_name": "Hilal Shaath", "author_url": "", "post_date": "2020-06-15T00:05:39.510000", "content": "Congrats @selimsef
", "votes": 1, "replies": [] }, { "id": 1014700, "author_name": "Sasree", "author_url": "", "post_date": "2020-09-17T16:42:25.893000", "content": "Congrats @selimsef for winning the competition! Is your code available publicly somewhere? I am doing some research on deepfakes and would like to test some videos against it?
", "votes": 1, "replies": [] } ] }, { "id": 883470, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2020-06-12T16:52:31.690000", "content": "\n\n\nWhile the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique. As a result of an extensive review of the context and licensing of the prior first-place team’s external data sources, it was determined that team removal was not warranted and their second selected submission should stand. This has resulted in their final standing being seventh place.
\n
\"different and unique\" Translation: they actually didn't violate any official competition rules, but we didn't like their submission.
", "votes": 9, "replies": [ { "id": 883500, "author_name": "wayfarer", "author_url": "", "post_date": "2020-06-12T17:32:00.123000", "content": "please, show some respect to hosts, thanks
", "votes": -21, "replies": [] }, { "id": 893108, "author_name": "Hector", "author_url": "", "post_date": "2020-06-19T11:53:31.893000", "content": "Ohhhhh dear
", "votes": 0, "replies": [] } ] }, { "id": 883535, "author_name": "ivan", "author_url": "", "post_date": "2020-06-12T18:04:58.320000", "content": "@cristiancanton did you deliberately mislead the participants with voice manipulations? After reading the paper I still do not understand why you did not clarify this in the competition description:
\n\n\n", "votes": 6, "replies": [ { "id": 887413, "author_name": "David", "author_url": "", "post_date": "2020-06-15T16:59:15.577000", "content": "Although these audio manipulations were not considered ‘Deepfakes’ for the competition, we included them in the set to provide more breadth of manipulation type, and they may be of use in further research.
\n
It is very misleading if audio manipulations were not labeled as Deepfake in the private dataset considering the subtitle of the competition states \"Identify videos with facial or voice manipulations\". It is even more misleading considering that audio analysis significantly helps in the provided dataset and also in public LB, implying that videos with audio alterations were labeled as FAKE in both datasets. Whereas in the private dataset, this is clearly not the case, as people who have reported including audio analysis in the private solution had a massive performance hit (for instance, I went from 1st public to 905th private) which statistically could not be explained by generalization error (at least in my case).
", "votes": 1, "replies": [] }, { "id": 887669, "author_name": "ivan", "author_url": "", "post_date": "2020-06-15T19:48:22.503000", "content": "Few hours ago we emailed to the authors and @cristiancanton answered that audio manipulations are not considered as DeepFakes, but they were labelled as FAKE in the dataset. He also mentioned that the data will be published this week. I hope it will shed some light on the audio analysis, and we will be able to reproduce the public/private LB scores.
", "votes": 3, "replies": [] } ] }, { "id": 883411, "author_name": "Jan Bre", "author_url": "", "post_date": "2020-06-12T16:03:47.257000", "content": "Out of curiosity: Would it be possible to know what external datasets were used that are not allowed?
", "votes": 4, "replies": [ { "id": 883415, "author_name": "Yifan Xie", "author_url": "", "post_date": "2020-06-12T16:07:16.207000", "content": "\n\n\nJan Bre wrote:
\n \nOut of curiosity: Would it be possible to know what external datasets were used that are not allowed?
\n
from our point of view, it is quite a story, we will share when we are ready
", "votes": 14, "replies": [] }, { "id": 883418, "author_name": "Jan Bre", "author_url": "", "post_date": "2020-06-12T16:08:34.723000", "content": "Thank you 🙌
", "votes": 1, "replies": [] } ] }, { "id": 883518, "author_name": "Arsenii Mustafin", "author_url": "", "post_date": "2020-06-12T17:54:23.270000", "content": "Congratulations to the winners!
\n\n@cristiancanton are you going to open late submissions and/or give and access to the test data? I wan't to research the problem further. I already got fairly good result on validation set and wish I can try it on test set.
", "votes": -1, "replies": [] }, { "id": 886395, "author_name": "Yinglin Zheng", "author_url": "", "post_date": "2020-06-15T02:42:19.630000", "content": "I wonder if there will be a certificate for the medal winners(Gold, Silver, Bronze), not for money, but to commemorate their work in the past few months. Most people have taken months of times in this challenge, but gain nothing after that. @cristiancanton
", "votes": -1, "replies": [ { "id": 896248, "author_name": "mjsML", "author_url": "", "post_date": "2020-06-22T02:42:06.803000", "content": "mate you earned a gold medal already :) ... check your profile ...
", "votes": 1, "replies": [] } ] }, { "id": 3463875, "author_name": "haiam hamed", "author_url": "", "post_date": "2026-05-28T06:48:53.287000", "content": "Please, I want to download videos for deepfake detection. Can you help me
", "votes": 0, "replies": [] }, { "id": 3463871, "author_name": "haiam hamed", "author_url": "", "post_date": "2026-05-28T06:46:39.737000", "content": "Please, I want to download the dataset deepfake detection video
", "votes": 0, "replies": [] }, { "id": 971443, "author_name": "__n1kshaN__", "author_url": "", "post_date": "2020-08-15T14:23:47.657000", "content": "*Nice one *
", "votes": 0, "replies": [] }, { "id": 916012, "author_name": "Peeyush Kant Misra", "author_url": "", "post_date": "2020-07-05T09:25:03.690000", "content": "Congrats to the winner
", "votes": 0, "replies": [] }, { "id": 891288, "author_name": "Al Warrior", "author_url": "", "post_date": "2020-06-18T04:33:58.713000", "content": "Congratulation to the winners. It was indeed a touch challenge and required lot of time and resources. But I learnt a lot while working on this challenge.
", "votes": 0, "replies": [ { "id": 891290, "author_name": "Al Warrior", "author_url": "", "post_date": "2020-06-18T04:38:42.473000", "content": "Congrats @selimsef
", "votes": 0, "replies": [] } ] }, { "id": 886908, "author_name": "MICADEE", "author_url": "", "post_date": "2020-06-15T11:03:17.773000", "content": "Yesssss..... I am so happy to see Selim won this competition eventually. Congratulations bro👍. Well done....
", "votes": 0, "replies": [] }, { "id": 886331, "author_name": "Hilal Shaath", "author_url": "", "post_date": "2020-06-15T00:03:09.970000", "content": "It's my hope that with post, that we move on. First, I think there were many issues in this competition, especially at the evaluation, but also in earlier stages (height and width leak ..etc). I think thought that this was not the intention of the organizers or Kaggle. Second, I think that this could have been resolved in a better way, there were many suggestions. Ultimately the goal should have always been to find the best model for this important competition, given the circumstances.
\n\nI think the organizers, in all due fairness should have applied the clause below to everyone in the competition:
\n\n\"While the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique.\"
\n\nThat is something which I hope they consider in the future, I think many teams deserve a similar consideration (I don't think I am one of those though, since I don't think my model is good enough), and the hope is that this was the case . Or I hope they have already done that, and the winning model is indeed the best one.
\n\nI am also happy to learn that the all start teams initially revealed were not disqualified. Indeed those are some of the best data scientists on the planet, and we need them to continue to be in this community, and help solve interesting problems in the future.
\n\nCongratulations for the winners, and I also congratulate every one who participated in this competition. For being so civil and considerate for every one. Indeed Its a honor to be a member of this community. I have learned so much.
\n\nI do hope that Kaggle never looses it's initial spirit :), and I hope that they have learned a lot from this competition, and I am sure in the future we will have better organization.
\n\nThank you, and I am sorry for the long post.
", "votes": 0, "replies": [] }, { "id": 2995839, "author_name": "", "author_url": "", "post_date": "2024-09-22T16:40:13.833000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "883409": "congratulation @selimsef, @jamesphoward and all the prize winners, we will release a statement later to further explain the situation regarding our team", "885420": "It is quite unusual at kaggle that the winner announcement post has more downvotes than upvotes.\n\n> We invested all necessary resources to make sure we were scoring everyone fairly\n\n Fairness is the last thing that comes to my mind... @cristiancanton as you have PhD in Computer Vision probably you see how absurd/questionable is the reasoning behind disqualifying the best submission.\n I don't expect that facebook changes the decision because of anything written in the forum but\n I still recommend you to read the other thread and try to understand the frustration of competitors.\n You spent two months but you decided this way anyway (which has probably more to Legal/PR/Marketing departments).", "907369": "The public and half private LB datasets are now available for [download](https://ai.facebook.com/datasets/dfdc/).\n\nUnfortunately, after analysing this data and reran our best submission, I had even more questions to the organizer @cristiancanton and authors of the [paper](https://arxiv.org/abs/2006.07397), Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, Cristian Canton Ferrer:\n\n1. How did you collect the real ”in-the-wild” Deepfake videos and, more importantly, how did you label it?\n\n2. Why didn't you warn the participants that voice manipulation will not be evaluated in this competition? Instead, you made it very [clear](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121694#708946) that we should detect both audio and video modifications.\n\n3. If you silently decide not to include voice manipulations into DFDC-like half of private dataset, how can you guarantee that the real \"in-the-wild\" videos do not have voice manipulation?", "883368": "So pleased with our final placing, and we've uploaded our full source code to GitHub [here](https://github.com/jphdotam/DFDC).\n\nMany thanks to the team at Kaggle and Facebook!", "883521": "Congrats to the winners!\n\n**BUT**\nThe semi-disqualification is fishy... I am very disappointed about the decision and it undermines trust from competitor point of view. Waiting for further details from `All Faces Are Real` before drawing too harsh conclusions.", "883609": "This competition was honestly a mess especially considering the importance of this problem and the large prize. Hopefully the organizers got something somewhat helpful and I hope they'll release the full data set for future research.\nCongrats to the winners!", "886431": "we are 2nd removed Team. if anyone is interested in our solution, welcome to this [link](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/158506) \n\nThis competition draws huge attention from both industry and academia, it will definitely boost the development of deepfake detection techniques to fight against fake videos. \n\nBut the rules are complicated to new Kagglers and we believe **it would be much clearer to state that no external data is allowed for this competition at the very beginning**. We are deeply disappointed by the final decision that both our scores are removed from the leaderboards. This simply erased all our efforts devoted to this competition.\n\nAnyway, we still like to thank Facebook and Kaggle for organizing this event", "889534": "please release the full dataset (public/private test set) for future deepfake research 🙏 ", "883509": "I don't know what just happened. Congrats @selimsef. You must be super happy. 2nd GM after b. e. s. from Belarus :)", "883470": "> While the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique. As a result of an extensive review of the context and licensing of the prior first-place team’s external data sources, it was determined that team removal was not warranted and their second selected submission should stand. This has resulted in their final standing being seventh place.\n\n\"different and unique\" Translation: they actually **didn't** violate any official competition rules, but we didn't like their submission.\n\n", "883535": "@cristiancanton did you deliberately mislead the participants with voice manipulations? After reading the [paper](https://scontent.fhen1-1.fna.fbcdn.net/v/t39.8562-6/10000000_2949149428508489_1422548062431945722_n.pdf?_nc_cat=111&_nc_sid=ae5e01&_nc_ohc=B5PFMIJniQkAX9Uee--&_nc_ht=scontent.fhen1-1.fna&oh=73fe20850318a06f3e7309cd611fdbfd&oe=5F07F184) I still do not understand why you did not clarify this in the competition description:\n> Although these audio manipulations were not considered ‘Deepfakes’ for the competition, we included them in the set to provide more breadth of manipulation type, and they may be of use in further research.\n\n", "883411": "Out of curiosity: Would it be possible to know what external datasets were used that are not allowed?", "883340": "Hello DFDC competitors,\n\nWe’re happy to announce the winners of the Deepfake Detection Challenge. Over the course of 3.5 months, there were 2,114 participants who submitted over 35,000 models to the DFDC.\n\nWe want to extend a huge congratulations to the following teams who placed in the top five and will receive prizes:\n\n- Selim Seferbekov\n- \\WM/\n- NtechLab\n- Eighteen years old\n- The Medics\n\nWe took extreme care when running the final submissions against the private test set. We invested all necessary resources to make sure we were scoring everyone fairly, ruling out any execution errors related to environment issues. Faulty submissions stemmed mostly from coding bugs and timeouts, among other factors. Unfortunately, we can’t provide a detailed log for every participant.\n\nYou may notice that the top rankings have changed. Unfortunately, the top two teams in the preliminary standings used external data sources in their winning submissions that were not allowed under the rules of this competition. \n\nWhile the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique. As a result of an extensive review of the context and licensing of the prior first-place team’s external data sources, it was determined that team removal was not warranted and their second selected submission should stand. This has resulted in their final standing being seventh place.\n\nWe’re grateful for the contributions of all participants in this challenge. We know that many of you have dedicated a lot of time to develop high quality and unique approaches.\n\nWe recognize that no single organization or individual can solve the issue of deepfakes on their own. This open and collaborative effort has enabled us to work together, accelerate progress, and ultimately we hope will help prevent people from being deceived by the images and videos they see online.\n\nThanks,\nCristian Canton Ferrer, Research Manager, Facebook \n", "883518": "Congratulations to the winners! \n\n@cristiancanton are you going to open late submissions and/or give and access to the test data? I wan't to research the problem further. I already got fairly good result on validation set and wish I can try it on test set.", "886395": "I wonder if there will be a certificate for the medal winners(Gold, Silver, Bronze), not for money, but to commemorate their work in the past few months. Most people have taken months of times in this challenge, but gain nothing after that. @cristiancanton", "3463875": "Please, I want to download videos for deepfake detection. Can you help me\n", "3463871": "Please, I want to download the dataset deepfake detection video ", "971443": "*Nice one *", "916012": "Congrats to the winner", "891288": "Congratulation to the winners. It was indeed a touch challenge and required lot of time and resources. But I learnt a lot while working on this challenge.", "886908": "Yesssss..... I am so happy to see Selim won this competition eventually. Congratulations bro👍. Well done....", "886331": "It's my hope that with post, that we move on. First, I think there were many issues in this competition, especially at the evaluation, but also in earlier stages (height and width leak ..etc). I think thought that this was not the intention of the organizers or Kaggle. Second, I think that this could have been resolved in a better way, there were many suggestions. Ultimately the goal should have always been to find the best model for this important competition, given the circumstances.\n\nI think the organizers, in all due fairness should have applied the clause below to everyone in the competition:\n\n\"While the standard consequence of a rules violation is team disqualification and removal, the circumstances surrounding the two teams’ submissions were different and unique.\"\n\nThat is something which I hope they consider in the future, I think many teams deserve a similar consideration (I don't think I am one of those though, since I don't think my model is good enough), and the hope is that this was the case . Or I hope they have already done that, and the winning model is indeed the best one.\n\nI am also happy to learn that the all start teams initially revealed were not disqualified. Indeed those are some of the best data scientists on the planet, and we need them to continue to be in this community, and help solve interesting problems in the future.\n\nCongratulations for the winners, and I also congratulate every one who participated in this competition. For being so civil and considerate for every one. Indeed Its a honor to be a member of this community. I have learned so much.\n\nI do hope that Kaggle never looses it's initial spirit :), and I hope that they have learned a lot from this competition, and I am sure in the future we will have better organization.\n\nThank you, and I am sorry for the long post.", "2995839": "" } }