Hi all!
\n\nSeveral members of the community have discovered an issue with the metadata in this competition, namely that two fields - display_aspect_ratio and sample_aspect_ratio - are leaking information about the target. We are working on resolving this - we are fixing the videos in the test set so that those fields no longer leak, and then we will rescore the existing submissions.
Due to the holidays, this process will not take place immediately. We aim to replace the video dataset later this week at the earliest, and begin the rescore at roughly the same time. We will post notifications as the process begins and ends. In the meantime please rest assured that this leak will be going away.
\n\nThanks to @jarfo1 for initially reporting this in great detail, and to @zaharch, @tivfrvqhs5, and others for corroborating and providing additional work to reproduce the problem. Your contributions are very much appreciated.
\n\nThank you all, and happy holidays!
", "messages": [ { "id": 702393, "postDate": "2019-12-24T16:21:29.460Z", "content": "Hi all!
\n\nSeveral members of the community have discovered an issue with the metadata in this competition, namely that two fields - display_aspect_ratio and sample_aspect_ratio - are leaking information about the target. We are working on resolving this - we are fixing the videos in the test set so that those fields no longer leak, and then we will rescore the existing submissions.
Due to the holidays, this process will not take place immediately. We aim to replace the video dataset later this week at the earliest, and begin the rescore at roughly the same time. We will post notifications as the process begins and ends. In the meantime please rest assured that this leak will be going away.
\n\nThanks to @jarfo1 for initially reporting this in great detail, and to @zaharch, @tivfrvqhs5, and others for corroborating and providing additional work to reproduce the problem. Your contributions are very much appreciated.
\n\nThank you all, and happy holidays!
", "rawMarkdown": "Hi all!\n\nSeveral members of the community have discovered an issue with the metadata in this competition, namely that two fields - `display_aspect_ratio` and `sample_aspect_ratio` - are leaking information about the target. We are working on resolving this - we are fixing the videos in the test set so that those fields no longer leak, and then we will rescore the existing submissions.\n\nDue to the holidays, this process will not take place immediately. We aim to replace the video dataset later this week at the earliest, and begin the rescore at roughly the same time. We will post notifications as the process begins and ends. In the meantime please rest assured that this leak *will* be going away.\n\nThanks to @jarfo1 for initially reporting this in great detail, and to @zaharch, @tivfrvqhs5, and others for corroborating and providing additional work to reproduce the problem. Your contributions are very much appreciated.\n\nThank you all, and happy holidays!", "votes": 19 }, { "id": 704719, "postDate": "2019-12-27T22:40:48.230Z", "content": "In correcting the problem, are you creating an entirely new test data set? Surely that is the only way to make the correction. Wouldn't just changing the metadata fields of the existing set leave the leaked information out there.
", "rawMarkdown": "In correcting the problem, are you creating an entirely new test data set? Surely that is the only way to make the correction. Wouldn't just changing the metadata fields of the existing set leave the leaked information out there.", "votes": 3 }, { "id": 702520, "postDate": "2019-12-24T19:28:02.377Z", "content": "As aspect ratio came out to be acknowledged as a leak, should we also expect a fix for videos that have the aspect_ratio leaking a modification and the a-sync between sound and image is visually evident, although the label still states that the video is \"REAL\" (examples are blqgpxokwq.mp4, fwoclhwwjg.mp4 from the 49th train set)?
", "rawMarkdown": "As aspect ratio came out to be acknowledged as a leak, should we also expect a fix for videos that have the aspect_ratio leaking a modification and the a-sync between sound and image is visually evident, although the label still states that the video is \"REAL\" (examples are blqgpxokwq.mp4, fwoclhwwjg.mp4 from the 49th train set)?", "votes": 1 }, { "id": 703281, "postDate": "2019-12-25T23:50:50.267Z", "content": "Is aspect ratio is leakage from video metadata as a string (and that's all)? Or real aspect ratio is real discriminator real/fake? (sorry for my English, I hope you understand the question)
", "rawMarkdown": "Is aspect ratio is leakage from video metadata as a string (and that's all)? Or real aspect ratio is real discriminator real/fake? (sorry for my English, I hope you understand the question)", "votes": 2 }, { "id": 708319, "postDate": "2020-01-02T07:21:39.900Z", "content": "Is it possible to change the file names? It is very important since some people could \"remember\" the answers for them to produce better LB result.
", "rawMarkdown": "Is it possible to change the file names? It is very important since some people could \"remember\" the answers for them to produce better LB result." }, { "id": 702459, "postDate": "2019-12-24T17:39:12.953Z", "content": "Will be interesting to see if JiangJun survives this change. On googling that name it leads to a LinkedIn of a Facebook employee. Just to confirm, Facebook employees(and a few other companies) are excluded from this competition, correct?
", "rawMarkdown": "Will be interesting to see if JiangJun survives this change. On googling that name it leads to a LinkedIn of a Facebook employee. Just to confirm, Facebook employees(and a few other companies) are excluded from this competition, correct? ", "replies": [ { "id": 702638, "postDate": "2019-12-25T00:32:48.220Z", "content": "I thought I read that they will be participating but not eligible for prize money. Will look to see if I can find that piece of info.
", "rawMarkdown": "I thought I read that they will be participating but not eligible for prize money. Will look to see if I can find that piece of info." }, { "id": 703902, "postDate": "2019-12-26T19:23:30.723Z", "content": "Yes, Facebook employees (and employees of other entities associated with the challenge) are permitted to enter but not eligible to win prize money.
", "rawMarkdown": "Yes, Facebook employees (and employees of other entities associated with the challenge) are permitted to enter but not eligible to win prize money." }, { "id": 703974, "postDate": "2019-12-26T22:07:25.457Z", "content": "Are these people still eligible for ranking points? It would be nice if these people were in some way differentiated on the leaderboard. I'd assume they would have to self-report and change their team name on the lb or something though.
", "rawMarkdown": "Are these people still eligible for ranking points? It would be nice if these people were in some way differentiated on the leaderboard. I'd assume they would have to self-report and change their team name on the lb or something though. " }, { "id": 704617, "postDate": "2019-12-27T18:04:34.637Z", "content": "@ryches We don't have capability to perform any systematic leaderboard differentiation of these individuals, so yes, they will be eligible for points. They can self-report with a team name change, but that is entirely voluntary and will not be policed.
", "rawMarkdown": "@ryches We don't have capability to perform any systematic leaderboard differentiation of these individuals, so yes, they will be eligible for points. They can self-report with a team name change, but that is entirely voluntary and will not be policed." } ] }, { "id": 754870, "postDate": "2020-02-24T06:42:49.723Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 704719, "author_name": "pete", "author_url": "", "post_date": "2019-12-27T22:40:48.230000", "content": "In correcting the problem, are you creating an entirely new test data set? Surely that is the only way to make the correction. Wouldn't just changing the metadata fields of the existing set leave the leaked information out there.
", "votes": 3, "replies": [] }, { "id": 702520, "author_name": "RaAV", "author_url": "", "post_date": "2019-12-24T19:28:02.377000", "content": "As aspect ratio came out to be acknowledged as a leak, should we also expect a fix for videos that have the aspect_ratio leaking a modification and the a-sync between sound and image is visually evident, although the label still states that the video is \"REAL\" (examples are blqgpxokwq.mp4, fwoclhwwjg.mp4 from the 49th train set)?
", "votes": 1, "replies": [] }, { "id": 703281, "author_name": "Jonas Matuzas", "author_url": "", "post_date": "2019-12-25T23:50:50.267000", "content": "Is aspect ratio is leakage from video metadata as a string (and that's all)? Or real aspect ratio is real discriminator real/fake? (sorry for my English, I hope you understand the question)
", "votes": 2, "replies": [] }, { "id": 708319, "author_name": "Victor Paslay", "author_url": "", "post_date": "2020-01-02T07:21:39.900000", "content": "Is it possible to change the file names? It is very important since some people could \"remember\" the answers for them to produce better LB result.
", "votes": 0, "replies": [] }, { "id": 702459, "author_name": "ryches", "author_url": "", "post_date": "2019-12-24T17:39:12.953000", "content": "Will be interesting to see if JiangJun survives this change. On googling that name it leads to a LinkedIn of a Facebook employee. Just to confirm, Facebook employees(and a few other companies) are excluded from this competition, correct?
", "votes": 0, "replies": [ { "id": 702638, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2019-12-25T00:32:48.220000", "content": "I thought I read that they will be participating but not eligible for prize money. Will look to see if I can find that piece of info.
", "votes": 0, "replies": [] }, { "id": 703902, "author_name": "Julia Elliott", "author_url": "", "post_date": "2019-12-26T19:23:30.723000", "content": "Yes, Facebook employees (and employees of other entities associated with the challenge) are permitted to enter but not eligible to win prize money.
", "votes": 0, "replies": [] }, { "id": 703974, "author_name": "ryches", "author_url": "", "post_date": "2019-12-26T22:07:25.457000", "content": "Are these people still eligible for ranking points? It would be nice if these people were in some way differentiated on the leaderboard. I'd assume they would have to self-report and change their team name on the lb or something though.
", "votes": 0, "replies": [] }, { "id": 704617, "author_name": "Julia Elliott", "author_url": "", "post_date": "2019-12-27T18:04:34.637000", "content": "@ryches We don't have capability to perform any systematic leaderboard differentiation of these individuals, so yes, they will be eligible for points. They can self-report with a team name change, but that is entirely voluntary and will not be policed.
", "votes": 0, "replies": [] } ] }, { "id": 754870, "author_name": "", "author_url": "", "post_date": "2020-02-24T06:42:49.723000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "702393": "Hi all!\n\nSeveral members of the community have discovered an issue with the metadata in this competition, namely that two fields - `display_aspect_ratio` and `sample_aspect_ratio` - are leaking information about the target. We are working on resolving this - we are fixing the videos in the test set so that those fields no longer leak, and then we will rescore the existing submissions.\n\nDue to the holidays, this process will not take place immediately. We aim to replace the video dataset later this week at the earliest, and begin the rescore at roughly the same time. We will post notifications as the process begins and ends. In the meantime please rest assured that this leak *will* be going away.\n\nThanks to @jarfo1 for initially reporting this in great detail, and to @zaharch, @tivfrvqhs5, and others for corroborating and providing additional work to reproduce the problem. Your contributions are very much appreciated.\n\nThank you all, and happy holidays!", "704719": "In correcting the problem, are you creating an entirely new test data set? Surely that is the only way to make the correction. Wouldn't just changing the metadata fields of the existing set leave the leaked information out there.", "702520": "As aspect ratio came out to be acknowledged as a leak, should we also expect a fix for videos that have the aspect_ratio leaking a modification and the a-sync between sound and image is visually evident, although the label still states that the video is \"REAL\" (examples are blqgpxokwq.mp4, fwoclhwwjg.mp4 from the 49th train set)?", "703281": "Is aspect ratio is leakage from video metadata as a string (and that's all)? Or real aspect ratio is real discriminator real/fake? (sorry for my English, I hope you understand the question)", "708319": "Is it possible to change the file names? It is very important since some people could \"remember\" the answers for them to produce better LB result.", "702459": "Will be interesting to see if JiangJun survives this change. On googling that name it leads to a LinkedIn of a Facebook employee. Just to confirm, Facebook employees(and a few other companies) are excluded from this competition, correct? ", "754870": "" } }