{ "id": 138452, "title": "Trust CV? LB?", "url": "/competitions/deepfake-detection-challenge/discussion/138452", "author_name": "SeshuRaju 🧘‍♂️", "post_date": "2020-03-25T01:41:30.250000", "votes": 3, "comment_count": 6, "views": 0, "content": "

As per this paper, public test set and private test set have 1/3 of distribution common.

\n\n

https://arxiv.org/pdf/1910.08854.pdf\na) 2/9 of videos: reduce the FPS of the video to 15;\nb) 2/9 of videos: reduce the resolution of the video to 1/4 of its original size;\nc) 2/9 of videos: reduce the overall encoding quality.\nd) 3/9 of videos: unmodified

\n\n

is it safe to build models and trust CV even LB score low?

", "messages": [ { "id": 785352, "postDate": "2020-03-25T01:41:30.250Z", "content": "

As per this paper, public test set and private test set have 1/3 of distribution common.

\n\n

https://arxiv.org/pdf/1910.08854.pdf\na) 2/9 of videos: reduce the FPS of the video to 15;\nb) 2/9 of videos: reduce the resolution of the video to 1/4 of its original size;\nc) 2/9 of videos: reduce the overall encoding quality.\nd) 3/9 of videos: unmodified

\n\n

is it safe to build models and trust CV even LB score low?

", "rawMarkdown": "As per this paper, public test set and private test set have 1/3 of distribution common. \n\nhttps://arxiv.org/pdf/1910.08854.pdf\na) 2/9 of videos: reduce the FPS of the video to 15;\nb) 2/9 of videos: reduce the resolution of the video to 1/4 of its original size;\nc) 2/9 of videos: reduce the overall encoding quality.\nd) 3/9 of videos: unmodified\n\nis it safe to build models and trust CV even LB score low?", "votes": 3 }, { "id": 786088, "postDate": "2020-03-25T16:11:29.663Z", "content": "

Local val loss is absolutely useless in this compete.

", "rawMarkdown": "Local val loss is absolutely useless in this compete.", "votes": 2, "replies": [ { "id": 786772, "postDate": "2020-03-26T07:39:18.017Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ] }, { "id": 785926, "postDate": "2020-03-25T14:16:07.987Z", "content": "

I've been wondering about this too after the BengaliAi comp. Is the train data broad enough for the hidden test set? Are we looking at a test set that has completely different videos/situations to the train data? Have the top teams looked into this and have generalized models? Does external data help?

", "rawMarkdown": "I've been wondering about this too after the BengaliAi comp. Is the train data broad enough for the hidden test set? Are we looking at a test set that has completely different videos/situations to the train data? Have the top teams looked into this and have generalized models? Does external data help?" }, { "id": 785719, "postDate": "2020-03-25T09:56:18.350Z", "content": "

About LB score:

\n\n
\n

I tried so hard and got so far \n But in the end it doesn't even matter

\n
", "rawMarkdown": "About LB score: \n> I tried so hard and got so far \nBut in the end it doesn't even matter", "votes": 12, "isDeleted": true, "replies": [ { "id": 785723, "postDate": "2020-03-25T10:03:58.850Z", "content": "

Nice Linkin Park quote 👍

", "rawMarkdown": "Nice Linkin Park quote 👍 " }, { "id": 785783, "postDate": "2020-03-25T11:43:33.163Z", "content": "

I feel bad for you Vlad. You had the lead for such a long time. Hope the private LB works out for you.

", "rawMarkdown": "I feel bad for you Vlad. You had the lead for such a long time. Hope the private LB works out for you." } ] } ], "comments": [ { "id": 786088, "author_name": "_KunlinLiu_", "author_url": "", "post_date": "2020-03-25T16:11:29.663000", "content": "

Local val loss is absolutely useless in this compete.

", "votes": 2, "replies": [ { "id": 786772, "author_name": "", "author_url": "", "post_date": "2020-03-26T07:39:18.017000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 785926, "author_name": "GreatGameDota", "author_url": "", "post_date": "2020-03-25T14:16:07.987000", "content": "

I've been wondering about this too after the BengaliAi comp. Is the train data broad enough for the hidden test set? Are we looking at a test set that has completely different videos/situations to the train data? Have the top teams looked into this and have generalized models? Does external data help?

", "votes": 0, "replies": [] }, { "id": 785719, "author_name": "", "author_url": "", "post_date": "2020-03-25T09:56:18.350000", "content": "

About LB score:

\n\n
\n

I tried so hard and got so far \n But in the end it doesn't even matter

\n
", "votes": 12, "replies": [ { "id": 785723, "author_name": "Stefan Mandl", "author_url": "", "post_date": "2020-03-25T10:03:58.850000", "content": "

Nice Linkin Park quote 👍

", "votes": 0, "replies": [] }, { "id": 785783, "author_name": "maralski", "author_url": "", "post_date": "2020-03-25T11:43:33.163000", "content": "

I feel bad for you Vlad. You had the lead for such a long time. Hope the private LB works out for you.

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "785352": "As per this paper, public test set and private test set have 1/3 of distribution common. \n\nhttps://arxiv.org/pdf/1910.08854.pdf\na) 2/9 of videos: reduce the FPS of the video to 15;\nb) 2/9 of videos: reduce the resolution of the video to 1/4 of its original size;\nc) 2/9 of videos: reduce the overall encoding quality.\nd) 3/9 of videos: unmodified\n\nis it safe to build models and trust CV even LB score low?", "786088": "Local val loss is absolutely useless in this compete.", "785926": "I've been wondering about this too after the BengaliAi comp. Is the train data broad enough for the hidden test set? Are we looking at a test set that has completely different videos/situations to the train data? Have the top teams looked into this and have generalized models? Does external data help?", "785719": "About LB score: \n> I tried so hard and got so far \nBut in the end it doesn't even matter" } }