{ "id": 129040, "title": "How did we get our current score(0.38504)", "url": "/competitions/deepfake-detection-challenge/discussion/129040", "author_name": "Shangqiu Li", "post_date": "2020-02-05T02:37:41.857000", "votes": 71, "comment_count": 39, "views": 0, "content": "

I noticed lots of people stuck around 0.68-0.69 or just simply following the best public notebook score so I decided to post such topic to help these people get better scores. I will not include any further details about our model structure.

\n\n

First, we were stuck at around 0.68 and can't get up any further no matter what. Soon, we noticed a pattern that our submission finishes so quickly in kaggle's black box environment.

\n\n

We posted a topic to get some help (link) and @harangdev kindly reminded us to add error catching.

\n\n

After we fixed the bug of submission finishing too fast, the score boosted from ~0.68 straight to ~0.48.\nThen adjustment of input size, model structure, clipping values, we were able to get to our score right now.

\n\n

Also check out my topic about submission error combat strategies link.

\n\n

Any further more details will be released after the competition ends(hopefully on the \"1st place solution\" topic).

", "messages": [ { "id": 737177, "postDate": "2020-02-05T02:37:41.857Z", "content": "

I noticed lots of people stuck around 0.68-0.69 or just simply following the best public notebook score so I decided to post such topic to help these people get better scores. I will not include any further details about our model structure.

\n\n

First, we were stuck at around 0.68 and can't get up any further no matter what. Soon, we noticed a pattern that our submission finishes so quickly in kaggle's black box environment.

\n\n

We posted a topic to get some help (link) and @harangdev kindly reminded us to add error catching.

\n\n

After we fixed the bug of submission finishing too fast, the score boosted from ~0.68 straight to ~0.48.\nThen adjustment of input size, model structure, clipping values, we were able to get to our score right now.

\n\n

Also check out my topic about submission error combat strategies link.

\n\n

Any further more details will be released after the competition ends(hopefully on the \"1st place solution\" topic).

", "rawMarkdown": "I noticed lots of people stuck around 0.68-0.69 or just simply following the best public notebook score so I decided to post such topic to help these people get better scores. I will not include any further details about our model structure.\n\nFirst, we were stuck at around 0.68 and can't get up any further no matter what. Soon, we noticed a pattern that our submission finishes so quickly in kaggle's black box environment. \n\nWe posted a topic to get some help ([link](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/127702)) and @harangdev kindly reminded us to add error catching. \n\nAfter we fixed the bug of submission finishing too fast, the score boosted from ~0.68 straight to ~0.48.\nThen adjustment of input size, model structure, clipping values, we were able to get to our score right now.\n\nAlso check out my topic about submission error combat strategies [link](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/127824).\n\nAny further more details will be released after the competition ends(hopefully on the \"1st place solution\" topic).", "votes": 71 }, { "id": 738739, "postDate": "2020-02-07T00:04:24.213Z", "content": "

I would warn against clipping as a post-process operation for this competition, remember:\n1. it is hard to set up a solid local validation set, due to overlapping actors\n2. It has been said by the host that the private test set for the final score will contain different types of videos than what we are training on.

\n\n

given the above, I fear clipping has a high chance of leading people to some optimum public LB score that would hurt private LB. Even small clipping value let's 0.95 or 0.05 might still hurt your score enough to mean the differents between prize/gold, or gold/silver, so far and so forth.

\n\n

Thought I would put it out here :)

", "rawMarkdown": "I would warn against clipping as a post-process operation for this competition, remember:\n1. it is hard to set up a solid local validation set, due to overlapping actors\n2. It has been said by the host that the private test set for the final score will contain different types of videos than what we are training on.\n\ngiven the above, I fear clipping has a high chance of leading people to some optimum public LB score that would hurt private LB. Even small clipping value let's 0.95 or 0.05 might still hurt your score enough to mean the differents between prize/gold, or gold/silver, so far and so forth.\n\nThought I would put it out here :)", "votes": 10, "replies": [ { "id": 738928, "postDate": "2020-02-07T07:25:07.543Z", "content": "

It is true, but it turns out it is simple to solve this issue, keep thinking about the solution, you will get it eventually.

", "rawMarkdown": "It is true, but it turns out it is simple to solve this issue, keep thinking about the solution, you will get it eventually." }, { "id": 752937, "postDate": "2020-02-21T14:59:45.957Z", "content": "

Fair arguments. Do you think clipping between 0.01 and 0.99 is still dangerous? It seems to me that this makes the model more robust as logarithmic loss is very unforgiving of confident, but wrong predictions.

", "rawMarkdown": "Fair arguments. Do you think clipping between 0.01 and 0.99 is still dangerous? It seems to me that this makes the model more robust as logarithmic loss is very unforgiving of confident, but wrong predictions.", "votes": 1 }, { "id": 753019, "postDate": "2020-02-21T16:08:13.273Z", "content": "

The clipping can be directly related to accuracy, just think about it, you will find a perfect way.

", "rawMarkdown": "The clipping can be directly related to accuracy, just think about it, you will find a perfect way." }, { "id": 753092, "postDate": "2020-02-21T17:43:49.150Z", "content": "
\n

Carlo Lepelaars wrote:

\n \n

Fair arguments. Do you think clipping between 0.01 and 0.99 is still dangerous? It seems to me that this makes the model more robust as logarithmic loss is very unforgiving of confident, but wrong predictions.

\n
\n\n

yeah that would work - or you can even scale your solution by some educated number - or even better (IMO) integrate such post-processing into your model training routine.

\n\n

I am clearly not the best person at postproc in our team, I will leave it to the expert :D

", "rawMarkdown": "> **Carlo Lepelaars wrote:**\n> \n> Fair arguments. Do you think clipping between 0.01 and 0.99 is still dangerous? It seems to me that this makes the model more robust as logarithmic loss is very unforgiving of confident, but wrong predictions.\n\nyeah that would work - or you can even scale your solution by some educated number - or even better (IMO) integrate such post-processing into your model training routine. \n\nI am clearly not the best person at postproc in our team, I will leave it to the expert :D", "votes": 1 }, { "id": 753104, "postDate": "2020-02-21T18:03:02.310Z", "content": "

Thanks for the insights! We will think about post-processing during model training. @yifanxie I have no doubt your team will crush this competition with Anokas and Giba! Really curious to see your solution at the end of the competition. 🙂

", "rawMarkdown": "Thanks for the insights! We will think about post-processing during model training. @yifanxie I have no doubt your team will crush this competition with Anokas and Giba! Really curious to see your solution at the end of the competition. 🙂 ", "votes": 1 }, { "id": 753120, "postDate": "2020-02-21T18:21:29.300Z", "content": "

just struggling like everyone else, and sinking with tensorflow :p

", "rawMarkdown": "just struggling like everyone else, and sinking with tensorflow :p", "votes": 1 }, { "id": 754759, "postDate": "2020-02-24T02:50:57.263Z", "content": "

Try isotonic regression to calibrate your NN models \"probabilities\".

", "rawMarkdown": "Try isotonic regression to calibrate your NN models \"probabilities\"." }, { "id": 798471, "postDate": "2020-04-05T14:28:19.883Z", "content": "

Thanks for the advice. Can it also be a bad choice setting the no-face to 0.5 or even 0.48 as in some kernels, or is it a minor impact/change to the private lb vs public lb?

", "rawMarkdown": "Thanks for the advice. Can it also be a bad choice setting the no-face to 0.5 or even 0.48 as in some kernels, or is it a minor impact/change to the private lb vs public lb?" } ] }, { "id": 748729, "postDate": "2020-02-17T22:42:40.643Z", "content": "

Edit: Added model stacking which got our score.

", "rawMarkdown": "Edit: Added model stacking which got our score.", "votes": 3, "replies": [ { "id": 753975, "postDate": "2020-02-22T22:07:18.857Z", "content": "

Nice. What was the LB score of your models, and how much did the stacking improve your final model?

", "rawMarkdown": "Nice. What was the LB score of your models, and how much did the stacking improve your final model?" }, { "id": 753977, "postDate": "2020-02-22T22:09:48.243Z", "content": "

we got ~0.34 for model stacking two ~0.38 models. improved ~0.04 but still a lot for this compacted LB.

", "rawMarkdown": "we got ~0.34 for model stacking two ~0.38 models. improved ~0.04 but still a lot for this compacted LB.", "votes": 2 }, { "id": 753982, "postDate": "2020-02-22T22:26:07.833Z", "content": "

0.04 is a nice improvement.

", "rawMarkdown": "0.04 is a nice improvement." } ] }, { "id": 738823, "postDate": "2020-02-07T04:14:45.030Z", "content": "

Thanks for sharing！ Can you provide some details about the “try catch” .

", "rawMarkdown": "Thanks for sharing！ Can you provide some details about the “try catch” .", "replies": [ { "id": 738825, "postDate": "2020-02-07T04:17:51.177Z", "content": "

try except blocks.

", "rawMarkdown": "try except blocks." }, { "id": 738833, "postDate": "2020-02-07T04:31:43.077Z", "content": "

Thanks , But I want to ask how to deal with a video that fails to read

", "rawMarkdown": "Thanks , But I want to ask how to deal with a video that fails to read" }, { "id": 738839, "postDate": "2020-02-07T04:36:56.677Z", "content": "

set the prediction of that video to 0.5

", "rawMarkdown": "set the prediction of that video to 0.5", "votes": 1 }, { "id": 759557, "postDate": "2020-02-29T07:24:27.800Z", "content": "

0.4855380058288574 to be exact

", "rawMarkdown": " 0.4855380058288574 to be exact" }, { "id": 760043, "postDate": "2020-02-29T19:11:01.583Z", "content": "

@computerguy Why it is 0.4855380058288574 instead of 0.5?

", "rawMarkdown": "@computerguy Why it is 0.4855380058288574 instead of 0.5?" }, { "id": 760313, "postDate": "2020-03-01T05:30:32.500Z", "content": "

Used maths to calculate that thing.

", "rawMarkdown": "Used maths to calculate that thing. " } ] }, { "id": 738668, "postDate": "2020-02-06T21:02:46.707Z", "content": "

BTW, our highest scoring kernel is just a modification of my public kernel link. I can't say just a little modification, but is still based on the whole outline.

", "rawMarkdown": "BTW, our highest scoring kernel is just a modification of my public kernel [link](https://www.kaggle.com/unkownhihi/starter-kernel-with-cnn-model-ll-lb-0-69235/). I can't say just a little modification, but is still based on the whole outline.", "replies": [ { "id": 738900, "postDate": "2020-02-07T06:40:27.820Z", "content": "

EDIT: It is like almost everything changed but the base is in there and there are a lot of hints to our new work. Good Luck finding them.

", "rawMarkdown": "EDIT: It is like almost everything changed but the base is in there and there are a lot of hints to our new work. Good Luck finding them." } ] }, { "id": 737893, "postDate": "2020-02-05T21:48:54.433Z", "content": "

Yes, I do that now. Be interesting to predict an optimal clipping from the LB score.

", "rawMarkdown": "Yes, I do that now. Be interesting to predict an optimal clipping from the LB score." }, { "id": 737291, "postDate": "2020-02-05T06:27:59.687Z", "content": "

Thanks. What does clipping values mean?

", "rawMarkdown": "Thanks. What does clipping values mean?", "replies": [ { "id": 737347, "postDate": "2020-02-05T08:08:40.197Z", "content": "

Clipping values means that values will be capped at max and min like clip (0.10, 0.90) means that anything below 0.10 in csv will be written as 0.1 and anything above 0.9 will be written as 0.9.

", "rawMarkdown": "Clipping values means that values will be capped at max and min like clip (0.10, 0.90) means that anything below 0.10 in csv will be written as 0.1 and anything above 0.9 will be written as 0.9." }, { "id": 737349, "postDate": "2020-02-05T08:11:00.037Z", "content": "

Thanks!

", "rawMarkdown": "Thanks!", "votes": -1 }, { "id": 737742, "postDate": "2020-02-05T17:59:02.593Z", "content": "

Just tried that but it made my score worse, so your mileage may vary... don't assume that clipping will be always good. 😄

", "rawMarkdown": "Just tried that but it made my score worse, so your mileage may vary... don't assume that clipping will be always good. 😄 " }, { "id": 737752, "postDate": "2020-02-05T18:11:35.633Z", "content": "

It's utility degrades as your model gets better.

", "rawMarkdown": "It's utility degrades as your model gets better.", "votes": 2 }, { "id": 737775, "postDate": "2020-02-05T19:03:43.570Z", "content": "

@humananalog, I see well, clipping is a method which is highly sensitive, even changing our values slights, we have got a super considerable performance boost.

", "rawMarkdown": "@humananalog, I see well, clipping is a method which is highly sensitive, even changing our values slights, we have got a super considerable performance boost." }, { "id": 737776, "postDate": "2020-02-05T19:05:53.670Z", "content": "

@petewills Well, it is not true, there is a way to make it even more use of the clipping as our model gets better, our model has accuracy through the roof, I can say that because we are not also using technically full data and still getting our best score.

", "rawMarkdown": "@petewills Well, it is not true, there is a way to make it even more use of the clipping as our model gets better, our model has accuracy through the roof, I can say that because we are not also using technically full data and still getting our best score." }, { "id": 737797, "postDate": "2020-02-05T19:33:12.110Z", "content": "

The advantage of clipping is that it makes really bad predictions hurt your score less, but it also makes really good predictions help the score less. For my model it apparently has more really good predictions than really bad ones. :-)

", "rawMarkdown": "The advantage of clipping is that it makes really bad predictions hurt your score less, but it also makes really good predictions help the score less. For my model it apparently has more really good predictions than really bad ones. :-)", "votes": 2 }, { "id": 737821, "postDate": "2020-02-05T20:08:49.963Z", "content": "

@harshitsheoran Interesting. I'll have to think about that.

", "rawMarkdown": "@harshitsheoran Interesting. I'll have to think about that." }, { "id": 737871, "postDate": "2020-02-05T21:22:48.790Z", "content": "

If you're model is good enough(that means can get most of the answer right and won't predict high probability for wrong answer), try higher clipping value will help.

", "rawMarkdown": "If you're model is good enough(that means can get most of the answer right and won't predict high probability for wrong answer), try higher clipping value will help." } ] }, { "id": 737180, "postDate": "2020-02-05T02:42:57.980Z", "content": "

Thanks, @unkownhihi. I'll try out to see whether I can climb up a little bit on the Public Leaderboard 😆.

", "rawMarkdown": "Thanks, @unkownhihi. I'll try out to see whether I can climb up a little bit on the Public Leaderboard 😆." }, { "id": 739504, "postDate": "2020-02-07T22:11:39.370Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 739540, "postDate": "2020-02-07T23:41:48.760Z", "content": "

Yeah. Dropped a ton of places after sharing. Was 18th now 24th. 😭

", "rawMarkdown": "Yeah. Dropped a ton of places after sharing. Was 18th now 24th. 😭", "votes": 1 }, { "id": 739873, "postDate": "2020-02-08T13:59:52.257Z", "content": "

Did you expect something else ?)

", "rawMarkdown": "Did you expect something else ?)", "votes": 2, "isDeleted": true }, { "id": 739920, "postDate": "2020-02-08T15:00:26.683Z", "content": "

Well, Yes we thought we would be improving our score again, which is not the case for now. 😅

", "rawMarkdown": "Well, Yes we thought we would be improving our score again, which is not the case for now. 😅" }, { "id": 740181, "postDate": "2020-02-09T01:47:02.273Z", "content": "

Dropping a ton of place in the current stage of this competition is better than in the end stage of this competition.

", "rawMarkdown": "Dropping a ton of place in the current stage of this competition is better than in the end stage of this competition." } ] } ], "comments": [ { "id": 738739, "author_name": "Yifan Xie", "author_url": "", "post_date": "2020-02-07T00:04:24.213000", "content": "

I would warn against clipping as a post-process operation for this competition, remember:\n1. it is hard to set up a solid local validation set, due to overlapping actors\n2. It has been said by the host that the private test set for the final score will contain different types of videos than what we are training on.

\n\n

given the above, I fear clipping has a high chance of leading people to some optimum public LB score that would hurt private LB. Even small clipping value let's 0.95 or 0.05 might still hurt your score enough to mean the differents between prize/gold, or gold/silver, so far and so forth.

\n\n

Thought I would put it out here :)

", "votes": 10, "replies": [ { "id": 738928, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-07T07:25:07.543000", "content": "

It is true, but it turns out it is simple to solve this issue, keep thinking about the solution, you will get it eventually.

", "votes": 0, "replies": [] }, { "id": 752937, "author_name": "Carlo", "author_url": "", "post_date": "2020-02-21T14:59:45.957000", "content": "

Fair arguments. Do you think clipping between 0.01 and 0.99 is still dangerous? It seems to me that this makes the model more robust as logarithmic loss is very unforgiving of confident, but wrong predictions.

", "votes": 1, "replies": [] }, { "id": 753019, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-21T16:08:13.273000", "content": "

The clipping can be directly related to accuracy, just think about it, you will find a perfect way.

", "votes": 0, "replies": [] }, { "id": 753092, "author_name": "Yifan Xie", "author_url": "", "post_date": "2020-02-21T17:43:49.150000", "content": "
\n

Carlo Lepelaars wrote:

\n \n

Fair arguments. Do you think clipping between 0.01 and 0.99 is still dangerous? It seems to me that this makes the model more robust as logarithmic loss is very unforgiving of confident, but wrong predictions.

\n
\n\n

yeah that would work - or you can even scale your solution by some educated number - or even better (IMO) integrate such post-processing into your model training routine.

\n\n

I am clearly not the best person at postproc in our team, I will leave it to the expert :D

", "votes": 1, "replies": [] }, { "id": 753104, "author_name": "Carlo", "author_url": "", "post_date": "2020-02-21T18:03:02.310000", "content": "

Thanks for the insights! We will think about post-processing during model training. @yifanxie I have no doubt your team will crush this competition with Anokas and Giba! Really curious to see your solution at the end of the competition. 🙂

", "votes": 1, "replies": [] }, { "id": 753120, "author_name": "Yifan Xie", "author_url": "", "post_date": "2020-02-21T18:21:29.300000", "content": "

just struggling like everyone else, and sinking with tensorflow :p

", "votes": 1, "replies": [] }, { "id": 754759, "author_name": "maralski", "author_url": "", "post_date": "2020-02-24T02:50:57.263000", "content": "

Try isotonic regression to calibrate your NN models \"probabilities\".

", "votes": 0, "replies": [] }, { "id": 798471, "author_name": "Kirderf", "author_url": "", "post_date": "2020-04-05T14:28:19.883000", "content": "

Thanks for the advice. Can it also be a bad choice setting the no-face to 0.5 or even 0.48 as in some kernels, or is it a minor impact/change to the private lb vs public lb?

", "votes": 0, "replies": [] } ] }, { "id": 748729, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-17T22:42:40.643000", "content": "

Edit: Added model stacking which got our score.

", "votes": 3, "replies": [ { "id": 753975, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-02-22T22:07:18.857000", "content": "

Nice. What was the LB score of your models, and how much did the stacking improve your final model?

", "votes": 0, "replies": [] }, { "id": 753977, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-22T22:09:48.243000", "content": "

we got ~0.34 for model stacking two ~0.38 models. improved ~0.04 but still a lot for this compacted LB.

", "votes": 2, "replies": [] }, { "id": 753982, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-02-22T22:26:07.833000", "content": "

0.04 is a nice improvement.

", "votes": 0, "replies": [] } ] }, { "id": 738823, "author_name": "beiweizai", "author_url": "", "post_date": "2020-02-07T04:14:45.030000", "content": "

Thanks for sharing！ Can you provide some details about the “try catch” .

", "votes": 0, "replies": [ { "id": 738825, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-07T04:17:51.177000", "content": "

try except blocks.

", "votes": 0, "replies": [] }, { "id": 738833, "author_name": "beiweizai", "author_url": "", "post_date": "2020-02-07T04:31:43.077000", "content": "

Thanks , But I want to ask how to deal with a video that fails to read

", "votes": 0, "replies": [] }, { "id": 738839, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-07T04:36:56.677000", "content": "

set the prediction of that video to 0.5

", "votes": 1, "replies": [] }, { "id": 759557, "author_name": "", "author_url": "", "post_date": "2020-02-29T07:24:27.800000", "content": "

0.4855380058288574 to be exact

", "votes": 0, "replies": [] }, { "id": 760043, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-29T19:11:01.583000", "content": "

@computerguy Why it is 0.4855380058288574 instead of 0.5?

", "votes": 0, "replies": [] }, { "id": 760313, "author_name": "", "author_url": "", "post_date": "2020-03-01T05:30:32.500000", "content": "

Used maths to calculate that thing.

", "votes": 0, "replies": [] } ] }, { "id": 738668, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-06T21:02:46.707000", "content": "

BTW, our highest scoring kernel is just a modification of my public kernel link. I can't say just a little modification, but is still based on the whole outline.

", "votes": 0, "replies": [ { "id": 738900, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-07T06:40:27.820000", "content": "

EDIT: It is like almost everything changed but the base is in there and there are a lot of hints to our new work. Good Luck finding them.

", "votes": 0, "replies": [] } ] }, { "id": 737893, "author_name": "pete", "author_url": "", "post_date": "2020-02-05T21:48:54.433000", "content": "

Yes, I do that now. Be interesting to predict an optimal clipping from the LB score.

", "votes": 0, "replies": [] }, { "id": 737291, "author_name": "Chason", "author_url": "", "post_date": "2020-02-05T06:27:59.687000", "content": "

Thanks. What does clipping values mean?

", "votes": 0, "replies": [ { "id": 737347, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-05T08:08:40.197000", "content": "

Clipping values means that values will be capped at max and min like clip (0.10, 0.90) means that anything below 0.10 in csv will be written as 0.1 and anything above 0.9 will be written as 0.9.

", "votes": 0, "replies": [] }, { "id": 737349, "author_name": "Chason", "author_url": "", "post_date": "2020-02-05T08:11:00.037000", "content": "

Thanks!

", "votes": -1, "replies": [] }, { "id": 737742, "author_name": "Human Analog", "author_url": "", "post_date": "2020-02-05T17:59:02.593000", "content": "

Just tried that but it made my score worse, so your mileage may vary... don't assume that clipping will be always good. 😄

", "votes": 0, "replies": [] }, { "id": 737752, "author_name": "pete", "author_url": "", "post_date": "2020-02-05T18:11:35.633000", "content": "

It's utility degrades as your model gets better.

", "votes": 2, "replies": [] }, { "id": 737775, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-05T19:03:43.570000", "content": "

@humananalog, I see well, clipping is a method which is highly sensitive, even changing our values slights, we have got a super considerable performance boost.

", "votes": 0, "replies": [] }, { "id": 737776, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-05T19:05:53.670000", "content": "

@petewills Well, it is not true, there is a way to make it even more use of the clipping as our model gets better, our model has accuracy through the roof, I can say that because we are not also using technically full data and still getting our best score.

", "votes": 0, "replies": [] }, { "id": 737797, "author_name": "Human Analog", "author_url": "", "post_date": "2020-02-05T19:33:12.110000", "content": "

The advantage of clipping is that it makes really bad predictions hurt your score less, but it also makes really good predictions help the score less. For my model it apparently has more really good predictions than really bad ones. :-)

", "votes": 2, "replies": [] }, { "id": 737821, "author_name": "pete", "author_url": "", "post_date": "2020-02-05T20:08:49.963000", "content": "

@harshitsheoran Interesting. I'll have to think about that.

", "votes": 0, "replies": [] }, { "id": 737871, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-05T21:22:48.790000", "content": "

If you're model is good enough(that means can get most of the answer right and won't predict high probability for wrong answer), try higher clipping value will help.

", "votes": 0, "replies": [] } ] }, { "id": 737180, "author_name": "Hieu Phung", "author_url": "", "post_date": "2020-02-05T02:42:57.980000", "content": "

Thanks, @unkownhihi. I'll try out to see whether I can climb up a little bit on the Public Leaderboard 😆.

", "votes": 0, "replies": [] }, { "id": 739504, "author_name": "", "author_url": "", "post_date": "2020-02-07T22:11:39.370000", "content": "", "votes": 0, "replies": [ { "id": 739540, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-07T23:41:48.760000", "content": "

Yeah. Dropped a ton of places after sharing. Was 18th now 24th. 😭

", "votes": 1, "replies": [] }, { "id": 739873, "author_name": "", "author_url": "", "post_date": "2020-02-08T13:59:52.257000", "content": "

Did you expect something else ?)

", "votes": 2, "replies": [] }, { "id": 739920, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-02-08T15:00:26.683000", "content": "

Well, Yes we thought we would be improving our score again, which is not the case for now. 😅

", "votes": 0, "replies": [] }, { "id": 740181, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2020-02-09T01:47:02.273000", "content": "

Dropping a ton of place in the current stage of this competition is better than in the end stage of this competition.

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "737177": "I noticed lots of people stuck around 0.68-0.69 or just simply following the best public notebook score so I decided to post such topic to help these people get better scores. I will not include any further details about our model structure.\n\nFirst, we were stuck at around 0.68 and can't get up any further no matter what. Soon, we noticed a pattern that our submission finishes so quickly in kaggle's black box environment. \n\nWe posted a topic to get some help ([link](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/127702)) and @harangdev kindly reminded us to add error catching. \n\nAfter we fixed the bug of submission finishing too fast, the score boosted from ~0.68 straight to ~0.48.\nThen adjustment of input size, model structure, clipping values, we were able to get to our score right now.\n\nAlso check out my topic about submission error combat strategies [link](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/127824).\n\nAny further more details will be released after the competition ends(hopefully on the \"1st place solution\" topic).", "738739": "I would warn against clipping as a post-process operation for this competition, remember:\n1. it is hard to set up a solid local validation set, due to overlapping actors\n2. It has been said by the host that the private test set for the final score will contain different types of videos than what we are training on.\n\ngiven the above, I fear clipping has a high chance of leading people to some optimum public LB score that would hurt private LB. Even small clipping value let's 0.95 or 0.05 might still hurt your score enough to mean the differents between prize/gold, or gold/silver, so far and so forth.\n\nThought I would put it out here :)", "748729": "Edit: Added model stacking which got our score.", "738823": "Thanks for sharing！ Can you provide some details about the “try catch” .", "738668": "BTW, our highest scoring kernel is just a modification of my public kernel [link](https://www.kaggle.com/unkownhihi/starter-kernel-with-cnn-model-ll-lb-0-69235/). I can't say just a little modification, but is still based on the whole outline.", "737893": "Yes, I do that now. Be interesting to predict an optimal clipping from the LB score.", "737291": "Thanks. What does clipping values mean?", "737180": "Thanks, @unkownhihi. I'll try out to see whether I can climb up a little bit on the Public Leaderboard 😆.", "739504": "" } }