{ "id": 55771, "title": "Shake up is coming?", "url": "/competitions/talkingdata-adtracking-fraud-detection/discussion/55771", "author_name": "", "post_date": "2018-05-01T14:01:38.854541200Z", "votes": 8, "comment_count": 6, "views": 0, "content": "

The 18% is only noon.\ntry

\n\n
sub.click_time += pd.offsets.Hour(8) # Chinese time zones\nsub.loc[sub.click_time.dt.hour!=12, 'is_attributed'] = 0\n
\n\n

Just believe your local validation. 😞

", "messages": [ { "id": "321536", "postDate": "05/01/2018 14:01:38", "content": "

The 18% is only noon.\ntry

\n\n
sub.click_time += pd.offsets.Hour(8) # Chinese time zones\nsub.loc[sub.click_time.dt.hour!=12, 'is_attributed'] = 0\n
\n\n

Just believe your local validation. 😞

", "rawMarkdown": "The 18% is only noon.\ntry\n\n sub.click_time += pd.offsets.Hour(8) # Chinese time zones\n sub.loc[sub.click_time.dt.hour!=12, 'is_attributed'] = 0\n\nJust believe your local validation. 😞", "votes": null }, { "id": "321594", "postDate": "05/01/2018 16:11:06", "content": "

That the final absolute numbers on private LB will be very different from the public one is pretty clear by this point (several people reported a shift in validation performance on the first hour - corresponding to the public / priv split - vs remaining period), the question is: by how much. I mean, The distance from my location to the very top is < 0.003 in absolute terms - so there is indeed potential for a bumpy ride.

", "rawMarkdown": "That the final absolute numbers on private LB will be very different from the public one is pretty clear by this point (several people reported a shift in validation performance on the first hour - corresponding to the public / priv split - vs remaining period), the question is: by how much. I mean, The distance from my location to the very top is < 0.003 in absolute terms - so there is indeed potential for a bumpy ride.", "votes": null }, { "id": "321962", "postDate": "05/02/2018 08:01:34", "content": "

I don't expect much shakeup near the top, based on my local validation on various dataset. They evolve consistently with LB score. I expect the same is true for most of the top teams.

\n\n

What is true, is that the validation score varies depending on what data you use for it. I expect private LB scores to be significantly different from public LB score. But ordering should be similar.

", "rawMarkdown": "I don't expect much shakeup near the top, based on my local validation on various dataset. They evolve consistently with LB score. I expect the same is true for most of the top teams. \n\nWhat is true, is that the validation score varies depending on what data you use for it. I expect private LB scores to be significantly different from public LB score. But ordering should be similar.", "votes": null }, { "id": "322056", "postDate": "05/02/2018 11:20:21", "content": "

Agree, the difference between the validation scores and leaderboard scores have been pretty consistent during the entire competition, which I suppose is also the case for most of the top teams.

", "rawMarkdown": "Agree, the difference between the validation scores and leaderboard scores have been pretty consistent during the entire competition, which I suppose is also the case for most of the top teams.", "votes": null }, { "id": "322210", "postDate": "05/02/2018 14:58:17", "content": "

Maybe I'll regret my previous answer when I will drop significantly on the private LB... Crossing fingers ;)

", "rawMarkdown": "Maybe I'll regret my previous answer when I will drop significantly on the private LB... Crossing fingers ;)", "votes": null }, { "id": "322211", "postDate": "05/02/2018 14:59:47", "content": "

I agree there will be shakeup when lots of people have similar scores. A 0.0001 difference is noise IMHO. But near the top differences are significant.

", "rawMarkdown": "I agree there will be shakeup when lots of people have similar scores. A 0.0001 difference is noise IMHO. But near the top differences are significant.", "votes": null }, { "id": "322648", "postDate": "05/03/2018 11:21:07", "content": "

I'm not sure I agree.\nThere is a large variance between the test hours. There might be algorithms that works differently on different hours and could cause a shakeup. I'm not saying that it would happen, but it could happen :).

", "rawMarkdown": "I'm not sure I agree.\nThere is a large variance between the test hours. There might be algorithms that works differently on different hours and could cause a shakeup. I'm not saying that it would happen, but it could happen :).", "votes": null } ], "comments": [ { "id": 321594, "author_name": "konradb", "author_url": "", "post_date": "05/01/2018 16:11:06", "content": "

That the final absolute numbers on private LB will be very different from the public one is pretty clear by this point (several people reported a shift in validation performance on the first hour - corresponding to the public / priv split - vs remaining period), the question is: by how much. I mean, The distance from my location to the very top is < 0.003 in absolute terms - so there is indeed potential for a bumpy ride.

", "votes": null, "replies": [ { "id": 322211, "author_name": "cpmpml", "author_url": "", "post_date": "05/02/2018 14:59:47", "content": "

I agree there will be shakeup when lots of people have similar scores. A 0.0001 difference is noise IMHO. But near the top differences are significant.

", "votes": null, "replies": [] } ] }, { "id": 321962, "author_name": "cpmpml", "author_url": "", "post_date": "05/02/2018 08:01:34", "content": "

I don't expect much shakeup near the top, based on my local validation on various dataset. They evolve consistently with LB score. I expect the same is true for most of the top teams.

\n\n

What is true, is that the validation score varies depending on what data you use for it. I expect private LB scores to be significantly different from public LB score. But ordering should be similar.

", "votes": null, "replies": [ { "id": 322056, "author_name": "kevinbonnes", "author_url": "", "post_date": "05/02/2018 11:20:21", "content": "

Agree, the difference between the validation scores and leaderboard scores have been pretty consistent during the entire competition, which I suppose is also the case for most of the top teams.

", "votes": null, "replies": [] }, { "id": 322210, "author_name": "cpmpml", "author_url": "", "post_date": "05/02/2018 14:58:17", "content": "

Maybe I'll regret my previous answer when I will drop significantly on the private LB... Crossing fingers ;)

", "votes": null, "replies": [] }, { "id": 322648, "author_name": "mrbeer", "author_url": "", "post_date": "05/03/2018 11:21:07", "content": "

I'm not sure I agree.\nThere is a large variance between the test hours. There might be algorithms that works differently on different hours and could cause a shakeup. I'm not saying that it would happen, but it could happen :).

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "321536": "The 18% is only noon.\ntry\n\n sub.click_time += pd.offsets.Hour(8) # Chinese time zones\n sub.loc[sub.click_time.dt.hour!=12, 'is_attributed'] = 0\n\nJust believe your local validation. 😞", "321594": "That the final absolute numbers on private LB will be very different from the public one is pretty clear by this point (several people reported a shift in validation performance on the first hour - corresponding to the public / priv split - vs remaining period), the question is: by how much. I mean, The distance from my location to the very top is < 0.003 in absolute terms - so there is indeed potential for a bumpy ride.", "321962": "I don't expect much shakeup near the top, based on my local validation on various dataset. They evolve consistently with LB score. I expect the same is true for most of the top teams. \n\nWhat is true, is that the validation score varies depending on what data you use for it. I expect private LB scores to be significantly different from public LB score. But ordering should be similar.", "322056": "Agree, the difference between the validation scores and leaderboard scores have been pretty consistent during the entire competition, which I suppose is also the case for most of the top teams.", "322210": "Maybe I'll regret my previous answer when I will drop significantly on the private LB... Crossing fingers ;)", "322211": "I agree there will be shakeup when lots of people have similar scores. A 0.0001 difference is noise IMHO. But near the top differences are significant.", "322648": "I'm not sure I agree.\nThere is a large variance between the test hours. There might be algorithms that works differently on different hours and could cause a shakeup. I'm not saying that it would happen, but it could happen :)." }, "source": "meta" }