I think it is hard to guess if the shakeup will happen or not especially after the tweet sentiment competition even ~80% of competitor are using public lb as local validation .
", "messages": [ { "id": 895938, "postDate": "2020-06-21T17:42:19.553Z", "content": "There will definitely be some shakeup but I dont think it will be massive. We are all operating under the same restrictions in terms of validation. It sounds like a few at the top have hyper optimized for public lb but hard to say exactly how things will shuffle out. Margins are very small here
", "rawMarkdown": "There will definitely be some shakeup but I dont think it will be massive. We are all operating under the same restrictions in terms of validation. It sounds like a few at the top have hyper optimized for public lb but hard to say exactly how things will shuffle out. Margins are very small here", "votes": 3 }, { "id": 896174, "postDate": "2020-06-21T23:43:49.337Z", "content": "Shake? My team went from 29th in public LB on Tweet to 297th. Hello shake!!! Hopefully I won't shake this time!
", "rawMarkdown": "Shake? My team went from **29th** in public LB on ***Tweet*** to **297th**. Hello shake!!! Hopefully I won't shake this time!", "votes": 4, "replies": [ { "id": 896218, "postDate": "2020-06-22T01:36:28.473Z", "content": "My team went from 95th in public LB on Tweet to almost 800th. Hello shake!!!
", "rawMarkdown": "My team went from 95th in public LB on Tweet to almost 800th. Hello shake!!!", "votes": 1 }, { "id": 896225, "postDate": "2020-06-22T01:58:29.360Z", "content": "went from 200 to 1900 on Tweet =) lets' go SHAKE !
", "rawMarkdown": "went from 200 to 1900 on Tweet =) lets' go SHAKE !", "votes": 1 }, { "id": 896699, "postDate": "2020-06-22T11:39:03.540Z", "content": "L O L.\nI am scared now :P
", "rawMarkdown": "L O L.\nI am scared now :P", "votes": 1 }, { "id": 896702, "postDate": "2020-06-22T11:43:22.640Z", "content": "Don't be. Let's have some faith!
", "rawMarkdown": "Don't be. Let's have some faith!", "votes": 1 }, { "id": 897522, "postDate": "2020-06-23T00:02:21.863Z", "content": "Well the shakeup was not big :D
", "rawMarkdown": "Well the shakeup was not big :D" }, { "id": 897530, "postDate": "2020-06-23T00:15:31.220Z", "content": "no shakeup 😜 @veryrobustperson : congrats !!
", "rawMarkdown": "no shakeup 😜 @veryrobustperson : congrats !! ", "votes": 1 }, { "id": 897550, "postDate": "2020-06-23T00:31:16.410Z", "content": "thanks!!!!
", "rawMarkdown": "thanks!!!!" }, { "id": 897556, "postDate": "2020-06-23T00:38:20.930Z", "content": "A very heartily congratulations to @veryrobustperson well deserved
", "rawMarkdown": "A very heartily congratulations to @veryrobustperson well deserved" }, { "id": 897560, "postDate": "2020-06-23T00:43:55.367Z", "content": "@tanulsingh077 thanks you so much! Congratulations to you as well.
", "rawMarkdown": "@tanulsingh077 thanks you so much! Congratulations to you as well." }, { "id": 897581, "postDate": "2020-06-23T01:07:19.827Z", "content": "congratulations bois, I'm still 3 positions short of a medal :(
", "rawMarkdown": "congratulations bois, I'm still 3 positions short of a medal :(" } ] }, { "id": 895395, "postDate": "2020-06-21T10:06:04.257Z", "content": "I think it is hard to guess if the shakeup will happen or not especially after the tweet sentiment competition even ~80% of competitor are using public lb as local validation .
", "rawMarkdown": "I think it is hard to guess if the shakeup will happen or not especially after the tweet sentiment competition even ~80% of competitor are using public lb as local validation .", "votes": 4 }, { "id": 895449, "postDate": "2020-06-21T10:55:26.887Z", "content": "Also this competition can be biased to french, russian, turkish ... native speakers. Wait competition from Jigsaw where 100 langs in testset with equal parts between languages =)
", "rawMarkdown": "Also this competition can be biased to french, russian, turkish ... native speakers. Wait competition from Jigsaw where 100 langs in testset with equal parts between languages =) ", "votes": 1, "replies": [ { "id": 895864, "postDate": "2020-06-21T16:45:52.367Z", "content": "Yes but we are overfitting to public lb
", "rawMarkdown": "Yes but we are overfitting to public lb " } ] }, { "id": 895642, "postDate": "2020-06-21T14:16:00.990Z", "content": "It should be a shake. My one fold valid score is 0.9614, but lb is only 0.9454. There is a big gap between valid score and lb. So there is a big chance for shaking. : )
", "rawMarkdown": "It should be a shake. My one fold valid score is 0.9614, but lb is only 0.9454. There is a big gap between valid score and lb. So there is a big chance for shaking. : )", "votes": 2, "replies": [ { "id": 895863, "postDate": "2020-06-21T16:45:11.127Z", "content": "Yes but i mean approximately all the competior are overfitting so the shakeup will happen but not that big
", "rawMarkdown": "Yes but i mean approximately all the competior are overfitting so the shakeup will happen but not that big " } ] }, { "id": 896839, "postDate": "2020-06-22T13:19:30.727Z", "content": "Someone in the high ranks(top 10) said there is no private dataset , so that means no shuffle?
", "rawMarkdown": "Someone in the high ranks(top 10) said there is no private dataset , so that means no shuffle?\n", "votes": 1, "replies": [ { "id": 896843, "postDate": "2020-06-22T13:22:27.763Z", "content": "the public score is based on only 30% of the total test dataset, the rest 70% will be used for private lb, so we may see some shaking!
", "rawMarkdown": "the public score is based on only 30% of the total test dataset, the rest 70% will be used for private lb, so we may see some shaking!", "votes": 1 }, { "id": 896964, "postDate": "2020-06-22T14:42:37.550Z", "content": "I hope no major shakeup
", "rawMarkdown": "I hope no major shakeup ", "votes": 1 } ] }, { "id": 897033, "postDate": "2020-06-22T15:28:13.843Z", "content": "Is there any topic about public test lang %?\nI recently trained a model with few languages missing and got a higher score than expected. Just curious.
", "rawMarkdown": "Is there any topic about public test lang %?\nI recently trained a model with few languages missing and got a higher score than expected. Just curious.", "replies": [ { "id": 897195, "postDate": "2020-06-22T17:33:11.073Z", "content": "I don't think but i guess there is no major difference between public test and private ..
", "rawMarkdown": "I don't think but i guess there is no major difference between public test and private .." } ] }, { "id": 895440, "postDate": "2020-06-21T10:49:43.880Z", "content": "Also interesting the proportions of langs in public and private lb. If they too different then shakeup can be very broad
", "rawMarkdown": "Also interesting the proportions of langs in public and private lb. If they too different then shakeup can be very broad" }, { "id": 896561, "postDate": "2020-06-22T09:07:01.657Z", "rawMarkdown": "", "votes": -4, "isDeleted": true, "replies": [ { "id": 896579, "postDate": "2020-06-22T09:35:37.690Z", "content": "I disagree with the second. They are not good indictors of overfitting.
", "rawMarkdown": "I disagree with the second. They are not good indictors of overfitting.", "votes": 5 }, { "id": 896632, "postDate": "2020-06-22T10:24:20.610Z", "content": "We have to select the best two submission wisely, i know it will not be easy :/
", "rawMarkdown": "We have to select the best two submission wisely, i know it will not be easy :/ " }, { "id": 896748, "postDate": "2020-06-22T12:12:10.353Z", "content": "Are you implying me? We will see tomorrow.
", "rawMarkdown": "Are you implying me? We will see tomorrow.", "votes": 3 }, { "id": 896845, "postDate": "2020-06-22T13:25:01.383Z", "content": "@underwearfitting I didn't imply anyone , this is the reason I don't pass controversial comments , as a matter of fact I did not imply anyone especially from the top 30 , and I also just said that it could a reason . And I am really sorry
\n@veryrobustperson I am sorry , I perhaps wrote in the wrong way , Will remove my comment
There will definitely be some shakeup but I dont think it will be massive. We are all operating under the same restrictions in terms of validation. It sounds like a few at the top have hyper optimized for public lb but hard to say exactly how things will shuffle out. Margins are very small here
", "votes": 3, "replies": [] }, { "id": 896174, "author_name": "cat", "author_url": "", "post_date": "2020-06-21T23:43:49.337000", "content": "Shake? My team went from 29th in public LB on Tweet to 297th. Hello shake!!! Hopefully I won't shake this time!
", "votes": 4, "replies": [ { "id": 896218, "author_name": "Liyan Tang", "author_url": "", "post_date": "2020-06-22T01:36:28.473000", "content": "My team went from 95th in public LB on Tweet to almost 800th. Hello shake!!!
", "votes": 1, "replies": [] }, { "id": 896225, "author_name": "Alin Cijov", "author_url": "", "post_date": "2020-06-22T01:58:29.360000", "content": "went from 200 to 1900 on Tweet =) lets' go SHAKE !
", "votes": 1, "replies": [] }, { "id": 896699, "author_name": "sajwankit", "author_url": "", "post_date": "2020-06-22T11:39:03.540000", "content": "L O L.\nI am scared now :P
", "votes": 1, "replies": [] }, { "id": 896702, "author_name": "cat", "author_url": "", "post_date": "2020-06-22T11:43:22.640000", "content": "Don't be. Let's have some faith!
", "votes": 1, "replies": [] }, { "id": 897522, "author_name": "sajwankit", "author_url": "", "post_date": "2020-06-23T00:02:21.863000", "content": "Well the shakeup was not big :D
", "votes": 0, "replies": [] }, { "id": 897530, "author_name": "Yassine Hamdaoui", "author_url": "", "post_date": "2020-06-23T00:15:31.220000", "content": "no shakeup 😜 @veryrobustperson : congrats !!
", "votes": 1, "replies": [] }, { "id": 897550, "author_name": "cat", "author_url": "", "post_date": "2020-06-23T00:31:16.410000", "content": "thanks!!!!
", "votes": 0, "replies": [] }, { "id": 897556, "author_name": "Mr_KnowNothing", "author_url": "", "post_date": "2020-06-23T00:38:20.930000", "content": "A very heartily congratulations to @veryrobustperson well deserved
", "votes": 0, "replies": [] }, { "id": 897560, "author_name": "cat", "author_url": "", "post_date": "2020-06-23T00:43:55.367000", "content": "@tanulsingh077 thanks you so much! Congratulations to you as well.
", "votes": 0, "replies": [] }, { "id": 897581, "author_name": "Alin Cijov", "author_url": "", "post_date": "2020-06-23T01:07:19.827000", "content": "congratulations bois, I'm still 3 positions short of a medal :(
", "votes": 0, "replies": [] } ] }, { "id": 895449, "author_name": "Sxwat", "author_url": "", "post_date": "2020-06-21T10:55:26.887000", "content": "Also this competition can be biased to french, russian, turkish ... native speakers. Wait competition from Jigsaw where 100 langs in testset with equal parts between languages =)
", "votes": 1, "replies": [ { "id": 895864, "author_name": "Haythem Tellili", "author_url": "", "post_date": "2020-06-21T16:45:52.367000", "content": "Yes but we are overfitting to public lb
", "votes": 0, "replies": [] } ] }, { "id": 895642, "author_name": "huiqin", "author_url": "", "post_date": "2020-06-21T14:16:00.990000", "content": "It should be a shake. My one fold valid score is 0.9614, but lb is only 0.9454. There is a big gap between valid score and lb. So there is a big chance for shaking. : )
", "votes": 2, "replies": [ { "id": 895863, "author_name": "Haythem Tellili", "author_url": "", "post_date": "2020-06-21T16:45:11.127000", "content": "Yes but i mean approximately all the competior are overfitting so the shakeup will happen but not that big
", "votes": 0, "replies": [] } ] }, { "id": 896839, "author_name": "yash chaudhary", "author_url": "", "post_date": "2020-06-22T13:19:30.727000", "content": "Someone in the high ranks(top 10) said there is no private dataset , so that means no shuffle?
", "votes": 1, "replies": [ { "id": 896843, "author_name": "sajwankit", "author_url": "", "post_date": "2020-06-22T13:22:27.763000", "content": "the public score is based on only 30% of the total test dataset, the rest 70% will be used for private lb, so we may see some shaking!
", "votes": 1, "replies": [] }, { "id": 896964, "author_name": "Haythem Tellili", "author_url": "", "post_date": "2020-06-22T14:42:37.550000", "content": "I hope no major shakeup
", "votes": 1, "replies": [] } ] }, { "id": 897033, "author_name": "Oleg Yaroshevskiy", "author_url": "", "post_date": "2020-06-22T15:28:13.843000", "content": "Is there any topic about public test lang %?\nI recently trained a model with few languages missing and got a higher score than expected. Just curious.
", "votes": 0, "replies": [ { "id": 897195, "author_name": "Haythem Tellili", "author_url": "", "post_date": "2020-06-22T17:33:11.073000", "content": "I don't think but i guess there is no major difference between public test and private ..
", "votes": 0, "replies": [] } ] }, { "id": 895440, "author_name": "Sxwat", "author_url": "", "post_date": "2020-06-21T10:49:43.880000", "content": "Also interesting the proportions of langs in public and private lb. If they too different then shakeup can be very broad
", "votes": 0, "replies": [] }, { "id": 896561, "author_name": "", "author_url": "", "post_date": "2020-06-22T09:07:01.657000", "content": "", "votes": -4, "replies": [ { "id": 896579, "author_name": "cat", "author_url": "", "post_date": "2020-06-22T09:35:37.690000", "content": "I disagree with the second. They are not good indictors of overfitting.
", "votes": 5, "replies": [] }, { "id": 896632, "author_name": "Haythem Tellili", "author_url": "", "post_date": "2020-06-22T10:24:20.610000", "content": "We have to select the best two submission wisely, i know it will not be easy :/
", "votes": 0, "replies": [] }, { "id": 896748, "author_name": "sin", "author_url": "", "post_date": "2020-06-22T12:12:10.353000", "content": "Are you implying me? We will see tomorrow.
", "votes": 3, "replies": [] }, { "id": 896845, "author_name": "Mr_KnowNothing", "author_url": "", "post_date": "2020-06-22T13:25:01.383000", "content": "@underwearfitting I didn't imply anyone , this is the reason I don't pass controversial comments , as a matter of fact I did not imply anyone especially from the top 30 , and I also just said that it could a reason . And I am really sorry
\n@veryrobustperson I am sorry , I perhaps wrote in the wrong way , Will remove my comment