{ "id": 205423, "title": "Are we overfitting lb", "url": "/competitions/riiid-test-answer-prediction/discussion/205423", "author_name": "", "post_date": "2020-12-20T05:46:52.790665900Z", "votes": 3, "comment_count": 12, "views": 0, "content": "

81.2 top score much higher than state of art .kudos to the person who have achieved this .
\nBut with just 20 pct data in public test set ,are we grossly overfitting the lb . What are chances of lb topple.

", "messages": [ { "id": "1119495", "postDate": "12/20/2020 05:46:52", "content": "

81.2 top score much higher than state of art .kudos to the person who have achieved this .
\nBut with just 20 pct data in public test set ,are we grossly overfitting the lb . What are chances of lb topple.

", "rawMarkdown": "81.2 top score much higher than state of art .kudos to the person who have achieved this .\nBut with just 20 pct data in public test set ,are we grossly overfitting the lb . What are chances of lb topple.", "votes": null }, { "id": "1119531", "postDate": "12/20/2020 06:32:38", "content": "

I highly doubt it. Too many samples, and nice cv/lb correlations have been reported.

", "rawMarkdown": "I highly doubt it. Too many samples, and nice cv/lb correlations have been reported.", "votes": null }, { "id": "1119575", "postDate": "12/20/2020 07:34:21", "content": "

SOTA is a single model. Top LB is probably an ensemble of a models.

", "rawMarkdown": "SOTA is a single model. Top LB is probably an ensemble of a models.", "votes": null }, { "id": "1119594", "postDate": "12/20/2020 08:02:09", "content": "

What is SOTA here? Saint+?

", "rawMarkdown": "What is SOTA here? Saint+?", "votes": null }, { "id": "1119598", "postDate": "12/20/2020 08:05:51", "content": "

No, .812 by a single model.

", "rawMarkdown": "No, .812 by a single model.", "votes": null }, { "id": "1119742", "postDate": "12/20/2020 10:43:59", "content": "

nice score. did you use both lecture & question as part of your input structure?

", "rawMarkdown": "nice score. did you use both lecture & question as part of your input structure?", "votes": null }, { "id": "1119874", "postDate": "12/20/2020 12:22:17", "content": "

@mamas nice work Hope you maintain your ranks.
\nWould you mind telling if your standing is result of twists n turns in Saint woth some more features ?

", "rawMarkdown": "mamas nice work Hope you maintain your ranks. \nWould you mind telling if your standing is result of twists n turns in Saint woth some more features ?", "votes": null }, { "id": "1121474", "postDate": "12/21/2020 17:04:59", "content": "

Amazing!!! Is it seq-seq model? I guess we will know after the competition :) but any hints welcome

", "rawMarkdown": "Amazing!!! Is it seq-seq model? I guess we will know after the competition :) but any hints welcome", "votes": null }, { "id": "1121760", "postDate": "12/21/2020 22:06:45", "content": "

It's not saint but transformer-like architecture. I promise that I will publicly share single model >0.812 notebook after the competition ends!

", "rawMarkdown": "It's not saint but transformer-like architecture. I promise that I will publicly share single model >0.812 notebook after the competition ends!", "votes": null }, { "id": "1121873", "postDate": "12/22/2020 01:37:47", "content": "

It's not the first time people here are beating the SoTA just FYI :) And as long as your CV is solid, I will Trust it, not the LB.

", "rawMarkdown": "It's not the first time people here are beating the SoTA just FYI :) And as long as your CV is solid, I will Trust it, not the LB.", "votes": null }, { "id": "1121936", "postDate": "12/22/2020 03:22:53", "content": "

@mamas good job ..I wish you only become winner .
\nSuggest to write a paper first before publishing…well done !

", "rawMarkdown": "mamas good job ..I wish you only become winner .\nSuggest to write a paper first before publishing...well done !", "votes": null }, { "id": "1121992", "postDate": "12/22/2020 05:15:55", "content": "

wishing you all the very best @mamasinkgs

", "rawMarkdown": "wishing you all the very best @mamasinkgs", "votes": null }, { "id": "1137340", "postDate": "01/03/2021 21:10:51", "content": "

it is crazy how transformers are being introduced to Tabular data probably breaking SOTA

", "rawMarkdown": "it is crazy how transformers are being introduced to Tabular data probably breaking SOTA", "votes": null } ], "comments": [ { "id": 1119531, "author_name": "authman", "author_url": "", "post_date": "12/20/2020 06:32:38", "content": "

I highly doubt it. Too many samples, and nice cv/lb correlations have been reported.

", "votes": null, "replies": [] }, { "id": 1119575, "author_name": "pavelvod", "author_url": "", "post_date": "12/20/2020 07:34:21", "content": "

SOTA is a single model. Top LB is probably an ensemble of a models.

", "votes": null, "replies": [ { "id": 1119598, "author_name": "mamasinkgs", "author_url": "", "post_date": "12/20/2020 08:05:51", "content": "

No, .812 by a single model.

", "votes": null, "replies": [] }, { "id": 1119742, "author_name": "projdev", "author_url": "", "post_date": "12/20/2020 10:43:59", "content": "

nice score. did you use both lecture & question as part of your input structure?

", "votes": null, "replies": [] }, { "id": 1119874, "author_name": "jaideepvalani", "author_url": "", "post_date": "12/20/2020 12:22:17", "content": "

@mamas nice work Hope you maintain your ranks.
\nWould you mind telling if your standing is result of twists n turns in Saint woth some more features ?

", "votes": null, "replies": [] }, { "id": 1121474, "author_name": "allohvk", "author_url": "", "post_date": "12/21/2020 17:04:59", "content": "

Amazing!!! Is it seq-seq model? I guess we will know after the competition :) but any hints welcome

", "votes": null, "replies": [] }, { "id": 1121760, "author_name": "mamasinkgs", "author_url": "", "post_date": "12/21/2020 22:06:45", "content": "

It's not saint but transformer-like architecture. I promise that I will publicly share single model >0.812 notebook after the competition ends!

", "votes": null, "replies": [] }, { "id": 1121936, "author_name": "jaideepvalani", "author_url": "", "post_date": "12/22/2020 03:22:53", "content": "

@mamas good job ..I wish you only become winner .
\nSuggest to write a paper first before publishing…well done !

", "votes": null, "replies": [] }, { "id": 1121992, "author_name": "allohvk", "author_url": "", "post_date": "12/22/2020 05:15:55", "content": "

wishing you all the very best @mamasinkgs

", "votes": null, "replies": [] } ] }, { "id": 1119594, "author_name": "nicohrubec", "author_url": "", "post_date": "12/20/2020 08:02:09", "content": "

What is SOTA here? Saint+?

", "votes": null, "replies": [] }, { "id": 1121873, "author_name": "adityaecdrid", "author_url": "", "post_date": "12/22/2020 01:37:47", "content": "

It's not the first time people here are beating the SoTA just FYI :) And as long as your CV is solid, I will Trust it, not the LB.

", "votes": null, "replies": [] }, { "id": 1137340, "author_name": "elvinagammed", "author_url": "", "post_date": "01/03/2021 21:10:51", "content": "

it is crazy how transformers are being introduced to Tabular data probably breaking SOTA

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1119495": "81.2 top score much higher than state of art .kudos to the person who have achieved this .\nBut with just 20 pct data in public test set ,are we grossly overfitting the lb . What are chances of lb topple.", "1119531": "I highly doubt it. Too many samples, and nice cv/lb correlations have been reported.", "1119575": "SOTA is a single model. Top LB is probably an ensemble of a models.", "1119594": "What is SOTA here? Saint+?", "1119598": "No, .812 by a single model.", "1119742": "nice score. did you use both lecture & question as part of your input structure?", "1119874": "mamas nice work Hope you maintain your ranks. \nWould you mind telling if your standing is result of twists n turns in Saint woth some more features ?", "1121474": "Amazing!!! Is it seq-seq model? I guess we will know after the competition :) but any hints welcome", "1121760": "It's not saint but transformer-like architecture. I promise that I will publicly share single model >0.812 notebook after the competition ends!", "1121873": "It's not the first time people here are beating the SoTA just FYI :) And as long as your CV is solid, I will Trust it, not the LB.", "1121936": "mamas good job ..I wish you only become winner .\nSuggest to write a paper first before publishing...well done !", "1121992": "wishing you all the very best @mamasinkgs", "1137340": "it is crazy how transformers are being introduced to Tabular data probably breaking SOTA" }, "source": "meta" }