{ "id": 216017, "title": "What do you think the degree of shake-up will be？", "url": "/competitions/cassava-leaf-disease-classification/discussion/216017", "author_name": "", "post_date": "2021-02-01T08:18:05.395555500Z", "votes": 11, "comment_count": 10, "views": 0, "content": "

The LB here https://www.kaggle.com/c/cassava-disease/overview in 2019 did not shake-up much. However, there were way fewer teams, only 86.

\n

For the top 10 teams in 2019, the shake is likely to be in the range [-0.006, +0.006].
\nWhich is quite a small number I believe, you could have [-0.001, +0.001] shake with the same submission impacted by the randomness of TTA even with a huge ensemble.

\n

Will the shake-up be an earthquake this year? What do you think?
\nPersonally, I would say the shake-up will slightly bigger than in 2019, somewhere inside [+-0.005, +-0.009].

", "messages": [ { "id": "1180363", "postDate": "02/01/2021 08:18:05", "content": "

The LB here https://www.kaggle.com/c/cassava-disease/overview in 2019 did not shake-up much. However, there were way fewer teams, only 86.

\n

For the top 10 teams in 2019, the shake is likely to be in the range [-0.006, +0.006].
\nWhich is quite a small number I believe, you could have [-0.001, +0.001] shake with the same submission impacted by the randomness of TTA even with a huge ensemble.

\n

Will the shake-up be an earthquake this year? What do you think?
\nPersonally, I would say the shake-up will slightly bigger than in 2019, somewhere inside [+-0.005, +-0.009].

", "rawMarkdown": "The LB here [https://www.kaggle.com/c/cassava-disease/overview](url) in 2019 did not shake-up much. However, there were way fewer teams, only 86.\n\nFor the top 10 teams in 2019, the shake is likely to be in the range [-0.006, +0.006].\nWhich is quite a small number I believe, you could have [-0.001, +0.001] shake with the same submission impacted by the randomness of TTA even with a huge ensemble.\n\nWill the shake-up be an earthquake this year? What do you think?\nPersonally, I would say the shake-up will slightly bigger than in 2019, somewhere inside [+-0.005, +-0.009].", "votes": null }, { "id": "1180378", "postDate": "02/01/2021 08:24:18", "content": "

The noise in test set may cause a big shake.

", "rawMarkdown": "The noise in test set may cause a big shake.", "votes": null }, { "id": "1180379", "postDate": "02/01/2021 08:24:18", "content": "

The noise in test set may cause a big shake.

", "rawMarkdown": "The noise in test set may cause a big shake.", "votes": null }, { "id": "1180477", "postDate": "02/01/2021 09:29:10", "content": "

you can test it if you have oof results. Randomly split the oof by 31% and 67% and run it many times and compare the std between 31% and 67%

", "rawMarkdown": "you can test it if you have oof results. Randomly split the oof by 31% and 67% and run it many times and compare the std between 31% and 67%", "votes": null }, { "id": "1180483", "postDate": "02/01/2021 09:33:00", "content": "

whats your result doing this?

", "rawMarkdown": "whats your result doing this?", "votes": null }, { "id": "1180487", "postDate": "02/01/2021 09:36:32", "content": "

[-0.005, +0.005], quite close to 2019 comp. It is not big deal for top teams, but quite large for the rest

", "rawMarkdown": "[-0.005, +0.005], quite close to 2019 comp. It is not big deal for top teams, but quite large for the rest", "votes": null }, { "id": "1180499", "postDate": "02/01/2021 09:50:10", "content": "

this gives a good insight https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/207410

", "rawMarkdown": "this gives a good insight https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/207410", "votes": null }, { "id": "1180512", "postDate": "02/01/2021 09:56:24", "content": "

I remember 2-3 private LB scores are shared because of the leak. Seems deleted now.
\nDid not record that, but I do remember these numbers did not shake much.

", "rawMarkdown": "I remember 2-3 private LB scores are shared because of the leak. Seems deleted now.\nDid not record that, but I do remember these numbers did not shake much.", "votes": null }, { "id": "1180544", "postDate": "02/01/2021 10:15:19", "content": "

Yep, most of them where nearly the same

", "rawMarkdown": "Yep, most of them where nearly the same", "votes": null }, { "id": "1181111", "postDate": "02/01/2021 16:50:21", "content": "

I feel the shake up will be much worse.
\nRemember this https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression ?
\nI think it will be on a similar scale :/

", "rawMarkdown": "I feel the shake up will be much worse.\nRemember this [https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression](url) ? \nI think it will be on a similar scale :/", "votes": null }, { "id": "1181123", "postDate": "02/01/2021 17:01:47", "content": "

why you think so? is there any evidence?
\nin this competition it's \"just\" images and their accuracy; so no additional features to evaluate and no special metric like \"forced vital capacity\"

", "rawMarkdown": "why you think so? is there any evidence? \nin this competition it's \"just\" images and their accuracy; so no additional features to evaluate and no special metric like \"forced vital capacity\"", "votes": null } ], "comments": [ { "id": 1180378, "author_name": "zekunn", "author_url": "", "post_date": "02/01/2021 08:24:18", "content": "

The noise in test set may cause a big shake.

", "votes": null, "replies": [] }, { "id": 1180379, "author_name": "zekunn", "author_url": "", "post_date": "02/01/2021 08:24:18", "content": "

The noise in test set may cause a big shake.

", "votes": null, "replies": [] }, { "id": 1180477, "author_name": "murphy89", "author_url": "", "post_date": "02/01/2021 09:29:10", "content": "

you can test it if you have oof results. Randomly split the oof by 31% and 67% and run it many times and compare the std between 31% and 67%

", "votes": null, "replies": [ { "id": 1180483, "author_name": "alexanderriedel", "author_url": "", "post_date": "02/01/2021 09:33:00", "content": "

whats your result doing this?

", "votes": null, "replies": [] }, { "id": 1180487, "author_name": "murphy89", "author_url": "", "post_date": "02/01/2021 09:36:32", "content": "

[-0.005, +0.005], quite close to 2019 comp. It is not big deal for top teams, but quite large for the rest

", "votes": null, "replies": [] } ] }, { "id": 1180499, "author_name": "mithilsalunkhe", "author_url": "", "post_date": "02/01/2021 09:50:10", "content": "

this gives a good insight https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/207410

", "votes": null, "replies": [ { "id": 1180512, "author_name": "zlanan", "author_url": "", "post_date": "02/01/2021 09:56:24", "content": "

I remember 2-3 private LB scores are shared because of the leak. Seems deleted now.
\nDid not record that, but I do remember these numbers did not shake much.

", "votes": null, "replies": [] }, { "id": 1180544, "author_name": "mithilsalunkhe", "author_url": "", "post_date": "02/01/2021 10:15:19", "content": "

Yep, most of them where nearly the same

", "votes": null, "replies": [] } ] }, { "id": 1181111, "author_name": "pratt3000", "author_url": "", "post_date": "02/01/2021 16:50:21", "content": "

I feel the shake up will be much worse.
\nRemember this https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression ?
\nI think it will be on a similar scale :/

", "votes": null, "replies": [ { "id": 1181123, "author_name": "alexanderriedel", "author_url": "", "post_date": "02/01/2021 17:01:47", "content": "

why you think so? is there any evidence?
\nin this competition it's \"just\" images and their accuracy; so no additional features to evaluate and no special metric like \"forced vital capacity\"

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1180363": "The LB here [https://www.kaggle.com/c/cassava-disease/overview](url) in 2019 did not shake-up much. However, there were way fewer teams, only 86.\n\nFor the top 10 teams in 2019, the shake is likely to be in the range [-0.006, +0.006].\nWhich is quite a small number I believe, you could have [-0.001, +0.001] shake with the same submission impacted by the randomness of TTA even with a huge ensemble.\n\nWill the shake-up be an earthquake this year? What do you think?\nPersonally, I would say the shake-up will slightly bigger than in 2019, somewhere inside [+-0.005, +-0.009].", "1180378": "The noise in test set may cause a big shake.", "1180379": "The noise in test set may cause a big shake.", "1180477": "you can test it if you have oof results. Randomly split the oof by 31% and 67% and run it many times and compare the std between 31% and 67%", "1180483": "whats your result doing this?", "1180487": "[-0.005, +0.005], quite close to 2019 comp. It is not big deal for top teams, but quite large for the rest", "1180499": "this gives a good insight https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/207410", "1180512": "I remember 2-3 private LB scores are shared because of the leak. Seems deleted now.\nDid not record that, but I do remember these numbers did not shake much.", "1180544": "Yep, most of them where nearly the same", "1181111": "I feel the shake up will be much worse.\nRemember this [https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression](url) ? \nI think it will be on a similar scale :/", "1181123": "why you think so? is there any evidence? \nin this competition it's \"just\" images and their accuracy; so no additional features to evaluate and no special metric like \"forced vital capacity\"" }, "source": "meta" }