{ "id": 117167, "title": "Why taking too much time verifying final result? ", "url": "/competitions/3d-object-detection-for-autonomous-vehicles/discussion/117167", "author_name": "Mobassir", "post_date": "2019-11-13T18:25:54.892000", "votes": 15, "comment_count": 20, "views": 0, "content": "

In my previous competitions within 6-7 hours of exposing private leaderboard organizers verified final result.but in this competition 19+ hours have been passed but the final result verification is not done yet.just curious to know,why?\nAre they waiting for rsna to finish or something? Thanks

", "messages": [ { "id": 672296, "postDate": "2019-11-13T18:25:54.893Z", "content": "

In my previous competitions within 6-7 hours of exposing private leaderboard organizers verified final result.but in this competition 19+ hours have been passed but the final result verification is not done yet.just curious to know,why?\nAre they waiting for rsna to finish or something? Thanks

", "rawMarkdown": "In my previous competitions within 6-7 hours of exposing private leaderboard organizers verified final result.but in this competition 19+ hours have been passed but the final result verification is not done yet.just curious to know,why?\nAre they waiting for rsna to finish or something? Thanks ", "votes": 15 }, { "id": 677462, "postDate": "2019-11-20T07:41:16.307Z", "content": "

A week has passed and not even a word from the host or kaggle ☚ī¸ At least a \"Thank you for your work, but we face problems and it will take about X days longer to verify the results.\" they could have given us IMO. We also invested a lot of time and just a small update message would make me feel our work is appreciated, but just leaving the competition unverified and uncommented gives me the impression, that they don't care. Even if this isn't the case. It's just about a minimum level of communication.

\n\n

Edit:\n@philculliton @iglovikov is it possible to please give us a small update with a rough estimation on when the competition will be finalized? I know you guys are busy, but please understand that this is a weird situation for us, checking every day the competition and seeing other competitions finalizing. Will the metric be recalculated due to the \"small classes problem\" (see thread)? Or have you contacted the winners and will publish the final Leaderboard during NeurIPS? Please talk to us.

", "rawMarkdown": "A week has passed and not even a word from the host or kaggle ☚ī¸ At least a \"Thank you for your work, but we face problems and it will take about X days longer to verify the results.\" they could have given us IMO. We also invested a lot of time and just a small update message would make me feel our work is appreciated, but just leaving the competition unverified and uncommented gives me the impression, that they don't care. Even if this isn't the case. It's just about a minimum level of communication.\n\nEdit:\n@philculliton @iglovikov is it possible to please give us a small update with a rough estimation on when the competition will be finalized? I know you guys are busy, but please understand that this is a weird situation for us, checking every day the competition and seeing other competitions finalizing. Will the metric be recalculated due to the \"small classes problem\" (see thread)? Or have you contacted the winners and will publish the final Leaderboard during NeurIPS? Please talk to us.", "votes": 14, "replies": [ { "id": 677468, "postDate": "2019-11-20T07:54:46.840Z", "content": "

Very well said. Waiting for an update from kaggle team 😟

", "rawMarkdown": "Very well said. Waiting for an update from kaggle team 😟 ", "votes": 4 }, { "id": 677488, "postDate": "2019-11-20T08:24:01.487Z", "content": "

you are right @marekwyborski \nand it hurts surely :(

", "rawMarkdown": "you are right @marekwyborski \nand it hurts surely :(", "votes": 4 }, { "id": 677643, "postDate": "2019-11-20T12:45:38.177Z", "content": "

Well said @marekwyborski . Almost 8 days since the competition got over, still no signs of when the leaderboard will be verified. RSNA, Clouds got completed in last 8 days and we got the LB verified almost on the same day.

", "rawMarkdown": "Well said @marekwyborski . Almost 8 days since the competition got over, still no signs of when the leaderboard will be verified. RSNA, Clouds got completed in last 8 days and we got the LB verified almost on the same day. ", "votes": 4 }, { "id": 677827, "postDate": "2019-11-20T16:23:51.383Z", "content": "

I tweeted to Vladimir regarding this issue, here's his reply : https://twitter.com/viglovikov/status/1197179150315872257?s=20

\n\n

(Sorry for formatting, posting from mobile rn)

", "rawMarkdown": "I tweeted to Vladimir regarding this issue, here's his reply : https://twitter.com/viglovikov/status/1197179150315872257?s=20\n\n(Sorry for formatting, posting from mobile rn)", "votes": 5 }, { "id": 678629, "postDate": "2019-11-21T17:00:30.737Z", "content": "

Thanks a lot for taking this initiative up @rishabhiitbhu . Hopefully Kaggle team will verify it before next set of competition gets over like NFL, Ashrae.

", "rawMarkdown": "Thanks a lot for taking this initiative up @rishabhiitbhu . Hopefully Kaggle team will verify it before next set of competition gets over like NFL, Ashrae. ", "votes": 4 }, { "id": 679359, "postDate": "2019-11-22T17:25:31.620Z", "content": "

Agree, you really hit the nail on the head!

", "rawMarkdown": "Agree, you really hit the nail on the head!", "votes": 3 } ] }, { "id": 679357, "postDate": "2019-11-22T17:21:48.407Z", "content": "

\nimport kaggle\nimport memegenerator\nvalidated = kaggle.lyft.check_if_validated()\nwhile True:\n if validated:\n break\n else:\n print(memegenerator.rand_meme(theme=\"waiting\"))\n validated = kaggle.lyft.check_if_validated()\n

\n\n

\"Still

", "rawMarkdown": "```\nimport kaggle\nimport memegenerator\nvalidated = kaggle.lyft.check_if_validated()\nwhile True:\n if validated:\n break\n else:\n print(memegenerator.rand_meme(theme=\"waiting\"))\n validated = kaggle.lyft.check_if_validated()\n```\n\n![Still Waiting](https://i.imgflip.com/185u4c.jpg)", "votes": 9, "replies": [ { "id": 679358, "postDate": "2019-11-22T17:23:21.030Z", "content": "

Best one so far!!! :)

", "rawMarkdown": "Best one so far!!! :)", "votes": 2 }, { "id": 679360, "postDate": "2019-11-22T17:26:04.373Z", "content": "

Haha! Thanks!

", "rawMarkdown": "Haha! Thanks!", "votes": 2 }, { "id": 679377, "postDate": "2019-11-22T17:54:11.227Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 672509, "postDate": "2019-11-14T00:36:32.500Z", "content": "

It might take several days. They have to contact every winner team and ask for their source codes. They have to make sure that both code and data do not break any rules. Teams and organizers might be in different time zones, so emails might be replied on the next day. If some team is removed in this process, they have to contact more teams, and so on.

\n\n

It might be faster in case of a kernel-only competition because source codes are available immediately.

", "rawMarkdown": "It might take several days. They have to contact every winner team and ask for their source codes. They have to make sure that both code and data do not break any rules. Teams and organizers might be in different time zones, so emails might be replied on the next day. If some team is removed in this process, they have to contact more teams, and so on.\n\nIt might be faster in case of a kernel-only competition because source codes are available immediately.", "votes": 7 }, { "id": 678018, "postDate": "2019-11-20T22:22:56.153Z", "content": "

\"\"\nThis is so tiring. Is the Kaggle team listening?

", "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1918280%2F1f0b26fb46385c3119cfaccfdcf6fdb6%2Fagent_skeleton_still_waiting_meme1.jpg?generation=1574288550663208&alt=media)\nThis is so tiring. Is the Kaggle team listening?", "votes": 7 }, { "id": 672351, "postDate": "2019-11-13T19:47:10.860Z", "content": "

Hope they don't wait when Clouds competition is finished 😂

", "rawMarkdown": "Hope they don't wait when Clouds competition is finished 😂", "votes": 6, "replies": [ { "id": 677242, "postDate": "2019-11-20T00:18:46.320Z", "content": "

Now the Cloud competitions is done and verified :)

", "rawMarkdown": "Now the Cloud competitions is done and verified :)", "votes": 4 } ] }, { "id": 674356, "postDate": "2019-11-16T10:24:52.813Z", "content": "

It's been 3 days now :P and still waiting for results to be finalized XD

", "rawMarkdown": "It's been 3 days now :P and still waiting for results to be finalized XD", "votes": 3, "replies": [ { "id": 674357, "postDate": "2019-11-16T10:26:31.043Z", "content": "

yeah,too much time

", "rawMarkdown": "yeah,too much time", "votes": 3 } ] }, { "id": 672314, "postDate": "2019-11-13T18:49:30.243Z", "content": "

At the start of the competition there were some problems with the leaderboard metric, I would guess they want to be extra careful in making sure the final results are correct.

", "rawMarkdown": "At the start of the competition there were some problems with the leaderboard metric, I would guess they want to be extra careful in making sure the final results are correct.", "votes": 3 }, { "id": 679380, "postDate": "2019-11-22T17:57:37.230Z", "content": "

\"Cat

", "rawMarkdown": "![Cat Waiting](https://www.memecreator.org/static/images/memes/5045664.jpg)", "votes": 4 }, { "id": 672317, "postDate": "2019-11-13T18:53:45.527Z", "content": "

You could be correct that RSNA is holding things up. It will be difficult to prove a score of 0.000 < 0.000 < 0.000 ... 😛

", "rawMarkdown": "You could be correct that RSNA is holding things up. It will be difficult to prove a score of 0.000 < 0.000 < 0.000 ... 😛 ", "votes": 4 } ], "comments": [ { "id": 677462, "author_name": "Marek Wyborski", "author_url": "", "post_date": "2019-11-20T07:41:16.307000", "content": "

A week has passed and not even a word from the host or kaggle ☚ī¸ At least a \"Thank you for your work, but we face problems and it will take about X days longer to verify the results.\" they could have given us IMO. We also invested a lot of time and just a small update message would make me feel our work is appreciated, but just leaving the competition unverified and uncommented gives me the impression, that they don't care. Even if this isn't the case. It's just about a minimum level of communication.

\n\n

Edit:\n@philculliton @iglovikov is it possible to please give us a small update with a rough estimation on when the competition will be finalized? I know you guys are busy, but please understand that this is a weird situation for us, checking every day the competition and seeing other competitions finalizing. Will the metric be recalculated due to the \"small classes problem\" (see thread)? Or have you contacted the winners and will publish the final Leaderboard during NeurIPS? Please talk to us.

", "votes": 14, "replies": [ { "id": 677468, "author_name": "Deepshad", "author_url": "", "post_date": "2019-11-20T07:54:46.840000", "content": "

Very well said. Waiting for an update from kaggle team 😟

", "votes": 4, "replies": [] }, { "id": 677488, "author_name": "Mobassir", "author_url": "", "post_date": "2019-11-20T08:24:01.487000", "content": "

you are right @marekwyborski \nand it hurts surely :(

", "votes": 4, "replies": [] }, { "id": 677643, "author_name": "Manoj Prabhakar", "author_url": "", "post_date": "2019-11-20T12:45:38.177000", "content": "

Well said @marekwyborski . Almost 8 days since the competition got over, still no signs of when the leaderboard will be verified. RSNA, Clouds got completed in last 8 days and we got the LB verified almost on the same day.

", "votes": 4, "replies": [] }, { "id": 677827, "author_name": "Rishabh Agrahari", "author_url": "", "post_date": "2019-11-20T16:23:51.383000", "content": "

I tweeted to Vladimir regarding this issue, here's his reply : https://twitter.com/viglovikov/status/1197179150315872257?s=20

\n\n

(Sorry for formatting, posting from mobile rn)

", "votes": 5, "replies": [] }, { "id": 678629, "author_name": "Manoj Prabhakar", "author_url": "", "post_date": "2019-11-21T17:00:30.737000", "content": "

Thanks a lot for taking this initiative up @rishabhiitbhu . Hopefully Kaggle team will verify it before next set of competition gets over like NFL, Ashrae.

", "votes": 4, "replies": [] }, { "id": 679359, "author_name": "Carlo", "author_url": "", "post_date": "2019-11-22T17:25:31.620000", "content": "

Agree, you really hit the nail on the head!

", "votes": 3, "replies": [] } ] }, { "id": 679357, "author_name": "Carlo", "author_url": "", "post_date": "2019-11-22T17:21:48.407000", "content": "

\nimport kaggle\nimport memegenerator\nvalidated = kaggle.lyft.check_if_validated()\nwhile True:\n if validated:\n break\n else:\n print(memegenerator.rand_meme(theme=\"waiting\"))\n validated = kaggle.lyft.check_if_validated()\n

\n\n

\"Still

", "votes": 9, "replies": [ { "id": 679358, "author_name": "Mobassir", "author_url": "", "post_date": "2019-11-22T17:23:21.030000", "content": "

Best one so far!!! :)

", "votes": 2, "replies": [] }, { "id": 679360, "author_name": "Carlo", "author_url": "", "post_date": "2019-11-22T17:26:04.373000", "content": "

Haha! Thanks!

", "votes": 2, "replies": [] }, { "id": 679377, "author_name": "", "author_url": "", "post_date": "2019-11-22T17:54:11.227000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 672509, "author_name": "Artyom Palvelev", "author_url": "", "post_date": "2019-11-14T00:36:32.500000", "content": "

It might take several days. They have to contact every winner team and ask for their source codes. They have to make sure that both code and data do not break any rules. Teams and organizers might be in different time zones, so emails might be replied on the next day. If some team is removed in this process, they have to contact more teams, and so on.

\n\n

It might be faster in case of a kernel-only competition because source codes are available immediately.

", "votes": 7, "replies": [] }, { "id": 678018, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-11-20T22:22:56.153000", "content": "

\"\"\nThis is so tiring. Is the Kaggle team listening?

", "votes": 7, "replies": [] }, { "id": 672351, "author_name": "Mukharbek Organokov", "author_url": "", "post_date": "2019-11-13T19:47:10.860000", "content": "

Hope they don't wait when Clouds competition is finished 😂

", "votes": 6, "replies": [ { "id": 677242, "author_name": "student", "author_url": "", "post_date": "2019-11-20T00:18:46.320000", "content": "

Now the Cloud competitions is done and verified :)

", "votes": 4, "replies": [] } ] }, { "id": 674356, "author_name": "Ram Ramrakhya", "author_url": "", "post_date": "2019-11-16T10:24:52.813000", "content": "

It's been 3 days now :P and still waiting for results to be finalized XD

", "votes": 3, "replies": [ { "id": 674357, "author_name": "Mobassir", "author_url": "", "post_date": "2019-11-16T10:26:31.043000", "content": "

yeah,too much time

", "votes": 3, "replies": [] } ] }, { "id": 672314, "author_name": "Jack Vial", "author_url": "", "post_date": "2019-11-13T18:49:30.243000", "content": "

At the start of the competition there were some problems with the leaderboard metric, I would guess they want to be extra careful in making sure the final results are correct.

", "votes": 3, "replies": [] }, { "id": 679380, "author_name": "Carlo", "author_url": "", "post_date": "2019-11-22T17:57:37.230000", "content": "

\"Cat

", "votes": 4, "replies": [] }, { "id": 672317, "author_name": "Jack Vial", "author_url": "", "post_date": "2019-11-13T18:53:45.527000", "content": "

You could be correct that RSNA is holding things up. It will be difficult to prove a score of 0.000 < 0.000 < 0.000 ... 😛

", "votes": 4, "replies": [] } ], "raw_markdown_by_id": { "672296": "In my previous competitions within 6-7 hours of exposing private leaderboard organizers verified final result.but in this competition 19+ hours have been passed but the final result verification is not done yet.just curious to know,why?\nAre they waiting for rsna to finish or something? Thanks ", "677462": "A week has passed and not even a word from the host or kaggle ☚ī¸ At least a \"Thank you for your work, but we face problems and it will take about X days longer to verify the results.\" they could have given us IMO. We also invested a lot of time and just a small update message would make me feel our work is appreciated, but just leaving the competition unverified and uncommented gives me the impression, that they don't care. Even if this isn't the case. It's just about a minimum level of communication.\n\nEdit:\n@philculliton @iglovikov is it possible to please give us a small update with a rough estimation on when the competition will be finalized? I know you guys are busy, but please understand that this is a weird situation for us, checking every day the competition and seeing other competitions finalizing. Will the metric be recalculated due to the \"small classes problem\" (see thread)? Or have you contacted the winners and will publish the final Leaderboard during NeurIPS? Please talk to us.", "679357": "```\nimport kaggle\nimport memegenerator\nvalidated = kaggle.lyft.check_if_validated()\nwhile True:\n if validated:\n break\n else:\n print(memegenerator.rand_meme(theme=\"waiting\"))\n validated = kaggle.lyft.check_if_validated()\n```\n\n![Still Waiting](https://i.imgflip.com/185u4c.jpg)", "672509": "It might take several days. They have to contact every winner team and ask for their source codes. They have to make sure that both code and data do not break any rules. Teams and organizers might be in different time zones, so emails might be replied on the next day. If some team is removed in this process, they have to contact more teams, and so on.\n\nIt might be faster in case of a kernel-only competition because source codes are available immediately.", "678018": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1918280%2F1f0b26fb46385c3119cfaccfdcf6fdb6%2Fagent_skeleton_still_waiting_meme1.jpg?generation=1574288550663208&alt=media)\nThis is so tiring. Is the Kaggle team listening?", "672351": "Hope they don't wait when Clouds competition is finished 😂", "674356": "It's been 3 days now :P and still waiting for results to be finalized XD", "672314": "At the start of the competition there were some problems with the leaderboard metric, I would guess they want to be extra careful in making sure the final results are correct.", "679380": "![Cat Waiting](https://www.memecreator.org/static/images/memes/5045664.jpg)", "672317": "You could be correct that RSNA is holding things up. It will be difficult to prove a score of 0.000 < 0.000 < 0.000 ... 😛 " } }