{ "id": 58329, "title": "We will provide manual evaluation for anyone scores > 0.15", "url": "/competitions/cvpr-2018-autonomous-driving/discussion/58329", "author_name": "", "post_date": "2018-06-06T06:54:20.097477600Z", "votes": null, "comment_count": 19, "views": 0, "content": "

Hello All:

\n\n

Thank you for your enthusiasm for the WAD competition. I am the developer of this competition evaluation project, Charles Shao

\n\n

We have noticed that many people have reported mistakes. Some are due to limitations of the Kaggle framework, such as the 20 minute limit. Some are data submission issues, etc..

\n\n

For this reason, we have an open source C# evaluation code that we hope will help users better assess locally. However, we have found that many users do not use C# code.

\n\n

Here, I suggest that you limit the submitted data to less than 100MB as much as possible, otherwise there is a high probability of timeouts or other stability errors.

\n\n

In addition, considering that competition is coming to the end, we decided to provide additional manual evaluation services. It scores more than 0.15 for all users. You can email me by email and I will return results in up to 16 hours. You can submit 3 data sets at once. Due to the size of the submission, I recommend that you compress and provide links to cloud downloads instead of using email attachments.

\n\n

note:

\n\n

email to : shaochenye@baidu.com , apollo-scape@baidu.com\nMail Title: WAD Evaluation -- [your kaggel username]--[version]

\n\n

Thank you

\n\n

Charels shao\nProduct Architect\nBaidu

", "messages": [ { "id": "339020", "postDate": "06/06/2018 06:54:20", "content": "

Hello All:

\n\n

Thank you for your enthusiasm for the WAD competition. I am the developer of this competition evaluation project, Charles Shao

\n\n

We have noticed that many people have reported mistakes. Some are due to limitations of the Kaggle framework, such as the 20 minute limit. Some are data submission issues, etc..

\n\n

For this reason, we have an open source C# evaluation code that we hope will help users better assess locally. However, we have found that many users do not use C# code.

\n\n

Here, I suggest that you limit the submitted data to less than 100MB as much as possible, otherwise there is a high probability of timeouts or other stability errors.

\n\n

In addition, considering that competition is coming to the end, we decided to provide additional manual evaluation services. It scores more than 0.15 for all users. You can email me by email and I will return results in up to 16 hours. You can submit 3 data sets at once. Due to the size of the submission, I recommend that you compress and provide links to cloud downloads instead of using email attachments.

\n\n

note:

\n\n

email to : shaochenye@baidu.com , apollo-scape@baidu.com\nMail Title: WAD Evaluation -- [your kaggel username]--[version]

\n\n

Thank you

\n\n

Charels shao\nProduct Architect\nBaidu

", "rawMarkdown": "Hello All:\n\nThank you for your enthusiasm for the WAD competition. I am the developer of this competition evaluation project, Charles Shao\n\nWe have noticed that many people have reported mistakes. Some are due to limitations of the Kaggle framework, such as the 20 minute limit. Some are data submission issues, etc..\n\nFor this reason, we have an open source C# evaluation code that we hope will help users better assess locally. However, we have found that many users do not use C# code.\n\nHere, I suggest that you limit the submitted data to less than 100MB as much as possible, otherwise there is a high probability of timeouts or other stability errors.\n\nIn addition, considering that competition is coming to the end, we decided to provide additional manual evaluation services. It scores more than 0.15 for all users. You can email me by email and I will return results in up to 16 hours. You can submit 3 data sets at once. Due to the size of the submission, I recommend that you compress and provide links to cloud downloads instead of using email attachments.\n\nnote:\n\nemail to : shaochenye@baidu.com , apollo-scape@baidu.com\nMail Title: WAD Evaluation -- [your kaggel username]--[version]\n\nThank you\n\nCharels shao\nProduct Architect\nBaidu", "votes": null }, { "id": "339022", "postDate": "06/06/2018 06:59:54", "content": "

Important note:\n1. The result of the competition is still based on the final kaggel online result.\n2. Our manual evaluation service only helps users to temporarily skip those bugs and accelerate everyone's algorithmic iteration.

", "rawMarkdown": "Important note:\n1. The result of the competition is still based on the final kaggel online result.\n2. Our manual evaluation service only helps users to temporarily skip those bugs and accelerate everyone's algorithmic iteration.", "votes": null }, { "id": "339023", "postDate": "06/06/2018 07:00:41", "content": "

贵公司还招人不，很想加入

", "rawMarkdown": "贵公司还招人不，很想加入", "votes": null }, { "id": "339033", "postDate": "06/06/2018 07:11:38", "content": "

Yes, Baidu's algorithm department is hiring. Please tell me your team name. I will see your achievements.

", "rawMarkdown": "Yes, Baidu's algorithm department is hiring. Please tell me your team name. I will see your achievements.", "votes": null }, { "id": "339040", "postDate": "06/06/2018 07:24:02", "content": "

I found a very strange issue: the file size cannot larger than 43M, otherwise cause timeouts and I had to delete some results. How to handle this problem? Thanks

", "rawMarkdown": "I found a very strange issue: the file size cannot larger than 43M, otherwise cause timeouts and I had to delete some results. How to handle this problem? Thanks", "votes": null }, { "id": "339045", "postDate": "06/06/2018 07:35:37", "content": "

It's relative with your proposals, not size.

", "rawMarkdown": "It's relative with your proposals, not size.", "votes": null }, { "id": "339056", "postDate": "06/06/2018 07:58:56", "content": "

maybe, but how did you know that?

", "rawMarkdown": "maybe, but how did you know that?", "votes": null }, { "id": "339059", "postDate": "06/06/2018 08:05:45", "content": "

because the error info~

", "rawMarkdown": "because the error info~", "votes": null }, { "id": "339062", "postDate": "06/06/2018 08:11:01", "content": "

I only got timeout info

", "rawMarkdown": "I only got timeout info", "votes": null }, { "id": "339063", "postDate": "06/06/2018 08:13:40", "content": "

I also got.

", "rawMarkdown": "I also got.", "votes": null }, { "id": "339070", "postDate": "06/06/2018 08:38:43", "content": "", "rawMarkdown": "", "votes": null }, { "id": "339071", "postDate": "06/06/2018 08:39:00", "content": "

the GodEye. Now I am 30th now.I am busy in searching for job this period.

", "rawMarkdown": "the GodEye. Now I am 30th now.I am busy in searching for job this period.", "votes": null }, { "id": "339434", "postDate": "06/07/2018 00:21:39", "content": "

So this service is only available for people who scored more than 0.15 on the public leaderboard? (The title is clear, however in the text the limitation is not mentioned)

", "rawMarkdown": "So this service is only available for people who scored more than 0.15 on the public leaderboard? (The title is clear, however in the text the limitation is not mentioned)", "votes": null }, { "id": "339468", "postDate": "06/07/2018 02:15:39", "content": "

If we were to provide you with data to test, does this mean that it won't be limited by the 20 minutes timeout ? Will this data be available on the leaderboard?

", "rawMarkdown": "If we were to provide you with data to test, does this mean that it won't be limited by the 20 minutes timeout ? Will this data be available on the leaderboard?", "votes": null }, { "id": "339723", "postDate": "06/07/2018 14:14:06", "content": "

100MB is my experience when debugging the system. When the CSV file is less than 100MB, your timeout probability is very small. But if the file is larger than 120MB, the probability of timeout will be great.

", "rawMarkdown": "100MB is my experience when debugging the system. When the CSV file is less than 100MB, your timeout probability is very small. But if the file is larger than 120MB, the probability of timeout will be great.", "votes": null }, { "id": "339724", "postDate": "06/07/2018 14:15:37", "content": "

you can sent you resume to apollo-scape@baidu.com

", "rawMarkdown": "you can sent you resume to apollo-scape@baidu.com", "votes": null }, { "id": "339725", "postDate": "06/07/2018 14:16:18", "content": "

sorry. what do you mean?

", "rawMarkdown": "sorry. what do you mean?", "votes": null }, { "id": "339728", "postDate": "06/07/2018 14:19:27", "content": "

manual evaluation has 60 minutes limit. The result of the competition is still based on the final kaggel online result. It means your final submission online for competition is still in 20 minutes limited

", "rawMarkdown": "manual evaluation has 60 minutes limit. The result of the competition is still based on the final kaggel online result. It means your final submission online for competition is still in 20 minutes limited", "votes": null }, { "id": "339810", "postDate": "06/07/2018 17:58:42", "content": "

Anyway, I improved my score to above 0.15. I guess you will be hearing from me :)

\n\n

It this service also available over the weekend?

", "rawMarkdown": "Anyway, I improved my score to above 0.15. I guess you will be hearing from me :)\n\nIt this service also available over the weekend?", "votes": null }, { "id": "339820", "postDate": "06/07/2018 18:14:14", "content": "

I can confirm that for me the boundary seems to be around 42MB as well (that still passes with 1150 seconds). I guess the only way is to fit better quality predictions into those 42 MB :)

", "rawMarkdown": "I can confirm that for me the boundary seems to be around 42MB as well (that still passes with 1150 seconds). I guess the only way is to fit better quality predictions into those 42 MB :)", "votes": null } ], "comments": [ { "id": 339022, "author_name": "shaochenye", "author_url": "", "post_date": "06/06/2018 06:59:54", "content": "

Important note:\n1. The result of the competition is still based on the final kaggel online result.\n2. Our manual evaluation service only helps users to temporarily skip those bugs and accelerate everyone's algorithmic iteration.

", "votes": null, "replies": [ { "id": 339040, "author_name": "the16throute", "author_url": "", "post_date": "06/06/2018 07:24:02", "content": "

I found a very strange issue: the file size cannot larger than 43M, otherwise cause timeouts and I had to delete some results. How to handle this problem? Thanks

", "votes": null, "replies": [] }, { "id": 339045, "author_name": "zhuangyq", "author_url": "", "post_date": "06/06/2018 07:35:37", "content": "

It's relative with your proposals, not size.

", "votes": null, "replies": [] }, { "id": 339056, "author_name": "the16throute", "author_url": "", "post_date": "06/06/2018 07:58:56", "content": "

maybe, but how did you know that?

", "votes": null, "replies": [] }, { "id": 339059, "author_name": "zhuangyq", "author_url": "", "post_date": "06/06/2018 08:05:45", "content": "

because the error info~

", "votes": null, "replies": [] }, { "id": 339062, "author_name": "the16throute", "author_url": "", "post_date": "06/06/2018 08:11:01", "content": "

I only got timeout info

", "votes": null, "replies": [] }, { "id": 339063, "author_name": "zhuangyq", "author_url": "", "post_date": "06/06/2018 08:13:40", "content": "

I also got.

", "votes": null, "replies": [] }, { "id": 339723, "author_name": "shaochenye", "author_url": "", "post_date": "06/07/2018 14:14:06", "content": "

100MB is my experience when debugging the system. When the CSV file is less than 100MB, your timeout probability is very small. But if the file is larger than 120MB, the probability of timeout will be great.

", "votes": null, "replies": [] }, { "id": 339820, "author_name": "michaelheinzer", "author_url": "", "post_date": "06/07/2018 18:14:14", "content": "

I can confirm that for me the boundary seems to be around 42MB as well (that still passes with 1150 seconds). I guess the only way is to fit better quality predictions into those 42 MB :)

", "votes": null, "replies": [] } ] }, { "id": 339023, "author_name": "andeyeluguo", "author_url": "", "post_date": "06/06/2018 07:00:41", "content": "

贵公司还招人不，很想加入

", "votes": null, "replies": [ { "id": 339033, "author_name": "shaochenye", "author_url": "", "post_date": "06/06/2018 07:11:38", "content": "

Yes, Baidu's algorithm department is hiring. Please tell me your team name. I will see your achievements.

", "votes": null, "replies": [] }, { "id": 339070, "author_name": "andeyeluguo", "author_url": "", "post_date": "06/06/2018 08:38:43", "content": "", "votes": null, "replies": [] }, { "id": 339071, "author_name": "andeyeluguo", "author_url": "", "post_date": "06/06/2018 08:39:00", "content": "

the GodEye. Now I am 30th now.I am busy in searching for job this period.

", "votes": null, "replies": [] }, { "id": 339724, "author_name": "shaochenye", "author_url": "", "post_date": "06/07/2018 14:15:37", "content": "

you can sent you resume to apollo-scape@baidu.com

", "votes": null, "replies": [] } ] }, { "id": 339434, "author_name": "michaelheinzer", "author_url": "", "post_date": "06/07/2018 00:21:39", "content": "

So this service is only available for people who scored more than 0.15 on the public leaderboard? (The title is clear, however in the text the limitation is not mentioned)

", "votes": null, "replies": [ { "id": 339725, "author_name": "shaochenye", "author_url": "", "post_date": "06/07/2018 14:16:18", "content": "

sorry. what do you mean?

", "votes": null, "replies": [] }, { "id": 339810, "author_name": "michaelheinzer", "author_url": "", "post_date": "06/07/2018 17:58:42", "content": "

Anyway, I improved my score to above 0.15. I guess you will be hearing from me :)

\n\n

It this service also available over the weekend?

", "votes": null, "replies": [] } ] }, { "id": 339468, "author_name": "nvadlr", "author_url": "", "post_date": "06/07/2018 02:15:39", "content": "

If we were to provide you with data to test, does this mean that it won't be limited by the 20 minutes timeout ? Will this data be available on the leaderboard?

", "votes": null, "replies": [ { "id": 339728, "author_name": "shaochenye", "author_url": "", "post_date": "06/07/2018 14:19:27", "content": "

manual evaluation has 60 minutes limit. The result of the competition is still based on the final kaggel online result. It means your final submission online for competition is still in 20 minutes limited

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "339020": "Hello All:\n\nThank you for your enthusiasm for the WAD competition. I am the developer of this competition evaluation project, Charles Shao\n\nWe have noticed that many people have reported mistakes. Some are due to limitations of the Kaggle framework, such as the 20 minute limit. Some are data submission issues, etc..\n\nFor this reason, we have an open source C# evaluation code that we hope will help users better assess locally. However, we have found that many users do not use C# code.\n\nHere, I suggest that you limit the submitted data to less than 100MB as much as possible, otherwise there is a high probability of timeouts or other stability errors.\n\nIn addition, considering that competition is coming to the end, we decided to provide additional manual evaluation services. It scores more than 0.15 for all users. You can email me by email and I will return results in up to 16 hours. You can submit 3 data sets at once. Due to the size of the submission, I recommend that you compress and provide links to cloud downloads instead of using email attachments.\n\nnote:\n\nemail to : shaochenye@baidu.com , apollo-scape@baidu.com\nMail Title: WAD Evaluation -- [your kaggel username]--[version]\n\nThank you\n\nCharels shao\nProduct Architect\nBaidu", "339022": "Important note:\n1. The result of the competition is still based on the final kaggel online result.\n2. Our manual evaluation service only helps users to temporarily skip those bugs and accelerate everyone's algorithmic iteration.", "339023": "贵公司还招人不，很想加入", "339033": "Yes, Baidu's algorithm department is hiring. Please tell me your team name. I will see your achievements.", "339040": "I found a very strange issue: the file size cannot larger than 43M, otherwise cause timeouts and I had to delete some results. How to handle this problem? Thanks", "339045": "It's relative with your proposals, not size.", "339056": "maybe, but how did you know that?", "339059": "because the error info~", "339062": "I only got timeout info", "339063": "I also got.", "339070": "", "339071": "the GodEye. Now I am 30th now.I am busy in searching for job this period.", "339434": "So this service is only available for people who scored more than 0.15 on the public leaderboard? (The title is clear, however in the text the limitation is not mentioned)", "339468": "If we were to provide you with data to test, does this mean that it won't be limited by the 20 minutes timeout ? Will this data be available on the leaderboard?", "339723": "100MB is my experience when debugging the system. When the CSV file is less than 100MB, your timeout probability is very small. But if the file is larger than 120MB, the probability of timeout will be great.", "339724": "you can sent you resume to apollo-scape@baidu.com", "339725": "sorry. what do you mean?", "339728": "manual evaluation has 60 minutes limit. The result of the competition is still based on the final kaggel online result. It means your final submission online for competition is still in 20 minutes limited", "339810": "Anyway, I improved my score to above 0.15. I guess you will be hearing from me :)\n\nIt this service also available over the weekend?", "339820": "I can confirm that for me the boundary seems to be around 42MB as well (that still passes with 1150 seconds). I guess the only way is to fit better quality predictions into those 42 MB :)" }, "source": "meta" }