{ "id": 5313, "title": "validation data", "url": "/competitions/multi-modal-gesture-recognition/discussion/5313", "author_name": "", "post_date": "2013-08-06T09:46:11.107Z", "votes": null, "comment_count": 7, "views": 1712, "content": "

do we have to submit results on validation data right now?

", "messages": [ { "id": "28266", "postDate": "08/06/2013 09:46:11", "content": "

do we have to submit results on validation data right now?

", "rawMarkdown": "", "votes": null }, { "id": "28268", "postDate": "08/06/2013 10:11:41", "content": "

Dear Abhishek,

\n

you can submit results on validation data until the publication of the test decryption key, when the final evaluation labels will be uploaded to the Kaggle system in order to start the final evaluation of all team models.

\n

We will put a message when we change the data on Kaggle.

\n

 

\n

Xavier

", "rawMarkdown": "", "votes": null }, { "id": "28309", "postDate": "08/06/2013 18:31:24", "content": "

Hi,

\n

     I don't get it, don't we already have the labels on the validation data. I have downloaded the new validation datasets which seem to have the labels in the .mat files. Are these labels incorrect and should we ignore them? Thanks.

", "rawMarkdown": "", "votes": null }, { "id": "28334", "postDate": "08/07/2013 00:56:34", "content": "

Dear xbaro,

\n

 

\n

I have a question about the leaderboard scoring system. At the moment, the score is based on the groundtruth labels of validation dataset. Will this change after the publication of the decryption key of the final evaluation dataset (Aug 10th)? Or will it remain the same until the end of this competition (Aug 20th), and the score on the final evaluation dataset will be announced after that?

\n

 

\n

Jiaxiang Wu

", "rawMarkdown": "", "votes": null }, { "id": "28349", "postDate": "08/07/2013 09:21:16", "content": "

Dear iamkhader,

\n

we provided the labels to avoid downloading again the validation data. If you have downloaded the labeled data, you have the correct data with the correct labels. The script allows to add the labels to the unlabeled validation data.

\n

Thanks

\n

Xavier

", "rawMarkdown": "", "votes": null }, { "id": "28350", "postDate": "08/07/2013 09:34:30", "content": "

Dear Jiaxiang,

\n

we will publish an update on the timeline clarifying this information. The last modification we did, which is now visible at the timeline, says that the 10th of August we publish the decryption key and you have until the 20th of August to submit everything. Nevertheless, it is possible that we need to advance everything. We are preparing the information and analyzing all the possibilities in order to give you as much as time we can to produce the predictions on test data, but following as much as we can the external restrictions. I will publish this new timeline this evening, and it is possible that we publish the decryption key today.

\n

We apologies for this situation.

\n

Xavier

", "rawMarkdown": "", "votes": null }, { "id": "28358", "postDate": "08/07/2013 12:35:09", "content": "

Thanks Xavier,

\n

                       So for now we can submit results on the validation data until the key is released? Thanks.

", "rawMarkdown": "", "votes": null }, { "id": "28360", "postDate": "08/07/2013 12:45:18", "content": "

Yes, in fact, probably you will be able to submit results on validation data until the deadline for submission of your predictions in test data. Take into account that if you use the validation labels in the training stage, your results will be no representative of the ones you will get on the final predictions.

\n

Xavier

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 28268, "author_name": "xbaro100272", "author_url": "", "post_date": "08/06/2013 10:11:41", "content": "

Dear Abhishek,

\n

you can submit results on validation data until the publication of the test decryption key, when the final evaluation labels will be uploaded to the Kaggle system in order to start the final evaluation of all team models.

\n

We will put a message when we change the data on Kaggle.

\n

 

\n

Xavier

", "votes": null, "replies": [] }, { "id": 28309, "author_name": "iamkhader", "author_url": "", "post_date": "08/06/2013 18:31:24", "content": "

Hi,

\n

     I don't get it, don't we already have the labels on the validation data. I have downloaded the new validation datasets which seem to have the labels in the .mat files. Are these labels incorrect and should we ignore them? Thanks.

", "votes": null, "replies": [] }, { "id": 28334, "author_name": "jiaxiangwu", "author_url": "", "post_date": "08/07/2013 00:56:34", "content": "

Dear xbaro,

\n

 

\n

I have a question about the leaderboard scoring system. At the moment, the score is based on the groundtruth labels of validation dataset. Will this change after the publication of the decryption key of the final evaluation dataset (Aug 10th)? Or will it remain the same until the end of this competition (Aug 20th), and the score on the final evaluation dataset will be announced after that?

\n

 

\n

Jiaxiang Wu

", "votes": null, "replies": [] }, { "id": 28349, "author_name": "xbaro100272", "author_url": "", "post_date": "08/07/2013 09:21:16", "content": "

Dear iamkhader,

\n

we provided the labels to avoid downloading again the validation data. If you have downloaded the labeled data, you have the correct data with the correct labels. The script allows to add the labels to the unlabeled validation data.

\n

Thanks

\n

Xavier

", "votes": null, "replies": [] }, { "id": 28350, "author_name": "xbaro100272", "author_url": "", "post_date": "08/07/2013 09:34:30", "content": "

Dear Jiaxiang,

\n

we will publish an update on the timeline clarifying this information. The last modification we did, which is now visible at the timeline, says that the 10th of August we publish the decryption key and you have until the 20th of August to submit everything. Nevertheless, it is possible that we need to advance everything. We are preparing the information and analyzing all the possibilities in order to give you as much as time we can to produce the predictions on test data, but following as much as we can the external restrictions. I will publish this new timeline this evening, and it is possible that we publish the decryption key today.

\n

We apologies for this situation.

\n

Xavier

", "votes": null, "replies": [] }, { "id": 28358, "author_name": "iamkhader", "author_url": "", "post_date": "08/07/2013 12:35:09", "content": "

Thanks Xavier,

\n

                       So for now we can submit results on the validation data until the key is released? Thanks.

", "votes": null, "replies": [] }, { "id": 28360, "author_name": "xbaro100272", "author_url": "", "post_date": "08/07/2013 12:45:18", "content": "

Yes, in fact, probably you will be able to submit results on validation data until the deadline for submission of your predictions in test data. Take into account that if you use the validation labels in the training stage, your results will be no representative of the ones you will get on the final predictions.

\n

Xavier

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "28266": "", "28268": "", "28309": "", "28334": "", "28349": "", "28350": "", "28358": "", "28360": "" }, "source": "meta" }