Hi, all. I have encountered a scoring error problem. Could you please help me? After I committed my notebook (it's implemented with pytorch huggingface transformers, it could run successful and output a submission.csv in /kaggle/working/submission.csv). I put the submission.csv here. I am new to this competition and not sure whether the format of submission.csv was wrong or The Gpu was used over 3hours (When I tested on the public dataset, it would use about 30 minues) or other important errors. Thank you very much!
", "messages": [ { "id": "705149", "postDate": "12/28/2019 14:23:45", "content": "Hi, all. I have encountered a scoring error problem. Could you please help me? After I committed my notebook (it's implemented with pytorch huggingface transformers, it could run successful and output a submission.csv in /kaggle/working/submission.csv). I put the submission.csv here. I am new to this competition and not sure whether the format of submission.csv was wrong or The Gpu was used over 3hours (When I tested on the public dataset, it would use about 30 minues) or other important errors. Thank you very much!
", "rawMarkdown": "Hi, all. I have encountered a scoring error problem. Could you please help me? After I committed my notebook (it's implemented with pytorch huggingface transformers, it could run successful and output a submission.csv in /kaggle/working/submission.csv). I put the submission.csv [here](https://drive.google.com/file/d/1UpkcAFEdlwskp5mBAaZ4MB-S9GOlY_uq/view?usp=sharing). I am new to this competition and not sure whether the format of submission.csv was wrong or The Gpu was used over 3hours (When I tested on the public dataset, it would use about 30 minues) or other important errors. Thank you very much!", "votes": null }, { "id": "705624", "postDate": "12/29/2019 07:11:59", "content": "@pingnie One of possibility is running time problem as you mentioned. The amount of private dataset is almost 9 times of public dataset. If it takes 30 minutes to predict answers for public dataset, it goes to around 5 hours to run on private dataset.
", "rawMarkdown": "pingnie One of possibility is running time problem as you mentioned. The amount of private dataset is almost 9 times of public dataset. If it takes 30 minutes to predict answers for public dataset, it goes to around 5 hours to run on private dataset.", "votes": null }, { "id": "706746", "postDate": "12/30/2019 19:08:11", "content": "@pingnie Other than the run time, you may want to verify that your code is able to evaluate on a private test set and, for example, is not hard-coding the submission.csv from the public test set.
@kentaronakanishi Thank you very much. It is the problem you mentioned!
", "rawMarkdown": "kentaronakanishi Thank you very much. It is the problem you mentioned!", "votes": null }, { "id": "706947", "postDate": "12/31/2019 03:54:07", "content": "@juliaelliott Thank you very much. I have also checked it. It's helpful! Thank you!
", "rawMarkdown": "juliaelliott Thank you very much. I have also checked it. It's helpful! Thank you!", "votes": null } ], "comments": [ { "id": 705624, "author_name": "kentaronakanishi", "author_url": "", "post_date": "12/29/2019 07:11:59", "content": "@pingnie One of possibility is running time problem as you mentioned. The amount of private dataset is almost 9 times of public dataset. If it takes 30 minutes to predict answers for public dataset, it goes to around 5 hours to run on private dataset.
", "votes": null, "replies": [] }, { "id": 706746, "author_name": "juliaelliott", "author_url": "", "post_date": "12/30/2019 19:08:11", "content": "@pingnie Other than the run time, you may want to verify that your code is able to evaluate on a private test set and, for example, is not hard-coding the submission.csv from the public test set.
@kentaronakanishi Thank you very much. It is the problem you mentioned!
", "votes": null, "replies": [] }, { "id": 706947, "author_name": "pingnie", "author_url": "", "post_date": "12/31/2019 03:54:07", "content": "@juliaelliott Thank you very much. I have also checked it. It's helpful! Thank you!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "705149": "Hi, all. I have encountered a scoring error problem. Could you please help me? After I committed my notebook (it's implemented with pytorch huggingface transformers, it could run successful and output a submission.csv in /kaggle/working/submission.csv). I put the submission.csv [here](https://drive.google.com/file/d/1UpkcAFEdlwskp5mBAaZ4MB-S9GOlY_uq/view?usp=sharing). I am new to this competition and not sure whether the format of submission.csv was wrong or The Gpu was used over 3hours (When I tested on the public dataset, it would use about 30 minues) or other important errors. Thank you very much!", "705624": "pingnie One of possibility is running time problem as you mentioned. The amount of private dataset is almost 9 times of public dataset. If it takes 30 minutes to predict answers for public dataset, it goes to around 5 hours to run on private dataset.", "706746": "pingnie Other than the run time, you may want to verify that your code is able to evaluate on a private test set and, for example, is not hard-coding the `submission.csv` from the public test set.", "706946": "kentaronakanishi Thank you very much. It is the problem you mentioned!", "706947": "juliaelliott Thank you very much. I have also checked it. It's helpful! Thank you!" }, "source": "meta" }