![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1013033%2F37704cc5e4b75fa654bf7df3eca72f95%2FScreenshot_2019-11-29%20TensorFlow%202%200%20Question%20Answering%20Kaggle.png?generation=1575018411106617&alt=media\")
I have created a notebook that simply hard codes the predictions. The notebook writes these to submission.csv. Notebook is shared at https://www.kaggle.com/arvindpdmn/basic-submission
I created this mainly to figure out how scoring works.
\n\nNot sure why there's a submission error.
\n\n
I have created a notebook that simply hard codes the predictions. The notebook writes these to submission.csv. Notebook is shared at https://www.kaggle.com/arvindpdmn/basic-submission
I created this mainly to figure out how scoring works.
\n\nNot sure why there's a submission error.
\n\n
That's expected. Your Kernel shouldn't submit hard-coded predictions. Instead, in should process whatever test set is there. So your code needs to read the test JSON file and produce a submission file.
\n\nWhen you hit \"Submit\", your code is run against a private test set (it's length is between 2500 and 3000), so this particular error means that you try to produce a 692-long prediction for a 4x larger test set.
", "rawMarkdown": "That's expected. Your Kernel shouldn't submit hard-coded predictions. Instead, in should process whatever test set is there. So your code needs to read the test JSON file and produce a submission file. \n\nWhen you hit \"Submit\", your code is run against a private test set (it's length is between 2500 and 3000), so this particular error means that you try to produce a 692-long prediction for a 4x larger test set.", "votes": null }, { "id": "684504", "postDate": "11/29/2019 23:21:08", "content": "Although submitting this public kernel : https://www.kaggle.com/prokaj/bert-joint-baseline-notebook\nstill produces an error.
", "rawMarkdown": "Although submitting this public kernel : https://www.kaggle.com/prokaj/bert-joint-baseline-notebook\nstill produces an error.", "votes": null }, { "id": "684549", "postDate": "11/30/2019 02:15:33", "content": "I was able to submit this without errors. Got 0.7 as score.
", "rawMarkdown": "I was able to submit this without errors. Got 0.7 as score.", "votes": null }, { "id": "689188", "postDate": "12/06/2019 15:53:57", "content": "Hi, Should I write my inference code based on \"test JSON\" or based on \"sample_submission.csv\" ? I write a inference code based on \"test JSON\", when I click the \"commit\" button, I got my \"submission.csv\", when I submit it to competition, my code runs about 10 times longer than \"commit\", everything looks well, but I got an \"error\", there is no score for my code, I'm very confused about this.
", "rawMarkdown": "Hi, Should I write my inference code based on \"test JSON\" or based on \"sample_submission.csv\" ? I write a inference code based on \"test JSON\", when I click the \"commit\" button, I got my \"submission.csv\", when I submit it to competition, my code runs about 10 times longer than \"commit\", everything looks well, but I got an \"error\", there is no score for my code, I'm very confused about this.", "votes": null }, { "id": "690979", "postDate": "12/09/2019 12:42:47", "content": "it should be based on sample_submission.csv, Make sure the qa_id is present in the output dataset and name of the output file is submission.csv
", "rawMarkdown": "it should be based on sample_submission.csv, Make sure the qa_id is present in the output dataset and name of the output file is submission.csv", "votes": null }, { "id": "704821", "postDate": "12/28/2019 03:39:45", "content": "@arvindpdmn Were you finally able to solve this error? Could you share what the fix was? If you could share a working notebook (doesn't need to have any ML in it, just I/O and submission), I would really appreciate it?
\n\nAlternatively, this is my notebook: https://www.kaggle.com/ankurgupta1985/randomly-selected\nI don't know what I am doing wrong or how to fix it. Any help would be very generous. Thanks!
", "rawMarkdown": "arvindpdmn Were you finally able to solve this error? Could you share what the fix was? If you could share a working notebook (doesn't need to have any ML in it, just I/O and submission), I would really appreciate it?\n\nAlternatively, this is my notebook: https://www.kaggle.com/ankurgupta1985/randomly-selected\nI don't know what I am doing wrong or how to fix it. Any help would be very generous. Thanks!", "votes": null }, { "id": "705117", "postDate": "12/28/2019 13:14:16", "content": "@ankurgupta1985 Yury has replied that hard-coded predictions are not accepted for submission. I was only testing if this works. I suspect the submission process actually checks if the kernel executes prediction code.
\n\nAlso, I looked at your CSV file. I checked a few sample entries. They're not in the public test dataset.
", "rawMarkdown": "ankurgupta1985 Yury has replied that hard-coded predictions are not accepted for submission. I was only testing if this works. I suspect the submission process actually checks if the kernel executes prediction code.\n\nAlso, I looked at your CSV file. I checked a few sample entries. They're not in the public test dataset.", "votes": null }, { "id": "705407", "postDate": "12/28/2019 22:22:00", "content": "Thanks @arvindpdmn for taking the time! I think I was able to get it to work. https://www.kaggle.com/c/tensorflow2-question-answering/discussion/123466
", "rawMarkdown": "Thanks @arvindpdmn for taking the time! I think I was able to get it to work. https://www.kaggle.com/c/tensorflow2-question-answering/discussion/123466", "votes": null } ], "comments": [ { "id": 684168, "author_name": "kashnitsky", "author_url": "", "post_date": "11/29/2019 10:40:20", "content": "That's expected. Your Kernel shouldn't submit hard-coded predictions. Instead, in should process whatever test set is there. So your code needs to read the test JSON file and produce a submission file.
\n\nWhen you hit \"Submit\", your code is run against a private test set (it's length is between 2500 and 3000), so this particular error means that you try to produce a 692-long prediction for a 4x larger test set.
", "votes": null, "replies": [ { "id": 684504, "author_name": "omarshaabanamer", "author_url": "", "post_date": "11/29/2019 23:21:08", "content": "Although submitting this public kernel : https://www.kaggle.com/prokaj/bert-joint-baseline-notebook\nstill produces an error.
", "votes": null, "replies": [] }, { "id": 684549, "author_name": "arvindpdmn", "author_url": "", "post_date": "11/30/2019 02:15:33", "content": "I was able to submit this without errors. Got 0.7 as score.
", "votes": null, "replies": [] }, { "id": 689188, "author_name": "", "author_url": "", "post_date": "12/06/2019 15:53:57", "content": "Hi, Should I write my inference code based on \"test JSON\" or based on \"sample_submission.csv\" ? I write a inference code based on \"test JSON\", when I click the \"commit\" button, I got my \"submission.csv\", when I submit it to competition, my code runs about 10 times longer than \"commit\", everything looks well, but I got an \"error\", there is no score for my code, I'm very confused about this.
", "votes": null, "replies": [] }, { "id": 690979, "author_name": "omarshaabanamer", "author_url": "", "post_date": "12/09/2019 12:42:47", "content": "it should be based on sample_submission.csv, Make sure the qa_id is present in the output dataset and name of the output file is submission.csv
", "votes": null, "replies": [] }, { "id": 704821, "author_name": "ankurgupta1985", "author_url": "", "post_date": "12/28/2019 03:39:45", "content": "@arvindpdmn Were you finally able to solve this error? Could you share what the fix was? If you could share a working notebook (doesn't need to have any ML in it, just I/O and submission), I would really appreciate it?
\n\nAlternatively, this is my notebook: https://www.kaggle.com/ankurgupta1985/randomly-selected\nI don't know what I am doing wrong or how to fix it. Any help would be very generous. Thanks!
", "votes": null, "replies": [] } ] }, { "id": 705117, "author_name": "arvindpdmn", "author_url": "", "post_date": "12/28/2019 13:14:16", "content": "@ankurgupta1985 Yury has replied that hard-coded predictions are not accepted for submission. I was only testing if this works. I suspect the submission process actually checks if the kernel executes prediction code.
\n\nAlso, I looked at your CSV file. I checked a few sample entries. They're not in the public test dataset.
", "votes": null, "replies": [ { "id": 705407, "author_name": "ankurgupta1985", "author_url": "", "post_date": "12/28/2019 22:22:00", "content": "Thanks @arvindpdmn for taking the time! I think I was able to get it to work. https://www.kaggle.com/c/tensorflow2-question-answering/discussion/123466
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "684122": "I have created a notebook that simply hard codes the predictions. The notebook writes these to `submission.csv`. Notebook is shared at https://www.kaggle.com/arvindpdmn/basic-submission\n\nI created this mainly to figure out how scoring works.\n\nNot sure why there's a submission error.\n\n", "684168": "That's expected. Your Kernel shouldn't submit hard-coded predictions. Instead, in should process whatever test set is there. So your code needs to read the test JSON file and produce a submission file. \n\nWhen you hit \"Submit\", your code is run against a private test set (it's length is between 2500 and 3000), so this particular error means that you try to produce a 692-long prediction for a 4x larger test set.", "684504": "Although submitting this public kernel : https://www.kaggle.com/prokaj/bert-joint-baseline-notebook\nstill produces an error.", "684549": "I was able to submit this without errors. Got 0.7 as score.", "689188": "Hi, Should I write my inference code based on \"test JSON\" or based on \"sample_submission.csv\" ? I write a inference code based on \"test JSON\", when I click the \"commit\" button, I got my \"submission.csv\", when I submit it to competition, my code runs about 10 times longer than \"commit\", everything looks well, but I got an \"error\", there is no score for my code, I'm very confused about this.", "690979": "it should be based on sample_submission.csv, Make sure the qa_id is present in the output dataset and name of the output file is submission.csv", "704821": "arvindpdmn Were you finally able to solve this error? Could you share what the fix was? If you could share a working notebook (doesn't need to have any ML in it, just I/O and submission), I would really appreciate it?\n\nAlternatively, this is my notebook: https://www.kaggle.com/ankurgupta1985/randomly-selected\nI don't know what I am doing wrong or how to fix it. Any help would be very generous. Thanks!", "705117": "ankurgupta1985 Yury has replied that hard-coded predictions are not accepted for submission. I was only testing if this works. I suspect the submission process actually checks if the kernel executes prediction code.\n\nAlso, I looked at your CSV file. I checked a few sample entries. They're not in the public test dataset.", "705407": "Thanks @arvindpdmn for taking the time! I think I was able to get it to work. https://www.kaggle.com/c/tensorflow2-question-answering/discussion/123466" }, "source": "meta" }