{ "id": 400313, "title": "Changes to the Competition API", "url": "/competitions/predict-student-performance-from-game-play/discussion/400313", "author_name": "", "post_date": "2023-04-07T17:56:44.570907300Z", "votes": 12, "comment_count": 14, "views": 0, "content": "

Hi Kagglers,

\n

The API was recently updated to fix reported bugs (this fix has been finalized), however the change requires all submissions to switch the order of the arguments to the iter_test object.

\n

Specifically, you will need to make the following change to any previous inference notebooks:

\n

Change
\nfor (sample_submission, test) in iter_test:
\nTo
\nfor (test, sample_submission) in iter_test:

\n

Any code using the previous order will run into an error, which will unfortunately cause issues when rerunning old submissions. However, this change was necessary to fix the issues we were seeing.

\n

We acknowledge that the change caused problems for a number of people, and we apologize for the frustration this caused! Although it doesn’t fully compensate for lost time, we will be extending the competition by a week to make up for lost submissions due to the update.

\n

If you have any questions or continue to run into errors, please respond in this thread.

\n

Many thanks!

", "messages": [ { "id": "2213621", "postDate": "04/07/2023 17:56:44", "content": "

Hi Kagglers,

\n

The API was recently updated to fix reported bugs (this fix has been finalized), however the change requires all submissions to switch the order of the arguments to the iter_test object.

\n

Specifically, you will need to make the following change to any previous inference notebooks:

\n

Change
\nfor (sample_submission, test) in iter_test:
\nTo
\nfor (test, sample_submission) in iter_test:

\n

Any code using the previous order will run into an error, which will unfortunately cause issues when rerunning old submissions. However, this change was necessary to fix the issues we were seeing.

\n

We acknowledge that the change caused problems for a number of people, and we apologize for the frustration this caused! Although it doesn’t fully compensate for lost time, we will be extending the competition by a week to make up for lost submissions due to the update.

\n

If you have any questions or continue to run into errors, please respond in this thread.

\n

Many thanks!

", "rawMarkdown": "Hi Kagglers,\n\nThe API was recently updated to fix reported bugs (this fix has been finalized), however the change requires all submissions to switch the order of the arguments to the iter_test object.\n\nSpecifically, you will need to make the following change to any previous inference notebooks:\n\nChange \n`for (sample_submission, test) in iter_test:`\nTo\n`for (test, sample_submission) in iter_test:`\n\nAny code using the previous order will run into an error, which will unfortunately cause issues when rerunning old submissions. However, this change was necessary to fix the issues we were seeing.\n\nWe acknowledge that the change caused problems for a number of people, and we apologize for the frustration this caused! Although it doesn’t fully compensate for lost time, we will be extending the competition by a week to make up for lost submissions due to the update.\n\nIf you have any questions or continue to run into errors, please respond in this thread.\n\nMany thanks!", "votes": null }, { "id": "2213780", "postDate": "04/07/2023 20:58:45", "content": "

@philculliton Hi Phil, it would be great if you could fix the sample data in the Kaggle API so that it only provides 1 session_id per for-loop iteration.

\n

Currently the Kaggle API provides 3 session_id per for-loop during commit and the Kaggle API provides 1 session_id per for-loop during submit. And the old API provided 1 session_id per for-loop during commit.

\n

All old notebook are broken because two reasons (not one reason like you say):

\n\n

Additionally old notebooks break because the double train data size causes memory errors, but that isn't an error regarding the Kaggle API.

", "rawMarkdown": "philculliton Hi Phil, it would be great if you could fix the sample data in the Kaggle API so that it only provides **1 session_id** per for-loop iteration. \n\nCurrently the Kaggle API provides **3 session_id** per for-loop during commit and the Kaggle API provides **1 session_id** per for-loop during submit. And the old API provided **1 session_id** per for-loop during commit.\n\nAll old notebook are broken because **two reasons** (not one reason like you say):\n* `sample_submission, test` has changed to `test, sample_submission`\n* commit Kaggle API provides **3 session_id** instead of **1 session_id** like it used to \n\nAdditionally old notebooks break because the double train data size causes memory errors, but that isn't an error regarding the Kaggle API.", "votes": null }, { "id": "2213855", "postDate": "04/07/2023 22:33:56", "content": "

It would also be great if the basic submission demo is updated.
\nhttps://www.kaggle.com/code/philculliton/basic-submission-demo

", "rawMarkdown": "It would also be great if the basic submission demo is updated.\n[https://www.kaggle.com/code/philculliton/basic-submission-demo](https://www.kaggle.com/code/philculliton/basic-submission-demo)", "votes": null }, { "id": "2213935", "postDate": "04/08/2023 01:03:29", "content": "

In regards to this, I was getting some errors on submissions to the hidden test set that were not present when I ran with the API on the test set. Do we expect 3 total loops in the iteration, one for each level group? Or are there many more loops on the hidden test set?

", "rawMarkdown": "In regards to this, I was getting some errors on submissions to the hidden test set that were not present when I ran with the API on the test set. Do we expect 3 total loops in the iteration, one for each level group? Or are there many more loops on the hidden test set?", "votes": null }, { "id": "2213936", "postDate": "04/08/2023 01:05:05", "content": "

Hi @cdeotte, can I confirm with you that during submission it is still 1 session_id, 1 level_group per for-loop iteration?
\nThat means, it is the same as before except the order of test, sample_submission?
\nThanks a lot!

", "rawMarkdown": "Hi @cdeotte, can I confirm with you that during **submission** it is still **1 session_id, 1 level_group** per for-loop iteration? \nThat means, it is the same as before except the order of `test, sample_submission`?\nThanks a lot!", "votes": null }, { "id": "2214000", "postDate": "04/08/2023 04:25:40", "content": "

When the change is done, you can see the 3 different session_ids in sample_submission
\n\"\"

", "rawMarkdown": "When the change is done, you can see the 3 different session_ids in sample_submission\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1903540%2Fcfb4695ef21dc9ce147169c9c6a0835b%2Fsample_problem.jpg?generation=1680927882047910&alt=media)", "votes": null }, { "id": "2214154", "postDate": "04/08/2023 07:33:11", "content": "

Thank you, Chris. You expressed very precisely what I was struggling with.

\n

It means that there are the following difficulties:

\n\n

We need to write the submission code for both 1 session_id and 3 session_id.
\nThis is a bit unnatural.

", "rawMarkdown": "Thank you, Chris. You expressed very precisely what I was struggling with.\n\nIt means that there are the following difficulties:\n\n- If the submission code assumes 1 session_id, **an error occurs when saving the notebook.**\n- If the submission code assumes 3 session_id, **an error occurs after submission.**\n\nWe need to write the submission code for both 1 session_id and 3 session_id.\nThis is a bit unnatural.", "votes": null }, { "id": "2214871", "postDate": "04/08/2023 20:19:47", "content": "

Hello, @danielphalen
\nThe hidden test looks like 1 session_id and 1 level_group in every loop.
\nTherefore, I assume that the number of loops is the number of session_id * 3.
\nYes, it is so many loops.
\nI hope this will help.

", "rawMarkdown": "Hello, @danielphalen \nThe hidden test looks like 1 session_id and 1 level_group in every loop.\nTherefore, I assume that the number of loops is the number of session_id * 3. \nYes, it is so many loops.\nI hope this will help.", "votes": null }, { "id": "2214873", "postDate": "04/08/2023 20:22:08", "content": "

Hello, @boscoyung
\nI have confirmed that the hidden test is almost certainly 1 session_id and 1 level_group in every loop.
\nThe way I did it is the following code.

\n
 (test, sample_submission)  iter_test:\n     (test[].nunique() == ) & (test[].nunique() == ):\n        *****my prediction code*****\n    :\n        sample_submission[] = \n        env.predict(sample_submission)\n
\n

The submission was successful and I got the normal score. My score did not decrease.
\nI hope this helps you.

", "rawMarkdown": "Hello, @boscoyung \nI have confirmed that the hidden test is almost certainly 1 session_id and 1 level_group in every loop.\nThe way I did it is the following code.\n```python\nfor (test, sample_submission) in iter_test:\n if (test['level_group'].nunique() == 1) & (test['session_id'].nunique() == 1):\n *****my prediction code*****\n else:\n sample_submission['correct'] = 2\n env.predict(sample_submission)\n```\n\nThe submission was successful and I got the normal score. My score did not decrease.\nI hope this helps you.", "votes": null }, { "id": "2214940", "postDate": "04/08/2023 22:15:45", "content": "

Thanks for that. Do you know if the order is
\nfor session_id in session_ids:
\nfor level_group in level_groups:
\nor the other way around?

", "rawMarkdown": "Thanks for that. Do you know if the order is\n`for session_id in session_ids:`\n` for level_group in level_groups:`\nor the other way around?", "votes": null }, { "id": "2214955", "postDate": "04/08/2023 23:10:36", "content": "

Thanks @tanakaakinori !

", "rawMarkdown": "Thanks @tanakaakinori !", "votes": null }, { "id": "2215651", "postDate": "04/09/2023 13:37:58", "content": "

I'm sorry, I do not know the order in the Hidden test.
\nBut I just know that.
\nBefore the Kaggle API updated, the Sample test worked like this in the Notebook.

\n
 (sample_submission, test)  iter_test:\n    (test.level_group.values[])\n    (session_id = test.session_id.values[])\n
\n

0-4
\n20090109393214576
\n0-4
\n20090312143683264
\n0-4
\n20090312331414616
\n5-12
\n20090109393214576
\n5-12
\n20090312143683264
\n5-12
\n20090312331414616
\n13-22
\n20090109393214576
\n13-22
\n20090312143683264
\n13-22
\n20090312331414616

", "rawMarkdown": "I'm sorry, I do not know the order in the Hidden test.\nBut I just know that.\nBefore the Kaggle API updated, the Sample test worked like this in the Notebook.\n```python\nfor (sample_submission, test) in iter_test:\n print(test.level_group.values[0])\n print(session_id = test.session_id.values[0])\n```\n0-4\n20090109393214576\n0-4\n20090312143683264\n0-4\n20090312331414616\n5-12\n20090109393214576\n5-12\n20090312143683264\n5-12\n20090312331414616\n13-22\n20090109393214576\n13-22\n20090312143683264\n13-22\n20090312331414616", "votes": null }, { "id": "2273789", "postDate": "05/25/2023 12:07:59", "content": "

Could you please explain what u said in detail.
\nI am having Submission Scoring Error, which i couldn't solve for two days.

\n
limits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nfor (test, sample_submission) in iter_test:\n    test = test.sort_values(by='index')\n    grp = test.level_group.values[0]\n    session_id = test.session_id.values[0]\n    a,b = limits[grp]\n    df = model_data_1(test, grp)\n    preds = []\n    for q in range(a, b):\n        model = models[q-1]   \n        pred = model.predict_proba(df)[0, 1]\n        mask = sample_submission.session_id.str.contains(f'q{q}')\n        sample_submission.loc[mask,'correct'] = int(pred > threshold[q-1]) \n    #sample_submission = sample_submission[['session_id', 'correct']]\n    print(preds)\n    env.predict(sample_submission)\n
\n

could you help me in solving the error, by pointing out mistake if any.
\nThank You

", "rawMarkdown": "Could you please explain what u said in detail.\nI am having Submission Scoring Error, which i couldn't solve for two days.\n```\nlimits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nfor (test, sample_submission) in iter_test:\n test = test.sort_values(by='index')\n grp = test.level_group.values[0]\n session_id = test.session_id.values[0]\n a,b = limits[grp]\n df = model_data_1(test, grp)\n preds = []\n for q in range(a, b):\n model = models[q-1] \n pred = model.predict_proba(df)[0, 1]\n mask = sample_submission.session_id.str.contains(f'q{q}')\n sample_submission.loc[mask,'correct'] = int(pred > threshold[q-1]) \n #sample_submission = sample_submission[['session_id', 'correct']]\n print(preds)\n env.predict(sample_submission)\n \n```\ncould you help me in solving the error, by pointing out mistake if any.\nThank You", "votes": null }, { "id": "2273793", "postDate": "05/25/2023 12:11:05", "content": "

Is the above problem in API fixed?
\nI am having Submission Scoring Error, which i couldn't solve for two days.

\n
limits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nfor (test, sample_submission) in iter_test:\n    test = test.sort_values(by='index')\n    grp = test.level_group.values[0]\n    session_id = test.session_id.values[0]\n    a,b = limits[grp]\n    df = model_data_1(test, grp)\n    preds = []\n    for q in range(a, b):\n        model = models[q-1]   \n        pred = model.predict_proba(df)[0, 1]\n        mask = sample_submission.session_id.str.contains(f'q{q}')\n        sample_submission.loc[mask,'correct'] = int(pred > threshold[q-1]) \n    #sample_submission = sample_submission[['session_id', 'correct']]\n    print(preds)\n    env.predict(sample_submission)\n
\n

could you help me in solving the error, by pointing out mistake if any.
\nThank You

", "rawMarkdown": "Is the above problem in API fixed?\nI am having Submission Scoring Error, which i couldn't solve for two days.\n```\nlimits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nfor (test, sample_submission) in iter_test:\n test = test.sort_values(by='index')\n grp = test.level_group.values[0]\n session_id = test.session_id.values[0]\n a,b = limits[grp]\n df = model_data_1(test, grp)\n preds = []\n for q in range(a, b):\n model = models[q-1] \n pred = model.predict_proba(df)[0, 1]\n mask = sample_submission.session_id.str.contains(f'q{q}')\n sample_submission.loc[mask,'correct'] = int(pred > threshold[q-1]) \n #sample_submission = sample_submission[['session_id', 'correct']]\n print(preds)\n env.predict(sample_submission)\n```\ncould you help me in solving the error, by pointing out mistake if any.\nThank You", "votes": null }, { "id": "2301568", "postDate": "06/14/2023 03:16:15", "content": "

The change of API caused much more effect than expected. For example, I have one normal submission which scored 0.703 before, and I select it as final submission. However, with the new API running, it went wrong with the hint \"Submission Scoring Error\". But I can not deselect it.

\n

Hope Kaggle team can fix this problem!

\n

\"\"

", "rawMarkdown": "The change of API caused much more effect than expected. For example, I have one normal submission which scored 0.703 before, and I select it as final submission. However, with the new API running, it went wrong with the hint \"Submission Scoring Error\". But I can not deselect it.\n\nHope Kaggle team can fix this problem! \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14268627%2Ff0d0e088d5b1972c890c3e18c4df5a5f%2Fsssss.png?generation=1686712469412276&alt=media)", "votes": null } ], "comments": [ { "id": 2213780, "author_name": "cdeotte", "author_url": "", "post_date": "04/07/2023 20:58:45", "content": "

@philculliton Hi Phil, it would be great if you could fix the sample data in the Kaggle API so that it only provides 1 session_id per for-loop iteration.

\n

Currently the Kaggle API provides 3 session_id per for-loop during commit and the Kaggle API provides 1 session_id per for-loop during submit. And the old API provided 1 session_id per for-loop during commit.

\n

All old notebook are broken because two reasons (not one reason like you say):

\n\n

Additionally old notebooks break because the double train data size causes memory errors, but that isn't an error regarding the Kaggle API.

", "votes": null, "replies": [ { "id": 2213935, "author_name": "danielphalen", "author_url": "", "post_date": "04/08/2023 01:03:29", "content": "

In regards to this, I was getting some errors on submissions to the hidden test set that were not present when I ran with the API on the test set. Do we expect 3 total loops in the iteration, one for each level group? Or are there many more loops on the hidden test set?

", "votes": null, "replies": [ { "id": 2214871, "author_name": "tanakaakinori", "author_url": "", "post_date": "04/08/2023 20:19:47", "content": "

Hello, @danielphalen
\nThe hidden test looks like 1 session_id and 1 level_group in every loop.
\nTherefore, I assume that the number of loops is the number of session_id * 3.
\nYes, it is so many loops.
\nI hope this will help.

", "votes": null, "replies": [ { "id": 2214940, "author_name": "danielphalen", "author_url": "", "post_date": "04/08/2023 22:15:45", "content": "

Thanks for that. Do you know if the order is
\nfor session_id in session_ids:
\nfor level_group in level_groups:
\nor the other way around?

", "votes": null, "replies": [ { "id": 2215651, "author_name": "tanakaakinori", "author_url": "", "post_date": "04/09/2023 13:37:58", "content": "

I'm sorry, I do not know the order in the Hidden test.
\nBut I just know that.
\nBefore the Kaggle API updated, the Sample test worked like this in the Notebook.

\n
 (sample_submission, test)  iter_test:\n    (test.level_group.values[])\n    (session_id = test.session_id.values[])\n
\n

0-4
\n20090109393214576
\n0-4
\n20090312143683264
\n0-4
\n20090312331414616
\n5-12
\n20090109393214576
\n5-12
\n20090312143683264
\n5-12
\n20090312331414616
\n13-22
\n20090109393214576
\n13-22
\n20090312143683264
\n13-22
\n20090312331414616

", "votes": null, "replies": [] } ] } ] } ] }, { "id": 2213936, "author_name": "boscoyung", "author_url": "", "post_date": "04/08/2023 01:05:05", "content": "

Hi @cdeotte, can I confirm with you that during submission it is still 1 session_id, 1 level_group per for-loop iteration?
\nThat means, it is the same as before except the order of test, sample_submission?
\nThanks a lot!

", "votes": null, "replies": [ { "id": 2214873, "author_name": "tanakaakinori", "author_url": "", "post_date": "04/08/2023 20:22:08", "content": "

Hello, @boscoyung
\nI have confirmed that the hidden test is almost certainly 1 session_id and 1 level_group in every loop.
\nThe way I did it is the following code.

\n
 (test, sample_submission)  iter_test:\n     (test[].nunique() == ) & (test[].nunique() == ):\n        *****my prediction code*****\n    :\n        sample_submission[] = \n        env.predict(sample_submission)\n
\n

The submission was successful and I got the normal score. My score did not decrease.
\nI hope this helps you.

", "votes": null, "replies": [ { "id": 2214955, "author_name": "boscoyung", "author_url": "", "post_date": "04/08/2023 23:10:36", "content": "

Thanks @tanakaakinori !

", "votes": null, "replies": [] } ] } ] }, { "id": 2214154, "author_name": "tanakaakinori", "author_url": "", "post_date": "04/08/2023 07:33:11", "content": "

Thank you, Chris. You expressed very precisely what I was struggling with.

\n

It means that there are the following difficulties:

\n\n

We need to write the submission code for both 1 session_id and 3 session_id.
\nThis is a bit unnatural.

", "votes": null, "replies": [ { "id": 2273789, "author_name": "navinkumarmnk", "author_url": "", "post_date": "05/25/2023 12:07:59", "content": "

Could you please explain what u said in detail.
\nI am having Submission Scoring Error, which i couldn't solve for two days.

\n
limits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nfor (test, sample_submission) in iter_test:\n    test = test.sort_values(by='index')\n    grp = test.level_group.values[0]\n    session_id = test.session_id.values[0]\n    a,b = limits[grp]\n    df = model_data_1(test, grp)\n    preds = []\n    for q in range(a, b):\n        model = models[q-1]   \n        pred = model.predict_proba(df)[0, 1]\n        mask = sample_submission.session_id.str.contains(f'q{q}')\n        sample_submission.loc[mask,'correct'] = int(pred > threshold[q-1]) \n    #sample_submission = sample_submission[['session_id', 'correct']]\n    print(preds)\n    env.predict(sample_submission)\n
\n

could you help me in solving the error, by pointing out mistake if any.
\nThank You

", "votes": null, "replies": [] } ] }, { "id": 2273793, "author_name": "navinkumarmnk", "author_url": "", "post_date": "05/25/2023 12:11:05", "content": "

Is the above problem in API fixed?
\nI am having Submission Scoring Error, which i couldn't solve for two days.

\n
limits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nfor (test, sample_submission) in iter_test:\n    test = test.sort_values(by='index')\n    grp = test.level_group.values[0]\n    session_id = test.session_id.values[0]\n    a,b = limits[grp]\n    df = model_data_1(test, grp)\n    preds = []\n    for q in range(a, b):\n        model = models[q-1]   \n        pred = model.predict_proba(df)[0, 1]\n        mask = sample_submission.session_id.str.contains(f'q{q}')\n        sample_submission.loc[mask,'correct'] = int(pred > threshold[q-1]) \n    #sample_submission = sample_submission[['session_id', 'correct']]\n    print(preds)\n    env.predict(sample_submission)\n
\n

could you help me in solving the error, by pointing out mistake if any.
\nThank You

", "votes": null, "replies": [] } ] }, { "id": 2213855, "author_name": "darryldias", "author_url": "", "post_date": "04/07/2023 22:33:56", "content": "

It would also be great if the basic submission demo is updated.
\nhttps://www.kaggle.com/code/philculliton/basic-submission-demo

", "votes": null, "replies": [ { "id": 2214000, "author_name": "darryldias", "author_url": "", "post_date": "04/08/2023 04:25:40", "content": "

When the change is done, you can see the 3 different session_ids in sample_submission
\n\"\"

", "votes": null, "replies": [] } ] }, { "id": 2301568, "author_name": "littlstar123", "author_url": "", "post_date": "06/14/2023 03:16:15", "content": "

The change of API caused much more effect than expected. For example, I have one normal submission which scored 0.703 before, and I select it as final submission. However, with the new API running, it went wrong with the hint \"Submission Scoring Error\". But I can not deselect it.

\n

Hope Kaggle team can fix this problem!

\n

\"\"

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2213621": "Hi Kagglers,\n\nThe API was recently updated to fix reported bugs (this fix has been finalized), however the change requires all submissions to switch the order of the arguments to the iter_test object.\n\nSpecifically, you will need to make the following change to any previous inference notebooks:\n\nChange \n`for (sample_submission, test) in iter_test:`\nTo\n`for (test, sample_submission) in iter_test:`\n\nAny code using the previous order will run into an error, which will unfortunately cause issues when rerunning old submissions. However, this change was necessary to fix the issues we were seeing.\n\nWe acknowledge that the change caused problems for a number of people, and we apologize for the frustration this caused! Although it doesn’t fully compensate for lost time, we will be extending the competition by a week to make up for lost submissions due to the update.\n\nIf you have any questions or continue to run into errors, please respond in this thread.\n\nMany thanks!", "2213780": "philculliton Hi Phil, it would be great if you could fix the sample data in the Kaggle API so that it only provides **1 session_id** per for-loop iteration. \n\nCurrently the Kaggle API provides **3 session_id** per for-loop during commit and the Kaggle API provides **1 session_id** per for-loop during submit. And the old API provided **1 session_id** per for-loop during commit.\n\nAll old notebook are broken because **two reasons** (not one reason like you say):\n* `sample_submission, test` has changed to `test, sample_submission`\n* commit Kaggle API provides **3 session_id** instead of **1 session_id** like it used to \n\nAdditionally old notebooks break because the double train data size causes memory errors, but that isn't an error regarding the Kaggle API.", "2213855": "It would also be great if the basic submission demo is updated.\n[https://www.kaggle.com/code/philculliton/basic-submission-demo](https://www.kaggle.com/code/philculliton/basic-submission-demo)", "2213935": "In regards to this, I was getting some errors on submissions to the hidden test set that were not present when I ran with the API on the test set. Do we expect 3 total loops in the iteration, one for each level group? Or are there many more loops on the hidden test set?", "2213936": "Hi @cdeotte, can I confirm with you that during **submission** it is still **1 session_id, 1 level_group** per for-loop iteration? \nThat means, it is the same as before except the order of `test, sample_submission`?\nThanks a lot!", "2214000": "When the change is done, you can see the 3 different session_ids in sample_submission\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1903540%2Fcfb4695ef21dc9ce147169c9c6a0835b%2Fsample_problem.jpg?generation=1680927882047910&alt=media)", "2214154": "Thank you, Chris. You expressed very precisely what I was struggling with.\n\nIt means that there are the following difficulties:\n\n- If the submission code assumes 1 session_id, **an error occurs when saving the notebook.**\n- If the submission code assumes 3 session_id, **an error occurs after submission.**\n\nWe need to write the submission code for both 1 session_id and 3 session_id.\nThis is a bit unnatural.", "2214871": "Hello, @danielphalen \nThe hidden test looks like 1 session_id and 1 level_group in every loop.\nTherefore, I assume that the number of loops is the number of session_id * 3. \nYes, it is so many loops.\nI hope this will help.", "2214873": "Hello, @boscoyung \nI have confirmed that the hidden test is almost certainly 1 session_id and 1 level_group in every loop.\nThe way I did it is the following code.\n```python\nfor (test, sample_submission) in iter_test:\n if (test['level_group'].nunique() == 1) & (test['session_id'].nunique() == 1):\n *****my prediction code*****\n else:\n sample_submission['correct'] = 2\n env.predict(sample_submission)\n```\n\nThe submission was successful and I got the normal score. My score did not decrease.\nI hope this helps you.", "2214940": "Thanks for that. Do you know if the order is\n`for session_id in session_ids:`\n` for level_group in level_groups:`\nor the other way around?", "2214955": "Thanks @tanakaakinori !", "2215651": "I'm sorry, I do not know the order in the Hidden test.\nBut I just know that.\nBefore the Kaggle API updated, the Sample test worked like this in the Notebook.\n```python\nfor (sample_submission, test) in iter_test:\n print(test.level_group.values[0])\n print(session_id = test.session_id.values[0])\n```\n0-4\n20090109393214576\n0-4\n20090312143683264\n0-4\n20090312331414616\n5-12\n20090109393214576\n5-12\n20090312143683264\n5-12\n20090312331414616\n13-22\n20090109393214576\n13-22\n20090312143683264\n13-22\n20090312331414616", "2273789": "Could you please explain what u said in detail.\nI am having Submission Scoring Error, which i couldn't solve for two days.\n```\nlimits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nfor (test, sample_submission) in iter_test:\n test = test.sort_values(by='index')\n grp = test.level_group.values[0]\n session_id = test.session_id.values[0]\n a,b = limits[grp]\n df = model_data_1(test, grp)\n preds = []\n for q in range(a, b):\n model = models[q-1] \n pred = model.predict_proba(df)[0, 1]\n mask = sample_submission.session_id.str.contains(f'q{q}')\n sample_submission.loc[mask,'correct'] = int(pred > threshold[q-1]) \n #sample_submission = sample_submission[['session_id', 'correct']]\n print(preds)\n env.predict(sample_submission)\n \n```\ncould you help me in solving the error, by pointing out mistake if any.\nThank You", "2273793": "Is the above problem in API fixed?\nI am having Submission Scoring Error, which i couldn't solve for two days.\n```\nlimits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nfor (test, sample_submission) in iter_test:\n test = test.sort_values(by='index')\n grp = test.level_group.values[0]\n session_id = test.session_id.values[0]\n a,b = limits[grp]\n df = model_data_1(test, grp)\n preds = []\n for q in range(a, b):\n model = models[q-1] \n pred = model.predict_proba(df)[0, 1]\n mask = sample_submission.session_id.str.contains(f'q{q}')\n sample_submission.loc[mask,'correct'] = int(pred > threshold[q-1]) \n #sample_submission = sample_submission[['session_id', 'correct']]\n print(preds)\n env.predict(sample_submission)\n```\ncould you help me in solving the error, by pointing out mistake if any.\nThank You", "2301568": "The change of API caused much more effect than expected. For example, I have one normal submission which scored 0.703 before, and I select it as final submission. However, with the new API running, it went wrong with the hint \"Submission Scoring Error\". But I can not deselect it.\n\nHope Kaggle team can fix this problem! \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14268627%2Ff0d0e088d5b1972c890c3e18c4df5a5f%2Fsssss.png?generation=1686712469412276&alt=media)" }, "source": "meta" }