{
  "id": 416784,
  "title": "[solved] submission scoring error",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/416784",
  "author_name": "",
  "post_date": "2023-06-13T02:30:07.806837800Z",
  "votes": 1,
  "comment_count": 10,
  "views": 0,
  "content": "<p>Is it just me who's suffering from submission scoring error right now??</p>\n<p>Will this be re-scored again when the error's fixed? Or do I have to submit it again?</p>\n<p>I do not want to waste my submission try…</p>\n<p>I'm currently using jo wilder 3.7</p>\n<hr>\n<p>I found out the problem and I think it may help you.</p>\n<p>While predicting and submitting, the data only comes with one session id. I was making some meta data with certain columns, if this column is under some condition. And I realized that in my train / test set, there's a bunch of other session id so that the feature columns will still remains even with nan values. But if featurizing with a single session id with the condition not applicable, those features disappeared.</p>\n<p>Consider this case</p>\n<pre><code>column = \n\ncond = df[column] != \ngb = df[cond].groupby([, ])\nfeature = gb[].mean()\n</code></pre>\n<p>If cond is all False then feature will not be correctly generated. This happens while running jo_wilder because there is only one session id each time. </p>\n<p>I hope this can help.</p>",
  "messages": [
    {
      "id": "2300049",
      "postDate": "06/13/2023 02:30:07",
      "content": "<p>Is it just me who's suffering from submission scoring error right now??</p>\n<p>Will this be re-scored again when the error's fixed? Or do I have to submit it again?</p>\n<p>I do not want to waste my submission try…</p>\n<p>I'm currently using jo wilder 3.7</p>\n<hr>\n<p>I found out the problem and I think it may help you.</p>\n<p>While predicting and submitting, the data only comes with one session id. I was making some meta data with certain columns, if this column is under some condition. And I realized that in my train / test set, there's a bunch of other session id so that the feature columns will still remains even with nan values. But if featurizing with a single session id with the condition not applicable, those features disappeared.</p>\n<p>Consider this case</p>\n<pre><code>column = \n\ncond = df[column] != \ngb = df[cond].groupby([, ])\nfeature = gb[].mean()\n</code></pre>\n<p>If cond is all False then feature will not be correctly generated. This happens while running jo_wilder because there is only one session id each time. </p>\n<p>I hope this can help.</p>",
      "rawMarkdown": "Is it just me who's suffering from submission scoring error right now??\n\nWill this be re-scored again when the error's fixed? Or do I have to submit it again?\n\nI do not want to waste my submission try...\n\nI'm currently using jo wilder 3.7\n\n\n----\nI found out the problem and I think it may help you.\n\nWhile predicting and submitting, the data only comes with one session id. I was making some meta data with certain columns, if this column is under some condition. And I realized that in my train / test set, there's a bunch of other session id so that the feature columns will still remains even with nan values. But if featurizing with a single session id with the condition not applicable, those features disappeared.\n\nConsider this case\n\n\n```python\ncolumn = 'some_column'\n\ncond = df[column] != 0\ngb = df[cond].groupby(['session_id', 'level_group'])\nfeature = gb['elapsed_time'].mean()\n```\n\n\n\nIf cond is all False then feature will not be correctly generated. This happens while running jo_wilder because there is only one session id each time. \n\nI hope this can help.",
      "votes": null
    },
    {
      "id": "2300055",
      "postDate": "06/13/2023 02:38:01",
      "content": "<p>Scoring error same for me! My submission cannot get score since last night, just running and running over 15h by now!</p>",
      "rawMarkdown": "Scoring error same for me! My submission cannot get score since last night, just running and running over 15h by now!",
      "votes": null
    },
    {
      "id": "2300062",
      "postDate": "06/13/2023 02:48:32",
      "content": "<p>did you try to submit again with the ones you already succeeded?</p>",
      "rawMarkdown": "did you try to submit again with the ones you already succeeded?",
      "votes": null
    },
    {
      "id": "2300066",
      "postDate": "06/13/2023 02:50:47",
      "content": "<p>I'm experiencing the same issue. The scoring has been running for more than 12 hours.</p>",
      "rawMarkdown": "I'm experiencing the same issue. The scoring has been running for more than 12 hours.",
      "votes": null
    },
    {
      "id": "2300069",
      "postDate": "06/13/2023 02:52:13",
      "content": "<p>Really exhausted in this competition…</p>",
      "rawMarkdown": "Really exhausted in this competition...",
      "votes": null
    },
    {
      "id": "2300082",
      "postDate": "06/13/2023 03:04:42",
      "content": "<p>I’m with you…</p>",
      "rawMarkdown": "I’m with you...",
      "votes": null
    },
    {
      "id": "2300188",
      "postDate": "06/13/2023 04:35:33",
      "content": "<p>Did you check if rerun the ones you've succeeded not working as well??</p>",
      "rawMarkdown": "Did you check if rerun the ones you've succeeded not working as well??",
      "votes": null
    },
    {
      "id": "2300209",
      "postDate": "06/13/2023 04:53:14",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6189766%2F1887dab6d6adf2935c07b9fac0590f33%2F.PNG?generation=1686631611118609&amp;alt=media\" alt=\"\"></p>\n<p>I have.<br>\nIf you have submitted a successful submission, but after re-scored, it becomes a submission scoring error.<br>\nThat's most likely a bug in your original submit script that wasn't shown as a failure (at least I did)<br>\nAfter I found those bug and modified them, I can submit normally.</p>",
      "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6189766%2F1887dab6d6adf2935c07b9fac0590f33%2F.PNG?generation=1686631611118609&alt=media)\n\nI have.\nIf you have submitted a successful submission, but after re-scored, it becomes a submission scoring error.\nThat's most likely a bug in your original submit script that wasn't shown as a failure (at least I did)\nAfter I found those bug and modified them, I can submit normally.",
      "votes": null
    },
    {
      "id": "2300243",
      "postDate": "06/13/2023 05:15:37",
      "content": "<p>I'm not trying it.<br>\nIn the end, I received an unfamiliar message called \"Kaggle error.\"</p>",
      "rawMarkdown": "I'm not trying it.\nIn the end, I received an unfamiliar message called \"Kaggle error.\"",
      "votes": null
    },
    {
      "id": "2300277",
      "postDate": "06/13/2023 05:38:18",
      "content": "<p>Hey guys, I've found my error case. Check if the case is applicable to yours. Thanks.</p>",
      "rawMarkdown": "Hey guys, I've found my error case. Check if the case is applicable to yours. Thanks.",
      "votes": null
    },
    {
      "id": "2300577",
      "postDate": "06/13/2023 09:09:37",
      "content": "<p>I think your case might be different from my case…</p>",
      "rawMarkdown": "I think your case might be different from my case...",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2300055,
      "author_name": "brook1114",
      "author_url": "",
      "post_date": "06/13/2023 02:38:01",
      "content": "<p>Scoring error same for me! My submission cannot get score since last night, just running and running over 15h by now!</p>",
      "votes": null,
      "replies": [
        {
          "id": 2300062,
          "author_name": "gilgarad",
          "author_url": "",
          "post_date": "06/13/2023 02:48:32",
          "content": "<p>did you try to submit again with the ones you already succeeded?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2300066,
      "author_name": "occccn",
      "author_url": "",
      "post_date": "06/13/2023 02:50:47",
      "content": "<p>I'm experiencing the same issue. The scoring has been running for more than 12 hours.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2300069,
          "author_name": "brook1114",
          "author_url": "",
          "post_date": "06/13/2023 02:52:13",
          "content": "<p>Really exhausted in this competition…</p>",
          "votes": null,
          "replies": [
            {
              "id": 2300082,
              "author_name": "occccn",
              "author_url": "",
              "post_date": "06/13/2023 03:04:42",
              "content": "<p>I’m with you…</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2300188,
                  "author_name": "gilgarad",
                  "author_url": "",
                  "post_date": "06/13/2023 04:35:33",
                  "content": "<p>Did you check if rerun the ones you've succeeded not working as well??</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2300243,
                      "author_name": "occccn",
                      "author_url": "",
                      "post_date": "06/13/2023 05:15:37",
                      "content": "<p>I'm not trying it.<br>\nIn the end, I received an unfamiliar message called \"Kaggle error.\"</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2300209,
      "author_name": "jimmyliao86204",
      "author_url": "",
      "post_date": "06/13/2023 04:53:14",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6189766%2F1887dab6d6adf2935c07b9fac0590f33%2F.PNG?generation=1686631611118609&amp;alt=media\" alt=\"\"></p>\n<p>I have.<br>\nIf you have submitted a successful submission, but after re-scored, it becomes a submission scoring error.<br>\nThat's most likely a bug in your original submit script that wasn't shown as a failure (at least I did)<br>\nAfter I found those bug and modified them, I can submit normally.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2300577,
          "author_name": "gilgarad",
          "author_url": "",
          "post_date": "06/13/2023 09:09:37",
          "content": "<p>I think your case might be different from my case…</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2300277,
      "author_name": "gilgarad",
      "author_url": "",
      "post_date": "06/13/2023 05:38:18",
      "content": "<p>Hey guys, I've found my error case. Check if the case is applicable to yours. Thanks.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2300049": "Is it just me who's suffering from submission scoring error right now??\n\nWill this be re-scored again when the error's fixed? Or do I have to submit it again?\n\nI do not want to waste my submission try...\n\nI'm currently using jo wilder 3.7\n\n\n----\nI found out the problem and I think it may help you.\n\nWhile predicting and submitting, the data only comes with one session id. I was making some meta data with certain columns, if this column is under some condition. And I realized that in my train / test set, there's a bunch of other session id so that the feature columns will still remains even with nan values. But if featurizing with a single session id with the condition not applicable, those features disappeared.\n\nConsider this case\n\n\n```python\ncolumn = 'some_column'\n\ncond = df[column] != 0\ngb = df[cond].groupby(['session_id', 'level_group'])\nfeature = gb['elapsed_time'].mean()\n```\n\n\n\nIf cond is all False then feature will not be correctly generated. This happens while running jo_wilder because there is only one session id each time. \n\nI hope this can help.",
    "2300055": "Scoring error same for me! My submission cannot get score since last night, just running and running over 15h by now!",
    "2300062": "did you try to submit again with the ones you already succeeded?",
    "2300066": "I'm experiencing the same issue. The scoring has been running for more than 12 hours.",
    "2300069": "Really exhausted in this competition...",
    "2300082": "I’m with you...",
    "2300188": "Did you check if rerun the ones you've succeeded not working as well??",
    "2300209": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6189766%2F1887dab6d6adf2935c07b9fac0590f33%2F.PNG?generation=1686631611118609&alt=media)\n\nI have.\nIf you have submitted a successful submission, but after re-scored, it becomes a submission scoring error.\nThat's most likely a bug in your original submit script that wasn't shown as a failure (at least I did)\nAfter I found those bug and modified them, I can submit normally.",
    "2300243": "I'm not trying it.\nIn the end, I received an unfamiliar message called \"Kaggle error.\"",
    "2300277": "Hey guys, I've found my error case. Check if the case is applicable to yours. Thanks.",
    "2300577": "I think your case might be different from my case..."
  },
  "source": "meta"
}