{
  "id": 191417,
  "title": "Submission Scoring Error",
  "url": "/competitions/riiid-test-answer-prediction/discussion/191417",
  "author_name": "",
  "post_date": "2020-10-16T11:23:28.390886600Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi everyone,</p>\n<p>I'm trying to get started with the challenge. I made a notebook containing a few simple features and an untuned model, just to get a baseline submission in there. However, I don't think I understand riiideducation well enough. I keep getting 'Submission Scoring Error' each time I submit the notebook. The only 2 features used are the % of questions answered right from the training set, and the % of questions answered right by a user in the training set. From the way my functions work, it should also work when a user in the test is non existent in the training set, as well for a question.</p>\n<p>Can anyone help me out? I added the notebook to the attachments and the following <a href=\"https://www.kaggle.com/jesseschouten/model\" target=\"_blank\">link</a>. If anyone has any ideas, or perhaps questions, please let me know, I would really like to get this out of the way.</p>\n<p>[EDIT] I just found out that this <a href=\"https://www.kaggle.com/jesseschouten/model-new\" target=\"_blank\">notebook</a> has the same error. In this notebook, I don't do any feature engineering and the model contains a single raw feature from the dataset. What is going on?! </p>",
  "messages": [
    {
      "id": "1051309",
      "postDate": "10/16/2020 11:23:28",
      "content": "<p>Hi everyone,</p>\n<p>I'm trying to get started with the challenge. I made a notebook containing a few simple features and an untuned model, just to get a baseline submission in there. However, I don't think I understand riiideducation well enough. I keep getting 'Submission Scoring Error' each time I submit the notebook. The only 2 features used are the % of questions answered right from the training set, and the % of questions answered right by a user in the training set. From the way my functions work, it should also work when a user in the test is non existent in the training set, as well for a question.</p>\n<p>Can anyone help me out? I added the notebook to the attachments and the following <a href=\"https://www.kaggle.com/jesseschouten/model\" target=\"_blank\">link</a>. If anyone has any ideas, or perhaps questions, please let me know, I would really like to get this out of the way.</p>\n<p>[EDIT] I just found out that this <a href=\"https://www.kaggle.com/jesseschouten/model-new\" target=\"_blank\">notebook</a> has the same error. In this notebook, I don't do any feature engineering and the model contains a single raw feature from the dataset. What is going on?! </p>",
      "rawMarkdown": "Hi everyone,\n\nI'm trying to get started with the challenge. I made a notebook containing a few simple features and an untuned model, just to get a baseline submission in there. However, I don't think I understand riiideducation well enough. I keep getting 'Submission Scoring Error' each time I submit the notebook. The only 2 features used are the % of questions answered right from the training set, and the % of questions answered right by a user in the training set. From the way my functions work, it should also work when a user in the test is non existent in the training set, as well for a question.\n\nCan anyone help me out? I added the notebook to the attachments and the following [link](https://www.kaggle.com/jesseschouten/model). If anyone has any ideas, or perhaps questions, please let me know, I would really like to get this out of the way.\n\n[EDIT] I just found out that this [notebook](https://www.kaggle.com/jesseschouten/model-new) has the same error. In this notebook, I don't do any feature engineering and the model contains a single raw feature from the dataset. What is going on?!",
      "votes": null
    },
    {
      "id": "1051967",
      "postDate": "10/17/2020 05:46:36",
      "content": "<p>I wish I had an idea. I took the code from a successful submission, copied it into a new notebook (just exchanged the prediction model) and boom it doesn't work anymore. I have genuinely no idea.<br>\nIf it fails rather quickly, then there's probably something causing an exception in your submission loop…</p>",
      "rawMarkdown": "I wish I had an idea. I took the code from a successful submission, copied it into a new notebook (just exchanged the prediction model) and boom it doesn't work anymore. I have genuinely no idea.\nIf it fails rather quickly, then there's probably something causing an exception in your submission loop...",
      "votes": null
    },
    {
      "id": "1052388",
      "postDate": "10/17/2020 17:00:16",
      "content": "<p>Its been a frustrating experience trying to find out reasons of Submission scoring error? I am still trying to debug the reason for my entries with no luck 😂 </p>",
      "rawMarkdown": "Its been a frustrating experience trying to find out reasons of Submission scoring error? I am still trying to debug the reason for my entries with no luck 😂",
      "votes": null
    },
    {
      "id": "1052396",
      "postDate": "10/17/2020 17:08:57",
      "content": "<p>I just got one to work. It has a <code>try...catch</code> block around the preprocessing but still scores quite well, so there might be a weird group which breaks my code in there…</p>",
      "rawMarkdown": "I just got one to work. It has a `try...catch` block around the preprocessing but still scores quite well, so there might be a weird group which breaks my code in there...",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1051967,
      "author_name": "spacelx",
      "author_url": "",
      "post_date": "10/17/2020 05:46:36",
      "content": "<p>I wish I had an idea. I took the code from a successful submission, copied it into a new notebook (just exchanged the prediction model) and boom it doesn't work anymore. I have genuinely no idea.<br>\nIf it fails rather quickly, then there's probably something causing an exception in your submission loop…</p>",
      "votes": null,
      "replies": [
        {
          "id": 1052388,
          "author_name": "sandy1112",
          "author_url": "",
          "post_date": "10/17/2020 17:00:16",
          "content": "<p>Its been a frustrating experience trying to find out reasons of Submission scoring error? I am still trying to debug the reason for my entries with no luck 😂 </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1052396,
          "author_name": "spacelx",
          "author_url": "",
          "post_date": "10/17/2020 17:08:57",
          "content": "<p>I just got one to work. It has a <code>try...catch</code> block around the preprocessing but still scores quite well, so there might be a weird group which breaks my code in there…</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1051309": "Hi everyone,\n\nI'm trying to get started with the challenge. I made a notebook containing a few simple features and an untuned model, just to get a baseline submission in there. However, I don't think I understand riiideducation well enough. I keep getting 'Submission Scoring Error' each time I submit the notebook. The only 2 features used are the % of questions answered right from the training set, and the % of questions answered right by a user in the training set. From the way my functions work, it should also work when a user in the test is non existent in the training set, as well for a question.\n\nCan anyone help me out? I added the notebook to the attachments and the following [link](https://www.kaggle.com/jesseschouten/model). If anyone has any ideas, or perhaps questions, please let me know, I would really like to get this out of the way.\n\n[EDIT] I just found out that this [notebook](https://www.kaggle.com/jesseschouten/model-new) has the same error. In this notebook, I don't do any feature engineering and the model contains a single raw feature from the dataset. What is going on?!",
    "1051967": "I wish I had an idea. I took the code from a successful submission, copied it into a new notebook (just exchanged the prediction model) and boom it doesn't work anymore. I have genuinely no idea.\nIf it fails rather quickly, then there's probably something causing an exception in your submission loop...",
    "1052388": "Its been a frustrating experience trying to find out reasons of Submission scoring error? I am still trying to debug the reason for my entries with no luck 😂",
    "1052396": "I just got one to work. It has a `try...catch` block around the preprocessing but still scores quite well, so there might be a weird group which breaks my code in there..."
  },
  "source": "meta"
}