{
  "id": 209911,
  "title": "Can Riiid! benefit from this Challenge?",
  "url": "/competitions/riiid-test-answer-prediction/discussion/209911",
  "author_name": "Mike L.",
  "post_date": "2021-01-09T04:53:16.877000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Thank you Kaggle for hosting such an interesting challenge.</p>\n<p>The Riiid! type of data is becoming more  common as online learning expands. Typically, questions are asked of students that are somewhat easy, so that the students have a 70%-80% chance of success. If the questions are made much harder, then students tend to quit or guess. If they are much easier, then students become careless and may not take learning seriously.</p>\n<p>For these data, using a simple \"Rasch\" prediction model, the chance of student success was around 70% (AUC = 0.7570), the bottom end of the typical range. The top of the Leaderboard, keetar (AUC = 0.820), raised this prediction to 75.4% (assuming equal success predicting right and wrong answers) - the middle of the 70%-80% range. Either way, Riiid! can be reassured that the difficulty level of their questions is appropriate for these students.</p>\n<p>We did discover that Riiid!'s question selection algorithm is problematic. It selects the same question for some students many times. Riiid! could limit this by avoiding asking a question that has already been answered correctly by the student.</p>\n<p>Does anyone have other insights or advice for Riiid! ?</p>",
  "messages": [
    {
      "id": 1145353,
      "postDate": "2021-01-09T04:53:16.877Z",
      "content": "<p>Thank you Kaggle for hosting such an interesting challenge.</p>\n<p>The Riiid! type of data is becoming more  common as online learning expands. Typically, questions are asked of students that are somewhat easy, so that the students have a 70%-80% chance of success. If the questions are made much harder, then students tend to quit or guess. If they are much easier, then students become careless and may not take learning seriously.</p>\n<p>For these data, using a simple \"Rasch\" prediction model, the chance of student success was around 70% (AUC = 0.7570), the bottom end of the typical range. The top of the Leaderboard, keetar (AUC = 0.820), raised this prediction to 75.4% (assuming equal success predicting right and wrong answers) - the middle of the 70%-80% range. Either way, Riiid! can be reassured that the difficulty level of their questions is appropriate for these students.</p>\n<p>We did discover that Riiid!'s question selection algorithm is problematic. It selects the same question for some students many times. Riiid! could limit this by avoiding asking a question that has already been answered correctly by the student.</p>\n<p>Does anyone have other insights or advice for Riiid! ?</p>",
      "rawMarkdown": "Thank you Kaggle for hosting such an interesting challenge.\n\nThe Riiid! type of data is becoming more  common as online learning expands. Typically, questions are asked of students that are somewhat easy, so that the students have a 70%-80% chance of success. If the questions are made much harder, then students tend to quit or guess. If they are much easier, then students become careless and may not take learning seriously.\n\nFor these data, using a simple \"Rasch\" prediction model, the chance of student success was around 70% (AUC = 0.7570), the bottom end of the typical range. The top of the Leaderboard, keetar (AUC = 0.820), raised this prediction to 75.4% (assuming equal success predicting right and wrong answers) - the middle of the 70%-80% range. Either way, Riiid! can be reassured that the difficulty level of their questions is appropriate for these students.\n\nWe did discover that Riiid!'s question selection algorithm is problematic. It selects the same question for some students many times. Riiid! could limit this by avoiding asking a question that has already been answered correctly by the student.\n\nDoes anyone have other insights or advice for Riiid! ?",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1145353": "Thank you Kaggle for hosting such an interesting challenge.\n\nThe Riiid! type of data is becoming more  common as online learning expands. Typically, questions are asked of students that are somewhat easy, so that the students have a 70%-80% chance of success. If the questions are made much harder, then students tend to quit or guess. If they are much easier, then students become careless and may not take learning seriously.\n\nFor these data, using a simple \"Rasch\" prediction model, the chance of student success was around 70% (AUC = 0.7570), the bottom end of the typical range. The top of the Leaderboard, keetar (AUC = 0.820), raised this prediction to 75.4% (assuming equal success predicting right and wrong answers) - the middle of the 70%-80% range. Either way, Riiid! can be reassured that the difficulty level of their questions is appropriate for these students.\n\nWe did discover that Riiid!'s question selection algorithm is problematic. It selects the same question for some students many times. Riiid! could limit this by avoiding asking a question that has already been answered correctly by the student.\n\nDoes anyone have other insights or advice for Riiid! ?"
  }
}