{
  "id": 205212,
  "title": "How info of \"prior_question_had_explanation\" is collected in SANTA app?",
  "url": "/competitions/riiid-test-answer-prediction/discussion/205212",
  "author_name": "",
  "post_date": "2020-12-19T02:48:15.974920800Z",
  "votes": 5,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I'm trying to understand <code>prior_question_had_explanation</code> via actual app. I thought there would be option like \"Do you want to see explanation? Yes/No\" after answering every question.<br>\nBut seeing picture below, explanation comes up every answering. This seemingly suggests you can see explanation regardless of user's consent. </p>\n<p>So, I'm wondering what is the criteria as to <code>whether or not the user saw an explanation and the correct response(s) after answering the previous question bundle</code>. In other words, how can we say this user actually saw an explanation?</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2182051%2Fa1048b72dd14d3cc431e49e3d51a1d01%2FScreen%20Shot%202020-12-19%20at%2011.33.45.png?generation=1608345253959459&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": "1118378",
      "postDate": "12/19/2020 02:48:15",
      "content": "<p>I'm trying to understand <code>prior_question_had_explanation</code> via actual app. I thought there would be option like \"Do you want to see explanation? Yes/No\" after answering every question.<br>\nBut seeing picture below, explanation comes up every answering. This seemingly suggests you can see explanation regardless of user's consent. </p>\n<p>So, I'm wondering what is the criteria as to <code>whether or not the user saw an explanation and the correct response(s) after answering the previous question bundle</code>. In other words, how can we say this user actually saw an explanation?</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2182051%2Fa1048b72dd14d3cc431e49e3d51a1d01%2FScreen%20Shot%202020-12-19%20at%2011.33.45.png?generation=1608345253959459&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "I'm trying to understand `prior_question_had_explanation` via actual app. I thought there would be option like \"Do you want to see explanation? Yes/No\" after answering every question.\nBut seeing picture below, explanation comes up every answering. This seemingly suggests you can see explanation regardless of user's consent. \n\nSo, I'm wondering what is the criteria as to `whether or not the user saw an explanation and the correct response(s) after answering the previous question bundle`. In other words, how can we say this user actually saw an explanation?\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2182051%2Fa1048b72dd14d3cc431e49e3d51a1d01%2FScreen%20Shot%202020-12-19%20at%2011.33.45.png?generation=1608345253959459&alt=media)",
      "votes": null
    },
    {
      "id": "1121484",
      "postDate": "12/21/2020 17:15:53",
      "content": "<p>Interesting…Since there are false for this flag, we can only assume some action was taken..maybe if the user scrolled down it is treated as true? </p>",
      "rawMarkdown": "Interesting...Since there are false for this flag, we can only assume some action was taken..maybe if the user scrolled down it is treated as true?",
      "votes": null
    },
    {
      "id": "1126921",
      "postDate": "12/26/2020 04:56:47",
      "content": "<p>From the description for this data field:</p>\n<blockquote>\n  <p>Typically the first several questions a user sees were part of an onboarding diagnostic test where they did not get any feedback.</p>\n</blockquote>\n<p>So, the first few questions for each user do not have an explanation. These are a short \"pre-test\" the user takes so that the app can decide what the user knows/doesn't know already. All other questions typically do have explanations. </p>\n<p>Although I've seen at least one user who seemingly got \"reset\" after a long absence and got a second diagnostic series of questions, that is:</p>\n<ul>\n<li>The user firsts answered some questions with <code>prior_question_had_explanation</code>=0</li>\n<li>The user then answered a few questions with <code>prior_question_had_explanation</code>=1</li>\n<li>The user did not use the app for a while (the gap in timestamps to next question is <strong>very</strong> large)</li>\n<li>The user started seeing questions with <code>prior_question_had_explanation</code>=0 again for a few questions</li>\n</ul>",
      "rawMarkdown": "From the description for this data field:\n\n>  Typically the first several questions a user sees were part of an onboarding diagnostic test where they did not get any feedback.\n\nSo, the first few questions for each user do not have an explanation. These are a short \"pre-test\" the user takes so that the app can decide what the user knows/doesn't know already. All other questions typically do have explanations. \n\nAlthough I've seen at least one user who seemingly got \"reset\" after a long absence and got a second diagnostic series of questions, that is:\n- The user firsts answered some questions with `prior_question_had_explanation`=0\n- The user then answered a few questions with `prior_question_had_explanation`=1\n- The user did not use the app for a while (the gap in timestamps to next question is **very** large)\n- The user started seeing questions with `prior_question_had_explanation`=0 again for a few questions",
      "votes": null
    },
    {
      "id": "1126959",
      "postDate": "12/26/2020 06:03:00",
      "content": "<p><a href=\"https://www.kaggle.com/yanamal\" target=\"_blank\">@yanamal</a> <br>\nYou mean user is supposed to see explanation regardless of their choice? If so, it is intuitive to me, because I also had \"pre-test with no explanation\" when opening app first.</p>",
      "rawMarkdown": "yanamal \nYou mean user is supposed to see explanation regardless of their choice? If so, it is intuitive to me, because I also had \"pre-test with no explanation\" when opening app first.",
      "votes": null
    },
    {
      "id": "1127055",
      "postDate": "12/26/2020 08:01:04",
      "content": "<p>Yes, I'm pretty sure that the user's choice as to whether to swipe down and actually look at the explanation isn't recorded at all in the data. It's more whether they offered it. </p>\n<p>Even if the user doesn't swipe down, from your screenshots, they seem to get more info than what they had when they were answering the question (e.g. what was actually said, in two languages). So if <code>prior_question_had_explanation</code> is 1, then the user got at least <em>some</em> feedback for that question.</p>\n<p>(I also haven't tried using the app, since I don't speak Japanese or Korean, so I wasn't even aware there was an option to ignore the explanation, really).</p>",
      "rawMarkdown": "Yes, I'm pretty sure that the user's choice as to whether to swipe down and actually look at the explanation isn't recorded at all in the data. It's more whether they offered it. \n\nEven if the user doesn't swipe down, from your screenshots, they seem to get more info than what they had when they were answering the question (e.g. what was actually said, in two languages). So if `prior_question_had_explanation` is 1, then the user got at least *some* feedback for that question.\n\n(I also haven't tried using the app, since I don't speak Japanese or Korean, so I wasn't even aware there was an option to ignore the explanation, really).",
      "votes": null
    },
    {
      "id": "1127328",
      "postDate": "12/26/2020 12:53:28",
      "content": "<p>thanks <a href=\"https://www.kaggle.com/yanamal\" target=\"_blank\">@yanamal</a> . nice insights as always</p>",
      "rawMarkdown": "thanks @yanamal . nice insights as always",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1121484,
      "author_name": "allohvk",
      "author_url": "",
      "post_date": "12/21/2020 17:15:53",
      "content": "<p>Interesting…Since there are false for this flag, we can only assume some action was taken..maybe if the user scrolled down it is treated as true? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1126921,
      "author_name": "yanamal",
      "author_url": "",
      "post_date": "12/26/2020 04:56:47",
      "content": "<p>From the description for this data field:</p>\n<blockquote>\n  <p>Typically the first several questions a user sees were part of an onboarding diagnostic test where they did not get any feedback.</p>\n</blockquote>\n<p>So, the first few questions for each user do not have an explanation. These are a short \"pre-test\" the user takes so that the app can decide what the user knows/doesn't know already. All other questions typically do have explanations. </p>\n<p>Although I've seen at least one user who seemingly got \"reset\" after a long absence and got a second diagnostic series of questions, that is:</p>\n<ul>\n<li>The user firsts answered some questions with <code>prior_question_had_explanation</code>=0</li>\n<li>The user then answered a few questions with <code>prior_question_had_explanation</code>=1</li>\n<li>The user did not use the app for a while (the gap in timestamps to next question is <strong>very</strong> large)</li>\n<li>The user started seeing questions with <code>prior_question_had_explanation</code>=0 again for a few questions</li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 1126959,
          "author_name": "ant3ng",
          "author_url": "",
          "post_date": "12/26/2020 06:03:00",
          "content": "<p><a href=\"https://www.kaggle.com/yanamal\" target=\"_blank\">@yanamal</a> <br>\nYou mean user is supposed to see explanation regardless of their choice? If so, it is intuitive to me, because I also had \"pre-test with no explanation\" when opening app first.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1127055,
          "author_name": "yanamal",
          "author_url": "",
          "post_date": "12/26/2020 08:01:04",
          "content": "<p>Yes, I'm pretty sure that the user's choice as to whether to swipe down and actually look at the explanation isn't recorded at all in the data. It's more whether they offered it. </p>\n<p>Even if the user doesn't swipe down, from your screenshots, they seem to get more info than what they had when they were answering the question (e.g. what was actually said, in two languages). So if <code>prior_question_had_explanation</code> is 1, then the user got at least <em>some</em> feedback for that question.</p>\n<p>(I also haven't tried using the app, since I don't speak Japanese or Korean, so I wasn't even aware there was an option to ignore the explanation, really).</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1127328,
          "author_name": "allohvk",
          "author_url": "",
          "post_date": "12/26/2020 12:53:28",
          "content": "<p>thanks <a href=\"https://www.kaggle.com/yanamal\" target=\"_blank\">@yanamal</a> . nice insights as always</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1118378": "I'm trying to understand `prior_question_had_explanation` via actual app. I thought there would be option like \"Do you want to see explanation? Yes/No\" after answering every question.\nBut seeing picture below, explanation comes up every answering. This seemingly suggests you can see explanation regardless of user's consent. \n\nSo, I'm wondering what is the criteria as to `whether or not the user saw an explanation and the correct response(s) after answering the previous question bundle`. In other words, how can we say this user actually saw an explanation?\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2182051%2Fa1048b72dd14d3cc431e49e3d51a1d01%2FScreen%20Shot%202020-12-19%20at%2011.33.45.png?generation=1608345253959459&alt=media)",
    "1121484": "Interesting...Since there are false for this flag, we can only assume some action was taken..maybe if the user scrolled down it is treated as true?",
    "1126921": "From the description for this data field:\n\n>  Typically the first several questions a user sees were part of an onboarding diagnostic test where they did not get any feedback.\n\nSo, the first few questions for each user do not have an explanation. These are a short \"pre-test\" the user takes so that the app can decide what the user knows/doesn't know already. All other questions typically do have explanations. \n\nAlthough I've seen at least one user who seemingly got \"reset\" after a long absence and got a second diagnostic series of questions, that is:\n- The user firsts answered some questions with `prior_question_had_explanation`=0\n- The user then answered a few questions with `prior_question_had_explanation`=1\n- The user did not use the app for a while (the gap in timestamps to next question is **very** large)\n- The user started seeing questions with `prior_question_had_explanation`=0 again for a few questions",
    "1126959": "yanamal \nYou mean user is supposed to see explanation regardless of their choice? If so, it is intuitive to me, because I also had \"pre-test with no explanation\" when opening app first.",
    "1127055": "Yes, I'm pretty sure that the user's choice as to whether to swipe down and actually look at the explanation isn't recorded at all in the data. It's more whether they offered it. \n\nEven if the user doesn't swipe down, from your screenshots, they seem to get more info than what they had when they were answering the question (e.g. what was actually said, in two languages). So if `prior_question_had_explanation` is 1, then the user got at least *some* feedback for that question.\n\n(I also haven't tried using the app, since I don't speak Japanese or Korean, so I wasn't even aware there was an option to ignore the explanation, really).",
    "1127328": "thanks @yanamal . nice insights as always"
  },
  "source": "meta"
}