![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5589062%2Fb25f3e890ad72c5d919fcc03f91d891d%2Fkaggle.png?generation=1602090463430829&alt=media\")
It seems that there is an inconsistency between the timestamp and task_container_id columns in the training set (or my understanding is incorrect). The definitions for both are:
timestamp: (int64) the time between this user interaction and the first event from that user.task_container_id: (int16) Id code for the batch of questions or lectures. For example, a user might see three questions in a row before seeing the explanations for any of them. Those three would all share a task_container_id. Monotonically increasing for each user.From my understanding, task_container_id should start from 0 and monotonically increase with the timestamp. However, it is not the case for some users (e.g. user 115).
It would be great to confirm with the hosts of this competition if this is an actual mistake. In the meantime, I would like to share a fix suggested by @maxhalford from the original discussion.
\n\n\nTo fix this, I suggest renumbering the tasks to make sure they're monotonically increasing for each user:
\n\ntrain['task_container_id'] = (\n train\n .groupby('user_id')['task_container_id']\n .transform(lambda x: pd.factorize(x)[0])\n .astype('int16')\n)\n
I am also interested to know how much applying this fix will impact the training and score so feel free to give your insights in the comments.
", "messages": [ { "id": 1041289, "postDate": "2020-10-07T17:12:00.533Z", "content": "It seems that there is an inconsistency between the timestamp and task_container_id columns in the training set (or my understanding is incorrect). The definitions for both are:
timestamp: (int64) the time between this user interaction and the first event from that user.task_container_id: (int16) Id code for the batch of questions or lectures. For example, a user might see three questions in a row before seeing the explanations for any of them. Those three would all share a task_container_id. Monotonically increasing for each user.From my understanding, task_container_id should start from 0 and monotonically increase with the timestamp. However, it is not the case for some users (e.g. user 115).
It would be great to confirm with the hosts of this competition if this is an actual mistake. In the meantime, I would like to share a fix suggested by @maxhalford from the original discussion.
\n\n\nTo fix this, I suggest renumbering the tasks to make sure they're monotonically increasing for each user:
\n\ntrain['task_container_id'] = (\n train\n .groupby('user_id')['task_container_id']\n .transform(lambda x: pd.factorize(x)[0])\n .astype('int16')\n)\n
I am also interested to know how much applying this fix will impact the training and score so feel free to give your insights in the comments.
", "rawMarkdown": "It seems that there is an inconsistency between the `timestamp` and `task_container_id` columns in the training set (or my understanding is incorrect). The definitions for both are:\n\n- `timestamp`: (int64) the time between this user interaction and the first event from that user.\n- `task_container_id`: (int16) Id code for the batch of questions or lectures. For example, a user might see three questions in a row before seeing the explanations for any of them. Those three would all share a `task_container_id`. Monotonically increasing for each user.\n\nFrom my understanding, `task_container_id` should start from 0 and monotonically increase with the `timestamp`. However, it is not the case for some users (e.g. user 115).\n\n\n\nIt would be great to confirm with the hosts of this competition if this is an actual mistake. In the meantime, I would like to share a fix suggested by @maxhalford from the original discussion.\n\n> To fix this, I suggest renumbering the tasks to make sure they're monotonically increasing for each user:\n> \n> ```python\n> train['task_container_id'] = (\n> train\n> .groupby('user_id')['task_container_id']\n> .transform(lambda x: pd.factorize(x)[0])\n> .astype('int16')\n> )\n> ```\n\nI am also interested to know how much applying this fix will impact the training and score so feel free to give your insights in the comments.", "votes": 33 }, { "id": 1041597, "postDate": "2020-10-07T20:42:15.730Z", "content": "From @sohier in this other thread
\n\n\n\n
\n- The
\ntimestampcolumn shows when an activity is finished, not when it started. A user could have spent an arbitrary amount of time working on the first problem before the firsttimestampvalue would get logged.
To me this suggests that while task_container_id captures the order that a user first sees tasks in, timestamp captures the order in which a user actually completes tasks. A user can start one task, start a second, and then finish the second before finishing the first -- this results in the later timestamp for the first task.
Okay with this additional information on timestamp, it makes sense now! Thanks!
@zhaochow could you help in understanding what do we exactly term as task container id.. is it pre fixed value by learning platfor, like user will be served some xyz question together so they would belong to some task container id , or it is not fixed then on what does it depends on
\nsecondly, how can we related bundle id to task container id if there is any relation between two ?
", "rawMarkdown": "@zhaochow could you help in understanding what do we exactly term as task container id.. is it pre fixed value by learning platfor, like user will be served some xyz question together so they would belong to some task container id , or it is not fixed then on what does it depends on\n\nsecondly, how can we related bundle id to task container id if there is any relation between two ?\n\n" } ] }, { "id": 1041603, "postDate": "2020-10-07T20:46:31.153Z", "content": "I won't have time to dig in to confirm this today, but I believe this is just a data artifact. Imagine someone logging in from both their phone and a tablet; the first batch of questions they started working on (lowest container ID) wouldn't necessarily actually be the first one to finish.
\nGenerally speaking, time tracked across multiple devices is unlikely to be precisely correct unless you take extreme measures like providing each device with an atomic clock .
", "rawMarkdown": "I won't have time to dig in to confirm this today, but I believe this is just a data artifact. Imagine someone logging in from both their phone and a tablet; the first batch of questions they started working on (lowest container ID) wouldn't necessarily actually be the first one to finish.\n\nGenerally speaking, time tracked across multiple devices is unlikely to be precisely correct unless you take extreme measures like providing each device [with an atomic clock ](https://www.wired.com/2012/11/google-spanner-time/).", "votes": 5, "replies": [ { "id": 1041977, "postDate": "2020-10-08T02:06:12.117Z", "content": "Thank you for your comment! If it is just a data artifact, we can thus leave the task_container_id column untouched. It is as @aquatic said, task_container_id captures the order in which a user starts/sees tasks whereas the timestamp captures the order of completion.
You used logging from multiple devices as an example. Just to be sure, could this situation happen even on a single device? For example, users can start a new task before finishing the previous one?
", "rawMarkdown": "Thank you for your comment! If it is just a data artifact, we can thus leave the `task_container_id` column untouched. It is as @aquatic said, `task_container_id` captures the order in which a user starts/sees tasks whereas the `timestamp` captures the order of completion. \n\nYou used logging from multiple devices as an example. Just to be sure, could this situation happen even on a single device? For example, users can start a new task before finishing the previous one?" }, { "id": 1099684, "postDate": "2020-12-02T14:31:59.223Z", "content": "@sohier could you clarify on this doubbt regarding Timestamp.
\nI believe it starts from 0 for every user when they start first their interaction with learning program.
But what i find that Number of Unique Records where TImestamp=0 is differs by Unique user dis by few thousands .Can this be possiility , i expected them to be equal .
", "rawMarkdown": "@sohier could you clarify on this doubbt regarding Timestamp.\nI believe it starts from 0 for every user when they start first their interaction with learning program. \n\nBut what i find that Number of Unique Records where TImestamp=0 is differs by Unique user dis by few thousands .Can this be possiility , i expected them to be equal . ", "votes": 1 } ] }, { "id": 1041604, "postDate": "2020-10-07T20:46:55.660Z", "content": "I guess that users can choose which groups of questions to work on. One user could do task 1, 2, and 4, then comes back to task 3.
", "rawMarkdown": "I guess that users can choose which groups of questions to work on. One user could do task 1, 2, and 4, then comes back to task 3." }, { "id": 1041588, "postDate": "2020-10-07T20:35:05.023Z", "content": "I also found this issue. The host should clarify this …BTW, where is the discussion you mentioned in a fix suggested by @maxhalford from the original discussion. ? Thanks.
Do you mean this discussion?
\nhttps://www.kaggle.com/c/riiid-test-answer-prediction/discussion/189351
Thank you for bringing this issue and I apologize for any inconvenience. Indeed, the non-increasing values of task_container_id is data artifact due to internal workings of the service, and it serves the purpose of distinguishing different bundles without monotonicity assumption. We took this into account by dropping the original note about monotonic increase.
@jineonjinbaek @sohier I understood task_container_id is not monotonically increasing in the training set due to internal workings of the service. Then, is task_container_id not monotonically increasing in the test set? I mean, when you provide a group using generator env.iter_test(), is the sample with the smallest task_container_id always selected first, or is the sample with the smallest timestamp selected first? I'm looking forward to the clarification, thanks!
I have the same question!
", "rawMarkdown": "I have the same question!" }, { "id": 1100420, "postDate": "2020-12-03T04:41:28.057Z", "content": "Hi I am late to the party, has this been clariified in any way?
", "rawMarkdown": "Hi I am late to the party, has this been clariified in any way?", "votes": 1 } ] } ], "comments": [ { "id": 1041597, "author_name": "Joe Eddy", "author_url": "", "post_date": "2020-10-07T20:42:15.730000", "content": "From @sohier in this other thread
\n\n\n\n
\n- The
\ntimestampcolumn shows when an activity is finished, not when it started. A user could have spent an arbitrary amount of time working on the first problem before the firsttimestampvalue would get logged.
To me this suggests that while task_container_id captures the order that a user first sees tasks in, timestamp captures the order in which a user actually completes tasks. A user can start one task, start a second, and then finish the second before finishing the first -- this results in the later timestamp for the first task.
Okay with this additional information on timestamp, it makes sense now! Thanks!
@zhaochow could you help in understanding what do we exactly term as task container id.. is it pre fixed value by learning platfor, like user will be served some xyz question together so they would belong to some task container id , or it is not fixed then on what does it depends on
\nsecondly, how can we related bundle id to task container id if there is any relation between two ?
", "votes": 0, "replies": [] } ] }, { "id": 1041603, "author_name": "Sohier Dane", "author_url": "", "post_date": "2020-10-07T20:46:31.153000", "content": "I won't have time to dig in to confirm this today, but I believe this is just a data artifact. Imagine someone logging in from both their phone and a tablet; the first batch of questions they started working on (lowest container ID) wouldn't necessarily actually be the first one to finish.
\nGenerally speaking, time tracked across multiple devices is unlikely to be precisely correct unless you take extreme measures like providing each device with an atomic clock .
", "votes": 5, "replies": [ { "id": 1041977, "author_name": "Zhao Chow", "author_url": "", "post_date": "2020-10-08T02:06:12.117000", "content": "Thank you for your comment! If it is just a data artifact, we can thus leave the task_container_id column untouched. It is as @aquatic said, task_container_id captures the order in which a user starts/sees tasks whereas the timestamp captures the order of completion.
You used logging from multiple devices as an example. Just to be sure, could this situation happen even on a single device? For example, users can start a new task before finishing the previous one?
", "votes": 0, "replies": [] }, { "id": 1099684, "author_name": "Jaideep", "author_url": "", "post_date": "2020-12-02T14:31:59.223000", "content": "@sohier could you clarify on this doubbt regarding Timestamp.
\nI believe it starts from 0 for every user when they start first their interaction with learning program.
But what i find that Number of Unique Records where TImestamp=0 is differs by Unique user dis by few thousands .Can this be possiility , i expected them to be equal .
", "votes": 1, "replies": [] } ] }, { "id": 1041604, "author_name": "statlearning", "author_url": "", "post_date": "2020-10-07T20:46:55.660000", "content": "I guess that users can choose which groups of questions to work on. One user could do task 1, 2, and 4, then comes back to task 3.
", "votes": 0, "replies": [] }, { "id": 1041588, "author_name": "Yih-Dar SHIEH", "author_url": "", "post_date": "2020-10-07T20:35:05.023000", "content": "I also found this issue. The host should clarify this …BTW, where is the discussion you mentioned in a fix suggested by @maxhalford from the original discussion. ? Thanks.
Do you mean this discussion?
\nhttps://www.kaggle.com/c/riiid-test-answer-prediction/discussion/189351
Thank you for bringing this issue and I apologize for any inconvenience. Indeed, the non-increasing values of task_container_id is data artifact due to internal workings of the service, and it serves the purpose of distinguishing different bundles without monotonicity assumption. We took this into account by dropping the original note about monotonic increase.
@jineonjinbaek @sohier I understood task_container_id is not monotonically increasing in the training set due to internal workings of the service. Then, is task_container_id not monotonically increasing in the test set? I mean, when you provide a group using generator env.iter_test(), is the sample with the smallest task_container_id always selected first, or is the sample with the smallest timestamp selected first? I'm looking forward to the clarification, thanks!
I have the same question!
", "votes": 0, "replies": [] }, { "id": 1100420, "author_name": "Felipe Bivort Haiek", "author_url": "", "post_date": "2020-12-03T04:41:28.057000", "content": "Hi I am late to the party, has this been clariified in any way?
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1041289": "It seems that there is an inconsistency between the `timestamp` and `task_container_id` columns in the training set (or my understanding is incorrect). The definitions for both are:\n\n- `timestamp`: (int64) the time between this user interaction and the first event from that user.\n- `task_container_id`: (int16) Id code for the batch of questions or lectures. For example, a user might see three questions in a row before seeing the explanations for any of them. Those three would all share a `task_container_id`. Monotonically increasing for each user.\n\nFrom my understanding, `task_container_id` should start from 0 and monotonically increase with the `timestamp`. However, it is not the case for some users (e.g. user 115).\n\n\n\nIt would be great to confirm with the hosts of this competition if this is an actual mistake. In the meantime, I would like to share a fix suggested by @maxhalford from the original discussion.\n\n> To fix this, I suggest renumbering the tasks to make sure they're monotonically increasing for each user:\n> \n> ```python\n> train['task_container_id'] = (\n> train\n> .groupby('user_id')['task_container_id']\n> .transform(lambda x: pd.factorize(x)[0])\n> .astype('int16')\n> )\n> ```\n\nI am also interested to know how much applying this fix will impact the training and score so feel free to give your insights in the comments.", "1041597": "From @sohier in this other [thread](https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/189351) \n\n> - The `timestamp` column shows when an activity is finished, not when it started. A user could have spent an arbitrary amount of time working on the first problem before the first `timestamp` value would get logged.\n\nTo me this suggests that while `task_container_id` captures the order that a user **first sees** tasks in, `timestamp` captures the order in which a user actually **completes** tasks. A user can start one task, start a second, and then finish the second before finishing the first -- this results in the later timestamp for the first task.", "1041603": "I won't have time to dig in to confirm this today, but I believe this is just a data artifact. Imagine someone logging in from both their phone and a tablet; the first batch of questions they started working on (lowest container ID) wouldn't necessarily actually be the first one to finish.\n\nGenerally speaking, time tracked across multiple devices is unlikely to be precisely correct unless you take extreme measures like providing each device [with an atomic clock ](https://www.wired.com/2012/11/google-spanner-time/).", "1041604": "I guess that users can choose which groups of questions to work on. One user could do task 1, 2, and 4, then comes back to task 3.", "1041588": "I also found this issue. The host should clarify this ...BTW, where is the discussion you mentioned in ` a fix suggested by @maxhalford from the original discussion.` ? Thanks.", "1047038": "Thank you for bringing this issue and I apologize for any inconvenience. Indeed, the non-increasing values of `task_container_id` is data artifact due to internal workings of the service, and it serves the purpose of distinguishing different bundles without monotonicity assumption. We took this into account by dropping the original note about monotonic increase." } }