Hi, I am trying to train an RNN, but I am running into the problem that the data is not perfectly ordered. Here is what the test data looks like for the first iteration:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4282397%2F290bf343e1ac48b8c18f011cdf227248%2FScreenshot%20from%202020-11-09%2014-19-44.png?generation=1604960409533167&alt=media\")
\nAs you can see, the first student does have the first question shown, but every student afterwards does not. If they were perfectly ordered, it wouldn't be hard to create an RNN, but since they aren't this makes it impossible, since you don't know how many questions are before the current one. Is there a specific reason why this is?
Hi, I am trying to train an RNN, but I am running into the problem that the data is not perfectly ordered. Here is what the test data looks like for the first iteration:
\n
\nAs you can see, the first student does have the first question shown, but every student afterwards does not. If they were perfectly ordered, it wouldn't be hard to create an RNN, but since they aren't this makes it impossible, since you don't know how many questions are before the current one. Is there a specific reason why this is?
I'm not sure what you mean by first question. As far as I understand, every user interaction is chronologically ordered. Because the test has different users in a single group, it just looks like the timestamps are in random order. Also, keep in mind that the test data chronologically continues after the train dataset, meaning that users in the train dataset will appear with timestamps greater than their latest timestamp in the test dataset.
", "rawMarkdown": "I'm not sure what you mean by first question. As far as I understand, every user interaction is chronologically ordered. Because the test has different users in a single group, it just looks like the timestamps are in random order. Also, keep in mind that the test data chronologically continues after the train dataset, meaning that users in the train dataset will appear with timestamps greater than their latest timestamp in the test dataset.", "votes": null }, { "id": "1075268", "postDate": "11/11/2020 14:34:54", "content": "Otherwise it wouldn't be fun :)
", "rawMarkdown": "Otherwise it wouldn't be fun :)", "votes": null }, { "id": "1076644", "postDate": "11/12/2020 18:00:09", "content": "If the data is chronologically ordered why aren't the first few groups all just nan for the prior_question_elapsed_time? Doesn't this mean that there are rows for the users in the group that are before the data presented in the dataset?
Yes. There are users that have not been presented in the train dataset, and their first appearance is within the test dataset. Notice that for other users, prior_question_elapsed_time is not NaN.
", "rawMarkdown": "Yes. There are users that have not been presented in the train dataset, and their first appearance is within the test dataset. Notice that for other users, prior_question_elapsed_time is not NaN.", "votes": null } ], "comments": [ { "id": 1074848, "author_name": "wingtail", "author_url": "", "post_date": "11/11/2020 06:37:46", "content": "I'm not sure what you mean by first question. As far as I understand, every user interaction is chronologically ordered. Because the test has different users in a single group, it just looks like the timestamps are in random order. Also, keep in mind that the test data chronologically continues after the train dataset, meaning that users in the train dataset will appear with timestamps greater than their latest timestamp in the test dataset.
", "votes": null, "replies": [ { "id": 1076644, "author_name": "fatehaliyev", "author_url": "", "post_date": "11/12/2020 18:00:09", "content": "If the data is chronologically ordered why aren't the first few groups all just nan for the prior_question_elapsed_time? Doesn't this mean that there are rows for the users in the group that are before the data presented in the dataset?
Yes. There are users that have not been presented in the train dataset, and their first appearance is within the test dataset. Notice that for other users, prior_question_elapsed_time is not NaN.
", "votes": null, "replies": [] } ] }, { "id": 1075268, "author_name": "anonymerspastiker", "author_url": "", "post_date": "11/11/2020 14:34:54", "content": "Otherwise it wouldn't be fun :)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1073755": "Hi, I am trying to train an RNN, but I am running into the problem that the data is not perfectly ordered. Here is what the test data looks like for the first iteration:\n\nAs you can see, the first student does have the first question shown, but every student afterwards does not. If they were perfectly ordered, it wouldn't be hard to create an RNN, but since they aren't this makes it impossible, since you don't know how many questions are before the current one. Is there a specific reason why this is?", "1074848": "I'm not sure what you mean by first question. As far as I understand, every user interaction is chronologically ordered. Because the test has different users in a single group, it just looks like the timestamps are in random order. Also, keep in mind that the test data chronologically continues after the train dataset, meaning that users in the train dataset will appear with timestamps greater than their latest timestamp in the test dataset.", "1075268": "Otherwise it wouldn't be fun :)", "1076644": "If the data is chronologically ordered why aren't the first few groups all just `nan` for the `prior_question_elapsed_time`? Doesn't this mean that there are rows for the users in the group that are before the data presented in the dataset?", "1077182": "Yes. There are users that have not been presented in the train dataset, and their first appearance is within the test dataset. Notice that for other users, prior_question_elapsed_time is not NaN." }, "source": "meta" }