![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F933480%2Fc0e4f503358f71fa75e4958d7b0ca5c9%2FScreen%20Shot%202020-12-20%20at%207.56.21%20AM.png?generation=1608473002212557&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F933480%2Fd5de5988313f08d3787ebc95a557ef8b%2FScreen%20Shot%202020-12-20%20at%207.56.30%20AM.png?generation=1608473014102214&alt=media\")
When I attempt to generate these features from the dataset, the distributions look n-o-t-h-i-n-g like this. They are MUCH longer tailed, with less prominent peaks, even after doing the //1000 and //60000. I assume the units of timestamp = milliseconds. Are others experiencing the same thing? I am being careful to take into consideration the difference between lectures and questions for these features, e.g. lag time is forward filled onto questions before removing exercises.
", "messages": [ { "id": "1119981", "postDate": "12/20/2020 14:07:30", "content": "
When I attempt to generate these features from the dataset, the distributions look n-o-t-h-i-n-g like this. They are MUCH longer tailed, with less prominent peaks, even after doing the //1000 and //60000. I assume the units of timestamp = milliseconds. Are others experiencing the same thing? I am being careful to take into consideration the difference between lectures and questions for these features, e.g. lag time is forward filled onto questions before removing exercises.
", "rawMarkdown": "\n\n \n\nWhen I attempt to generate these features from the dataset, the distributions look n-o-t-h-i-n-g like this. They are MUCH longer tailed, with less prominent peaks, even after doing the //1000 and //60000. I assume the units of timestamp = milliseconds. Are others experiencing the same thing? I am being careful to take into consideration the difference between lectures and questions for these features, e.g. lag time is forward filled onto questions before removing exercises.", "votes": null }, { "id": "1120060", "postDate": "12/20/2020 15:07:19", "content": "This is what my distributions look like on aa sample of 1M Q's when I exclude ==0 and ==300 because there are a lot of values peaked there:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F933480%2F26cb2a65491d90592abe853510185c29%2Fmydistros.png?generation=1608476837488455&alt=media\")
I just realized something. In the SAINT+ paper, elapsed time is the delta between when a question is answered and when the prompt of the question is first shown. This is evidenced not by the text of the paper, but by the very figure I shared above. It has nothing to do with lectures whatsoever, and furthermore, is not calculable given the granularity of the dataset we currently have. A few people have mentioned they implemented SAINT+ as-is while reporting CV/LB scores—did you guys miss this, or did you guys have different interpretations of the above features? Similarly, lag time, as defined by the paper would be the delta between the completion of one question and when the next question's prompt is first shown. We don't have the later, so again, this feature in it's SAINT+ form isn't computable given our dataset.
Timestamp being defined as the time in milliseconds between this user interaction and the first event completion from that user, which starts at 0, the best we can do is calculate deltas between question completions.
\n\nTimestamp being defined as the time in milliseconds between this user interaction and the first event completion from that user, which starts at 0…
\n
What we are given for each user is the first TS (let's say reg time) subtracted from each TS further (future interactions) is my current understanding.
", "rawMarkdown": ">Timestamp being defined as the time in milliseconds between this user interaction and the first event completion from that user, which starts at 0...\n\nWhat we are given for each user is the first TS (let's say reg time) subtracted from each TS further (future interactions) is my current understanding.", "votes": null }, { "id": "1120225", "postDate": "12/20/2020 17:19:24", "content": "The 'un-calculable' of lag time as exactly in the paper is one reason I hesitated for so long to do it, but finally I just decided to do something similar (so lag time in some sense, but not the same as in the paper).
", "rawMarkdown": "The 'un-calculable' of lag time as exactly in the paper is one reason I hesitated for so long to do it, but finally I just decided to do something similar (so lag time in some sense, but not the same as in the paper).", "votes": null }, { "id": "1120250", "postDate": "12/20/2020 17:35:14", "content": "@yihdarshieh Thank you for the confirmation. If you don't mind me prying: in your implementation, do you discretize and embed your lag variable—or do you transform it with, e.g., a linear layer?
", "rawMarkdown": "yihdarshieh Thank you for the confirmation. If you don't mind me prying: in your implementation, do you discretize and embed your lag variable—or do you transform it with, e.g., a linear layer?", "votes": null }, { "id": "1120701", "postDate": "12/21/2020 04:09:04", "content": "I have not read original paper, but the distribution of elapsed time you show seems to be different from that of prior_question_elapsed_time in this EDA notebook.
\nThe distribution in EDA seems to be similar to that of Fig. 3 you show.
\nCorrect me if I am wrong.
@yihdarshieh as per above time diagram..
\nI am unable to understand how we can calculate the Lag time which i think currently kinda approximated as Ts2-Ts1
The we can calculate for the current container a feature: prior_container_lag_time = prior_container_ts_diff - prior_container_elapsed_time * prior_container_num_items
\nbelow is my understanding looking at diagram above
\nE1- start time T1 - We dont know\nR1 (Response submitted time) - TIme-T2 - We know it as TS1\n\nLagtime Lt1- To calculate\n\nE2-Start TIme T3- We dont know\n\nR2-Stop Time T4 - We know TS2\nThis is what my distributions look like on aa sample of 1M Q's when I exclude ==0 and ==300 because there are a lot of values peaked there:
\n
I have not read original paper, but the distribution of elapsed time you show seems to be different from that of prior_question_elapsed_time in this EDA notebook.
\nThe distribution in EDA seems to be similar to that of Fig. 3 you show.
\nCorrect me if I am wrong.
I just realized something. In the SAINT+ paper, elapsed time is the delta between when a question is answered and when the prompt of the question is first shown. This is evidenced not by the text of the paper, but by the very figure I shared above. It has nothing to do with lectures whatsoever, and furthermore, is not calculable given the granularity of the dataset we currently have. A few people have mentioned they implemented SAINT+ as-is while reporting CV/LB scores—did you guys miss this, or did you guys have different interpretations of the above features? Similarly, lag time, as defined by the paper would be the delta between the completion of one question and when the next question's prompt is first shown. We don't have the later, so again, this feature in it's SAINT+ form isn't computable given our dataset.
Timestamp being defined as the time in milliseconds between this user interaction and the first event completion from that user, which starts at 0, the best we can do is calculate deltas between question completions.
\n\nTimestamp being defined as the time in milliseconds between this user interaction and the first event completion from that user, which starts at 0…
\n
What we are given for each user is the first TS (let's say reg time) subtracted from each TS further (future interactions) is my current understanding.
", "votes": null, "replies": [] }, { "id": 1120225, "author_name": "yihdarshieh", "author_url": "", "post_date": "12/20/2020 17:19:24", "content": "The 'un-calculable' of lag time as exactly in the paper is one reason I hesitated for so long to do it, but finally I just decided to do something similar (so lag time in some sense, but not the same as in the paper).
", "votes": null, "replies": [] }, { "id": 1120250, "author_name": "authman", "author_url": "", "post_date": "12/20/2020 17:35:14", "content": "@yihdarshieh Thank you for the confirmation. If you don't mind me prying: in your implementation, do you discretize and embed your lag variable—or do you transform it with, e.g., a linear layer?
", "votes": null, "replies": [] }, { "id": 1132802, "author_name": "jaideepvalani", "author_url": "", "post_date": "12/30/2020 17:40:26", "content": "@yihdarshieh as per above time diagram..
\nI am unable to understand how we can calculate the Lag time which i think currently kinda approximated as Ts2-Ts1
The we can calculate for the current container a feature: prior_container_lag_time = prior_container_ts_diff - prior_container_elapsed_time * prior_container_num_items
\nbelow is my understanding looking at diagram above
\nE1- start time T1 - We dont know\nR1 (Response submitted time) - TIme-T2 - We know it as TS1\n\nLagtime Lt1- To calculate\n\nE2-Start TIme T3- We dont know\n\nR2-Stop Time T4 - We know TS2\n