{ "id": 392131, "title": "Using previous guesses as a feature 🤔🤔", "url": "/competitions/predict-student-performance-from-game-play/discussion/392131", "author_name": "JEANMPIA", "post_date": "2023-03-03T18:53:48.799000", "votes": 17, "comment_count": 10, "views": 0, "content": "

I guess most of use use 18 different models to predict each question's answer individually,

\n

What do you think of a student that answered the first 17 questions right ?

\n

Easy guess no ? He probably got the 18th right.
\nWe obviously don't know if a student got the last questions right, but don't we have a value for that anyways ?
\nHas anyone tried to implement such a model where lets say you want to predict question i , the model takes into account predictions on questions <i ? If so did he find any luck doing so ?
\nI'm taking a break from this comp hence why I don't try myself but I still wanted to share this, I hope you will find this useful !

", "messages": [ { "id": 2167912, "postDate": "2023-03-03T18:53:48.800Z", "content": "

I guess most of use use 18 different models to predict each question's answer individually,

\n

What do you think of a student that answered the first 17 questions right ?

\n

Easy guess no ? He probably got the 18th right.
\nWe obviously don't know if a student got the last questions right, but don't we have a value for that anyways ?
\nHas anyone tried to implement such a model where lets say you want to predict question i , the model takes into account predictions on questions <i ? If so did he find any luck doing so ?
\nI'm taking a break from this comp hence why I don't try myself but I still wanted to share this, I hope you will find this useful !

", "rawMarkdown": "### I guess most of use use 18 different models to predict each question's answer individually,\nWhat do you think of a **student that answered the first 17 questions right** ?\n\nEasy guess no ? He probably got the 18th right.\nWe obviously don't know if a student got the last questions right, **but don't we have a value for that anyways ?**\nHas anyone tried to implement such a model where lets say you want to predict question `i` , the model takes into account predictions on questions `I do. It works. :)

", "rawMarkdown": "I do. It works. :)", "votes": 7, "replies": [ { "id": 2169248, "postDate": "2023-03-04T23:13:00.710Z", "content": "

Thx for letting everyone know ! I'll definitely add it when i come back to this comp !

", "rawMarkdown": "Thx for letting everyone know ! I'll definitely add it when i come back to this comp !" }, { "id": 2179150, "postDate": "2023-03-12T23:30:30.570Z", "content": "

Hi Dennis, may I ask you how many points you improved in your CV and LB?

", "rawMarkdown": "Hi Dennis, may I ask you how many points you improved in your CV and LB?" } ] }, { "id": 2169988, "postDate": "2023-03-05T15:37:08.863Z", "content": "

by the way, it is a bad competition design when they don't give us actual+correct answers for previous level group. That would probably by the most predictive feature (% correct in all previous level groups), and that information IS available in real world - forcing is to not use it will only lead to sub-optimal predictions.

", "rawMarkdown": "by the way, it is a bad competition design when they don't give us actual+correct answers for previous level group. That would probably by the most predictive feature (% correct in all previous level groups), and that information IS available in real world - forcing is to not use it will only lead to sub-optimal predictions.", "votes": 3, "replies": [ { "id": 2170444, "postDate": "2023-03-06T02:18:43.053Z", "content": "

I think that is the point, they could also tell us informations about the session's location, if it was in class or not, the grade the player selected and many more information that is available in real life. They problably want to see what we can do only with what they gave us here !

", "rawMarkdown": "I think that is the point, they could also tell us informations about the session's location, if it was in class or not, the grade the player selected and many more **information that is available in real life**. They problably want to see what we can do only with what they gave us here !" }, { "id": 2170606, "postDate": "2023-03-06T06:15:20.283Z", "content": "

Fully agree - missing out on the most important data is a bad design. However, if you look at the correlation matrix between different questions, the highest is around 0.25, which suggests that the accuracy of previous answers carry o only limited information as to the accuracy of future answers. But still that would probably be the strongest feature, right after the base rate for every question

", "rawMarkdown": "Fully agree - missing out on the most important data is a bad design. However, if you look at the correlation matrix between different questions, the highest is around 0.25, which suggests that the accuracy of previous answers carry o only limited information as to the accuracy of future answers. But still that would probably be the strongest feature, right after the base rate for every question", "votes": 2 } ] }, { "id": 2188982, "postDate": "2023-03-20T06:22:55.333Z", "content": "

Update: It worked well for me, and the good thing is for this, the better your predictions are, the better they become again. 👍

", "rawMarkdown": "*Update*: It worked well for me, and the good thing is for this, the better your predictions are, the better they become again. 👍", "votes": 1, "replies": [ { "id": 2192059, "postDate": "2023-03-22T11:59:06.817Z", "content": "

Hi, could you give a brief explanation on how it works well? I took all previous predictions as features for the next question. Although it boosts my CV a lot, I don't see any obvious improvement for LB.

", "rawMarkdown": "Hi, could you give a brief explanation on how it works well? I took all previous predictions as features for the next question. Although it boosts my CV a lot, I don't see any obvious improvement for LB. ", "votes": 1, "replies": [ { "id": 2192352, "postDate": "2023-03-22T15:51:39.917Z", "content": "

I can't explain why it didn't help your model sorry since I do'nt know what you are using, but it helped both CV and LB for me, level group per level group accuracy

", "rawMarkdown": "I can't explain why it didn't help your model sorry since I do'nt know what you are using, but it helped both CV and LB for me, level group per level group accuracy " }, { "id": 2274142, "postDate": "2023-05-25T16:43:16.677Z", "rawMarkdown": "", "isDeleted": true } ] } ] } ], "comments": [ { "id": 2168929, "author_name": "DennisSakva", "author_url": "", "post_date": "2023-03-04T16:40:01.233000", "content": "

I do. It works. :)

", "votes": 7, "replies": [ { "id": 2169248, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-03-04T23:13:00.710000", "content": "

Thx for letting everyone know ! I'll definitely add it when i come back to this comp !

", "votes": 0, "replies": [] }, { "id": 2179150, "author_name": "HARRISONLIANG", "author_url": "", "post_date": "2023-03-12T23:30:30.570000", "content": "

Hi Dennis, may I ask you how many points you improved in your CV and LB?

", "votes": 0, "replies": [] } ] }, { "id": 2169988, "author_name": "Youri Matiounine", "author_url": "", "post_date": "2023-03-05T15:37:08.863000", "content": "

by the way, it is a bad competition design when they don't give us actual+correct answers for previous level group. That would probably by the most predictive feature (% correct in all previous level groups), and that information IS available in real world - forcing is to not use it will only lead to sub-optimal predictions.

", "votes": 3, "replies": [ { "id": 2170444, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-03-06T02:18:43.053000", "content": "

I think that is the point, they could also tell us informations about the session's location, if it was in class or not, the grade the player selected and many more information that is available in real life. They problably want to see what we can do only with what they gave us here !

", "votes": 0, "replies": [] }, { "id": 2170606, "author_name": "narsil (jobs-in-data.com)", "author_url": "", "post_date": "2023-03-06T06:15:20.283000", "content": "

Fully agree - missing out on the most important data is a bad design. However, if you look at the correlation matrix between different questions, the highest is around 0.25, which suggests that the accuracy of previous answers carry o only limited information as to the accuracy of future answers. But still that would probably be the strongest feature, right after the base rate for every question

", "votes": 2, "replies": [] } ] }, { "id": 2188982, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-03-20T06:22:55.333000", "content": "

Update: It worked well for me, and the good thing is for this, the better your predictions are, the better they become again. 👍

", "votes": 1, "replies": [ { "id": 2192059, "author_name": "Moonlit", "author_url": "", "post_date": "2023-03-22T11:59:06.817000", "content": "

Hi, could you give a brief explanation on how it works well? I took all previous predictions as features for the next question. Although it boosts my CV a lot, I don't see any obvious improvement for LB.

", "votes": 1, "replies": [ { "id": 2192352, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-03-22T15:51:39.917000", "content": "

I can't explain why it didn't help your model sorry since I do'nt know what you are using, but it helped both CV and LB for me, level group per level group accuracy

", "votes": 0, "replies": [] }, { "id": 2274142, "author_name": "", "author_url": "", "post_date": "2023-05-25T16:43:16.677000", "content": "", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2167912": "### I guess most of use use 18 different models to predict each question's answer individually,\nWhat do you think of a **student that answered the first 17 questions right** ?\n\nEasy guess no ? He probably got the 18th right.\nWe obviously don't know if a student got the last questions right, **but don't we have a value for that anyways ?**\nHas anyone tried to implement such a model where lets say you want to predict question `i` , the model takes into account predictions on questions `