At the end of this competition, I tried to include the count and correctness ration as features for my transformer model. More precisely, for each user:
\nAdding these performance features took me a lot of time, and I got quite boost on CV.
\nHowever, in order to use it, I need to optimized the time / memory.
\nAfter all the efforts, it can't give the same improvement as my CV (running in Tensorflow dataset pipeline) when I run it with my submission pipeline.
A lot of bugs fixed, now I can't see anymore the input inconsistency, but the result is still bad. I am exhausted for now, but I am wondering if anyone makes it work …?
", "messages": [ { "id": "1143590", "postDate": "01/08/2021 00:48:34", "content": "At the end of this competition, I tried to include the count and correctness ration as features for my transformer model. More precisely, for each user:
\nAdding these performance features took me a lot of time, and I got quite boost on CV.
\nHowever, in order to use it, I need to optimized the time / memory.
\nAfter all the efforts, it can't give the same improvement as my CV (running in Tensorflow dataset pipeline) when I run it with my submission pipeline.
A lot of bugs fixed, now I can't see anymore the input inconsistency, but the result is still bad. I am exhausted for now, but I am wondering if anyone makes it work …?
", "rawMarkdown": "At the end of this competition, I tried to include the count and correctness ration as features for my transformer model. More precisely, for each user:\n\n- The number of questions seen / The number of questions correctly answered (as ratio)\n- For each part, again, the count and the correctness ratio\n- For each actual answer (of questions), the count and correctness ratio\n- For each question at the current time to predict, the number of previously seen it and correctly answered\n\nAdding these performance features took me a lot of time, and I got quite boost on CV.\nHowever, in order to use it, I need to optimized the time / memory.\nAfter all the efforts, it can't give the same improvement as my CV (running in Tensorflow dataset pipeline) when I run it with my submission pipeline.\n\nA lot of bugs fixed, now I can't see anymore the input inconsistency, but the result is still bad. I am exhausted for now, but I am wondering if anyone makes it work ...?", "votes": null }, { "id": "1143594", "postDate": "01/08/2021 00:52:51", "content": "I have used question accuracy, part accuracy, lsi accuracy, question lagtime, question elapsed time, user responses as part of my model aside from default features in SAINT+ model. I think they give me good boost. Without hyperparameter tuning I was able to get 0.782 CV/LB. I did used quantile transform to make sure that NNs converge faster. I have discussed it in my writeup.
", "rawMarkdown": "I have used question accuracy, part accuracy, lsi accuracy, question lagtime, question elapsed time, user responses as part of my model aside from default features in SAINT+ model. I think they give me good boost. Without hyperparameter tuning I was able to get 0.782 CV/LB. I did used quantile transform to make sure that NNs converge faster. I have discussed it [in my writeup](https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/209599).", "votes": null } ], "comments": [ { "id": 1143594, "author_name": "manikanthr5", "author_url": "", "post_date": "01/08/2021 00:52:51", "content": "I have used question accuracy, part accuracy, lsi accuracy, question lagtime, question elapsed time, user responses as part of my model aside from default features in SAINT+ model. I think they give me good boost. Without hyperparameter tuning I was able to get 0.782 CV/LB. I did used quantile transform to make sure that NNs converge faster. I have discussed it in my writeup.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1143590": "At the end of this competition, I tried to include the count and correctness ration as features for my transformer model. More precisely, for each user:\n\n- The number of questions seen / The number of questions correctly answered (as ratio)\n- For each part, again, the count and the correctness ratio\n- For each actual answer (of questions), the count and correctness ratio\n- For each question at the current time to predict, the number of previously seen it and correctly answered\n\nAdding these performance features took me a lot of time, and I got quite boost on CV.\nHowever, in order to use it, I need to optimized the time / memory.\nAfter all the efforts, it can't give the same improvement as my CV (running in Tensorflow dataset pipeline) when I run it with my submission pipeline.\n\nA lot of bugs fixed, now I can't see anymore the input inconsistency, but the result is still bad. I am exhausted for now, but I am wondering if anyone makes it work ...?", "1143594": "I have used question accuracy, part accuracy, lsi accuracy, question lagtime, question elapsed time, user responses as part of my model aside from default features in SAINT+ model. I think they give me good boost. Without hyperparameter tuning I was able to get 0.782 CV/LB. I did used quantile transform to make sure that NNs converge faster. I have discussed it [in my writeup](https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/209599)." }, "source": "meta" }