Hi, everyone, I have a question about using the previous answer.
\nAs we know, each user has to answer from question level 1 to level 18, and he can not answer the question afterward if he did not solve the previous one.
\nHere comes the assumption that the performance of the user could be inferred from his or her previous answers. There are mainly two methods for prediction:
Hi, everyone, I have a question about using the previous answer.
\nAs we know, each user has to answer from question level 1 to level 18, and he can not answer the question afterward if he did not solve the previous one.
\nHere comes the assumption that the performance of the user could be inferred from his or her previous answers. There are mainly two methods for prediction:
Both improved my cv but decreased my lb (maybe i did something wrong , I haven’t checked yet!)
", "rawMarkdown": "Both improved my cv but decreased my lb (maybe i did something wrong , I haven’t checked yet!)", "votes": null }, { "id": "2179543", "postDate": "03/13/2023 08:52:10", "content": "Hi, Reacher, may I ask you how many points you improved in your CV?
\nI got similar results by using the second methods.
\nMy cv improved a lot (0.004~0.006) but got little improve in lb (< 0.001).
It seems that the test data is not in order, after commin three data of level 5, it then goes up to level 13, and I cannot predict based on the result of level 6-12!
", "rawMarkdown": "It seems that the test data is not in order, after commin three data of level 5, it then goes up to level 13, and I cannot predict based on the result of level 6-12!", "votes": null }, { "id": "2179699", "postDate": "03/13/2023 10:35:49", "content": "Sorry I did it early in the competition and i don't remember .I will reimplement it and see what it gives.
", "rawMarkdown": "Sorry I did it early in the competition and i don't remember .I will reimplement it and see what it gives.", "votes": null }, { "id": "2180452", "postDate": "03/13/2023 20:16:36", "content": "Using predictions from previous level groups boosted my CV and LB scores by about +0.001, I've got the second method running on LB right now but on CV it increased by about +0.003
\nI've written a notebook that demonstrates both methods - hope that helps!
\nhttps://www.kaggle.com/code/judehunt23/saving-predictions-from-previous-levels-2-ways
Hi Jude, your code is so helpful to me and I learned a lot!
", "rawMarkdown": "Hi Jude, your code is so helpful to me and I learned a lot!", "votes": null }, { "id": "2180638", "postDate": "03/14/2023 02:04:11", "content": "Just to clear up my confusion: After reading the code of @judehunt23 I understood that the data is in order. It first feeds you a session, with the questions from level 1 to 3, and then it feeds you another session, level 1 to 3， after completing all the sessions and all the questions of level group 1-3, then it feeds you questions from 4-12, finally, it feeds you question 13-22.
", "rawMarkdown": "Just to clear up my confusion: After reading the code of @judehunt23 I understood that the data is in order. It first feeds you a session, with the questions from level 1 to 3, and then it feeds you another session, level 1 to 3， after completing all the sessions and all the questions of level group 1-3, then it feeds you questions from 4-12, finally, it feeds you question 13-22.", "votes": null }, { "id": "2180640", "postDate": "03/14/2023 02:07:20", "content": "And I think that the previous version of API is incorrect because it gives you a session, with levels 1-18, and then gives you the next session. This makes you infer the present result of session B based on the future of session A, and this is not relevant.
", "rawMarkdown": "And I think that the previous version of API is incorrect because it gives you a session, with levels 1-18, and then gives you the next session. This makes you infer the present result of session B based on the future of session A, and this is not relevant.", "votes": null }, { "id": "2181196", "postDate": "03/14/2023 11:37:47", "content": "I'm glad you found it useful @harrisonliang!
", "rawMarkdown": "I'm glad you found it useful @harrisonliang!", "votes": null }, { "id": "2183942", "postDate": "03/16/2023 03:51:14", "content": "Hi, danniel, I got the same result. Have you found what caused this situation?
", "rawMarkdown": "Hi, danniel, I got the same result. Have you found what caused this situation?", "votes": null }, { "id": "2184643", "postDate": "03/16/2023 14:06:30", "content": "Same as you.
\nMy cv imporved 0.002 but my lb decreased 0.001.
\nMaybe it has some cv leak…
", "rawMarkdown": "Same as you.\n\nMy cv imporved 0.002 but my lb decreased 0.001.\n\nMaybe it has some cv leak...", "votes": null }, { "id": "2185218", "postDate": "03/16/2023 22:11:38", "content": "In my case, my CV improved by 0.002 and my LB slightly improved by less than 0.001. I don't know why. May I ask you how many feature you used in the model?
", "rawMarkdown": "In my case, my CV improved by 0.002 and my LB slightly improved by less than 0.001. I don't know why. May I ask you how many feature you used in the model?", "votes": null }, { "id": "2187404", "postDate": "03/18/2023 17:38:49", "content": "I tried both, and I even tried both together. The first method deteriorated the score a bit, the second helped a tiny bit, but scoring became very long as the data need to be saved for the next iterations.
", "rawMarkdown": "I tried both, and I even tried both together. The first method deteriorated the score a bit, the second helped a tiny bit, but scoring became very long as the data need to be saved for the next iterations.", "votes": null } ], "comments": [ { "id": 2179391, "author_name": "ihebch", "author_url": "", "post_date": "03/13/2023 06:30:59", "content": "Both improved my cv but decreased my lb (maybe i did something wrong , I haven’t checked yet!)
", "votes": null, "replies": [ { "id": 2179543, "author_name": "dannielscut", "author_url": "", "post_date": "03/13/2023 08:52:10", "content": "Hi, Reacher, may I ask you how many points you improved in your CV?
\nI got similar results by using the second methods.
\nMy cv improved a lot (0.004~0.006) but got little improve in lb (< 0.001).
Sorry I did it early in the competition and i don't remember .I will reimplement it and see what it gives.
", "votes": null, "replies": [] }, { "id": 2183942, "author_name": "yunqicao", "author_url": "", "post_date": "03/16/2023 03:51:14", "content": "Hi, danniel, I got the same result. Have you found what caused this situation?
", "votes": null, "replies": [] } ] }, { "id": 2184643, "author_name": "clement1999", "author_url": "", "post_date": "03/16/2023 14:06:30", "content": "Same as you.
\nMy cv imporved 0.002 but my lb decreased 0.001.
\nMaybe it has some cv leak…
", "votes": null, "replies": [ { "id": 2185218, "author_name": "harrisonliang", "author_url": "", "post_date": "03/16/2023 22:11:38", "content": "In my case, my CV improved by 0.002 and my LB slightly improved by less than 0.001. I don't know why. May I ask you how many feature you used in the model?
", "votes": null, "replies": [] } ] } ] }, { "id": 2179632, "author_name": "harrisonliang", "author_url": "", "post_date": "03/13/2023 09:48:12", "content": "It seems that the test data is not in order, after commin three data of level 5, it then goes up to level 13, and I cannot predict based on the result of level 6-12!
", "votes": null, "replies": [ { "id": 2180638, "author_name": "harrisonliang", "author_url": "", "post_date": "03/14/2023 02:04:11", "content": "Just to clear up my confusion: After reading the code of @judehunt23 I understood that the data is in order. It first feeds you a session, with the questions from level 1 to 3, and then it feeds you another session, level 1 to 3， after completing all the sessions and all the questions of level group 1-3, then it feeds you questions from 4-12, finally, it feeds you question 13-22.
", "votes": null, "replies": [] }, { "id": 2180640, "author_name": "harrisonliang", "author_url": "", "post_date": "03/14/2023 02:07:20", "content": "And I think that the previous version of API is incorrect because it gives you a session, with levels 1-18, and then gives you the next session. This makes you infer the present result of session B based on the future of session A, and this is not relevant.
", "votes": null, "replies": [] } ] }, { "id": 2180452, "author_name": "judehunt23", "author_url": "", "post_date": "03/13/2023 20:16:36", "content": "Using predictions from previous level groups boosted my CV and LB scores by about +0.001, I've got the second method running on LB right now but on CV it increased by about +0.003
\nI've written a notebook that demonstrates both methods - hope that helps!
\nhttps://www.kaggle.com/code/judehunt23/saving-predictions-from-previous-levels-2-ways
Hi Jude, your code is so helpful to me and I learned a lot!
", "votes": null, "replies": [ { "id": 2181196, "author_name": "judehunt23", "author_url": "", "post_date": "03/14/2023 11:37:47", "content": "I'm glad you found it useful @harrisonliang!
", "votes": null, "replies": [] } ] } ] }, { "id": 2187404, "author_name": "gehallak", "author_url": "", "post_date": "03/18/2023 17:38:49", "content": "I tried both, and I even tried both together. The first method deteriorated the score a bit, the second helped a tiny bit, but scoring became very long as the data need to be saved for the next iterations.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2179344": "Hi, everyone, I have a question about using the previous answer.\nAs we know, each user has to answer from question level 1 to level 18, and he can not answer the question afterward if he did not solve the previous one. \nHere comes the assumption that the performance of the user could be inferred from his or her previous answers. There are mainly two methods for prediction:\n1. Divide those questions into three groups 0-4, 5-12, and 13-22, when predicting the second group 5-12, add the predicted result of 0-4 for training, when predicting 13-22, add the predicted result 0-4 and 5-12.\n2. Add all the information from the previous questions. for example, when predicting level 2, add the result of level 1, when predicting level 5, add the result of 1,2,3, and 4.\nI m wondering which method is better, and how to write the INFERENCE code.\nSome discussion says it did not improve their CV score, any reason for that?", "2179391": "Both improved my cv but decreased my lb (maybe i did something wrong , I haven’t checked yet!)", "2179543": "Hi, Reacher, may I ask you how many points you improved in your CV?\nI got similar results by using the second methods. \nMy cv improved a lot (0.004~0.006) but got little improve in lb (< 0.001).", "2179632": "It seems that the test data is not in order, after commin three data of level 5, it then goes up to level 13, and I cannot predict based on the result of level 6-12!", "2179699": "Sorry I did it early in the competition and i don't remember .I will reimplement it and see what it gives.", "2180452": "Using predictions from previous level groups boosted my CV and LB scores by about +0.001, I've got the second method running on LB right now but on CV it increased by about +0.003\n\nI've written a notebook that demonstrates both methods - hope that helps!\nhttps://www.kaggle.com/code/judehunt23/saving-predictions-from-previous-levels-2-ways", "2180636": "Hi Jude, your code is so helpful to me and I learned a lot!", "2180638": "Just to clear up my confusion: After reading the code of @judehunt23 I understood that the data is in order. It first feeds you a session, with the questions from level 1 to 3, and then it feeds you another session, level 1 to 3， after completing all the sessions and all the questions of level group 1-3, then it feeds you questions from 4-12, finally, it feeds you question 13-22.", "2180640": "And I think that the previous version of API is incorrect because it gives you a session, with levels 1-18, and then gives you the next session. This makes you infer the present result of session B based on the future of session A, and this is not relevant.", "2181196": "I'm glad you found it useful @harrisonliang!", "2183942": "Hi, danniel, I got the same result. Have you found what caused this situation?", "2184643": "Same as you.\n\nMy cv imporved 0.002 but my lb decreased 0.001.\n\nMaybe it has some cv leak...", "2185218": "In my case, my CV improved by 0.002 and my LB slightly improved by less than 0.001. I don't know why. May I ask you how many feature you used in the model?", "2187404": "I tried both, and I even tried both together. The first method deteriorated the score a bit, the second helped a tiny bit, but scoring became very long as the data need to be saved for the next iterations." }, "source": "meta" }