{ "id": 418325, "title": "Have new private dataset been included already?", "url": "/competitions/predict-student-performance-from-game-play/discussion/418325", "author_name": "", "post_date": "2023-06-20T02:34:38.194693100Z", "votes": 7, "comment_count": 16, "views": 0, "content": "

Based on discussion here, it says \"we are going to incorporate a new private dataset into the competition that has been gathered over the months since the leak\" and I guess the api need more time to infer. But the inference time didn't change too much before/after reruning, at least for me. So I want to ask whether new private dataset are included now.

", "messages": [ { "id": "2309951", "postDate": "06/20/2023 02:34:38", "content": "

Based on discussion here, it says \"we are going to incorporate a new private dataset into the competition that has been gathered over the months since the leak\" and I guess the api need more time to infer. But the inference time didn't change too much before/after reruning, at least for me. So I want to ask whether new private dataset are included now.

", "rawMarkdown": "Based on [discussion](https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/414109) here, it says \"we are going to incorporate a new private dataset into the competition that has been gathered over the months since the leak\" and I guess the api need more time to infer. But the inference time didn't change too much before/after reruning, at least for me. So I want to ask whether new private dataset are included now.", "votes": null }, { "id": "2309952", "postDate": "06/20/2023 02:35:38", "content": "

Hi @philculliton, could you confirm this?

", "rawMarkdown": "Hi @philculliton, could you confirm this?", "votes": null }, { "id": "2310108", "postDate": "06/20/2023 05:44:37", "content": "

i dont think he will reply … because too many ppls asked him the same question in his post .
\nBut what i think is a new data set is incorporated/merged/updated into the existing private data set ..
\nbecause i found my solution is totally not worked some days before. ( i am very sure it is not due to the data leakage and update from the first time. because i hv made the change from that time and the scoring is also back to the original score .) Even i reloaded a recently successful submition, the score is changed from 0.697 to 0.65xx..
\ni dont think a non-change of data set will made a sinificant difference in the scoring.
\nso , if u found the time is not changed too much , maybe the change is just a merging/update in the existing private data set.
\nps, this is a real 坑 competition…

", "rawMarkdown": "i dont think he will reply ... because too many ppls asked him the same question in his post . \nBut what i think is a new data set is incorporated/merged/updated into the existing private data set .. \nbecause i found my solution is totally not worked some days before. ( i am very sure it is not due to the data leakage and update from the first time. because i hv made the change from that time and the scoring is also back to the original score .) Even i reloaded a recently successful submition, the score is changed from 0.697 to 0.65xx.. \ni dont think a non-change of data set will made a sinificant difference in the scoring. \nso , if u found the time is not changed too much , maybe the change is just a merging/update in the existing private data set. \nps, this is a real 坑 competition...", "votes": null }, { "id": "2310161", "postDate": "06/20/2023 06:45:28", "content": "

Within the last several weeks neither Kaggle staff, nor host have ever answered at least one question. My feeling is that this competition is abandoned.

", "rawMarkdown": "Within the last several weeks neither Kaggle staff, nor host have ever answered at least one question. My feeling is that this competition is abandoned.", "votes": null }, { "id": "2310233", "postDate": "06/20/2023 08:12:10", "content": "

I think it is included now because I can confirm that there were reruns after that post

", "rawMarkdown": "I think it is included now because I can confirm that there were reruns after that post", "votes": null }, { "id": "2310398", "postDate": "06/20/2023 10:37:06", "content": "

Big drop on LB might be caused by api change. We need to sort index and question number to recover.

", "rawMarkdown": "Big drop on LB might be caused by api change. We need to sort index and question number to recover.", "votes": null }, { "id": "2310400", "postDate": "06/20/2023 10:38:40", "content": "

Yes, the api changes so it reruns but don't know about private dataset… I suppose new data would increase inference time but it didn't.

", "rawMarkdown": "Yes, the api changes so it reruns but don't know about private dataset... I suppose new data would increase inference time but it didn't.", "votes": null }, { "id": "2310786", "postDate": "06/20/2023 15:51:42", "content": "

Sorry to ask u question here , actually , i hv tried to sort the index in Test, could you give me more detial on sort the question number ?? Thanks a lot in advance .

", "rawMarkdown": "Sorry to ask u question here , actually , i hv tried to sort the index in Test, could you give me more detial on sort the question number ?? Thanks a lot in advance .", "votes": null }, { "id": "2310881", "postDate": "06/20/2023 17:08:55", "content": "

something like this: sample_submission is not sorted. you can see discussion here.

\n
 (test, sample_submission)  (iter_test):\n    test = test.sort_values(by = )\n    \n
", "rawMarkdown": "something like this: sample_submission is not sorted. you can see [discussion] (https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/412512#2274390) here. \n```python\nfor (test, sample_submission) in (iter_test):\n test = test.sort_values(by = 'index')\n # sample_submission is not sorted by question_num, we need to sort it by ourselves\n```", "votes": null }, { "id": "2311597", "postDate": "06/21/2023 09:21:09", "content": "

I am still unable to determine what the problem is with the new API. I obtained a CV with a score of 0.702, but the LB score is 0.695.🤕

", "rawMarkdown": "I am still unable to determine what the problem is with the new API. I obtained a CV with a score of 0.702, but the LB score is 0.695.🤕", "votes": null }, { "id": "2311788", "postDate": "06/21/2023 13:02:25", "content": "

maybe some leakage on your cv？

", "rawMarkdown": "maybe some leakage on your cv？", "votes": null }, { "id": "2312511", "postDate": "06/22/2023 01:22:28", "content": "

You are right. Now I think maybe the leakage happens in meta features process, even though I set special oof for it.

", "rawMarkdown": "You are right. Now I think maybe the leakage happens in meta features process, even though I set special oof for it.", "votes": null }, { "id": "2313630", "postDate": "06/22/2023 19:35:45", "content": "

Btw, not sure if this is related to your question, but I have noticed today that some teams disappeared from top-100 efficiency LB, meaning that their submissions were re-ran and either errored, or the score dropped down significantly.

", "rawMarkdown": "Btw, not sure if this is related to your question, but I have noticed today that some teams disappeared from top-100 efficiency LB, meaning that their submissions were re-ran and either errored, or the score dropped down significantly.", "votes": null }, { "id": "2313872", "postDate": "06/23/2023 02:17:49", "content": "

My teammate has same case here. The root cause of us is that efficiency leaderboard only take the selected submission into consideration and after we team up, we need to re-select our submission and then it will show in the efficiency leaderboard.

", "rawMarkdown": "My teammate has same case here. The root cause of us is that efficiency leaderboard only take the selected submission into consideration and after we team up, we need to re-select our submission and then it will show in the efficiency leaderboard.", "votes": null }, { "id": "2313874", "postDate": "06/23/2023 02:21:03", "content": "

Hm, for me it shows the best submission, even though I didn’t select it.

", "rawMarkdown": "Hm, for me it shows the best submission, even though I didn’t select it.", "votes": null }, { "id": "2313876", "postDate": "06/23/2023 02:24:06", "content": "

yes, I guess if you don't select, it will show your submission with best public leaderboard.

", "rawMarkdown": "yes, I guess if you don't select, it will show your submission with best public leaderboard.", "votes": null }, { "id": "2313883", "postDate": "06/23/2023 02:33:34", "content": "

I thought it is like a normal LB: whatever submission has the highest score is shown on the LB. It is a little bit confusing since it could take several days to get onto the efficiency LB.

", "rawMarkdown": "I thought it is like a normal LB: whatever submission has the highest score is shown on the LB. It is a little bit confusing since it could take several days to get onto the efficiency LB.", "votes": null } ], "comments": [ { "id": 2309952, "author_name": "hookman", "author_url": "", "post_date": "06/20/2023 02:35:38", "content": "

Hi @philculliton, could you confirm this?

", "votes": null, "replies": [ { "id": 2310108, "author_name": "johnfjliu", "author_url": "", "post_date": "06/20/2023 05:44:37", "content": "

i dont think he will reply … because too many ppls asked him the same question in his post .
\nBut what i think is a new data set is incorporated/merged/updated into the existing private data set ..
\nbecause i found my solution is totally not worked some days before. ( i am very sure it is not due to the data leakage and update from the first time. because i hv made the change from that time and the scoring is also back to the original score .) Even i reloaded a recently successful submition, the score is changed from 0.697 to 0.65xx..
\ni dont think a non-change of data set will made a sinificant difference in the scoring.
\nso , if u found the time is not changed too much , maybe the change is just a merging/update in the existing private data set.
\nps, this is a real 坑 competition…

", "votes": null, "replies": [ { "id": 2310398, "author_name": "hookman", "author_url": "", "post_date": "06/20/2023 10:37:06", "content": "

Big drop on LB might be caused by api change. We need to sort index and question number to recover.

", "votes": null, "replies": [ { "id": 2310786, "author_name": "johnfjliu", "author_url": "", "post_date": "06/20/2023 15:51:42", "content": "

Sorry to ask u question here , actually , i hv tried to sort the index in Test, could you give me more detial on sort the question number ?? Thanks a lot in advance .

", "votes": null, "replies": [ { "id": 2310881, "author_name": "hookman", "author_url": "", "post_date": "06/20/2023 17:08:55", "content": "

something like this: sample_submission is not sorted. you can see discussion here.

\n
 (test, sample_submission)  (iter_test):\n    test = test.sort_values(by = )\n    \n
", "votes": null, "replies": [] } ] } ] } ] } ] }, { "id": 2310161, "author_name": "kononenko", "author_url": "", "post_date": "06/20/2023 06:45:28", "content": "

Within the last several weeks neither Kaggle staff, nor host have ever answered at least one question. My feeling is that this competition is abandoned.

", "votes": null, "replies": [] }, { "id": 2310233, "author_name": "kvlmll", "author_url": "", "post_date": "06/20/2023 08:12:10", "content": "

I think it is included now because I can confirm that there were reruns after that post

", "votes": null, "replies": [ { "id": 2310400, "author_name": "hookman", "author_url": "", "post_date": "06/20/2023 10:38:40", "content": "

Yes, the api changes so it reruns but don't know about private dataset… I suppose new data would increase inference time but it didn't.

", "votes": null, "replies": [] } ] }, { "id": 2311597, "author_name": "mengvision", "author_url": "", "post_date": "06/21/2023 09:21:09", "content": "

I am still unable to determine what the problem is with the new API. I obtained a CV with a score of 0.702, but the LB score is 0.695.🤕

", "votes": null, "replies": [ { "id": 2311788, "author_name": "librauee", "author_url": "", "post_date": "06/21/2023 13:02:25", "content": "

maybe some leakage on your cv？

", "votes": null, "replies": [ { "id": 2312511, "author_name": "mengvision", "author_url": "", "post_date": "06/22/2023 01:22:28", "content": "

You are right. Now I think maybe the leakage happens in meta features process, even though I set special oof for it.

", "votes": null, "replies": [] } ] } ] }, { "id": 2313630, "author_name": "kononenko", "author_url": "", "post_date": "06/22/2023 19:35:45", "content": "

Btw, not sure if this is related to your question, but I have noticed today that some teams disappeared from top-100 efficiency LB, meaning that their submissions were re-ran and either errored, or the score dropped down significantly.

", "votes": null, "replies": [ { "id": 2313872, "author_name": "hookman", "author_url": "", "post_date": "06/23/2023 02:17:49", "content": "

My teammate has same case here. The root cause of us is that efficiency leaderboard only take the selected submission into consideration and after we team up, we need to re-select our submission and then it will show in the efficiency leaderboard.

", "votes": null, "replies": [ { "id": 2313874, "author_name": "kononenko", "author_url": "", "post_date": "06/23/2023 02:21:03", "content": "

Hm, for me it shows the best submission, even though I didn’t select it.

", "votes": null, "replies": [ { "id": 2313876, "author_name": "hookman", "author_url": "", "post_date": "06/23/2023 02:24:06", "content": "

yes, I guess if you don't select, it will show your submission with best public leaderboard.

", "votes": null, "replies": [ { "id": 2313883, "author_name": "kononenko", "author_url": "", "post_date": "06/23/2023 02:33:34", "content": "

I thought it is like a normal LB: whatever submission has the highest score is shown on the LB. It is a little bit confusing since it could take several days to get onto the efficiency LB.

", "votes": null, "replies": [] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2309951": "Based on [discussion](https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/414109) here, it says \"we are going to incorporate a new private dataset into the competition that has been gathered over the months since the leak\" and I guess the api need more time to infer. But the inference time didn't change too much before/after reruning, at least for me. So I want to ask whether new private dataset are included now.", "2309952": "Hi @philculliton, could you confirm this?", "2310108": "i dont think he will reply ... because too many ppls asked him the same question in his post . \nBut what i think is a new data set is incorporated/merged/updated into the existing private data set .. \nbecause i found my solution is totally not worked some days before. ( i am very sure it is not due to the data leakage and update from the first time. because i hv made the change from that time and the scoring is also back to the original score .) Even i reloaded a recently successful submition, the score is changed from 0.697 to 0.65xx.. \ni dont think a non-change of data set will made a sinificant difference in the scoring. \nso , if u found the time is not changed too much , maybe the change is just a merging/update in the existing private data set. \nps, this is a real 坑 competition...", "2310161": "Within the last several weeks neither Kaggle staff, nor host have ever answered at least one question. My feeling is that this competition is abandoned.", "2310233": "I think it is included now because I can confirm that there were reruns after that post", "2310398": "Big drop on LB might be caused by api change. We need to sort index and question number to recover.", "2310400": "Yes, the api changes so it reruns but don't know about private dataset... I suppose new data would increase inference time but it didn't.", "2310786": "Sorry to ask u question here , actually , i hv tried to sort the index in Test, could you give me more detial on sort the question number ?? Thanks a lot in advance .", "2310881": "something like this: sample_submission is not sorted. you can see [discussion] (https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/412512#2274390) here. \n```python\nfor (test, sample_submission) in (iter_test):\n test = test.sort_values(by = 'index')\n # sample_submission is not sorted by question_num, we need to sort it by ourselves\n```", "2311597": "I am still unable to determine what the problem is with the new API. I obtained a CV with a score of 0.702, but the LB score is 0.695.🤕", "2311788": "maybe some leakage on your cv？", "2312511": "You are right. Now I think maybe the leakage happens in meta features process, even though I set special oof for it.", "2313630": "Btw, not sure if this is related to your question, but I have noticed today that some teams disappeared from top-100 efficiency LB, meaning that their submissions were re-ran and either errored, or the score dropped down significantly.", "2313872": "My teammate has same case here. The root cause of us is that efficiency leaderboard only take the selected submission into consideration and after we team up, we need to re-select our submission and then it will show in the efficiency leaderboard.", "2313874": "Hm, for me it shows the best submission, even though I didn’t select it.", "2313876": "yes, I guess if you don't select, it will show your submission with best public leaderboard.", "2313883": "I thought it is like a normal LB: whatever submission has the highest score is shown on the LB. It is a little bit confusing since it could take several days to get onto the efficiency LB." }, "source": "meta" }