Hello, I have trained 18 lightgbm models and stored them in a list. There is no problem with my code in my test. But my submission always reports a Submission Scoring Error.
\nAnd my submission code is here. What should I do? If you need more info feel free to ask me.
limits = {'0-4':(0,3), '5-12':(3,13), '13-22':(13,18)}\nenv = jo_wilder.make_env()\ntest_env = env.iter_test()\n\nfor (sample_submission, test_) in test_env:\n\n test_=data_progress(test_)\n test=feature_engineering(test_,True)\n test=fill_disappear_feature(features,test)\n test.set_index(\"session_id\",inplace=True)\n\n a,b=limits[test_[\"level_group\"].values[0]]\n for i in range(a,b):\n model=model_group[i]\n ans = model.predict(test)\n\n mask = sample_submission[\"session_id\"].str.contains(f'q{i+1}')\n sample_submission.loc[mask,\"correct\"] = ans\n\n env.predict(sample_submission)\nHello, I have trained 18 lightgbm models and stored them in a list. There is no problem with my code in my test. But my submission always reports a Submission Scoring Error.
\nAnd my submission code is here. What should I do? If you need more info feel free to ask me.
limits = {'0-4':(0,3), '5-12':(3,13), '13-22':(13,18)}\nenv = jo_wilder.make_env()\ntest_env = env.iter_test()\n\nfor (sample_submission, test_) in test_env:\n\n test_=data_progress(test_)\n test=feature_engineering(test_,True)\n test=fill_disappear_feature(features,test)\n test.set_index(\"session_id\",inplace=True)\n\n a,b=limits[test_[\"level_group\"].values[0]]\n for i in range(a,b):\n model=model_group[i]\n ans = model.predict(test)\n\n mask = sample_submission[\"session_id\"].str.contains(f'q{i+1}')\n sample_submission.loc[mask,\"correct\"] = ans\n\n env.predict(sample_submission)\nSubmission scoring error is misleading and most likely means that your program get error (probably in one of your functions). Try to run your functions on some train data examples with single session_id selected to check if everything is fine.
", "rawMarkdown": "Submission scoring error is misleading and most likely means that your program get error (probably in one of your functions). Try to run your functions on some train data examples with single session_id selected to check if everything is fine.", "votes": null }, { "id": "2159032", "postDate": "02/25/2023 11:19:05", "content": "Thanks for reply. I will check all of my codes as soon as possible. But I also want to know whether the hidden test set will have abnormal values different from the training set, such as inf, which is not in the training set.
", "rawMarkdown": "Thanks for reply. I will check all of my codes as soon as possible. But I also want to know whether the hidden test set will have abnormal values different from the training set, such as inf, which is not in the training set.", "votes": null }, { "id": "2159066", "postDate": "02/25/2023 11:44:12", "content": "I do not know, but I think it is more likely that some events may be missing in individual sessions.
", "rawMarkdown": "I do not know, but I think it is more likely that some events may be missing in individual sessions.", "votes": null }, { "id": "2163769", "postDate": "03/01/2023 04:27:34", "content": "I have the same problem⦠I guess it is related to the run time ! Did you solve your problem
", "rawMarkdown": "I have the same problem... I guess it is related to the run time ! Did you solve your problem", "votes": null }, { "id": "2163781", "postDate": "03/01/2023 04:43:10", "content": "I had such error due to OOM. But there are many other variants
", "rawMarkdown": "I had such error due to OOM. But there are many other variants", "votes": null }, { "id": "2163785", "postDate": "03/01/2023 04:45:52", "content": "https://www.kaggle.com/code/zakopur0/psp-debug-inference try this, but pay attention to my comment there
", "rawMarkdown": "https://www.kaggle.com/code/zakopur0/psp-debug-inference try this, but pay attention to my comment there", "votes": null }, { "id": "2163824", "postDate": "03/01/2023 05:25:33", "content": "Thanks for reply. I have already solved this problem. As RyszardStaruch's said: \"Submission scoring error is misleading and most likely means that your program get error (probably in one of your functions)\", I found a bug in my feature engineering function. So I reduced the number of derivatives features and now it can work, but the score is low.π
", "rawMarkdown": "Thanks for reply. I have already solved this problem. As RyszardStaruch's said: \"**Submission scoring error is misleading and most likely means that your program get error (probably in one of your functions)**\", I found a bug in my feature engineering function. So I reduced the number of derivatives features and now it can work, but the score is low.π", "votes": null }, { "id": "2163825", "postDate": "03/01/2023 05:26:49", "content": "Thanks for your sharing!π
", "rawMarkdown": "Thanks for your sharing!π", "votes": null }, { "id": "2165083", "postDate": "03/02/2023 01:23:19", "content": "oh great ! it is still not working for me. how you are loading your model !
", "rawMarkdown": "oh great ! it is still not working for me. how you are loading your model !", "votes": null }, { "id": "2165236", "postDate": "03/02/2023 04:18:04", "content": "I train and submit my models in the same notebook. But I think you can create a dataset to store and load your models.
", "rawMarkdown": "I train and submit my models in the same notebook. But I think you can create a dataset to store and load your models.", "votes": null } ], "comments": [ { "id": 2158984, "author_name": "ryszardstaruch", "author_url": "", "post_date": "02/25/2023 10:29:35", "content": "Submission scoring error is misleading and most likely means that your program get error (probably in one of your functions). Try to run your functions on some train data examples with single session_id selected to check if everything is fine.
", "votes": null, "replies": [ { "id": 2159032, "author_name": "zhangjiexi", "author_url": "", "post_date": "02/25/2023 11:19:05", "content": "Thanks for reply. I will check all of my codes as soon as possible. But I also want to know whether the hidden test set will have abnormal values different from the training set, such as inf, which is not in the training set.
", "votes": null, "replies": [ { "id": 2159066, "author_name": "ryszardstaruch", "author_url": "", "post_date": "02/25/2023 11:44:12", "content": "I do not know, but I think it is more likely that some events may be missing in individual sessions.
", "votes": null, "replies": [] } ] } ] }, { "id": 2163769, "author_name": "mhirimohamed", "author_url": "", "post_date": "03/01/2023 04:27:34", "content": "I have the same problem⦠I guess it is related to the run time ! Did you solve your problem
", "votes": null, "replies": [ { "id": 2163824, "author_name": "zhangjiexi", "author_url": "", "post_date": "03/01/2023 05:25:33", "content": "Thanks for reply. I have already solved this problem. As RyszardStaruch's said: \"Submission scoring error is misleading and most likely means that your program get error (probably in one of your functions)\", I found a bug in my feature engineering function. So I reduced the number of derivatives features and now it can work, but the score is low.π
", "votes": null, "replies": [ { "id": 2165083, "author_name": "mhirimohamed", "author_url": "", "post_date": "03/02/2023 01:23:19", "content": "oh great ! it is still not working for me. how you are loading your model !
", "votes": null, "replies": [ { "id": 2165236, "author_name": "zhangjiexi", "author_url": "", "post_date": "03/02/2023 04:18:04", "content": "I train and submit my models in the same notebook. But I think you can create a dataset to store and load your models.
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 2163781, "author_name": "kvlmll", "author_url": "", "post_date": "03/01/2023 04:43:10", "content": "I had such error due to OOM. But there are many other variants
", "votes": null, "replies": [ { "id": 2163785, "author_name": "kvlmll", "author_url": "", "post_date": "03/01/2023 04:45:52", "content": "https://www.kaggle.com/code/zakopur0/psp-debug-inference try this, but pay attention to my comment there
", "votes": null, "replies": [ { "id": 2163825, "author_name": "zhangjiexi", "author_url": "", "post_date": "03/01/2023 05:26:49", "content": "Thanks for your sharing!π
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2158553": "Hello, I have trained 18 lightgbm models and stored them in a list. There is no problem with my code in my test. But my submission always reports a Submission Scoring Error.\nAnd my submission code is here. What should I do? If you need more info feel free to ask me.\n\n```\nlimits = {'0-4':(0,3), '5-12':(3,13), '13-22':(13,18)}\nenv = jo_wilder.make_env()\ntest_env = env.iter_test()\n\nfor (sample_submission, test_) in test_env:\n\n test_=data_progress(test_)\n test=feature_engineering(test_,True)\n test=fill_disappear_feature(features,test)\n test.set_index(\"session_id\",inplace=True)\n \n a,b=limits[test_[\"level_group\"].values[0]]\n for i in range(a,b):\n model=model_group[i]\n ans = model.predict(test)\n\n mask = sample_submission[\"session_id\"].str.contains(f'q{i+1}')\n sample_submission.loc[mask,\"correct\"] = ans\n \n env.predict(sample_submission)\n```", "2158984": "Submission scoring error is misleading and most likely means that your program get error (probably in one of your functions). Try to run your functions on some train data examples with single session_id selected to check if everything is fine.", "2159032": "Thanks for reply. I will check all of my codes as soon as possible. But I also want to know whether the hidden test set will have abnormal values different from the training set, such as inf, which is not in the training set.", "2159066": "I do not know, but I think it is more likely that some events may be missing in individual sessions.", "2163769": "I have the same problem... I guess it is related to the run time ! Did you solve your problem", "2163781": "I had such error due to OOM. But there are many other variants", "2163785": "https://www.kaggle.com/code/zakopur0/psp-debug-inference try this, but pay attention to my comment there", "2163824": "Thanks for reply. I have already solved this problem. As RyszardStaruch's said: \"**Submission scoring error is misleading and most likely means that your program get error (probably in one of your functions)**\", I found a bug in my feature engineering function. So I reduced the number of derivatives features and now it can work, but the score is low.π", "2163825": "Thanks for your sharing!π", "2165083": "oh great ! it is still not working for me. how you are loading your model !", "2165236": "I train and submit my models in the same notebook. But I think you can create a dataset to store and load your models." }, "source": "meta" }