I'm stuck creating a submission file.
\nI've trained 18 xgboost models, one for each question, and stored them in a list. Now i want to make a submission but i have no idea how (am new, sorry). To be precise, how do i select the right data to make a prediction with and how do i write the prediction in the submission file? Maybe someone can help? If you need more info feel free to ask me
Hi, I'm also new here and trained in ways similar to you. Hope my submission code could help. I mainly follows Chris notebook. I 'm not so sure how to add codes by markdown :) lol
\nmodels = {}\nfor t in range(1,19):\n if t<=3: grp = '0-4'\n elif t<=13: grp = '5-12'\n elif t<=18: grp = '13-22'\n clf = XGBClassifier()\n clf.load_model(work_path + f'XGB_question_{t}.xgb')\n models[f'{grp}_{t}'] = clf\n\nimport jo_wilder\nenv = jo_wilder.make_env()\niter_test = env.iter_test()\n\n\nlimits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nbest_threshold = {0: 0.631, 1: 0.613, 2: 0.607, 3: 0.653, 4: 0.625, 5: 0.637, 6: 0.637, 7: 0.605, 8: 0.631, 9: 0.602, 10: 0.636, 11: 0.66, 12: 0.612, 13: 0.647, 14: 0.626, 15: 0.63, 16: 0.61, 17: 0.602}\ncount = 0\nFEATURES_QUESTION = np.loadtxt(work_path + 'FEATURES_QUESTION.txt',dtype=str, delimiter = ' ').tolist()\n\n\nfor (sample_submission, test) in iter_test:\n df = feature_engineer(test)\n #if count == 0:\n #FEATURES = [c for c in df.columns if c != 'level_group']\n #count += 1\n grp = test.level_group.values[0]\n a,b = limits[grp]\n for t in range(a,b):\n clf = models[f'{grp}_{t}']\n p = clf.predict_proba(df[FEATURES_QUESTION[t-1]].astype('float32'))[:,1]\n mask = sample_submission.session_id.str.contains(f'q{t}')\n sample_submission.loc[mask,'correct'] = int(p.item()>best_threshold[t-1])\n env.predict(sample_submission)\nThanks! It does indeed look similar. I still get the error, though.. So i must do it wrong. Basically, i copied everything starting from \"import jo_wilder\" to the end. My models are already stored the same way you did it. Then I changed FEATURE_QUESTION to a list of the features my models are using. And I changed clf.predict_proba(df[FEATURES_QUESTION[t-1]].astype('float32'))[:,1] into clf.predict_proba(df[FEATURES_QUESTION].astype('float32'))[:,1], so that every model uses the same features. Still doesnt work :/
", "rawMarkdown": "Thanks! It does indeed look similar. I still get the error, though.. So i must do it wrong. Basically, i copied everything starting from \"import jo_wilder\" to the end. My models are already stored the same way you did it. Then I changed FEATURE_QUESTION to a list of the features my models are using. And I changed clf.predict_proba(df[FEATURES_QUESTION[t-1]].astype('float32'))[:,1] into clf.predict_proba(df[FEATURES_QUESTION].astype('float32'))[:,1], so that every model uses the same features. Still doesnt work :/" } ] }, { "id": 2156338, "postDate": "2023-02-23T09:25:55.650Z", "content": "Have a look at this great notebook by @carnozhao - it takes some pretrained models saved in a kaggle dataset, defines a feature engineering function (done with polars instead of pandas!), and then cells 4-6 set up the API needed for the submission and then loops through the data for submission to perform the relevant feature engineering and predict using the pretrained models https://www.kaggle.com/code/carnozhao/cpu-catboost-baseline-using-polars-inference
\nThere's also this notebook for a sample submission that shows how that API works! https://www.kaggle.com/code/carnozhao/cpu-catboost-baseline-using-polars-inference
", "rawMarkdown": "Have a look at this great notebook by @carnozhao - it takes some pretrained models saved in a kaggle dataset, defines a feature engineering function (done with polars instead of pandas!), and then cells 4-6 set up the API needed for the submission and then loops through the data for submission to perform the relevant feature engineering and predict using the pretrained models https://www.kaggle.com/code/carnozhao/cpu-catboost-baseline-using-polars-inference\n\nThere's also this notebook for a sample submission that shows how that API works! https://www.kaggle.com/code/carnozhao/cpu-catboost-baseline-using-polars-inference", "replies": [ { "id": 2156605, "postDate": "2023-02-23T12:37:51.757Z", "content": "Thanks for your reply! I tried doing it accordingly to the first notebook you provided but get an error.
\npredict() successfully before you can continue with iter_test():TypeError Traceback (most recent call last)
\n/tmp/ipykernel_25/3104309027.py in
\n 3 levels = {'0-4': (0, 5), '5-12': (5, 13), '13-22': (13, 23)}
\n 4 questions = {'0-4': (1, 4), '5-12': (4, 14), '13-22': (14, 19)}
\n----> 5 for (sample_submission, test) in iter_test:
\n 6 target_level_group = level_groups_reverse[test.level_group.iloc[0]]
\n 7 df = feature_engineer(test)
TypeError: cannot unpack non-iterable NoneType object
\nI did it like this.
\nlevel_groups = [\"0-4\", \"5-12\", \"13-22\"]
\nlevel_groups_reverse = {'0-4': 0, '5-12': 1, '13-22': 2}
\nlevels = {'0-4': (0, 5), '5-12': (5, 13), '13-22': (13, 23)}
\nquestions = {'0-4': (1, 4), '5-12': (4, 14), '13-22': (14, 19)}
\nfor (sample_submission, test) in iter_test:
\n target_level_group = level_groups_reverse[test.level_group.iloc[0]]
\n df = add_features(data_engineer(test))
fold = 0\npreds = []\nfor q in range(*questions[level_groups[target_level_group]]):\n model = models[q - 1]\n feature_cols = model.feature_names_\n pred = model.predict_proba(df[FEATURES].astype(np.float32))[0,1]\n preds.append(int(pred > 0.63))\n\nsample_submission[\"correct\"] = preds\n\nenv.predict(sample_submission)\"\nwhere models is the list of 18 models, and FEATURES is a list of column names used for predictions. df is created in the same way i created the df for training
", "rawMarkdown": "Thanks for your reply! I tried doing it accordingly to the first notebook you provided but get an error. \n\n\"You must call `predict()` successfully before you can continue with `iter_test()`:\n---------------------------------------------------------------------------\nTypeError Traceback (most recent call last)\n/tmp/ipykernel_25/3104309027.py inIt looks like you might have already started iterating over iter_test. Restarting your notebook kernel and rerunning it should work with the code that you've got there - essentially that env.predict line has to run before the API can move on to the next piece of data
", "rawMarkdown": "It looks like you might have already started iterating over iter_test. Restarting your notebook kernel and rerunning it should work with the code that you've got there - essentially that env.predict line has to run before the API can move on to the next piece of data" }, { "id": 2158729, "postDate": "2023-02-25T05:28:16.910Z", "content": "Ah, that was indeed the problem. I thought I could just restart the run of the cell.. Thanks a lot!
", "rawMarkdown": "Ah, that was indeed the problem. I thought I could just restart the run of the cell.. Thanks a lot!" } ] } ] }, { "id": 2156236, "postDate": "2023-02-23T07:24:38.413Z", "content": "I'm stuck creating a submission file.
\nI've trained 18 xgboost models, one for each question, and stored them in a list. Now i want to make a submission but i have no idea how (am new, sorry). To be precise, how do i select the right data to make a prediction with and how do i write the prediction in the submission file? Maybe someone can help? If you need more info feel free to ask me
Hi, I'm also new here and trained in ways similar to you. Hope my submission code could help. I mainly follows Chris notebook. I 'm not so sure how to add codes by markdown :) lol
\nmodels = {}\nfor t in range(1,19):\n if t<=3: grp = '0-4'\n elif t<=13: grp = '5-12'\n elif t<=18: grp = '13-22'\n clf = XGBClassifier()\n clf.load_model(work_path + f'XGB_question_{t}.xgb')\n models[f'{grp}_{t}'] = clf\n\nimport jo_wilder\nenv = jo_wilder.make_env()\niter_test = env.iter_test()\n\n\nlimits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nbest_threshold = {0: 0.631, 1: 0.613, 2: 0.607, 3: 0.653, 4: 0.625, 5: 0.637, 6: 0.637, 7: 0.605, 8: 0.631, 9: 0.602, 10: 0.636, 11: 0.66, 12: 0.612, 13: 0.647, 14: 0.626, 15: 0.63, 16: 0.61, 17: 0.602}\ncount = 0\nFEATURES_QUESTION = np.loadtxt(work_path + 'FEATURES_QUESTION.txt',dtype=str, delimiter = ' ').tolist()\n\n\nfor (sample_submission, test) in iter_test:\n df = feature_engineer(test)\n #if count == 0:\n #FEATURES = [c for c in df.columns if c != 'level_group']\n #count += 1\n grp = test.level_group.values[0]\n a,b = limits[grp]\n for t in range(a,b):\n clf = models[f'{grp}_{t}']\n p = clf.predict_proba(df[FEATURES_QUESTION[t-1]].astype('float32'))[:,1]\n mask = sample_submission.session_id.str.contains(f'q{t}')\n sample_submission.loc[mask,'correct'] = int(p.item()>best_threshold[t-1])\n env.predict(sample_submission)\nThanks! It does indeed look similar. I still get the error, though.. So i must do it wrong. Basically, i copied everything starting from \"import jo_wilder\" to the end. My models are already stored the same way you did it. Then I changed FEATURE_QUESTION to a list of the features my models are using. And I changed clf.predict_proba(df[FEATURES_QUESTION[t-1]].astype('float32'))[:,1] into clf.predict_proba(df[FEATURES_QUESTION].astype('float32'))[:,1], so that every model uses the same features. Still doesnt work :/
", "votes": 0, "replies": [] } ] }, { "id": 2156338, "author_name": "Jude Hunt", "author_url": "", "post_date": "2023-02-23T09:25:55.650000", "content": "Have a look at this great notebook by @carnozhao - it takes some pretrained models saved in a kaggle dataset, defines a feature engineering function (done with polars instead of pandas!), and then cells 4-6 set up the API needed for the submission and then loops through the data for submission to perform the relevant feature engineering and predict using the pretrained models https://www.kaggle.com/code/carnozhao/cpu-catboost-baseline-using-polars-inference
\nThere's also this notebook for a sample submission that shows how that API works! https://www.kaggle.com/code/carnozhao/cpu-catboost-baseline-using-polars-inference
", "votes": 0, "replies": [ { "id": 2156605, "author_name": "Julius Bohnen", "author_url": "", "post_date": "2023-02-23T12:37:51.757000", "content": "Thanks for your reply! I tried doing it accordingly to the first notebook you provided but get an error.
\npredict() successfully before you can continue with iter_test():TypeError Traceback (most recent call last)
\n/tmp/ipykernel_25/3104309027.py in
\n 3 levels = {'0-4': (0, 5), '5-12': (5, 13), '13-22': (13, 23)}
\n 4 questions = {'0-4': (1, 4), '5-12': (4, 14), '13-22': (14, 19)}
\n----> 5 for (sample_submission, test) in iter_test:
\n 6 target_level_group = level_groups_reverse[test.level_group.iloc[0]]
\n 7 df = feature_engineer(test)
TypeError: cannot unpack non-iterable NoneType object
\nI did it like this.
\nlevel_groups = [\"0-4\", \"5-12\", \"13-22\"]
\nlevel_groups_reverse = {'0-4': 0, '5-12': 1, '13-22': 2}
\nlevels = {'0-4': (0, 5), '5-12': (5, 13), '13-22': (13, 23)}
\nquestions = {'0-4': (1, 4), '5-12': (4, 14), '13-22': (14, 19)}
\nfor (sample_submission, test) in iter_test:
\n target_level_group = level_groups_reverse[test.level_group.iloc[0]]
\n df = add_features(data_engineer(test))
fold = 0\npreds = []\nfor q in range(*questions[level_groups[target_level_group]]):\n model = models[q - 1]\n feature_cols = model.feature_names_\n pred = model.predict_proba(df[FEATURES].astype(np.float32))[0,1]\n preds.append(int(pred > 0.63))\n\nsample_submission[\"correct\"] = preds\n\nenv.predict(sample_submission)\"\nwhere models is the list of 18 models, and FEATURES is a list of column names used for predictions. df is created in the same way i created the df for training
", "votes": 1, "replies": [ { "id": 2157044, "author_name": "Jude Hunt", "author_url": "", "post_date": "2023-02-23T17:37:40.007000", "content": "It looks like you might have already started iterating over iter_test. Restarting your notebook kernel and rerunning it should work with the code that you've got there - essentially that env.predict line has to run before the API can move on to the next piece of data
", "votes": 0, "replies": [] }, { "id": 2158729, "author_name": "Julius Bohnen", "author_url": "", "post_date": "2023-02-25T05:28:16.910000", "content": "Ah, that was indeed the problem. I thought I could just restart the run of the cell.. Thanks a lot!
", "votes": 0, "replies": [] } ] } ] }, { "id": 2156922, "author_name": "", "author_url": "", "post_date": "2023-02-23T16:06:29.427000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2156946": "Hi, I'm also new here and trained in ways similar to you. Hope my submission code could help. I mainly follows Chris [notebook](https://www.kaggle.com/code/cdeotte/xgboost-baseline-0-676). I 'm not so sure how to add codes by markdown :) lol\n\n\n```\nmodels = {}\nfor t in range(1,19):\n if t<=3: grp = '0-4'\n elif t<=13: grp = '5-12'\n elif t<=18: grp = '13-22'\n clf = XGBClassifier()\n clf.load_model(work_path + f'XGB_question_{t}.xgb')\n models[f'{grp}_{t}'] = clf\n\nimport jo_wilder\nenv = jo_wilder.make_env()\niter_test = env.iter_test()\n\n\nlimits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\nbest_threshold = {0: 0.631, 1: 0.613, 2: 0.607, 3: 0.653, 4: 0.625, 5: 0.637, 6: 0.637, 7: 0.605, 8: 0.631, 9: 0.602, 10: 0.636, 11: 0.66, 12: 0.612, 13: 0.647, 14: 0.626, 15: 0.63, 16: 0.61, 17: 0.602}\ncount = 0\nFEATURES_QUESTION = np.loadtxt(work_path + 'FEATURES_QUESTION.txt',dtype=str, delimiter = ' ').tolist()\n\n\nfor (sample_submission, test) in iter_test:\n df = feature_engineer(test)\n #if count == 0:\n #FEATURES = [c for c in df.columns if c != 'level_group']\n #count += 1\n grp = test.level_group.values[0]\n a,b = limits[grp]\n for t in range(a,b):\n clf = models[f'{grp}_{t}']\n p = clf.predict_proba(df[FEATURES_QUESTION[t-1]].astype('float32'))[:,1]\n mask = sample_submission.session_id.str.contains(f'q{t}')\n sample_submission.loc[mask,'correct'] = int(p.item()>best_threshold[t-1])\n env.predict(sample_submission)\n```", "2156338": "Have a look at this great notebook by @carnozhao - it takes some pretrained models saved in a kaggle dataset, defines a feature engineering function (done with polars instead of pandas!), and then cells 4-6 set up the API needed for the submission and then loops through the data for submission to perform the relevant feature engineering and predict using the pretrained models https://www.kaggle.com/code/carnozhao/cpu-catboost-baseline-using-polars-inference\n\nThere's also this notebook for a sample submission that shows how that API works! https://www.kaggle.com/code/carnozhao/cpu-catboost-baseline-using-polars-inference", "2156236": "I'm stuck creating a submission file. \nI've trained 18 xgboost models, one for each question, and stored them in a list. Now i want to make a submission but i have no idea how (am new, sorry). To be precise, how do i select the right data to make a prediction with and how do i write the prediction in the submission file? Maybe someone can help? If you need more info feel free to ask me\n", "2156922": "" } }