Hi guys,
\nThis is my third day of encountering this problem. I have tried to use a bunch of different models. So here while using catboost models in this loop the submission works perfectly fine. But when I changed it to my tabnet model submission fails despite the fact that the submission notebook works perfectly fine and the submission notebook is generated just like my catboost notebook.
(test,sample_submission) iter_test:\n target_level_group = level_groups_reverse[test.level_group.iloc[]]\n df = trans(test)\n preds = []\n q (*questions[level_groups[target_level_group]]):\n model = models_list[q - ][]\n pred = model.predict_proba(df[feature_cols].astype(np.float32).values)[,]\n preds.append((pred > ))\n sample_submission[] = preds\n env.predict(sample_submission)\nCan anyone please help me out I have wasted over 10 submissions to get over it but still can't get past it. 😔
", "messages": [ { "id": "2257184", "postDate": "05/13/2023 05:24:20", "content": "Hi guys,
\nThis is my third day of encountering this problem. I have tried to use a bunch of different models. So here while using catboost models in this loop the submission works perfectly fine. But when I changed it to my tabnet model submission fails despite the fact that the submission notebook works perfectly fine and the submission notebook is generated just like my catboost notebook.
(test,sample_submission) iter_test:\n target_level_group = level_groups_reverse[test.level_group.iloc[]]\n df = trans(test)\n preds = []\n q (*questions[level_groups[target_level_group]]):\n model = models_list[q - ][]\n pred = model.predict_proba(df[feature_cols].astype(np.float32).values)[,]\n preds.append((pred > ))\n sample_submission[] = preds\n env.predict(sample_submission)\nCan anyone please help me out I have wasted over 10 submissions to get over it but still can't get past it. 😔
", "rawMarkdown": "Hi guys, \nThis is my third day of encountering this problem. I have tried to use a bunch of different models. So here while using catboost models in this loop the submission works perfectly fine. But when I changed it to my tabnet model submission fails despite the fact that the submission notebook works perfectly fine and the submission notebook is generated just like my catboost notebook. \n\n```python\nfor (test,sample_submission) in iter_test:\n target_level_group = level_groups_reverse[test.level_group.iloc[0]]\n df = trans(test)\n preds = []\n for q in range(*questions[level_groups[target_level_group]]):\n model = models_list[q - 1][0]\n pred = model.predict_proba(df[feature_cols].astype(np.float32).values)[0,1]\n preds.append(int(pred > 0.60))\n sample_submission[\"correct\"] = preds\n env.predict(sample_submission)\n```\n\nCan anyone please help me out I have wasted over 10 submissions to get over it but still can't get past it. 😔", "votes": null }, { "id": "2257381", "postDate": "05/13/2023 10:00:14", "content": "It's difficult to determine the exact cause of the submission scoring error without more information or error messages. However, one possible issue could be the format of your predicted probabilities from the TabNet model. Make sure that the predicted probabilities are in the same format as those from the CatBoost model, and that they are being thresholded correctly to produce binary predictions.
\nYou may also want to check if there are any differences between the feature engineering or data preprocessing steps used for the TabNet and CatBoost models. These differences could potentially affect the performance of the model on the test set and lead to submission scoring errors.
\nIf you're still encountering issues, try reaching out to the competition host or posting on the competition discussion forum for further assistance.
", "rawMarkdown": "It's difficult to determine the exact cause of the submission scoring error without more information or error messages. However, one possible issue could be the format of your predicted probabilities from the TabNet model. Make sure that the predicted probabilities are in the same format as those from the CatBoost model, and that they are being thresholded correctly to produce binary predictions.\n\nYou may also want to check if there are any differences between the feature engineering or data preprocessing steps used for the TabNet and CatBoost models. These differences could potentially affect the performance of the model on the test set and lead to submission scoring errors.\n\nIf you're still encountering issues, try reaching out to the competition host or posting on the competition discussion forum for further assistance.", "votes": null }, { "id": "2257443", "postDate": "05/13/2023 10:51:12", "content": "Thanks @ericka42 for your detailed answer.
\nMy submission csv looks like this :
session_id correct
\n0 20090109393214576_q1 0
\n1 20090109393214576_q2 1
\n2 20090109393214576_q3 1
\n3 20090109393214576_q4 1
\n4 20090109393214576_q5 0
\n5 20090109393214576_q6 0
\n6 20090109393214576_q7 0
\n7 20090109393214576_q8 1
\n8 20090109393214576_q9 0
\n9 20090109393214576_q10 0
\n10 20090109393214576_q11 0
\n11 20090109393214576_q12 1
\n12 20090109393214576_q13 1
\n13 20090109393214576_q14 0
\n14 20090109393214576_q15 0
\n15 20090109393214576_q16 0
\n16 20090109393214576_q17 0
\n17 20090109393214576_q18 0
\n18 20090312143683264_q1 1
\n19 20090312143683264_q2 1
\n20 20090312143683264_q3 0
\n21 20090312143683264_q4 1
\n22 20090312143683264_q5 0
\n23 20090312143683264_q6 0
\n24 20090312143683264_q7 0
\n25 20090312143683264_q8 1
\n26 20090312143683264_q9 0
\n27 20090312143683264_q10 0
\n28 20090312143683264_q11 0
\n29 20090312143683264_q12 1
\n30 20090312143683264_q13 1
\n31 20090312143683264_q14 1
\n32 20090312143683264_q15 0
\n33 20090312143683264_q16 1
\n34 20090312143683264_q17 0
\n35 20090312143683264_q18 1
\n36 20090312331414616_q1 1
\n37 20090312331414616_q2 1
\n38 20090312331414616_q3 0
\n39 20090312331414616_q4 1
\n40 20090312331414616_q5 1
\n41 20090312331414616_q6 1
\n42 20090312331414616_q7 1
\n43 20090312331414616_q8 1
\n44 20090312331414616_q9 1
\n45 20090312331414616_q10 1
\n46 20090312331414616_q11 1
\n47 20090312331414616_q12 1
\n48 20090312331414616_q13 1
\n49 20090312331414616_q14 1
\n50 20090312331414616_q15 0
\n51 20090312331414616_q16 1
\n52 20090312331414616_q17 0
\n53 20090312331414616_q18 1
the id order is exactly same as that of Catboost Classifier.
\nBoth are very different models so there is some deal of differences of preprocessing but I didn't get how would that effect my submission scoring. They might have if tabnet was exhausting the resources but I get he submission scoring failed immediately after the notebook has ran.
\nIf you're still encountering issues, try reaching out to the competition host or posting on the competition discussion forum for further assistance.
\nWould do that. Thanks!!!
Can anyone from the Kaggle team help me figure it out ?
", "rawMarkdown": "Can anyone from the Kaggle team help me figure it out ?", "votes": null }, { "id": "2263744", "postDate": "05/17/2023 21:33:02", "content": "harshit, did you figure this out? i have the same problem.
", "rawMarkdown": "harshit, did you figure this out? i have the same problem.", "votes": null } ], "comments": [ { "id": 2257381, "author_name": "ericka42", "author_url": "", "post_date": "05/13/2023 10:00:14", "content": "It's difficult to determine the exact cause of the submission scoring error without more information or error messages. However, one possible issue could be the format of your predicted probabilities from the TabNet model. Make sure that the predicted probabilities are in the same format as those from the CatBoost model, and that they are being thresholded correctly to produce binary predictions.
\nYou may also want to check if there are any differences between the feature engineering or data preprocessing steps used for the TabNet and CatBoost models. These differences could potentially affect the performance of the model on the test set and lead to submission scoring errors.
\nIf you're still encountering issues, try reaching out to the competition host or posting on the competition discussion forum for further assistance.
", "votes": null, "replies": [ { "id": 2257443, "author_name": "harshitkmr", "author_url": "", "post_date": "05/13/2023 10:51:12", "content": "Thanks @ericka42 for your detailed answer.
\nMy submission csv looks like this :
session_id correct
\n0 20090109393214576_q1 0
\n1 20090109393214576_q2 1
\n2 20090109393214576_q3 1
\n3 20090109393214576_q4 1
\n4 20090109393214576_q5 0
\n5 20090109393214576_q6 0
\n6 20090109393214576_q7 0
\n7 20090109393214576_q8 1
\n8 20090109393214576_q9 0
\n9 20090109393214576_q10 0
\n10 20090109393214576_q11 0
\n11 20090109393214576_q12 1
\n12 20090109393214576_q13 1
\n13 20090109393214576_q14 0
\n14 20090109393214576_q15 0
\n15 20090109393214576_q16 0
\n16 20090109393214576_q17 0
\n17 20090109393214576_q18 0
\n18 20090312143683264_q1 1
\n19 20090312143683264_q2 1
\n20 20090312143683264_q3 0
\n21 20090312143683264_q4 1
\n22 20090312143683264_q5 0
\n23 20090312143683264_q6 0
\n24 20090312143683264_q7 0
\n25 20090312143683264_q8 1
\n26 20090312143683264_q9 0
\n27 20090312143683264_q10 0
\n28 20090312143683264_q11 0
\n29 20090312143683264_q12 1
\n30 20090312143683264_q13 1
\n31 20090312143683264_q14 1
\n32 20090312143683264_q15 0
\n33 20090312143683264_q16 1
\n34 20090312143683264_q17 0
\n35 20090312143683264_q18 1
\n36 20090312331414616_q1 1
\n37 20090312331414616_q2 1
\n38 20090312331414616_q3 0
\n39 20090312331414616_q4 1
\n40 20090312331414616_q5 1
\n41 20090312331414616_q6 1
\n42 20090312331414616_q7 1
\n43 20090312331414616_q8 1
\n44 20090312331414616_q9 1
\n45 20090312331414616_q10 1
\n46 20090312331414616_q11 1
\n47 20090312331414616_q12 1
\n48 20090312331414616_q13 1
\n49 20090312331414616_q14 1
\n50 20090312331414616_q15 0
\n51 20090312331414616_q16 1
\n52 20090312331414616_q17 0
\n53 20090312331414616_q18 1
the id order is exactly same as that of Catboost Classifier.
\nBoth are very different models so there is some deal of differences of preprocessing but I didn't get how would that effect my submission scoring. They might have if tabnet was exhausting the resources but I get he submission scoring failed immediately after the notebook has ran.
\nIf you're still encountering issues, try reaching out to the competition host or posting on the competition discussion forum for further assistance.
\nWould do that. Thanks!!!
Can anyone from the Kaggle team help me figure it out ?
", "votes": null, "replies": [ { "id": 2263744, "author_name": "yaobvs", "author_url": "", "post_date": "05/17/2023 21:33:02", "content": "harshit, did you figure this out? i have the same problem.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2257184": "Hi guys, \nThis is my third day of encountering this problem. I have tried to use a bunch of different models. So here while using catboost models in this loop the submission works perfectly fine. But when I changed it to my tabnet model submission fails despite the fact that the submission notebook works perfectly fine and the submission notebook is generated just like my catboost notebook. \n\n```python\nfor (test,sample_submission) in iter_test:\n target_level_group = level_groups_reverse[test.level_group.iloc[0]]\n df = trans(test)\n preds = []\n for q in range(*questions[level_groups[target_level_group]]):\n model = models_list[q - 1][0]\n pred = model.predict_proba(df[feature_cols].astype(np.float32).values)[0,1]\n preds.append(int(pred > 0.60))\n sample_submission[\"correct\"] = preds\n env.predict(sample_submission)\n```\n\nCan anyone please help me out I have wasted over 10 submissions to get over it but still can't get past it. 😔", "2257381": "It's difficult to determine the exact cause of the submission scoring error without more information or error messages. However, one possible issue could be the format of your predicted probabilities from the TabNet model. Make sure that the predicted probabilities are in the same format as those from the CatBoost model, and that they are being thresholded correctly to produce binary predictions.\n\nYou may also want to check if there are any differences between the feature engineering or data preprocessing steps used for the TabNet and CatBoost models. These differences could potentially affect the performance of the model on the test set and lead to submission scoring errors.\n\nIf you're still encountering issues, try reaching out to the competition host or posting on the competition discussion forum for further assistance.", "2257443": "Thanks @ericka42 for your detailed answer.\nMy submission csv looks like this :\n\n session_id correct \n0\t 20090109393214576_q1 0\n1\t 20090109393214576_q2 1\n2\t 20090109393214576_q3 1\n3\t 20090109393214576_q4 1\n4\t 20090109393214576_q5 0\n5\t 20090109393214576_q6 0\n6\t 20090109393214576_q7 0\n7\t 20090109393214576_q8 1\n8\t 20090109393214576_q9 0\n9\t 20090109393214576_q10 0\n10\t 20090109393214576_q11 0\n11\t 20090109393214576_q12 1\n12\t 20090109393214576_q13 1\n13\t\t 20090109393214576_q14 0\n14\t 20090109393214576_q15 0\n15\t\t 20090109393214576_q16 0\n16\t\t 20090109393214576_q17 0\n17\t\t 20090109393214576_q18 0\n18\t\t 20090312143683264_q1 1\n19\t\t 20090312143683264_q2 1\n20\t\t 20090312143683264_q3 0\n21\t\t 20090312143683264_q4 1\n22\t\t 20090312143683264_q5 0\n23\t\t 20090312143683264_q6 0\n24\t\t 20090312143683264_q7 0\n25\t\t 20090312143683264_q8 1\n26\t\t 20090312143683264_q9 0\n27\t\t 20090312143683264_q10 0\n28\t\t 20090312143683264_q11 0\n29\t\t 20090312143683264_q12 1\n30\t\t 20090312143683264_q13 1\n31\t\t 20090312143683264_q14 1\n32\t\t 20090312143683264_q15 0\n33\t\t 20090312143683264_q16 1\n34\t\t 20090312143683264_q17 0\n35\t\t 20090312143683264_q18 1\n36\t\t 20090312331414616_q1 1\n37\t\t 20090312331414616_q2 1\n38\t\t 20090312331414616_q3 0\n39\t\t 20090312331414616_q4 1\n40\t\t 20090312331414616_q5 1\n41\t\t 20090312331414616_q6 1\n42\t\t 20090312331414616_q7 1\n43\t\t 20090312331414616_q8 1\n44\t\t 20090312331414616_q9 1\n45\t\t 20090312331414616_q10 1\n46\t\t 20090312331414616_q11 1\n47\t\t 20090312331414616_q12 1\n48\t\t 20090312331414616_q13 1\n49\t\t 20090312331414616_q14 1\n50\t\t 20090312331414616_q15 0\n51\t\t 20090312331414616_q16 1\n52\t\t 20090312331414616_q17 0\n53\t\t 20090312331414616_q18 1 \n\nthe id order is exactly same as that of Catboost Classifier.\n\nBoth are very different models so there is some deal of differences of preprocessing but I didn't get how would that effect my submission scoring. They might have if tabnet was exhausting the resources but I get he submission scoring failed immediately after the notebook has ran. \n\n`If you're still encountering issues, try reaching out to the competition host or posting on the competition discussion forum for further assistance.`\nWould do that. Thanks!!!", "2258527": "Can anyone from the Kaggle team help me figure it out ?", "2263744": "harshit, did you figure this out? i have the same problem." }, "source": "meta" }