{ "id": 484568, "title": "About submission", "url": "/competitions/home-credit-credit-risk-model-stability/discussion/484568", "author_name": "", "post_date": "2024-03-17T08:13:20.131153700Z", "votes": 3, "comment_count": 16, "views": 0, "content": "

Hello,
\nMy team's notebook is ready but I tried to submit it a few times but all of them it failed (Error: Run out of memory). I have few questions about it.

\n
    \n

  1. Is there any way to connect a kaggle notebook with external server ?

    2. \n

  2. Can I use finalized data to train my model and then submit it (I will comment all lines for processing data then upload finalized data then train the model) ?

    3. \n
\n

Note: This is my first competition. Thank you for your guidance/support.

", "messages": [ { "id": "2701763", "postDate": "03/17/2024 08:13:20", "content": "

Hello,
\nMy team's notebook is ready but I tried to submit it a few times but all of them it failed (Error: Run out of memory). I have few questions about it.

\n
    \n

  1. Is there any way to connect a kaggle notebook with external server ?

    2. \n

  2. Can I use finalized data to train my model and then submit it (I will comment all lines for processing data then upload finalized data then train the model) ?

    3. \n
\n

Note: This is my first competition. Thank you for your guidance/support.

", "rawMarkdown": "Hello,\nMy team's notebook is ready but I tried to submit it a few times but all of them it failed (Error: Run out of memory). I have few questions about it.\n\n1. Is there any way to connect a kaggle notebook with external server ?\n\n2. Can I use finalized data to train my model and then submit it (I will comment all lines for processing data then upload finalized data then train the model) ?\n\n**Note:** This is my first competition. Thank you for your guidance/support.", "votes": null }, { "id": "2701787", "postDate": "03/17/2024 08:25:38", "content": "

@elvinrustam
\nI don't think we can connect a Kaggle notebook to an external server, so this is not possible.
\nYou can try and use Polars lazy execution and perhaps help yourself with the feature generation process.
\nYou can very well train the model outside Kaggle and import the pickled files as datasets and use the inference process in the test predictions. Also, you can try and predict in batches too, another way to save memory as below-

\n
    \n
  1. Import the saved model as a dataset component
    2. \n
  2. Use it to predict a batch of the test data (say 10000 rows)
    3. \n
  3. Save the batch result on the disk (/kaggle/working)
    4. \n
  4. Purge the interim files to conserve space and RAM
    5. \n
  5. Continue the steps 3-4 till the entire test set is covered
    6. \n
  6. Importing one disk prediction file at a time and appending
    7. \n
  7. Saving the submission file and submitting to the LB
    8. \n
  8. I did this successfully and my process works. You may peruse this from my public materials to know more of the code process in this regard.
    9. \n
\n

Best wishes @elvinrustam

", "rawMarkdown": "elvinrustam \nI don't think we can connect a Kaggle notebook to an external server, so this is not possible. \nYou can try and use Polars lazy execution and perhaps help yourself with the feature generation process. \nYou can very well train the model outside Kaggle and import the pickled files as datasets and use the inference process in the test predictions. Also, you can try and predict in batches too, another way to save memory as below-\n1. Import the saved model as a dataset component\n2. Use it to predict a batch of the test data (say 10000 rows)\n3. Save the batch result on the disk (/kaggle/working)\n4. Purge the interim files to conserve space and RAM\n5. Continue the steps 3-4 till the entire test set is covered \n6. Importing one disk prediction file at a time and appending \n7. Saving the submission file and submitting to the LB\n8. I did this successfully and my process works. You may peruse this from my public materials to know more of the code process in this regard. \n\nBest wishes @elvinrustam", "votes": null }, { "id": "2701809", "postDate": "03/17/2024 08:38:52", "content": "

Thank you, I will try!

", "rawMarkdown": "Thank you, I will try!", "votes": null }, { "id": "2702089", "postDate": "03/17/2024 11:43:04", "content": "

1) Is there any way to connect a kaggle notebook with external server ? - the moment this would be allowed, the whole test set would leak and that is something we don't want
\n2) Can I use finalized data to train my model and then submit it (I will comment all lines for processing data then upload finalized data then train the model) ? - I believe you can upload the trained model as dataset and then add it to the notebook.

", "rawMarkdown": "1) Is there any way to connect a kaggle notebook with external server ? - the moment this would be allowed, the whole test set would leak and that is something we don't want\n2) Can I use finalized data to train my model and then submit it (I will comment all lines for processing data then upload finalized data then train the model) ? - I believe you can upload the trained model as dataset and then add it to the notebook.", "votes": null }, { "id": "2702872", "postDate": "03/17/2024 21:23:19", "content": "

Can external Python libraries be used (by wheel)? which are not present by default in Kaggle server

", "rawMarkdown": "Can external Python libraries be used (by wheel)? which are not present by default in Kaggle server", "votes": null }, { "id": "2702881", "postDate": "03/17/2024 21:28:12", "content": "

There will be limit of size, but apart from that yes. Just remember that you can not use proprietary or licensed code.

", "rawMarkdown": "There will be limit of size, but apart from that yes. Just remember that you can not use proprietary or licensed code.", "votes": null }, { "id": "2703630", "postDate": "03/18/2024 10:08:17", "content": "

I did everything and in kaggle notebook it worked without any errors. But when I try to sumbit it, it gives me run out of memory. I don't understand one thing if my notebook works in kaggle without any error how can error occurs when I submit it?

", "rawMarkdown": "I did everything and in kaggle notebook it worked without any errors. But when I try to sumbit it, it gives me run out of memory. I don't understand one thing if my notebook works in kaggle without any error how can error occurs when I submit it?", "votes": null }, { "id": "2706996", "postDate": "03/20/2024 08:28:14", "content": "

The same thing just happened to me. The weirdest thing was that I only preprocessed three features and gave a try, but it showed \"out of memory\". I am considering narrowing down the training datasets by picking some case_ids inside them randomly at the very beginning, but I don't know if it will influence the accuracy and AUC significantly.

", "rawMarkdown": "The same thing just happened to me. The weirdest thing was that I only preprocessed three features and gave a try, but it showed \"out of memory\". I am considering narrowing down the training datasets by picking some case_ids inside them randomly at the very beginning, but I don't know if it will influence the accuracy and AUC significantly.", "votes": null }, { "id": "2707935", "postDate": "03/20/2024 18:37:26", "content": "

Thanks, I will try to picking some case_ids

", "rawMarkdown": "Thanks, I will try to picking some case_ids", "votes": null }, { "id": "2710134", "postDate": "03/22/2024 03:42:52", "content": "

Have you found any better way to solve the problem? The way I suggested earlier failed for me too.

", "rawMarkdown": "Have you found any better way to solve the problem? The way I suggested earlier failed for me too.", "votes": null }, { "id": "2711099", "postDate": "03/22/2024 18:04:23", "content": "

No and I did what you said and it failed again. @jetakow can you help us? My notebook works successfully (27 sec) but when I submit it just scoring for 18-20 minutes then gives me an error (run out of memory). I used 25-30% percent of train data just because of this error but i failed again.

", "rawMarkdown": "No and I did what you said and it failed again. @jetakow can you help us? My notebook works successfully (27 sec) but when I submit it just scoring for 18-20 minutes then gives me an error (run out of memory). I used 25-30% percent of train data just because of this error but i failed again.", "votes": null }, { "id": "2711488", "postDate": "03/22/2024 21:57:07", "content": "

It is hard for me to pinpoint what could be the issue without seeing the code. I suggest you test the scoring on your train set first and load it as instead of test set. After you are sure it all works, then submit it.

", "rawMarkdown": "It is hard for me to pinpoint what could be the issue without seeing the code. I suggest you test the scoring on your train set first and load it as instead of test set. After you are sure it all works, then submit it.", "votes": null }, { "id": "2712062", "postDate": "03/23/2024 09:21:19", "content": "

Everything is working well, my notebook is working without error, it scores test set and create a submission file but when I submit it, an error occurs (run out of memory). I don't know how to solve this problem I used only 10% of train set just to submit but again I failed (notebook works in kaggle without errors, less than 20 sec)

", "rawMarkdown": "Everything is working well, my notebook is working without error, it scores test set and create a submission file but when I submit it, an error occurs (run out of memory). I don't know how to solve this problem I used only 10% of train set just to submit but again I failed (notebook works in kaggle without errors, less than 20 sec)", "votes": null }, { "id": "2712399", "postDate": "03/23/2024 14:35:39", "content": "

I suggest testing it with e.g. 0.5 score for every case_id from test set. If the error persists, that would imply it is not due operations within the notebook itself.

", "rawMarkdown": "I suggest testing it with e.g. 0.5 score for every case_id from test set. If the error persists, that would imply it is not due operations within the notebook itself.", "votes": null }, { "id": "2713696", "postDate": "03/24/2024 11:14:10", "content": "

@jetakow I don't use train data in my notebook just test data and pretrained model (model is 243kb). Everything is working very well but I can't submit and can't find a solution

", "rawMarkdown": "jetakow I don't use train data in my notebook just test data and pretrained model (model is 243kb). Everything is working very well but I can't submit and can't find a solution", "votes": null }, { "id": "2713949", "postDate": "03/24/2024 14:48:24", "content": "

Please create a new post with minimal example to reproduce the error. Provide the the error messages if any and describe the behavior. I can't see any successful submissions by you, is that correct?

", "rawMarkdown": "Please create a new post with minimal example to reproduce the error. Provide the the error messages if any and describe the behavior. I can't see any successful submissions by you, is that correct?", "votes": null }, { "id": "2715069", "postDate": "03/25/2024 08:55:22", "content": "

Yes, it is correct i tried 9 times and all of them failed. Okay I will create a new post with screenshots

", "rawMarkdown": "Yes, it is correct i tried 9 times and all of them failed. Okay I will create a new post with screenshots", "votes": null } ], "comments": [ { "id": 2701787, "author_name": "ravi20076", "author_url": "", "post_date": "03/17/2024 08:25:38", "content": "

@elvinrustam
\nI don't think we can connect a Kaggle notebook to an external server, so this is not possible.
\nYou can try and use Polars lazy execution and perhaps help yourself with the feature generation process.
\nYou can very well train the model outside Kaggle and import the pickled files as datasets and use the inference process in the test predictions. Also, you can try and predict in batches too, another way to save memory as below-

\n
    \n
  1. Import the saved model as a dataset component
    2. \n
  2. Use it to predict a batch of the test data (say 10000 rows)
    3. \n
  3. Save the batch result on the disk (/kaggle/working)
    4. \n
  4. Purge the interim files to conserve space and RAM
    5. \n
  5. Continue the steps 3-4 till the entire test set is covered
    6. \n
  6. Importing one disk prediction file at a time and appending
    7. \n
  7. Saving the submission file and submitting to the LB
    8. \n
  8. I did this successfully and my process works. You may peruse this from my public materials to know more of the code process in this regard.
    9. \n
\n

Best wishes @elvinrustam

", "votes": null, "replies": [ { "id": 2701809, "author_name": "elvinrustam", "author_url": "", "post_date": "03/17/2024 08:38:52", "content": "

Thank you, I will try!

", "votes": null, "replies": [] } ] }, { "id": 2702089, "author_name": "jetakow", "author_url": "", "post_date": "03/17/2024 11:43:04", "content": "

1) Is there any way to connect a kaggle notebook with external server ? - the moment this would be allowed, the whole test set would leak and that is something we don't want
\n2) Can I use finalized data to train my model and then submit it (I will comment all lines for processing data then upload finalized data then train the model) ? - I believe you can upload the trained model as dataset and then add it to the notebook.

", "votes": null, "replies": [ { "id": 2702872, "author_name": "shreyas9181", "author_url": "", "post_date": "03/17/2024 21:23:19", "content": "

Can external Python libraries be used (by wheel)? which are not present by default in Kaggle server

", "votes": null, "replies": [ { "id": 2702881, "author_name": "jetakow", "author_url": "", "post_date": "03/17/2024 21:28:12", "content": "

There will be limit of size, but apart from that yes. Just remember that you can not use proprietary or licensed code.

", "votes": null, "replies": [] } ] }, { "id": 2703630, "author_name": "elvinrustam", "author_url": "", "post_date": "03/18/2024 10:08:17", "content": "

I did everything and in kaggle notebook it worked without any errors. But when I try to sumbit it, it gives me run out of memory. I don't understand one thing if my notebook works in kaggle without any error how can error occurs when I submit it?

", "votes": null, "replies": [ { "id": 2706996, "author_name": "tianjunmaftk", "author_url": "", "post_date": "03/20/2024 08:28:14", "content": "

The same thing just happened to me. The weirdest thing was that I only preprocessed three features and gave a try, but it showed \"out of memory\". I am considering narrowing down the training datasets by picking some case_ids inside them randomly at the very beginning, but I don't know if it will influence the accuracy and AUC significantly.

", "votes": null, "replies": [ { "id": 2707935, "author_name": "elvinrustam", "author_url": "", "post_date": "03/20/2024 18:37:26", "content": "

Thanks, I will try to picking some case_ids

", "votes": null, "replies": [ { "id": 2710134, "author_name": "tianjunmaftk", "author_url": "", "post_date": "03/22/2024 03:42:52", "content": "

Have you found any better way to solve the problem? The way I suggested earlier failed for me too.

", "votes": null, "replies": [ { "id": 2711099, "author_name": "elvinrustam", "author_url": "", "post_date": "03/22/2024 18:04:23", "content": "

No and I did what you said and it failed again. @jetakow can you help us? My notebook works successfully (27 sec) but when I submit it just scoring for 18-20 minutes then gives me an error (run out of memory). I used 25-30% percent of train data just because of this error but i failed again.

", "votes": null, "replies": [ { "id": 2711488, "author_name": "jetakow", "author_url": "", "post_date": "03/22/2024 21:57:07", "content": "

It is hard for me to pinpoint what could be the issue without seeing the code. I suggest you test the scoring on your train set first and load it as instead of test set. After you are sure it all works, then submit it.

", "votes": null, "replies": [ { "id": 2712062, "author_name": "elvinrustam", "author_url": "", "post_date": "03/23/2024 09:21:19", "content": "

Everything is working well, my notebook is working without error, it scores test set and create a submission file but when I submit it, an error occurs (run out of memory). I don't know how to solve this problem I used only 10% of train set just to submit but again I failed (notebook works in kaggle without errors, less than 20 sec)

", "votes": null, "replies": [ { "id": 2712399, "author_name": "jetakow", "author_url": "", "post_date": "03/23/2024 14:35:39", "content": "

I suggest testing it with e.g. 0.5 score for every case_id from test set. If the error persists, that would imply it is not due operations within the notebook itself.

", "votes": null, "replies": [ { "id": 2713696, "author_name": "elvinrustam", "author_url": "", "post_date": "03/24/2024 11:14:10", "content": "

@jetakow I don't use train data in my notebook just test data and pretrained model (model is 243kb). Everything is working very well but I can't submit and can't find a solution

", "votes": null, "replies": [ { "id": 2713949, "author_name": "jetakow", "author_url": "", "post_date": "03/24/2024 14:48:24", "content": "

Please create a new post with minimal example to reproduce the error. Provide the the error messages if any and describe the behavior. I can't see any successful submissions by you, is that correct?

", "votes": null, "replies": [ { "id": 2715069, "author_name": "elvinrustam", "author_url": "", "post_date": "03/25/2024 08:55:22", "content": "

Yes, it is correct i tried 9 times and all of them failed. Okay I will create a new post with screenshots

", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] } ] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2701763": "Hello,\nMy team's notebook is ready but I tried to submit it a few times but all of them it failed (Error: Run out of memory). I have few questions about it.\n\n1. Is there any way to connect a kaggle notebook with external server ?\n\n2. Can I use finalized data to train my model and then submit it (I will comment all lines for processing data then upload finalized data then train the model) ?\n\n**Note:** This is my first competition. Thank you for your guidance/support.", "2701787": "elvinrustam \nI don't think we can connect a Kaggle notebook to an external server, so this is not possible. \nYou can try and use Polars lazy execution and perhaps help yourself with the feature generation process. \nYou can very well train the model outside Kaggle and import the pickled files as datasets and use the inference process in the test predictions. Also, you can try and predict in batches too, another way to save memory as below-\n1. Import the saved model as a dataset component\n2. Use it to predict a batch of the test data (say 10000 rows)\n3. Save the batch result on the disk (/kaggle/working)\n4. Purge the interim files to conserve space and RAM\n5. Continue the steps 3-4 till the entire test set is covered \n6. Importing one disk prediction file at a time and appending \n7. Saving the submission file and submitting to the LB\n8. I did this successfully and my process works. You may peruse this from my public materials to know more of the code process in this regard. \n\nBest wishes @elvinrustam", "2701809": "Thank you, I will try!", "2702089": "1) Is there any way to connect a kaggle notebook with external server ? - the moment this would be allowed, the whole test set would leak and that is something we don't want\n2) Can I use finalized data to train my model and then submit it (I will comment all lines for processing data then upload finalized data then train the model) ? - I believe you can upload the trained model as dataset and then add it to the notebook.", "2702872": "Can external Python libraries be used (by wheel)? which are not present by default in Kaggle server", "2702881": "There will be limit of size, but apart from that yes. Just remember that you can not use proprietary or licensed code.", "2703630": "I did everything and in kaggle notebook it worked without any errors. But when I try to sumbit it, it gives me run out of memory. I don't understand one thing if my notebook works in kaggle without any error how can error occurs when I submit it?", "2706996": "The same thing just happened to me. The weirdest thing was that I only preprocessed three features and gave a try, but it showed \"out of memory\". I am considering narrowing down the training datasets by picking some case_ids inside them randomly at the very beginning, but I don't know if it will influence the accuracy and AUC significantly.", "2707935": "Thanks, I will try to picking some case_ids", "2710134": "Have you found any better way to solve the problem? The way I suggested earlier failed for me too.", "2711099": "No and I did what you said and it failed again. @jetakow can you help us? My notebook works successfully (27 sec) but when I submit it just scoring for 18-20 minutes then gives me an error (run out of memory). I used 25-30% percent of train data just because of this error but i failed again.", "2711488": "It is hard for me to pinpoint what could be the issue without seeing the code. I suggest you test the scoring on your train set first and load it as instead of test set. After you are sure it all works, then submit it.", "2712062": "Everything is working well, my notebook is working without error, it scores test set and create a submission file but when I submit it, an error occurs (run out of memory). I don't know how to solve this problem I used only 10% of train set just to submit but again I failed (notebook works in kaggle without errors, less than 20 sec)", "2712399": "I suggest testing it with e.g. 0.5 score for every case_id from test set. If the error persists, that would imply it is not due operations within the notebook itself.", "2713696": "jetakow I don't use train data in my notebook just test data and pretrained model (model is 243kb). Everything is working very well but I can't submit and can't find a solution", "2713949": "Please create a new post with minimal example to reproduce the error. Provide the the error messages if any and describe the behavior. I can't see any successful submissions by you, is that correct?", "2715069": "Yes, it is correct i tried 9 times and all of them failed. Okay I will create a new post with screenshots" }, "source": "meta" }