{ "id": 488257, "title": "How to install libraries during evaluation?", "url": "/competitions/home-credit-credit-risk-model-stability/discussion/488257", "author_name": "", "post_date": "2024-04-01T19:36:02.094838200Z", "votes": 2, "comment_count": 7, "views": 0, "content": "

I want to use some open-source libraries, and I can install them using pip. However, since the internet is disabled during evaluation I guess I won't be able to use them in submission. How can I use additional libraries?

", "messages": [ { "id": "2727481", "postDate": "04/01/2024 19:36:02", "content": "

I want to use some open-source libraries, and I can install them using pip. However, since the internet is disabled during evaluation I guess I won't be able to use them in submission. How can I use additional libraries?

", "rawMarkdown": "I want to use some open-source libraries, and I can install them using pip. However, since the internet is disabled during evaluation I guess I won't be able to use them in submission. How can I use additional libraries?", "votes": null }, { "id": "2727599", "postDate": "04/01/2024 20:42:43", "content": "

@nerkan you need to separately install these libraries in another dataset/ kernel using their .whl files and import them in your reference kernel.

", "rawMarkdown": "nerkan you need to separately install these libraries in another dataset/ kernel using their .whl files and import them in your reference kernel.", "votes": null }, { "id": "2727614", "postDate": "04/01/2024 20:58:25", "content": "

Just follow directions as outlined in pip-install-no-internet notebook, @nerkan.

", "rawMarkdown": "Just follow directions as outlined in [pip-install-no-internet](https://www.kaggle.com/code/kononenko/pip-install-no-internet) notebook, @nerkan.", "votes": null }, { "id": "2728557", "postDate": "04/02/2024 10:04:49", "content": "

I did as it is outlined. However:

\n
    \n
  1. I can't see the \"data\" section on the right panel. Only input. Are they the same?
    2. \n
  2. In file section I can see the option \"add utility script\". It opens the panel \"Add data\" on the right panel. But it is completely empty despite I saved the notebook with installation and imports.
    3. \n
  3. I have added auxiliary notebook as input and it is shown as utility script. But import of desired libraries in the main script still doesn't work.
    \nCan you help me with that?
    4. \n
", "rawMarkdown": "I did as it is outlined. However:\n1. I can't see the \"data\" section on the right panel. Only input. Are they the same?\n2. In file section I can see the option \"add utility script\". It opens the panel \"Add data\" on the right panel. But it is completely empty despite I saved the notebook with installation and imports.\n3. I have added auxiliary notebook as input and it is shown as utility script. But import of desired libraries in the main script still doesn't work. \nCan you help me with that?", "votes": null }, { "id": "2731674", "postDate": "04/02/2024 20:20:25", "content": "

Dear @nerkan

\n
    \n
  1. Yes, they're the same. It just seems that Kaggle changed \"Data\" to \"Input\" at some point, I've updated my notebook text to reflect that change.
    2. \n
  2. You don't need to do \"Add utility script\" at all. You only need to click \"Set as Utility Script\" in your auxiliary notebook and in the main notebook you add that notebook as the input data.
    3. \n
  3. Have you installed the required packages and saved the auxiliary notebook version?
    4. \n
\n

You can share the sample notebooks you're using and I can take a look.

", "rawMarkdown": "Dear @nerkan\n\n1. Yes, they're the same. It just seems that Kaggle changed \"Data\" to \"Input\" at some point, I've updated my notebook text to reflect that change.\n2. You don't need to do \"Add utility script\" at all. You only need to click \"Set as Utility Script\" in your auxiliary notebook and in the main notebook you add that notebook as the input data.\n3. Have you installed the required packages and saved the auxiliary notebook version? \n\nYou can share the sample notebooks you're using and I can take a look.", "votes": null }, { "id": "2731782", "postDate": "04/02/2024 21:20:34", "content": "

Dear @kononenko,
\nThis is the target notebook, where I need to do the imports
\nhttps://www.kaggle.com/code/nerkan/target-notebook
\nThis is the notebook where I do installation
\nhttps://www.kaggle.com/code/nerkan/import-notebook

\n

In the import notebook I set persistence level on variables and files and set it as utility script. In target notebook I added import notebook as input and disabled internet. And it fails to import library (optbinning in that case) directly, and if it is imported from notebook, it tries to reinstall it and fails due to lack if internet. Also, if I try to remove !pip install lines, it fails also with \"module not found\" error

\n

I would greatly appreciate your help.

", "rawMarkdown": "Dear @kononenko,\nThis is the target notebook, where I need to do the imports\nhttps://www.kaggle.com/code/nerkan/target-notebook\nThis is the notebook where I do installation\nhttps://www.kaggle.com/code/nerkan/import-notebook\n\nIn the import notebook I set persistence level on variables and files and set it as utility script. In target notebook I added import notebook as input and disabled internet. And it fails to import library (optbinning in that case) directly, and if it is imported from notebook, it tries to reinstall it and fails due to lack if internet. Also, if I try to remove !pip install lines, it fails also with \"module not found\" error\n\nI would greatly appreciate your help.", "votes": null }, { "id": "2731862", "postDate": "04/02/2024 22:15:24", "content": "

You are missing --target=/kaggle/working/ when installing the packages, it should be

\n
!pip install ==9.5.2237 =/kaggle/working/\n!pip install optbinning =/kaggle/working/\n
", "rawMarkdown": "You are missing `--target=/kaggle/working/` when installing the packages, it should be\n```\n!pip install ortools==9.5.2237 --target=/kaggle/working/\n!pip install optbinning --target=/kaggle/working/\n```", "votes": null }, { "id": "2732572", "postDate": "04/03/2024 07:49:08", "content": "

Yes, it worked! Don't know how I could miss that. Thanks a lot!
\nI also noticed that that notebook with instructions is also yours so thanks for it as well!

", "rawMarkdown": "Yes, it worked! Don't know how I could miss that. Thanks a lot!\nI also noticed that that notebook with instructions is also yours so thanks for it as well!", "votes": null } ], "comments": [ { "id": 2727599, "author_name": "ravi20076", "author_url": "", "post_date": "04/01/2024 20:42:43", "content": "

@nerkan you need to separately install these libraries in another dataset/ kernel using their .whl files and import them in your reference kernel.

", "votes": null, "replies": [] }, { "id": 2727614, "author_name": "kononenko", "author_url": "", "post_date": "04/01/2024 20:58:25", "content": "

Just follow directions as outlined in pip-install-no-internet notebook, @nerkan.

", "votes": null, "replies": [ { "id": 2728557, "author_name": "nerkan", "author_url": "", "post_date": "04/02/2024 10:04:49", "content": "

I did as it is outlined. However:

\n
    \n
  1. I can't see the \"data\" section on the right panel. Only input. Are they the same?
    2. \n
  2. In file section I can see the option \"add utility script\". It opens the panel \"Add data\" on the right panel. But it is completely empty despite I saved the notebook with installation and imports.
    3. \n
  3. I have added auxiliary notebook as input and it is shown as utility script. But import of desired libraries in the main script still doesn't work.
    \nCan you help me with that?
    4. \n
", "votes": null, "replies": [ { "id": 2731674, "author_name": "kononenko", "author_url": "", "post_date": "04/02/2024 20:20:25", "content": "

Dear @nerkan

\n
    \n
  1. Yes, they're the same. It just seems that Kaggle changed \"Data\" to \"Input\" at some point, I've updated my notebook text to reflect that change.
    2. \n
  2. You don't need to do \"Add utility script\" at all. You only need to click \"Set as Utility Script\" in your auxiliary notebook and in the main notebook you add that notebook as the input data.
    3. \n
  3. Have you installed the required packages and saved the auxiliary notebook version?
    4. \n
\n

You can share the sample notebooks you're using and I can take a look.

", "votes": null, "replies": [ { "id": 2731782, "author_name": "nerkan", "author_url": "", "post_date": "04/02/2024 21:20:34", "content": "

Dear @kononenko,
\nThis is the target notebook, where I need to do the imports
\nhttps://www.kaggle.com/code/nerkan/target-notebook
\nThis is the notebook where I do installation
\nhttps://www.kaggle.com/code/nerkan/import-notebook

\n

In the import notebook I set persistence level on variables and files and set it as utility script. In target notebook I added import notebook as input and disabled internet. And it fails to import library (optbinning in that case) directly, and if it is imported from notebook, it tries to reinstall it and fails due to lack if internet. Also, if I try to remove !pip install lines, it fails also with \"module not found\" error

\n

I would greatly appreciate your help.

", "votes": null, "replies": [ { "id": 2731862, "author_name": "kononenko", "author_url": "", "post_date": "04/02/2024 22:15:24", "content": "

You are missing --target=/kaggle/working/ when installing the packages, it should be

\n
!pip install ==9.5.2237 =/kaggle/working/\n!pip install optbinning =/kaggle/working/\n
", "votes": null, "replies": [ { "id": 2732572, "author_name": "nerkan", "author_url": "", "post_date": "04/03/2024 07:49:08", "content": "

Yes, it worked! Don't know how I could miss that. Thanks a lot!
\nI also noticed that that notebook with instructions is also yours so thanks for it as well!

", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2727481": "I want to use some open-source libraries, and I can install them using pip. However, since the internet is disabled during evaluation I guess I won't be able to use them in submission. How can I use additional libraries?", "2727599": "nerkan you need to separately install these libraries in another dataset/ kernel using their .whl files and import them in your reference kernel.", "2727614": "Just follow directions as outlined in [pip-install-no-internet](https://www.kaggle.com/code/kononenko/pip-install-no-internet) notebook, @nerkan.", "2728557": "I did as it is outlined. However:\n1. I can't see the \"data\" section on the right panel. Only input. Are they the same?\n2. In file section I can see the option \"add utility script\". It opens the panel \"Add data\" on the right panel. But it is completely empty despite I saved the notebook with installation and imports.\n3. I have added auxiliary notebook as input and it is shown as utility script. But import of desired libraries in the main script still doesn't work. \nCan you help me with that?", "2731674": "Dear @nerkan\n\n1. Yes, they're the same. It just seems that Kaggle changed \"Data\" to \"Input\" at some point, I've updated my notebook text to reflect that change.\n2. You don't need to do \"Add utility script\" at all. You only need to click \"Set as Utility Script\" in your auxiliary notebook and in the main notebook you add that notebook as the input data.\n3. Have you installed the required packages and saved the auxiliary notebook version? \n\nYou can share the sample notebooks you're using and I can take a look.", "2731782": "Dear @kononenko,\nThis is the target notebook, where I need to do the imports\nhttps://www.kaggle.com/code/nerkan/target-notebook\nThis is the notebook where I do installation\nhttps://www.kaggle.com/code/nerkan/import-notebook\n\nIn the import notebook I set persistence level on variables and files and set it as utility script. In target notebook I added import notebook as input and disabled internet. And it fails to import library (optbinning in that case) directly, and if it is imported from notebook, it tries to reinstall it and fails due to lack if internet. Also, if I try to remove !pip install lines, it fails also with \"module not found\" error\n\nI would greatly appreciate your help.", "2731862": "You are missing `--target=/kaggle/working/` when installing the packages, it should be\n```\n!pip install ortools==9.5.2237 --target=/kaggle/working/\n!pip install optbinning --target=/kaggle/working/\n```", "2732572": "Yes, it worked! Don't know how I could miss that. Thanks a lot!\nI also noticed that that notebook with instructions is also yours so thanks for it as well!" }, "source": "meta" }