{
  "id": 425776,
  "title": "is there a way to use docker image to avoid installation in submission?",
  "url": "/competitions/bengaliai-speech/discussion/425776",
  "author_name": "",
  "post_date": "2023-07-20T10:33:54.064966100Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>it is a hassle to install modules for submission. I wonder is there a way to:</p>\n<ol>\n<li>prepare the image from internet-connected notebook</li>\n<li>for the submission notebook, just setup docker and load image from database. <br>\ninput test directory in docker commandline and product ouput csv.</li>\n</ol>\n<p>has anyone done something smiliar before?<br>\nin short, how to let the kaggle notebook use our docker image?</p>",
  "messages": [
    {
      "id": "2351691",
      "postDate": "07/20/2023 10:33:54",
      "content": "<p>it is a hassle to install modules for submission. I wonder is there a way to:</p>\n<ol>\n<li>prepare the image from internet-connected notebook</li>\n<li>for the submission notebook, just setup docker and load image from database. <br>\ninput test directory in docker commandline and product ouput csv.</li>\n</ol>\n<p>has anyone done something smiliar before?<br>\nin short, how to let the kaggle notebook use our docker image?</p>",
      "rawMarkdown": "it is a hassle to install modules for submission. I wonder is there a way to:\n1. prepare the image from internet-connected notebook\n2. for the submission notebook, just setup docker and load image from database. \ninput test directory in docker commandline and product ouput csv.\n\nhas anyone done something smiliar before?\nin short, how to let the kaggle notebook use our docker image?",
      "votes": null
    },
    {
      "id": "2352229",
      "postDate": "07/20/2023 18:19:38",
      "content": "<p>Probably not. Kaggle doesn't let users to mess with the Notebook environments. So much hassle!</p>",
      "rawMarkdown": "Probably not. Kaggle doesn't let users to mess with the Notebook environments. So much hassle!",
      "votes": null
    },
    {
      "id": "2352288",
      "postDate": "07/20/2023 19:13:29",
      "content": "<p>For me the easiest way to prepare an environment for submission is to create the wheels from a requirements.txt, export to a dataset that's synced to the the notebook.<br>\nAnd then install in the offline the same requirements file from the wheels</p>\n<p>For the online notebook<br>\n<code>pip wheel -r {dataset-with-requirements}/requirements.txt</code></p>\n<p>Export the dataset to <strong>dataset-with-wheels</strong></p>\n<p>For the offline submission<br>\n<code>pip install --no-index --find-links=/kaggle/input/{dataset-with-wheels}/wheels -r {dataset-with-requirements}/requirements.txt</code></p>\n<p>And if you are version controlling you're code with github you can set up an action that runs the online notebook on push to main and will automatically update the wheels so you only have to focuse on the online inference notebook</p>\n<pre><code>    \n\n\n  \n     [ ,  ]\n\n  \n     \n    \n         \n         \n\n          \n         \n        \n           \n           \n          \n         \n        \n             \n             \n        \n           \n          \n         \n        \n             \n             \n         </code></pre>",
      "rawMarkdown": "For me the easiest way to prepare an environment for submission is to create the wheels from a requirements.txt, export to a dataset that's synced to the the notebook.\nAnd then install in the offline the same requirements file from the wheels\n\nFor the online notebook\n`pip wheel -r {dataset-with-requirements}/requirements.txt`\n\nExport the dataset to **dataset-with-wheels**\n\nFor the offline submission\n```pip install --no-index --find-links=/kaggle/input/{dataset-with-wheels}/wheels -r {dataset-with-requirements}/requirements.txt```\n\nAnd if you are version controlling you're code with github you can set up an action that runs the online notebook on push to main and will automatically update the wheels so you only have to focuse on the online inference notebook\n\n```\nname: Update Kaggle Code Dependencies\n\non:\n  push:\n    branches: [ main, master ]\njobs:\n  upload_code_to_kaggle:\n    runs-on: ubuntu-latest\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v3\n\n      - name: Setup Python version\n        uses: actions/setup-python@v3\n        with:\n          python-version: '3.9'\n          cache: 'pip'\n      - name: Install the packages\n        uses: BSFishy/pip-action@v1\n        env:\n          KAGGLE_USERNAME: ${{ secrets.KAGGLE_USERNAME }}\n          KAGGLE_KEY: ${{ secrets.KAGGLE_KEY }}\n        with:\n          packages: kaggle\n      - name: Update Kaggle Dependencies\n        shell: bash\n        env:\n          KAGGLE_USERNAME: ${{ secrets.KAGGLE_USERNAME }}\n          KAGGLE_KEY: ${{ secrets.KAGGLE_KEY }}\n        run: |\n          kaggle kernels pull -m $KAGGLE_USERNAME/dependencies-notebook\n          kaggle kernels push -p .\n```",
      "votes": null
    },
    {
      "id": "2352408",
      "postDate": "07/20/2023 23:30:16",
      "content": "<p>thanks a lot!<br>\ni include a link here for those who are interested in using kaggle CLI to push notebook</p>\n<p><a href=\"https://www.kaggle.com/docs/api\" target=\"_blank\">https://www.kaggle.com/docs/api</a></p>\n<hr>\n<p>Creating and Running a New Notebook</p>\n<p>\"The Kaggle API can be used to to create new Notebooks and Notebook versions on Kaggle from the comfort of the command-line. This can make executing and sharing code on Kaggle a simple part of your workflow.\"</p>",
      "rawMarkdown": "thanks a lot!\ni include a link here for those who are interested in using kaggle CLI to push notebook\n\nhttps://www.kaggle.com/docs/api\n\n----\n\nCreating and Running a New Notebook\n\n\"The Kaggle API can be used to to create new Notebooks and Notebook versions on Kaggle from the comfort of the command-line. This can make executing and sharing code on Kaggle a simple part of your workflow.\"",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2352229,
      "author_name": "mbmmurad",
      "author_url": "",
      "post_date": "07/20/2023 18:19:38",
      "content": "<p>Probably not. Kaggle doesn't let users to mess with the Notebook environments. So much hassle!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2352288,
      "author_name": "ksmcg90",
      "author_url": "",
      "post_date": "07/20/2023 19:13:29",
      "content": "<p>For me the easiest way to prepare an environment for submission is to create the wheels from a requirements.txt, export to a dataset that's synced to the the notebook.<br>\nAnd then install in the offline the same requirements file from the wheels</p>\n<p>For the online notebook<br>\n<code>pip wheel -r {dataset-with-requirements}/requirements.txt</code></p>\n<p>Export the dataset to <strong>dataset-with-wheels</strong></p>\n<p>For the offline submission<br>\n<code>pip install --no-index --find-links=/kaggle/input/{dataset-with-wheels}/wheels -r {dataset-with-requirements}/requirements.txt</code></p>\n<p>And if you are version controlling you're code with github you can set up an action that runs the online notebook on push to main and will automatically update the wheels so you only have to focuse on the online inference notebook</p>\n<pre><code>    \n\n\n  \n     [ ,  ]\n\n  \n     \n    \n         \n         \n\n          \n         \n        \n           \n           \n          \n         \n        \n             \n             \n        \n           \n          \n         \n        \n             \n             \n         </code></pre>",
      "votes": null,
      "replies": [
        {
          "id": 2352408,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "07/20/2023 23:30:16",
          "content": "<p>thanks a lot!<br>\ni include a link here for those who are interested in using kaggle CLI to push notebook</p>\n<p><a href=\"https://www.kaggle.com/docs/api\" target=\"_blank\">https://www.kaggle.com/docs/api</a></p>\n<hr>\n<p>Creating and Running a New Notebook</p>\n<p>\"The Kaggle API can be used to to create new Notebooks and Notebook versions on Kaggle from the comfort of the command-line. This can make executing and sharing code on Kaggle a simple part of your workflow.\"</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2351691": "it is a hassle to install modules for submission. I wonder is there a way to:\n1. prepare the image from internet-connected notebook\n2. for the submission notebook, just setup docker and load image from database. \ninput test directory in docker commandline and product ouput csv.\n\nhas anyone done something smiliar before?\nin short, how to let the kaggle notebook use our docker image?",
    "2352229": "Probably not. Kaggle doesn't let users to mess with the Notebook environments. So much hassle!",
    "2352288": "For me the easiest way to prepare an environment for submission is to create the wheels from a requirements.txt, export to a dataset that's synced to the the notebook.\nAnd then install in the offline the same requirements file from the wheels\n\nFor the online notebook\n`pip wheel -r {dataset-with-requirements}/requirements.txt`\n\nExport the dataset to **dataset-with-wheels**\n\nFor the offline submission\n```pip install --no-index --find-links=/kaggle/input/{dataset-with-wheels}/wheels -r {dataset-with-requirements}/requirements.txt```\n\nAnd if you are version controlling you're code with github you can set up an action that runs the online notebook on push to main and will automatically update the wheels so you only have to focuse on the online inference notebook\n\n```\nname: Update Kaggle Code Dependencies\n\non:\n  push:\n    branches: [ main, master ]\njobs:\n  upload_code_to_kaggle:\n    runs-on: ubuntu-latest\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v3\n\n      - name: Setup Python version\n        uses: actions/setup-python@v3\n        with:\n          python-version: '3.9'\n          cache: 'pip'\n      - name: Install the packages\n        uses: BSFishy/pip-action@v1\n        env:\n          KAGGLE_USERNAME: ${{ secrets.KAGGLE_USERNAME }}\n          KAGGLE_KEY: ${{ secrets.KAGGLE_KEY }}\n        with:\n          packages: kaggle\n      - name: Update Kaggle Dependencies\n        shell: bash\n        env:\n          KAGGLE_USERNAME: ${{ secrets.KAGGLE_USERNAME }}\n          KAGGLE_KEY: ${{ secrets.KAGGLE_KEY }}\n        run: |\n          kaggle kernels pull -m $KAGGLE_USERNAME/dependencies-notebook\n          kaggle kernels push -p .\n```",
    "2352408": "thanks a lot!\ni include a link here for those who are interested in using kaggle CLI to push notebook\n\nhttps://www.kaggle.com/docs/api\n\n----\n\nCreating and Running a New Notebook\n\n\"The Kaggle API can be used to to create new Notebooks and Notebook versions on Kaggle from the comfort of the command-line. This can make executing and sharing code on Kaggle a simple part of your workflow.\""
  },
  "source": "meta"
}