{
  "id": 425889,
  "title": "How to submit w/o internet : brief tutorial and resources",
  "url": "/competitions/bengaliai-speech/discussion/425889",
  "author_name": "",
  "post_date": "2023-07-20T19:48:18.797399200Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>A big challenge of this competition is that, since it is a code competition you have to submit a notebook rather than any submission.csv file. The notebook will have some restrictions, one of them is that it needs to have the internet disabled. This introduces the problem of installing external packages. <br>\nInstalling packages using wheels is a solution to this problem. I have demonstrated in these notebooks how to install required packages and generate the .whl files and then how to install the packages offline using the wheels.</p>\n<ul>\n<li><a href=\"https://www.kaggle.com/code/mbmmurad/how-to-submit-code-w-o-internet-creating-wheels\" target=\"_blank\">how-to-submit-code-w-o-internet-creating-wheels </a></li>\n<li><a href=\"https://www.kaggle.com/code/mbmmurad/how-to-submit-code-w-o-internet-creating-wheels\" target=\"_blank\">NeMo Conformer baseline w/o internet: Inference NB</a></li>\n</ul>\n<p>Hope this helps! 😁</p>\n<p>Some other resource :</p>\n<blockquote>\n  <p><a href=\"https://www.kaggle.com/code/truthr/install-python-packages-without-internet-access/notebook\" target=\"_blank\">https://www.kaggle.com/code/truthr/install-python-packages-without-internet-access/notebook</a><br>\n  <a href=\"https://www.kaggle.com/code/narnaoot/installing-packages-without-internet-for-kaggle\" target=\"_blank\">https://www.kaggle.com/code/narnaoot/installing-packages-without-internet-for-kaggle</a><br>\n  <a href=\"https://www.kaggle.com/code/samuelepino/pip-downloading-packages-to-your-local-machine/notebook?scriptVersionId=29576961\" target=\"_blank\">https://www.kaggle.com/code/samuelepino/pip-downloading-packages-to-your-local-machine/notebook?scriptVersionId=29576961</a></p>\n</blockquote>",
  "messages": [
    {
      "id": "2352320",
      "postDate": "07/20/2023 19:48:18",
      "content": "<p>A big challenge of this competition is that, since it is a code competition you have to submit a notebook rather than any submission.csv file. The notebook will have some restrictions, one of them is that it needs to have the internet disabled. This introduces the problem of installing external packages. <br>\nInstalling packages using wheels is a solution to this problem. I have demonstrated in these notebooks how to install required packages and generate the .whl files and then how to install the packages offline using the wheels.</p>\n<ul>\n<li><a href=\"https://www.kaggle.com/code/mbmmurad/how-to-submit-code-w-o-internet-creating-wheels\" target=\"_blank\">how-to-submit-code-w-o-internet-creating-wheels </a></li>\n<li><a href=\"https://www.kaggle.com/code/mbmmurad/how-to-submit-code-w-o-internet-creating-wheels\" target=\"_blank\">NeMo Conformer baseline w/o internet: Inference NB</a></li>\n</ul>\n<p>Hope this helps! 😁</p>\n<p>Some other resource :</p>\n<blockquote>\n  <p><a href=\"https://www.kaggle.com/code/truthr/install-python-packages-without-internet-access/notebook\" target=\"_blank\">https://www.kaggle.com/code/truthr/install-python-packages-without-internet-access/notebook</a><br>\n  <a href=\"https://www.kaggle.com/code/narnaoot/installing-packages-without-internet-for-kaggle\" target=\"_blank\">https://www.kaggle.com/code/narnaoot/installing-packages-without-internet-for-kaggle</a><br>\n  <a href=\"https://www.kaggle.com/code/samuelepino/pip-downloading-packages-to-your-local-machine/notebook?scriptVersionId=29576961\" target=\"_blank\">https://www.kaggle.com/code/samuelepino/pip-downloading-packages-to-your-local-machine/notebook?scriptVersionId=29576961</a></p>\n</blockquote>",
      "rawMarkdown": "A big challenge of this competition is that, since it is a code competition you have to submit a notebook rather than any submission.csv file. The notebook will have some restrictions, one of them is that it needs to have the internet disabled. This introduces the problem of installing external packages. \nInstalling packages using wheels is a solution to this problem. I have demonstrated in these notebooks how to install required packages and generate the .whl files and then how to install the packages offline using the wheels.\n\n- [how-to-submit-code-w-o-internet-creating-wheels ](https://www.kaggle.com/code/mbmmurad/how-to-submit-code-w-o-internet-creating-wheels)\n- [NeMo Conformer baseline w/o internet: Inference NB](https://www.kaggle.com/code/mbmmurad/how-to-submit-code-w-o-internet-creating-wheels)\n\n\nHope this helps! 😁\n\nSome other resource :\n>https://www.kaggle.com/code/truthr/install-python-packages-without-internet-access/notebook\n>https://www.kaggle.com/code/narnaoot/installing-packages-without-internet-for-kaggle\n>https://www.kaggle.com/code/samuelepino/pip-downloading-packages-to-your-local-machine/notebook?scriptVersionId=29576961",
      "votes": null
    },
    {
      "id": "2353288",
      "postDate": "07/21/2023 15:02:38",
      "content": "<p>fyi, hugging face use this to read mp3 file:</p>\n<pre><code> open(mp3_file, )  f:\n               bpayload = f.read()\n\n\ndef ffmpeg_read(bpayload: bytes, sampling_rate: int) -&gt; np.array:\n    \n    ar = f\n    ac = \n    format_for_conversion = \n    ffmpeg_command = [\n        ,\n        ,\n        ,\n        ,\n        ac,\n        ,\n        ar,\n        ,\n        format_for_conversion,\n        ,\n        ,\n        ,\n        ,\n    ]\n\n    :\n         subprocess.(ffmpeg_command, stdin=subprocess., stdout=subprocess.)  ffmpeg_process:\n            output_stream = ffmpeg_process.communicate(bpayload)\n    except   :\n        raise () from \n    out_bytes = output_stream[]\n    audio = np.frombuffer(out_bytes, np.float32)\n     audio.shape[] == :\n        raise ()\n     audio\n</code></pre>",
      "rawMarkdown": "fyi, hugging face use this to read mp3 file:\n\n```\n\nwith open(mp3_file, \"rb\") as f:\n               bpayload = f.read()\n\n\ndef ffmpeg_read(bpayload: bytes, sampling_rate: int) -> np.array:\n    \"\"\"\n    Helper function to read an audio file through ffmpeg.\n    \"\"\"\n    ar = f\"{sampling_rate}\"\n    ac = \"1\"\n    format_for_conversion = \"f32le\"\n    ffmpeg_command = [\n        \"ffmpeg\",\n        \"-i\",\n        \"pipe:0\",\n        \"-ac\",\n        ac,\n        \"-ar\",\n        ar,\n        \"-f\",\n        format_for_conversion,\n        \"-hide_banner\",\n        \"-loglevel\",\n        \"quiet\",\n        \"pipe:1\",\n    ]\n\n    try:\n        with subprocess.Popen(ffmpeg_command, stdin=subprocess.PIPE, stdout=subprocess.PIPE) as ffmpeg_process:\n            output_stream = ffmpeg_process.communicate(bpayload)\n    except FileNotFoundError as error:\n        raise ValueError(\"ffmpeg was not found but is required to load audio files from filename\") from error\n    out_bytes = output_stream[0]\n    audio = np.frombuffer(out_bytes, np.float32)\n    if audio.shape[0] == 0:\n        raise ValueError(\"Malformed soundfile\")\n    return audio\n\n```",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2353288,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "07/21/2023 15:02:38",
      "content": "<p>fyi, hugging face use this to read mp3 file:</p>\n<pre><code> open(mp3_file, )  f:\n               bpayload = f.read()\n\n\ndef ffmpeg_read(bpayload: bytes, sampling_rate: int) -&gt; np.array:\n    \n    ar = f\n    ac = \n    format_for_conversion = \n    ffmpeg_command = [\n        ,\n        ,\n        ,\n        ,\n        ac,\n        ,\n        ar,\n        ,\n        format_for_conversion,\n        ,\n        ,\n        ,\n        ,\n    ]\n\n    :\n         subprocess.(ffmpeg_command, stdin=subprocess., stdout=subprocess.)  ffmpeg_process:\n            output_stream = ffmpeg_process.communicate(bpayload)\n    except   :\n        raise () from \n    out_bytes = output_stream[]\n    audio = np.frombuffer(out_bytes, np.float32)\n     audio.shape[] == :\n        raise ()\n     audio\n</code></pre>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2352320": "A big challenge of this competition is that, since it is a code competition you have to submit a notebook rather than any submission.csv file. The notebook will have some restrictions, one of them is that it needs to have the internet disabled. This introduces the problem of installing external packages. \nInstalling packages using wheels is a solution to this problem. I have demonstrated in these notebooks how to install required packages and generate the .whl files and then how to install the packages offline using the wheels.\n\n- [how-to-submit-code-w-o-internet-creating-wheels ](https://www.kaggle.com/code/mbmmurad/how-to-submit-code-w-o-internet-creating-wheels)\n- [NeMo Conformer baseline w/o internet: Inference NB](https://www.kaggle.com/code/mbmmurad/how-to-submit-code-w-o-internet-creating-wheels)\n\n\nHope this helps! 😁\n\nSome other resource :\n>https://www.kaggle.com/code/truthr/install-python-packages-without-internet-access/notebook\n>https://www.kaggle.com/code/narnaoot/installing-packages-without-internet-for-kaggle\n>https://www.kaggle.com/code/samuelepino/pip-downloading-packages-to-your-local-machine/notebook?scriptVersionId=29576961",
    "2353288": "fyi, hugging face use this to read mp3 file:\n\n```\n\nwith open(mp3_file, \"rb\") as f:\n               bpayload = f.read()\n\n\ndef ffmpeg_read(bpayload: bytes, sampling_rate: int) -> np.array:\n    \"\"\"\n    Helper function to read an audio file through ffmpeg.\n    \"\"\"\n    ar = f\"{sampling_rate}\"\n    ac = \"1\"\n    format_for_conversion = \"f32le\"\n    ffmpeg_command = [\n        \"ffmpeg\",\n        \"-i\",\n        \"pipe:0\",\n        \"-ac\",\n        ac,\n        \"-ar\",\n        ar,\n        \"-f\",\n        format_for_conversion,\n        \"-hide_banner\",\n        \"-loglevel\",\n        \"quiet\",\n        \"pipe:1\",\n    ]\n\n    try:\n        with subprocess.Popen(ffmpeg_command, stdin=subprocess.PIPE, stdout=subprocess.PIPE) as ffmpeg_process:\n            output_stream = ffmpeg_process.communicate(bpayload)\n    except FileNotFoundError as error:\n        raise ValueError(\"ffmpeg was not found but is required to load audio files from filename\") from error\n    out_bytes = output_stream[0]\n    audio = np.frombuffer(out_bytes, np.float32)\n    if audio.shape[0] == 0:\n        raise ValueError(\"Malformed soundfile\")\n    return audio\n\n```"
  },
  "source": "meta"
}