{
  "id": 492871,
  "title": "PYSPNF | 3X Faster PNG Decoding!",
  "url": "/competitions/birdclef-2024/discussion/492871",
  "author_name": "Mark Wijkhuizen",
  "post_date": "2024-04-11T08:19:40.995000",
  "votes": 10,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Another day, another efficiency trick.</p>\n<p>Since this competition has strict limitations when it comes to inference time, the models used in this competition will be rather small.</p>\n<p>The amount of training data is however large, with ~25K audio files and &gt;100K spectrograms of 5 second.</p>\n<p>It is thus important to have a fast dataloader to fully utilize the GPU and not make the GPU wait for the next training batch.</p>\n<p>My own dataloader was limited by the PNG decoding speed and the <a href=\"https://pypi.org/project/pyspng/\" target=\"_blank\">PYSPNG</a> increased the PNG decoding speed by 3x!</p>\n<p>All you need to do is replace your conventional PNG decoder by <code>pyspng.load</code>, which takes raw bytes as input, not the filename.</p>\n<p>Example:</p>\n<pre><code>import pyspng\n\n (, )  f:\n    \n    image= pyspng.(f.())  \n</code></pre>",
  "messages": [
    {
      "id": 2746458,
      "postDate": "2024-04-11T08:19:40.997Z",
      "content": "<p>Another day, another efficiency trick.</p>\n<p>Since this competition has strict limitations when it comes to inference time, the models used in this competition will be rather small.</p>\n<p>The amount of training data is however large, with ~25K audio files and &gt;100K spectrograms of 5 second.</p>\n<p>It is thus important to have a fast dataloader to fully utilize the GPU and not make the GPU wait for the next training batch.</p>\n<p>My own dataloader was limited by the PNG decoding speed and the <a href=\"https://pypi.org/project/pyspng/\" target=\"_blank\">PYSPNG</a> increased the PNG decoding speed by 3x!</p>\n<p>All you need to do is replace your conventional PNG decoder by <code>pyspng.load</code>, which takes raw bytes as input, not the filename.</p>\n<p>Example:</p>\n<pre><code>import pyspng\n\n (, )  f:\n    \n    image= pyspng.(f.())  \n</code></pre>",
      "rawMarkdown": "Another day, another efficiency trick.\n\nSince this competition has strict limitations when it comes to inference time, the models used in this competition will be rather small.\n\nThe amount of training data is however large, with ~25K audio files and >100K spectrograms of 5 second.\n\nIt is thus important to have a fast dataloader to fully utilize the GPU and not make the GPU wait for the next training batch.\n\nMy own dataloader was limited by the PNG decoding speed and the [PYSPNG](https://pypi.org/project/pyspng/) increased the PNG decoding speed by 3x!\n\nAll you need to do is replace your conventional PNG decoder by `pyspng.load`, which takes raw bytes as input, not the filename.\n\nExample:\n\n```\nimport pyspng\n\nwith open('image.png', 'rb') as f:\n    # bytes are passed to pyspng, not the filename!\n    image= pyspng.load(f.read())  \n```",
      "votes": 10
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2746458": "Another day, another efficiency trick.\n\nSince this competition has strict limitations when it comes to inference time, the models used in this competition will be rather small.\n\nThe amount of training data is however large, with ~25K audio files and >100K spectrograms of 5 second.\n\nIt is thus important to have a fast dataloader to fully utilize the GPU and not make the GPU wait for the next training batch.\n\nMy own dataloader was limited by the PNG decoding speed and the [PYSPNG](https://pypi.org/project/pyspng/) increased the PNG decoding speed by 3x!\n\nAll you need to do is replace your conventional PNG decoder by `pyspng.load`, which takes raw bytes as input, not the filename.\n\nExample:\n\n```\nimport pyspng\n\nwith open('image.png', 'rb') as f:\n    # bytes are passed to pyspng, not the filename!\n    image= pyspng.load(f.read())  \n```"
  }
}