Hi, all!
\nWe've released a new version of the Google Bird Vocalization Classifier. The only change is that it runs about 2x faster. The TFLite model run time with 4 threads reduced from ~260ms to 120ms on my home machine (CPU only). We're also seeing about a 40% speedup on GPU and TPU for the original Jax model, which I expect should be similar for the exported TF SavedModel.
\nNote that BirdNet still runs about twice as fast, once you account for the difference in window sizes (3s vs 5s).
\nWe reduced some over-computation in the spectrogram, trained up a new version of the model from scratch with the new spectrogram settings, and checked that there's no regressions on all of our existing annotated soundscape datasets. (well, except the Kenyan data, of course. :) )
\nHappy hacking!
", "messages": [ { "id": "2203449", "postDate": "03/30/2023 20:01:30", "content": "Hi, all!
\nWe've released a new version of the Google Bird Vocalization Classifier. The only change is that it runs about 2x faster. The TFLite model run time with 4 threads reduced from ~260ms to 120ms on my home machine (CPU only). We're also seeing about a 40% speedup on GPU and TPU for the original Jax model, which I expect should be similar for the exported TF SavedModel.
\nNote that BirdNet still runs about twice as fast, once you account for the difference in window sizes (3s vs 5s).
\nWe reduced some over-computation in the spectrogram, trained up a new version of the model from scratch with the new spectrogram settings, and checked that there's no regressions on all of our existing annotated soundscape datasets. (well, except the Kenyan data, of course. :) )
\nHappy hacking!
", "rawMarkdown": "Hi, all!\n\nWe've released a new version of the [Google Bird Vocalization Classifier](https://www.kaggle.com/models/google/bird-vocalization-classifier). The only change is that it runs about 2x faster. The TFLite model run time with 4 threads reduced from ~260ms to 120ms on my home machine (CPU only). We're also seeing about a 40% speedup on GPU and TPU for the original Jax model, which I expect should be similar for the exported TF SavedModel.\n\nNote that BirdNet still runs about twice as fast, once you account for the difference in window sizes (3s vs 5s).\n\nWe reduced some over-computation in the spectrogram, trained up a new version of the model from scratch with the new spectrogram settings, and checked that there's no regressions on all of our existing annotated soundscape datasets. *(well, except the Kenyan data, of course. :) )*\n\nHappy hacking!", "votes": null }, { "id": "2204211", "postDate": "03/31/2023 12:20:30", "content": "Thanks for the updated faster model Tom!!! That is great!
", "rawMarkdown": "Thanks for the updated faster model Tom!!! That is great!", "votes": null }, { "id": "2211179", "postDate": "04/05/2023 21:15:34", "content": "@tomdenton Thanks for sharing this update. Any idea how to run with batch, and not one by one sample ? I tried to run on small batch (N, 5*32000) but it raise some error related to signature
", "rawMarkdown": "tomdenton Thanks for sharing this update. Any idea how to run with batch, and not one by one sample ? I tried to run on small batch (N, 5*32000) but it raise some error related to signature", "votes": null }, { "id": "2211188", "postDate": "04/05/2023 21:32:02", "content": "Hi, Shiro!
\nUnfortunately, exporting with polymorphic shape support from jax can be difficult, so I don't have a good option for batched support at this time. On the somewhat-mitigating-if-not-actually-bright side, iterating over the batch and running sequentially should be about the same when executing on CPU.
", "rawMarkdown": "Hi, Shiro!\n\nUnfortunately, exporting with polymorphic shape support from jax can be difficult, so I don't have a good option for batched support at this time. On the somewhat-mitigating-if-not-actually-bright side, iterating over the batch and running sequentially should be about the same when executing on CPU.", "votes": null }, { "id": "2214185", "postDate": "04/08/2023 08:16:35", "content": "Hi Tom!
\nthank for the reply. I will investigate if we can use it for a \"feature extractor\" inside a TPU pipeline to train a MLP then!
@tomdenton Does the jax checkpoint are available somewhere?
", "rawMarkdown": "tomdenton Does the jax checkpoint are available somewhere?", "votes": null }, { "id": "2214662", "postDate": "04/08/2023 16:37:52", "content": "BTW: A really nice thing about fixed embeddings is that you can pre-compute them and experiment freely for cheap - similar to precomputing spectrograms.
", "rawMarkdown": "BTW: A really nice thing about fixed embeddings is that you can pre-compute them and experiment freely for cheap - similar to precomputing spectrograms.", "votes": null }, { "id": "2214667", "postDate": "04/08/2023 16:40:11", "content": "We're a bit reticent to release the Jax checkpoints because it creates a really nasty support problem - if there are code changes it can cause Jax checkpoint loading to fail, and then we'll need to include lots of overhead for backwards compatibility with old checkpoints. Exporting to TF + TFLite lets us totally decouple inference from the training pipeline.
", "rawMarkdown": "We're a bit reticent to release the Jax checkpoints because it creates a really nasty support problem - if there are code changes it can cause Jax checkpoint loading to fail, and then we'll need to include lots of overhead for backwards compatibility with old checkpoints. Exporting to TF + TFLite lets us totally decouple inference from the training pipeline.", "votes": null } ], "comments": [ { "id": 2204211, "author_name": "gusthema", "author_url": "", "post_date": "03/31/2023 12:20:30", "content": "Thanks for the updated faster model Tom!!! That is great!
", "votes": null, "replies": [] }, { "id": 2211179, "author_name": "ludovick", "author_url": "", "post_date": "04/05/2023 21:15:34", "content": "@tomdenton Thanks for sharing this update. Any idea how to run with batch, and not one by one sample ? I tried to run on small batch (N, 5*32000) but it raise some error related to signature
", "votes": null, "replies": [ { "id": 2211188, "author_name": "tomdenton", "author_url": "", "post_date": "04/05/2023 21:32:02", "content": "Hi, Shiro!
\nUnfortunately, exporting with polymorphic shape support from jax can be difficult, so I don't have a good option for batched support at this time. On the somewhat-mitigating-if-not-actually-bright side, iterating over the batch and running sequentially should be about the same when executing on CPU.
", "votes": null, "replies": [ { "id": 2214185, "author_name": "ludovick", "author_url": "", "post_date": "04/08/2023 08:16:35", "content": "Hi Tom!
\nthank for the reply. I will investigate if we can use it for a \"feature extractor\" inside a TPU pipeline to train a MLP then!
BTW: A really nice thing about fixed embeddings is that you can pre-compute them and experiment freely for cheap - similar to precomputing spectrograms.
", "votes": null, "replies": [] } ] }, { "id": 2214305, "author_name": "ludovick", "author_url": "", "post_date": "04/08/2023 10:52:08", "content": "@tomdenton Does the jax checkpoint are available somewhere?
", "votes": null, "replies": [ { "id": 2214667, "author_name": "tomdenton", "author_url": "", "post_date": "04/08/2023 16:40:11", "content": "We're a bit reticent to release the Jax checkpoints because it creates a really nasty support problem - if there are code changes it can cause Jax checkpoint loading to fail, and then we'll need to include lots of overhead for backwards compatibility with old checkpoints. Exporting to TF + TFLite lets us totally decouple inference from the training pipeline.
", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "2203449": "Hi, all!\n\nWe've released a new version of the [Google Bird Vocalization Classifier](https://www.kaggle.com/models/google/bird-vocalization-classifier). The only change is that it runs about 2x faster. The TFLite model run time with 4 threads reduced from ~260ms to 120ms on my home machine (CPU only). We're also seeing about a 40% speedup on GPU and TPU for the original Jax model, which I expect should be similar for the exported TF SavedModel.\n\nNote that BirdNet still runs about twice as fast, once you account for the difference in window sizes (3s vs 5s).\n\nWe reduced some over-computation in the spectrogram, trained up a new version of the model from scratch with the new spectrogram settings, and checked that there's no regressions on all of our existing annotated soundscape datasets. *(well, except the Kenyan data, of course. :) )*\n\nHappy hacking!", "2204211": "Thanks for the updated faster model Tom!!! That is great!", "2211179": "tomdenton Thanks for sharing this update. Any idea how to run with batch, and not one by one sample ? I tried to run on small batch (N, 5*32000) but it raise some error related to signature", "2211188": "Hi, Shiro!\n\nUnfortunately, exporting with polymorphic shape support from jax can be difficult, so I don't have a good option for batched support at this time. On the somewhat-mitigating-if-not-actually-bright side, iterating over the batch and running sequentially should be about the same when executing on CPU.", "2214185": "Hi Tom! \nthank for the reply. I will investigate if we can use it for a \"feature extractor\" inside a TPU pipeline to train a MLP then!", "2214305": "tomdenton Does the jax checkpoint are available somewhere?", "2214662": "BTW: A really nice thing about fixed embeddings is that you can pre-compute them and experiment freely for cheap - similar to precomputing spectrograms.", "2214667": "We're a bit reticent to release the Jax checkpoints because it creates a really nasty support problem - if there are code changes it can cause Jax checkpoint loading to fail, and then we'll need to include lots of overhead for backwards compatibility with old checkpoints. Exporting to TF + TFLite lets us totally decouple inference from the training pipeline." }, "source": "meta" }