I have attempted to add pseudo-labels by means of a binary classifier trained on external data. A portion of the visualization is available as a notebook[1] (unfortunately, the notebook is not reproducible for everyone, as the binary classifier is not publicly available. Please understand that this is only in the context of sharing ideas).
\nMy attempt at this experiment was intended to address weak labels. As is well known, we are only given labels in long clip units, so when we crop to shorter clips, the label features may not appear in the data (e.g., silent clips). This would give a positive label for the wrong data and would hinder the efficiency of learning.
\nPast competitions and related studies have addressed weak labels using various techniques. Here, I used the simplest and easiest to understand method, the pseudo-label method. The procedure is as follows.
\nAs you can see from the notebook visualization results[1], for some clips we are able to discriminate well between clips that contain squeals and those that do not. (On the other hand, as far as the experiment at hand is concerned, in some cases, on the contrary, the clips with calls were judged as silent. This is probably due to the fact that the competition data contains squeals with features that are not present in the external data used for training).
\nHowever, since the percentage of calls predicted by the binary classifier is high, about 75% on average, it may be possible to deal with this problem using mixups or other methods without using this kind of pseudo-label approach. (However, if we looking into each species, it may shows different results than this.)
\nI have attempted to add pseudo-labels by means of a binary classifier trained on external data. A portion of the visualization is available as a notebook[1] (unfortunately, the notebook is not reproducible for everyone, as the binary classifier is not publicly available. Please understand that this is only in the context of sharing ideas).
\nMy attempt at this experiment was intended to address weak labels. As is well known, we are only given labels in long clip units, so when we crop to shorter clips, the label features may not appear in the data (e.g., silent clips). This would give a positive label for the wrong data and would hinder the efficiency of learning.
\nPast competitions and related studies have addressed weak labels using various techniques. Here, I used the simplest and easiest to understand method, the pseudo-label method. The procedure is as follows.
\nAs you can see from the notebook visualization results[1], for some clips we are able to discriminate well between clips that contain squeals and those that do not. (On the other hand, as far as the experiment at hand is concerned, in some cases, on the contrary, the clips with calls were judged as silent. This is probably due to the fact that the competition data contains squeals with features that are not present in the external data used for training).
\nHowever, since the percentage of calls predicted by the binary classifier is high, about 75% on average, it may be possible to deal with this problem using mixups or other methods without using this kind of pseudo-label approach. (However, if we looking into each species, it may shows different results than this.)
\nThis is a great idea and I love the visualizations. It would be interesting to see how this approach would work with other types of weak labels, such as those generated by unsupervised methods.
", "rawMarkdown": "This is a great idea and I love the visualizations. It would be interesting to see how this approach would work with other types of weak labels, such as those generated by unsupervised methods.", "votes": null } ], "comments": [ { "id": 1780696, "author_name": "", "author_url": "", "post_date": "05/07/2022 17:50:11", "content": "This is a great idea and I love the visualizations. It would be interesting to see how this approach would work with other types of weak labels, such as those generated by unsupervised methods.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1780227": "# TL; DR\n\nI have attempted to add pseudo-labels by means of a binary classifier trained on external data. A portion of the visualization is available as a notebook[1] (unfortunately, the notebook is not reproducible for everyone, as the binary classifier is not publicly available. Please understand that this is only in the context of sharing ideas).\n\n# Experiment\n\nMy attempt at this experiment was intended to address weak labels. As is well known, we are only given labels in long clip units, so when we crop to shorter clips, the label features may not appear in the data (e.g., silent clips). This would give a positive label for the wrong data and would hinder the efficiency of learning.\n\nPast competitions and related studies have addressed weak labels using various techniques. Here, I used the simplest and easiest to understand method, the pseudo-label method. The procedure is as follows.\n\n1. train a binary classifier using external data (DCASE2018)\n2. feed the binary classifier a long clip (60sec in this case) and predict the probability of a CALL\n3. give the call probabilities as soft labels\n\n# Result & Discussion\n\nAs you can see from the notebook visualization results[1], for some clips we are able to discriminate well between clips that contain squeals and those that do not. (On the other hand, as far as the experiment at hand is concerned, in some cases, on the contrary, the clips with calls were judged as silent. This is probably due to the fact that the competition data contains squeals with features that are not present in the external data used for training).\n\nHowever, since the percentage of calls predicted by the binary classifier is high, about **75%** on average, it may be possible to deal with this problem using mixups or other methods without using this kind of pseudo-label approach. (However, if we looking into each species, it may shows different results than this.)\n\n# Reference\n\n- [1] https://www.kaggle.com/code/tatamikenn/birdclef22-visualization-of-pseudo-labels/notebook", "1780696": "This is a great idea and I love the visualizations. It would be interesting to see how this approach would work with other types of weak labels, such as those generated by unsupervised methods." }, "source": "meta" }