{
  "id": 173573,
  "title": "new paper addressing missing labels in AudioSet",
  "url": "/competitions/freesound-audio-tagging-2019/discussion/173573",
  "author_name": "",
  "post_date": "2020-08-09T20:23:15.069382900Z",
  "votes": 2,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Dear all, </p>\n\n<p>For those interested in sound event recognition and/or learning with noisy labels, we’ve just published a paper in IEEE Signal Processing Letters that may be of interest to you.</p>\n\n<p>The paper is a collaboration with folks at Google Research and it deals with AudioSet classification from the perspective of label noise. In particular, we address missing labels in AudioSet using a teacher-student framework with loss masking.</p>\n\n<p>You can find more details in our paper:</p>\n\n<blockquote>\n  <p>Eduardo Fonseca, Shawn Hershey, Manoj Plakal, Daniel P. W. Ellis, Aren Jansen, R. Channing Moore. <br>\n  <strong>Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking.</strong>\n  In IEEE Signal Processing Letters, Vol. 27, pages 1235 - 1239, 2020</p>\n</blockquote>\n\n<p>And you can also check out this blog with the main takeaways! <br>\n<a href=\"http://www.eduardofonseca.net/papers/2020/05/08/teacher-student-missing-labels.html\">http://www.eduardofonseca.net/papers/2020/05/08/teacher-student-missing-labels.html</a></p>\n\n<p>Best, </p>\n\n<p>Eduardo on behalf of the authors</p>",
  "messages": [
    {
      "id": "964403",
      "postDate": "08/09/2020 20:23:15",
      "content": "<p>Dear all, </p>\n\n<p>For those interested in sound event recognition and/or learning with noisy labels, we’ve just published a paper in IEEE Signal Processing Letters that may be of interest to you.</p>\n\n<p>The paper is a collaboration with folks at Google Research and it deals with AudioSet classification from the perspective of label noise. In particular, we address missing labels in AudioSet using a teacher-student framework with loss masking.</p>\n\n<p>You can find more details in our paper:</p>\n\n<blockquote>\n  <p>Eduardo Fonseca, Shawn Hershey, Manoj Plakal, Daniel P. W. Ellis, Aren Jansen, R. Channing Moore. <br>\n  <strong>Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking.</strong>\n  In IEEE Signal Processing Letters, Vol. 27, pages 1235 - 1239, 2020</p>\n</blockquote>\n\n<p>And you can also check out this blog with the main takeaways! <br>\n<a href=\"http://www.eduardofonseca.net/papers/2020/05/08/teacher-student-missing-labels.html\">http://www.eduardofonseca.net/papers/2020/05/08/teacher-student-missing-labels.html</a></p>\n\n<p>Best, </p>\n\n<p>Eduardo on behalf of the authors</p>",
      "rawMarkdown": "Dear all, \n\nFor those interested in sound event recognition and/or learning with noisy labels, we’ve just published a paper in IEEE Signal Processing Letters that may be of interest to you.\n\nThe paper is a collaboration with folks at Google Research and it deals with AudioSet classification from the perspective of label noise. In particular, we address missing labels in AudioSet using a teacher-student framework with loss masking.\n\nYou can find more details in our paper:\n&gt; Eduardo Fonseca, Shawn Hershey, Manoj Plakal, Daniel P. W. Ellis, Aren Jansen, R. Channing Moore.  \n<a href=\"https://arxiv.org/pdf/2005.00878.pdf\" target=\"_blank\">**Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking.**</a>\nIn IEEE Signal Processing Letters, Vol. 27, pages 1235 - 1239, 2020\n\nAnd you can also check out this blog with the main takeaways!  \nhttp://www.eduardofonseca.net/papers/2020/05/08/teacher-student-missing-labels.html\n\nBest, \n\nEduardo on behalf of the authors",
      "votes": null
    },
    {
      "id": "964493",
      "postDate": "08/10/2020 00:06:59",
      "content": "<p>Thanks for the notice, I read the paper and it is quite interesting!</p>\n\n<p>May I ask a question:\nWhy do you choose to <strong>ignore</strong> (potential) missing labels in loss calculation instead of re-labeling them? Maybe I've overlooked the description about that...but I could not find the answer in the paper.</p>\n\n<p>Thanks in advance!</p>",
      "rawMarkdown": "Thanks for the notice, I read the paper and it is quite interesting!\n\nMay I ask a question:\nWhy do you choose to **ignore** (potential) missing labels in loss calculation instead of re-labeling them? Maybe I've overlooked the description about that...but I could not find the answer in the paper.\n\nThanks in advance!",
      "votes": null
    },
    {
      "id": "975611",
      "postDate": "08/18/2020 11:32:15",
      "content": "<p>Thanks for the interest!</p>\n<p>Ignoring them is the simplest action to take (and probably the most conservative one). As you pointed out, the natural extension is to re-label those instances, as we mention at the end of Sec. IV. </p>\n<p>This can be done manually, or automatically by using some top-scored implicit negative labels as positive labels. However, this must be done carefully. Fig 2 shows that the method is sensitive to the amount of discarded labels---in particular, when we start ignoring presumably informative True Negatives. If these informative True Negatives are not ignored, but treated as positives, it could harm performance (potentially at a higher rate than just ignoring them).</p>\n<p>To prevent this, the simplest solution is to treat only a very small amount of top-scored negatives as positives. Estimating optimal per-class proportions automatically would be ideal.</p>\n<p>hope this helps!</p>",
      "rawMarkdown": "Thanks for the interest!\n\nIgnoring them is the simplest action to take (and probably the most conservative one). As you pointed out, the natural extension is to re-label those instances, as we mention at the end of Sec. IV. \n\nThis can be done manually, or automatically by using some top-scored implicit negative labels as positive labels. However, this must be done carefully. Fig 2 shows that the method is sensitive to the amount of discarded labels---in particular, when we start ignoring presumably informative True Negatives. If these informative True Negatives are not ignored, but treated as positives, it could harm performance (potentially at a higher rate than just ignoring them).\n\nTo prevent this, the simplest solution is to treat only a very small amount of top-scored negatives as positives. Estimating optimal per-class proportions automatically would be ideal.\n\nhope this helps!",
      "votes": null
    },
    {
      "id": "982075",
      "postDate": "08/23/2020 03:37:31",
      "content": "<p>Thank you for that! I get the idea behind the proposed method!</p>",
      "rawMarkdown": "Thank you for that! I get the idea behind the proposed method!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 975611,
      "author_name": "eduardofonseca",
      "author_url": "",
      "post_date": "08/18/2020 11:32:15",
      "content": "<p>Thanks for the interest!</p>\n<p>Ignoring them is the simplest action to take (and probably the most conservative one). As you pointed out, the natural extension is to re-label those instances, as we mention at the end of Sec. IV. </p>\n<p>This can be done manually, or automatically by using some top-scored implicit negative labels as positive labels. However, this must be done carefully. Fig 2 shows that the method is sensitive to the amount of discarded labels---in particular, when we start ignoring presumably informative True Negatives. If these informative True Negatives are not ignored, but treated as positives, it could harm performance (potentially at a higher rate than just ignoring them).</p>\n<p>To prevent this, the simplest solution is to treat only a very small amount of top-scored negatives as positives. Estimating optimal per-class proportions automatically would be ideal.</p>\n<p>hope this helps!</p>",
      "votes": null,
      "replies": [
        {
          "id": 982075,
          "author_name": "hidehisaarai1213",
          "author_url": "",
          "post_date": "08/23/2020 03:37:31",
          "content": "<p>Thank you for that! I get the idea behind the proposed method!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 964493,
      "author_name": "hidehisaarai1213",
      "author_url": "",
      "post_date": "08/10/2020 00:06:59",
      "content": "<p>Thanks for the notice, I read the paper and it is quite interesting!</p>\n\n<p>May I ask a question:\nWhy do you choose to <strong>ignore</strong> (potential) missing labels in loss calculation instead of re-labeling them? Maybe I've overlooked the description about that...but I could not find the answer in the paper.</p>\n\n<p>Thanks in advance!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "964403": "Dear all, \n\nFor those interested in sound event recognition and/or learning with noisy labels, we’ve just published a paper in IEEE Signal Processing Letters that may be of interest to you.\n\nThe paper is a collaboration with folks at Google Research and it deals with AudioSet classification from the perspective of label noise. In particular, we address missing labels in AudioSet using a teacher-student framework with loss masking.\n\nYou can find more details in our paper:\n&gt; Eduardo Fonseca, Shawn Hershey, Manoj Plakal, Daniel P. W. Ellis, Aren Jansen, R. Channing Moore.  \n<a href=\"https://arxiv.org/pdf/2005.00878.pdf\" target=\"_blank\">**Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking.**</a>\nIn IEEE Signal Processing Letters, Vol. 27, pages 1235 - 1239, 2020\n\nAnd you can also check out this blog with the main takeaways!  \nhttp://www.eduardofonseca.net/papers/2020/05/08/teacher-student-missing-labels.html\n\nBest, \n\nEduardo on behalf of the authors",
    "964493": "Thanks for the notice, I read the paper and it is quite interesting!\n\nMay I ask a question:\nWhy do you choose to **ignore** (potential) missing labels in loss calculation instead of re-labeling them? Maybe I've overlooked the description about that...but I could not find the answer in the paper.\n\nThanks in advance!",
    "975611": "Thanks for the interest!\n\nIgnoring them is the simplest action to take (and probably the most conservative one). As you pointed out, the natural extension is to re-label those instances, as we mention at the end of Sec. IV. \n\nThis can be done manually, or automatically by using some top-scored implicit negative labels as positive labels. However, this must be done carefully. Fig 2 shows that the method is sensitive to the amount of discarded labels---in particular, when we start ignoring presumably informative True Negatives. If these informative True Negatives are not ignored, but treated as positives, it could harm performance (potentially at a higher rate than just ignoring them).\n\nTo prevent this, the simplest solution is to treat only a very small amount of top-scored negatives as positives. Estimating optimal per-class proportions automatically would be ideal.\n\nhope this helps!",
    "982075": "Thank you for that! I get the idea behind the proposed method!"
  },
  "source": "meta"
}