{
  "id": 206012,
  "title": "What is the meaning of true positive and false positive here? (found several contradictions)",
  "url": "/competitions/rfcx-species-audio-detection/discussion/206012",
  "author_name": "",
  "post_date": "2020-12-22T19:45:18.467073400Z",
  "votes": 6,
  "comment_count": 2,
  "views": 0,
  "content": "<p>If a file_id / species_id combinations is not in tp csv does it mean that the respective species is definitely not in that file_id? </p>\n<p>When there is an entry in the fp csv does it mean that the respective species does not exist in the whole recording? That the species does not exist just in the time window specified? That the song type does not exist (but the species might)?</p>\n<p>+ all other variants of questions which aim to resolve the exact meaning of tp and fp.</p>\n<hr>\n<p><strong>EDIT</strong>:</p>\n<p>Some quick digging showed me that it's possible to have the an fp and a tp for the same species_id in the same recording_id:</p>\n<pre><code>df_tp = pd.read_csv(TP_CSV_PATH)\ndf_fp = pd.read_csv(FP_CSV_PATH)\ntps = set([r + '_' + str(s) for r, s in df_tp[['recording_id', 'species_id']].values])\nfps = set([r + '_' + str(s) for r, s in df_fp[['recording_id', 'species_id']].values])\ntps.intersection(fps)\n</code></pre>\n<p>Produces</p>\n<blockquote>\n  <p>{'0c48ed342_17',<br>\n   '178b835e3_16',<br>\n   '2c97a8020_17',<br>\n   '8f3d579c8_17',<br>\n   'a2441a74b_17',<br>\n   'bda762977_17'}</p>\n</blockquote>\n<p>Not sure if this is just a labelling error, or a consequence of the correct interpretation of tp and pf.</p>\n<p>Further adding the songtype_id into the mix:</p>\n<pre><code>tps = set([r + '_' + str(s) + '_' + str(t) for r, s, t in df_tp[['recording_id', 'species_id', 'songtype_id']].values])\nfps = set([r + '_' + str(s) + '_' + str(t) for r, s, t in df_fp[['recording_id', 'species_id', 'songtype_id']].values])\ntps.intersection(fps)\n</code></pre>\n<p>Produces:</p>\n<blockquote>\n  <p>{'178b835e3_16_4', 'a2441a74b_17_4'}</p>\n</blockquote>\n<p>In any case, given the total amount of data and the few intersections, <strong>I think this means we can mostly go with the simplest interpretation of tp and fp: ie, TP means \"This recording has this species\".  FP means \"This recording does not have the species\".</strong></p>",
  "messages": [
    {
      "id": "1122933",
      "postDate": "12/22/2020 19:45:18",
      "content": "<p>If a file_id / species_id combinations is not in tp csv does it mean that the respective species is definitely not in that file_id? </p>\n<p>When there is an entry in the fp csv does it mean that the respective species does not exist in the whole recording? That the species does not exist just in the time window specified? That the song type does not exist (but the species might)?</p>\n<p>+ all other variants of questions which aim to resolve the exact meaning of tp and fp.</p>\n<hr>\n<p><strong>EDIT</strong>:</p>\n<p>Some quick digging showed me that it's possible to have the an fp and a tp for the same species_id in the same recording_id:</p>\n<pre><code>df_tp = pd.read_csv(TP_CSV_PATH)\ndf_fp = pd.read_csv(FP_CSV_PATH)\ntps = set([r + '_' + str(s) for r, s in df_tp[['recording_id', 'species_id']].values])\nfps = set([r + '_' + str(s) for r, s in df_fp[['recording_id', 'species_id']].values])\ntps.intersection(fps)\n</code></pre>\n<p>Produces</p>\n<blockquote>\n  <p>{'0c48ed342_17',<br>\n   '178b835e3_16',<br>\n   '2c97a8020_17',<br>\n   '8f3d579c8_17',<br>\n   'a2441a74b_17',<br>\n   'bda762977_17'}</p>\n</blockquote>\n<p>Not sure if this is just a labelling error, or a consequence of the correct interpretation of tp and pf.</p>\n<p>Further adding the songtype_id into the mix:</p>\n<pre><code>tps = set([r + '_' + str(s) + '_' + str(t) for r, s, t in df_tp[['recording_id', 'species_id', 'songtype_id']].values])\nfps = set([r + '_' + str(s) + '_' + str(t) for r, s, t in df_fp[['recording_id', 'species_id', 'songtype_id']].values])\ntps.intersection(fps)\n</code></pre>\n<p>Produces:</p>\n<blockquote>\n  <p>{'178b835e3_16_4', 'a2441a74b_17_4'}</p>\n</blockquote>\n<p>In any case, given the total amount of data and the few intersections, <strong>I think this means we can mostly go with the simplest interpretation of tp and fp: ie, TP means \"This recording has this species\".  FP means \"This recording does not have the species\".</strong></p>",
      "rawMarkdown": "If a file_id / species_id combinations is not in tp csv does it mean that the respective species is definitely not in that file_id? \n\nWhen there is an entry in the fp csv does it mean that the respective species does not exist in the whole recording? That the species does not exist just in the time window specified? That the song type does not exist (but the species might)?\n\n\\+ all other variants of questions which aim to resolve the exact meaning of tp and fp.\n\n---\n\n**EDIT**:\n\nSome quick digging showed me that it's possible to have the an fp and a tp for the same species_id in the same recording_id:\n\n```python\ndf_tp = pd.read_csv(TP_CSV_PATH)\ndf_fp = pd.read_csv(FP_CSV_PATH)\ntps = set([r + '_' + str(s) for r, s in df_tp[['recording_id', 'species_id']].values])\nfps = set([r + '_' + str(s) for r, s in df_fp[['recording_id', 'species_id']].values])\ntps.intersection(fps)\n```\n\nProduces\n>{'0c48ed342_17',\n '178b835e3_16',\n '2c97a8020_17',\n '8f3d579c8_17',\n 'a2441a74b_17',\n 'bda762977_17'}\n\n\nNot sure if this is just a labelling error, or a consequence of the correct interpretation of tp and pf.\n\nFurther adding the songtype_id into the mix:\n\n```\ntps = set([r + '_' + str(s) + '_' + str(t) for r, s, t in df_tp[['recording_id', 'species_id', 'songtype_id']].values])\nfps = set([r + '_' + str(s) + '_' + str(t) for r, s, t in df_fp[['recording_id', 'species_id', 'songtype_id']].values])\ntps.intersection(fps)\n```\n\nProduces:\n\n> {'178b835e3_16_4', 'a2441a74b_17_4'}\n\nIn any case, given the total amount of data and the few intersections, **I think this means we can mostly go with the simplest interpretation of tp and fp: ie, TP means \"This recording has this species\".  FP means \"This recording does not have the species\".**",
      "votes": null
    },
    {
      "id": "1123741",
      "postDate": "12/23/2020 13:19:51",
      "content": "<p>In case anyone is interested, this post from the competition host helps with some of my questions <a href=\"https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197866\" target=\"_blank\">https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197866</a></p>",
      "rawMarkdown": "In case anyone is interested, this post from the competition host helps with some of my questions https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197866",
      "votes": null
    },
    {
      "id": "1125605",
      "postDate": "12/24/2020 21:10:04",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/alexandersoare\" target=\"_blank\">@alexandersoare</a> , good question!<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1557202%2F87045d264121a1b891a07a003f08d9d2%2FScreenshot%20from%202020-12-24%2015-01-41.png?generation=1608843866454291&amp;alt=media\" alt=\"\"></p>\n<p>In the above example, the pattern that repeats 3 times at 49, 54, 57 seconds is species three (I can eyeball it…). But it is not labeled by any box in <code>train_tp</code> and <code>train_fp</code>.</p>\n<p>If you look at and listen to many of the audio, you will find that it is almost impossible to label all that is happening in the rainforest.</p>\n<p>I don't know if I am correct, but I would take it to the narrowest sense:<br>\n<code>train_tp</code> says inside of the time-frequency box, there is at least one instance of this species. <br>\n<code>train_fp</code> says inside of the time-frequency box, there is no instance of this specific species.</p>",
      "rawMarkdown": "Hi, @alexandersoare , good question!\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1557202%2F87045d264121a1b891a07a003f08d9d2%2FScreenshot%20from%202020-12-24%2015-01-41.png?generation=1608843866454291&alt=media)\n\nIn the above example, the pattern that repeats 3 times at 49, 54, 57 seconds is species three (I can eyeball it...). But it is not labeled by any box in `train_tp` and `train_fp`.\n\nIf you look at and listen to many of the audio, you will find that it is almost impossible to label all that is happening in the rainforest.\n\nI don't know if I am correct, but I would take it to the narrowest sense:\n`train_tp` says inside of the time-frequency box, there is at least one instance of this species. \n`train_fp` says inside of the time-frequency box, there is no instance of this specific species.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1123741,
      "author_name": "alexandersoare",
      "author_url": "",
      "post_date": "12/23/2020 13:19:51",
      "content": "<p>In case anyone is interested, this post from the competition host helps with some of my questions <a href=\"https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197866\" target=\"_blank\">https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197866</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1125605,
      "author_name": "barnwellguy",
      "author_url": "",
      "post_date": "12/24/2020 21:10:04",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/alexandersoare\" target=\"_blank\">@alexandersoare</a> , good question!<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1557202%2F87045d264121a1b891a07a003f08d9d2%2FScreenshot%20from%202020-12-24%2015-01-41.png?generation=1608843866454291&amp;alt=media\" alt=\"\"></p>\n<p>In the above example, the pattern that repeats 3 times at 49, 54, 57 seconds is species three (I can eyeball it…). But it is not labeled by any box in <code>train_tp</code> and <code>train_fp</code>.</p>\n<p>If you look at and listen to many of the audio, you will find that it is almost impossible to label all that is happening in the rainforest.</p>\n<p>I don't know if I am correct, but I would take it to the narrowest sense:<br>\n<code>train_tp</code> says inside of the time-frequency box, there is at least one instance of this species. <br>\n<code>train_fp</code> says inside of the time-frequency box, there is no instance of this specific species.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1122933": "If a file_id / species_id combinations is not in tp csv does it mean that the respective species is definitely not in that file_id? \n\nWhen there is an entry in the fp csv does it mean that the respective species does not exist in the whole recording? That the species does not exist just in the time window specified? That the song type does not exist (but the species might)?\n\n\\+ all other variants of questions which aim to resolve the exact meaning of tp and fp.\n\n---\n\n**EDIT**:\n\nSome quick digging showed me that it's possible to have the an fp and a tp for the same species_id in the same recording_id:\n\n```python\ndf_tp = pd.read_csv(TP_CSV_PATH)\ndf_fp = pd.read_csv(FP_CSV_PATH)\ntps = set([r + '_' + str(s) for r, s in df_tp[['recording_id', 'species_id']].values])\nfps = set([r + '_' + str(s) for r, s in df_fp[['recording_id', 'species_id']].values])\ntps.intersection(fps)\n```\n\nProduces\n>{'0c48ed342_17',\n '178b835e3_16',\n '2c97a8020_17',\n '8f3d579c8_17',\n 'a2441a74b_17',\n 'bda762977_17'}\n\n\nNot sure if this is just a labelling error, or a consequence of the correct interpretation of tp and pf.\n\nFurther adding the songtype_id into the mix:\n\n```\ntps = set([r + '_' + str(s) + '_' + str(t) for r, s, t in df_tp[['recording_id', 'species_id', 'songtype_id']].values])\nfps = set([r + '_' + str(s) + '_' + str(t) for r, s, t in df_fp[['recording_id', 'species_id', 'songtype_id']].values])\ntps.intersection(fps)\n```\n\nProduces:\n\n> {'178b835e3_16_4', 'a2441a74b_17_4'}\n\nIn any case, given the total amount of data and the few intersections, **I think this means we can mostly go with the simplest interpretation of tp and fp: ie, TP means \"This recording has this species\".  FP means \"This recording does not have the species\".**",
    "1123741": "In case anyone is interested, this post from the competition host helps with some of my questions https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197866",
    "1125605": "Hi, @alexandersoare , good question!\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1557202%2F87045d264121a1b891a07a003f08d9d2%2FScreenshot%20from%202020-12-24%2015-01-41.png?generation=1608843866454291&alt=media)\n\nIn the above example, the pattern that repeats 3 times at 49, 54, 57 seconds is species three (I can eyeball it...). But it is not labeled by any box in `train_tp` and `train_fp`.\n\nIf you look at and listen to many of the audio, you will find that it is almost impossible to label all that is happening in the rainforest.\n\nI don't know if I am correct, but I would take it to the narrowest sense:\n`train_tp` says inside of the time-frequency box, there is at least one instance of this species. \n`train_fp` says inside of the time-frequency box, there is no instance of this specific species."
  },
  "source": "meta"
}