The Leaderboard metric was updated to include FN. Although I knew TP and FP from official explanation, but what's specific definition of FN in this competition ?\nI guess it's the number of unpaired solution objects.
", "messages": [ { "id": "691423", "postDate": "12/10/2019 05:58:10", "content": "The Leaderboard metric was updated to include FN. Although I knew TP and FP from official explanation, but what's specific definition of FN in this competition ?\nI guess it's the number of unpaired solution objects.
", "rawMarkdown": "The Leaderboard metric was updated to include FN. Although I knew TP and FP from official explanation, but what's specific definition of FN in this competition ?\nI guess it's the number of unpaired solution objects.", "votes": null }, { "id": "691459", "postDate": "12/10/2019 06:59:38", "content": "*F*alse *N*egatives: cars, that have not been identified.
\n\nEarlier, FN were not in the metric. Thus , a model that only predicts one car with perfect match to ground truth would score 1.000. Of course, this was not intended by the hosts.
\n\nI agree, the metric page must be updated to reflect the new behavior.
", "rawMarkdown": "**F**alse **N**egatives: cars, that have not been identified.\n\nEarlier, FN were not in the metric. Thus , a model that only predicts one car with perfect match to ground truth would score 1.000. Of course, this was not intended by the hosts. \n\nI agree, the metric page must be updated to reflect the new behavior.", "votes": null }, { "id": "691508", "postDate": "12/10/2019 08:03:31", "content": "Thanks!\nIt's confusing that mAP, which is used as the metric, or AP already includes FN, and if FN was not included in the previous metric, how will recall parts in mAP be calculated?
", "rawMarkdown": "Thanks!\nIt's confusing that [mAP](https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Mean_average_precision), which is used as the metric, or [AP](https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Average%20precision) already includes FN, and if FN was not included in the previous metric, how will recall parts in mAP be calculated?", "votes": null }, { "id": "691521", "postDate": "12/10/2019 08:15:38", "content": "Or there might be a mistake, the previous metric was not mAP, it was just mP or mean precision.
", "rawMarkdown": "Or there might be a mistake, the previous metric was not mAP, it was just mP or mean precision.", "votes": null }, { "id": "691897", "postDate": "12/10/2019 16:09:41", "content": "No one knows! I cannot understand why kaggle keep hiding what they want us to do...🙄
", "rawMarkdown": "No one knows! I cannot understand why kaggle keep hiding what they want us to do...🙄", "votes": null } ], "comments": [ { "id": 691459, "author_name": "ilu000", "author_url": "", "post_date": "12/10/2019 06:59:38", "content": "*F*alse *N*egatives: cars, that have not been identified.
\n\nEarlier, FN were not in the metric. Thus , a model that only predicts one car with perfect match to ground truth would score 1.000. Of course, this was not intended by the hosts.
\n\nI agree, the metric page must be updated to reflect the new behavior.
", "votes": null, "replies": [ { "id": 691508, "author_name": "a1850961785", "author_url": "", "post_date": "12/10/2019 08:03:31", "content": "Thanks!\nIt's confusing that mAP, which is used as the metric, or AP already includes FN, and if FN was not included in the previous metric, how will recall parts in mAP be calculated?
", "votes": null, "replies": [] }, { "id": 691521, "author_name": "a1850961785", "author_url": "", "post_date": "12/10/2019 08:15:38", "content": "Or there might be a mistake, the previous metric was not mAP, it was just mP or mean precision.
", "votes": null, "replies": [] } ] }, { "id": 691897, "author_name": "bamps53", "author_url": "", "post_date": "12/10/2019 16:09:41", "content": "No one knows! I cannot understand why kaggle keep hiding what they want us to do...🙄
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "691423": "The Leaderboard metric was updated to include FN. Although I knew TP and FP from official explanation, but what's specific definition of FN in this competition ?\nI guess it's the number of unpaired solution objects.", "691459": "**F**alse **N**egatives: cars, that have not been identified.\n\nEarlier, FN were not in the metric. Thus , a model that only predicts one car with perfect match to ground truth would score 1.000. Of course, this was not intended by the hosts. \n\nI agree, the metric page must be updated to reflect the new behavior.", "691508": "Thanks!\nIt's confusing that [mAP](https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Mean_average_precision), which is used as the metric, or [AP](https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Average%20precision) already includes FN, and if FN was not included in the previous metric, how will recall parts in mAP be calculated?", "691521": "Or there might be a mistake, the previous metric was not mAP, it was just mP or mean precision.", "691897": "No one knows! I cannot understand why kaggle keep hiding what they want us to do...🙄" }, "source": "meta" }