The equation for calculating the F2 score is given in the challenge description section on evaluation as:
\n\nFβ(t) = ((1+β2)⋅TP(t)) / ((1+β2)⋅TP(t)+β2⋅FN(t)+FP(t))
\n\nWhere β = 2.
\n\nIf I have understood this correctly, where there are no ground truths in a sample (i.e. no boats) the F2 score equates to zero, irregardless of how many false positives are present in the prediction. In other words, you can predict as many boats as you like in a picture with no boats with no impact on your score.
\n\nThis really doesn't seem right. Am I missing something? Is something missing from the metric calculation description?
\n\nThanks in advance for any light anyone can shed on this.
\n\nHappy predicting
\n\nLeo
", "messages": [ { "id": "396258", "postDate": "09/30/2018 10:51:06", "content": "The equation for calculating the F2 score is given in the challenge description section on evaluation as:
\n\nFβ(t) = ((1+β2)⋅TP(t)) / ((1+β2)⋅TP(t)+β2⋅FN(t)+FP(t))
\n\nWhere β = 2.
\n\nIf I have understood this correctly, where there are no ground truths in a sample (i.e. no boats) the F2 score equates to zero, irregardless of how many false positives are present in the prediction. In other words, you can predict as many boats as you like in a picture with no boats with no impact on your score.
\n\nThis really doesn't seem right. Am I missing something? Is something missing from the metric calculation description?
\n\nThanks in advance for any light anyone can shed on this.
\n\nHappy predicting
\n\nLeo
", "rawMarkdown": "The equation for calculating the F2 score is given in the [challenge description section on evaluation][1] as:\n\nFβ(t) = ((1+β2)⋅TP(t)) / ((1+β2)⋅TP(t)+β2⋅FN(t)+FP(t))\n\nWhere β = 2.\n\nIf I have understood this correctly, where there are no ground truths in a sample (i.e. no boats) the F2 score equates to zero, irregardless of how many false positives are present in the prediction. In other words, **you can predict as many boats as you like in a picture with no boats with no impact on your score**. \n\nThis really doesn't seem right. Am I missing something? Is something missing from the metric calculation description?\n\nThanks in advance for any light anyone can shed on this.\n\nHappy predicting\n\nLeo\n\n\n [1]: https://www.kaggle.com/c/airbus-ship-detection#evaluation", "votes": null }, { "id": "396352", "postDate": "09/30/2018 14:51:29", "content": "No boats in ground truth means TP must be zero, thus any non-zero FP will cause F2 score to be zero. You get 1 if FP is also zero by the definition of F2 score. \nThus, to get 1, u need to not have any FP. This makes sense since the only correct prediction for an empty image is empty prediction.
", "rawMarkdown": "No boats in ground truth means TP must be zero, thus any non-zero FP will cause F2 score to be zero. You get 1 if FP is also zero by the definition of F2 score. \nThus, to get 1, u need to not have any FP. This makes sense since the only correct prediction for an empty image is empty prediction.", "votes": null }, { "id": "396373", "postDate": "09/30/2018 15:34:03", "content": "OK, now I understand, thanks. The prediction of no boats in a pic without boats counts as one TP for the sake of the metric.
", "rawMarkdown": "OK, now I understand, thanks. The prediction of no boats in a pic without boats counts as one TP for the sake of the metric.", "votes": null } ], "comments": [ { "id": 396352, "author_name": "woodywang", "author_url": "", "post_date": "09/30/2018 14:51:29", "content": "No boats in ground truth means TP must be zero, thus any non-zero FP will cause F2 score to be zero. You get 1 if FP is also zero by the definition of F2 score. \nThus, to get 1, u need to not have any FP. This makes sense since the only correct prediction for an empty image is empty prediction.
", "votes": null, "replies": [ { "id": 396373, "author_name": "leoauri", "author_url": "", "post_date": "09/30/2018 15:34:03", "content": "OK, now I understand, thanks. The prediction of no boats in a pic without boats counts as one TP for the sake of the metric.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "396258": "The equation for calculating the F2 score is given in the [challenge description section on evaluation][1] as:\n\nFβ(t) = ((1+β2)⋅TP(t)) / ((1+β2)⋅TP(t)+β2⋅FN(t)+FP(t))\n\nWhere β = 2.\n\nIf I have understood this correctly, where there are no ground truths in a sample (i.e. no boats) the F2 score equates to zero, irregardless of how many false positives are present in the prediction. In other words, **you can predict as many boats as you like in a picture with no boats with no impact on your score**. \n\nThis really doesn't seem right. Am I missing something? Is something missing from the metric calculation description?\n\nThanks in advance for any light anyone can shed on this.\n\nHappy predicting\n\nLeo\n\n\n [1]: https://www.kaggle.com/c/airbus-ship-detection#evaluation", "396352": "No boats in ground truth means TP must be zero, thus any non-zero FP will cause F2 score to be zero. You get 1 if FP is also zero by the definition of F2 score. \nThus, to get 1, u need to not have any FP. This makes sense since the only correct prediction for an empty image is empty prediction.", "396373": "OK, now I understand, thanks. The prediction of no boats in a pic without boats counts as one TP for the sake of the metric." }, "source": "meta" }