officatial evaluation score describe as this
", "messages": [ { "id": "403967", "postDate": "10/15/2018 02:12:54", "content": "officatial evaluation score describe as this
", "rawMarkdown": "[officatial evaluation score describe as this][1]\n\n\n [1]: https://www.kaggle.com/c/airbus-ship-detection#evaluation", "votes": null }, { "id": "404003", "postDate": "10/15/2018 04:59:34", "content": "You can check get_score function in https://www.kaggle.com/iafoss/unet34-submission-tta-0-699-new-public-lb . For images without ships, the score is 0 if you predict any nonzero pixel and 1 if you return an empty mask.
", "rawMarkdown": "You can check get_score function in https://www.kaggle.com/iafoss/unet34-submission-tta-0-699-new-public-lb . For images without ships, the score is 0 if you predict any nonzero pixel and 1 if you return an empty mask.", "votes": null }, { "id": "404227", "postDate": "10/15/2018 13:07:48", "content": "in this case, this kernel only calculate IOU score above threshold , not TP,FN,FP ,they regard as evaluation score for F0 (when beta = 0) , here is F2(when beta = 2)
\n\ndef get_iou_vector(A, B):\n batch_size = A.shape[0]\n metric = []\n for batch in range(batch_size):\n t, p = A[batch]>0, B[batch]>0\n intersection = np.logical_and(t, p)\n union = np.logical_or(t, p)\n iou = (np.sum(intersection > 0) + 1e-10 )/ (np.sum(union > 0) + 1e-10)\n thresholds = np.arange(0.5, 1, 0.05)\n s = []\n for thresh in thresholds:\n s.append(iou > thresh)\n metric.append(np.mean(s))\n\n return np.mean(metric)\nin your kernel, i consider wil get different value when apply different batch_size, they just compare predict sample with true sample in batch_size number
", "rawMarkdown": "in your kernel, i consider wil get different value when apply different batch_size, they just compare predict sample with true sample in batch_size number", "votes": null }, { "id": "404288", "postDate": "10/15/2018 14:18:45", "content": "That competition is a little bit different, here you need to predict an individual mask for each ship present in the image, and TP, TN, FP are calculated for each image individually based on your prediction. The result should not depend on the batch size.
", "rawMarkdown": "That competition is a little bit different, here you need to predict an individual mask for each ship present in the image, and TP, TN, FP are calculated for each image individually based on your prediction. The result should not depend on the batch size.", "votes": null } ], "comments": [ { "id": 404003, "author_name": "iafoss", "author_url": "", "post_date": "10/15/2018 04:59:34", "content": "You can check get_score function in https://www.kaggle.com/iafoss/unet34-submission-tta-0-699-new-public-lb . For images without ships, the score is 0 if you predict any nonzero pixel and 1 if you return an empty mask.
", "votes": null, "replies": [ { "id": 404227, "author_name": "liuchuanloong", "author_url": "", "post_date": "10/15/2018 13:07:48", "content": "in this case, this kernel only calculate IOU score above threshold , not TP,FN,FP ,they regard as evaluation score for F0 (when beta = 0) , here is F2(when beta = 2)
\n\ndef get_iou_vector(A, B):\n batch_size = A.shape[0]\n metric = []\n for batch in range(batch_size):\n t, p = A[batch]>0, B[batch]>0\n intersection = np.logical_and(t, p)\n union = np.logical_or(t, p)\n iou = (np.sum(intersection > 0) + 1e-10 )/ (np.sum(union > 0) + 1e-10)\n thresholds = np.arange(0.5, 1, 0.05)\n s = []\n for thresh in thresholds:\n s.append(iou > thresh)\n metric.append(np.mean(s))\n\n return np.mean(metric)\nin your kernel, i consider wil get different value when apply different batch_size, they just compare predict sample with true sample in batch_size number
", "votes": null, "replies": [] }, { "id": 404288, "author_name": "iafoss", "author_url": "", "post_date": "10/15/2018 14:18:45", "content": "That competition is a little bit different, here you need to predict an individual mask for each ship present in the image, and TP, TN, FP are calculated for each image individually based on your prediction. The result should not depend on the batch size.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "403967": "[officatial evaluation score describe as this][1]\n\n\n [1]: https://www.kaggle.com/c/airbus-ship-detection#evaluation", "404003": "You can check get_score function in https://www.kaggle.com/iafoss/unet34-submission-tta-0-699-new-public-lb . For images without ships, the score is 0 if you predict any nonzero pixel and 1 if you return an empty mask.", "404227": "in [this][1] case, this [kernel][2] only calculate IOU score above threshold , not TP,FN,FP ,they regard as evaluation score for F0 (when beta = 0) , here is F2(when beta = 2)\n \n def get_iou_vector(A, B):\n batch_size = A.shape[0]\n metric = []\n for batch in range(batch_size):\n t, p = A[batch]>0, B[batch]>0\n intersection = np.logical_and(t, p)\n union = np.logical_or(t, p)\n iou = (np.sum(intersection > 0) + 1e-10 )/ (np.sum(union > 0) + 1e-10)\n thresholds = np.arange(0.5, 1, 0.05)\n s = []\n for thresh in thresholds:\n s.append(iou > thresh)\n metric.append(np.mean(s))\n \n return np.mean(metric)\n\n\n\n [1]: https://www.kaggle.com/c/tgs-salt-identification-challenge#evaluation\n [2]: https://www.kaggle.com/shaojiaxin/u-net-with-simple-resnet-blocks-v2-new-loss", "404253": "in your kernel, i consider wil get different value when apply different batch_size, they just compare predict sample with true sample in batch_size number", "404288": "That competition is a little bit different, here you need to predict an individual mask for each ship present in the image, and TP, TN, FP are calculated for each image individually based on your prediction. The result should not depend on the batch size." }, "source": "meta" }