Fro anyone new to this type of competitions that did not now the Jaccard index metric (like myself), quite neat explanation with code on stack overflow and on wikipedia
", "messages": [ { "id": 364156, "postDate": "2018-07-30T22:02:55.733Z", "content": "Fro anyone new to this type of competitions that did not now the Jaccard index metric (like myself), quite neat explanation with code on stack overflow and on wikipedia
", "rawMarkdown": "Fro anyone new to this type of competitions that did not now the Jaccard index metric (like myself), quite neat explanation with code on [stack overflow][1] and on [wikipedia][2]\n\n\n [1]: https://stackoverflow.com/questions/28723670/intersection-over-union-between-two-detections\n [2]: https://en.wikipedia.org/wiki/Jaccard_index", "votes": 7 }, { "id": 364162, "postDate": "2018-07-30T22:29:39.750Z", "content": "The evaluation page https://www.kaggle.com/c/airbus-ship-detection#evaluation look very similar to evaluation page in other competition (TGS Salt Identification Challenge) https://www.kaggle.com/c/tgs-salt-identification-challenge#evaluation . Even there is similar string \" Each row in your submission represents a single predicted salt segmentation for the given image.\"
", "rawMarkdown": "The evaluation page https://www.kaggle.com/c/airbus-ship-detection#evaluation look very similar to evaluation page in other competition (TGS Salt Identification Challenge) https://www.kaggle.com/c/tgs-salt-identification-challenge#evaluation . Even there is similar string \" Each row in your submission represents a single predicted salt segmentation for the given image.\"", "votes": 4, "replies": [ { "id": 364285, "postDate": "2018-07-31T07:59:43.853Z", "content": "really funny :)
", "rawMarkdown": "really funny :)" }, { "id": 364512, "postDate": "2018-07-31T16:52:29.930Z", "content": "Good catch!
\n\nWe did this to see who reads the evaluation page closely. I think the reward should be a bag of brown M&Ms. ;-)
\n\nFixed.
", "rawMarkdown": "Good catch! \n\nWe did this to see who reads the evaluation page closely. I think the reward should be a bag of [brown M&Ms][1]. ;-)\n\nFixed.\n\n\n [1]: https://www.snopes.com/fact-check/brown-out/", "votes": 1 } ] }, { "id": 366575, "postDate": "2018-08-05T21:41:38.437Z", "content": "That doesn't explain why empty submissions give a quite high score >0.8. The intersection of anything to None is 0, isn't it?\nOr are they using the intersection on the 0's in case there's no boat in the image?
", "rawMarkdown": "That doesn't explain why empty submissions give a quite high score >0.8. The intersection of anything to None is 0, isn't it?\nOr are they using the intersection on the 0's in case there's no boat in the image?", "votes": 1 }, { "id": 383921, "postDate": "2018-09-10T00:30:12.600Z", "content": "Thanks for the links jaccard or IOU seems to be fairly easy to understand. Yet I have another question: how to match the predictions to ground truth objects ? Thanks
", "rawMarkdown": "Thanks for the links jaccard or IOU seems to be fairly easy to understand. Yet I have another question: how to match the predictions to ground truth objects ? Thanks\n", "replies": [ { "id": 387465, "postDate": "2018-09-15T01:33:22.550Z", "content": "I am hoping one of the organizers will answer this. It's not apparent how matching is done when there are multiple ground-truths and multiple predictions for the same image.
", "rawMarkdown": "I am hoping one of the organizers will answer this. It's not apparent how matching is done when there are multiple ground-truths and multiple predictions for the same image.", "isDeleted": true }, { "id": 387571, "postDate": "2018-09-15T07:31:53.577Z", "content": "My understanding is that for each predicted object, there should be a ground-truth object with an IOU greater than a threshold in order for that prediction to be considered a true positive.
\n\nCheck out https://www.kaggle.com/c/airbus-ship-detection#evaluation for the full description.
", "rawMarkdown": "My understanding is that for each predicted object, there should be a ground-truth object with an IOU greater than a threshold in order for that prediction to be considered a true positive.\n\nCheck out https://www.kaggle.com/c/airbus-ship-detection#evaluation for the full description." }, { "id": 387972, "postDate": "2018-09-16T02:14:09.913Z", "content": "I ended up implementing this https://www.kaggle.com/stkbailey/step-by-step-explanation-of-scoring-metric. It's an excellent notebook to understand the metric and it makes a lot of sense why this should be used to evaluate.
", "rawMarkdown": "I ended up implementing this https://www.kaggle.com/stkbailey/step-by-step-explanation-of-scoring-metric. It's an excellent notebook to understand the metric and it makes a lot of sense why this should be used to evaluate." } ] }, { "id": 364394, "postDate": "2018-07-31T13:03:33.233Z", "content": "Andrew Ng also has quite a nice explanation here!
", "rawMarkdown": "Andrew Ng also has quite a nice explanation [here][1]!\n\n\n [1]: https://www.youtube.com/watch?v=ANIzQ5G-XPE", "replies": [ { "id": 365003, "postDate": "2018-08-01T17:47:30.760Z", "content": "Especially this part of the video is nicely explained:\n\"IoU not to be confused with I owe you that much money\"
", "rawMarkdown": "Especially this part of the video is nicely explained:\n\"IoU not to be confused with I owe you that much money\"" } ] } ], "comments": [ { "id": 364162, "author_name": "waskita adijarto", "author_url": "", "post_date": "2018-07-30T22:29:39.750000", "content": "The evaluation page https://www.kaggle.com/c/airbus-ship-detection#evaluation look very similar to evaluation page in other competition (TGS Salt Identification Challenge) https://www.kaggle.com/c/tgs-salt-identification-challenge#evaluation . Even there is similar string \" Each row in your submission represents a single predicted salt segmentation for the given image.\"
", "votes": 4, "replies": [ { "id": 364285, "author_name": "Karthick", "author_url": "", "post_date": "2018-07-31T07:59:43.853000", "content": "really funny :)
", "votes": 0, "replies": [] }, { "id": 364512, "author_name": "inversion", "author_url": "", "post_date": "2018-07-31T16:52:29.930000", "content": "Good catch!
\n\nWe did this to see who reads the evaluation page closely. I think the reward should be a bag of brown M&Ms. ;-)
\n\nFixed.
", "votes": 1, "replies": [] } ] }, { "id": 366575, "author_name": "Henrique Mendonça", "author_url": "", "post_date": "2018-08-05T21:41:38.437000", "content": "That doesn't explain why empty submissions give a quite high score >0.8. The intersection of anything to None is 0, isn't it?\nOr are they using the intersection on the 0's in case there's no boat in the image?
", "votes": 1, "replies": [] }, { "id": 383921, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2018-09-10T00:30:12.600000", "content": "Thanks for the links jaccard or IOU seems to be fairly easy to understand. Yet I have another question: how to match the predictions to ground truth objects ? Thanks
", "votes": 0, "replies": [ { "id": 387465, "author_name": "", "author_url": "", "post_date": "2018-09-15T01:33:22.550000", "content": "I am hoping one of the organizers will answer this. It's not apparent how matching is done when there are multiple ground-truths and multiple predictions for the same image.
", "votes": 0, "replies": [] }, { "id": 387571, "author_name": "Cosmin", "author_url": "", "post_date": "2018-09-15T07:31:53.577000", "content": "My understanding is that for each predicted object, there should be a ground-truth object with an IOU greater than a threshold in order for that prediction to be considered a true positive.
\n\nCheck out https://www.kaggle.com/c/airbus-ship-detection#evaluation for the full description.
", "votes": 0, "replies": [] }, { "id": 387972, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2018-09-16T02:14:09.913000", "content": "I ended up implementing this https://www.kaggle.com/stkbailey/step-by-step-explanation-of-scoring-metric. It's an excellent notebook to understand the metric and it makes a lot of sense why this should be used to evaluate.
", "votes": 0, "replies": [] } ] }, { "id": 364394, "author_name": "differentialmind", "author_url": "", "post_date": "2018-07-31T13:03:33.233000", "content": "Andrew Ng also has quite a nice explanation here!
", "votes": 0, "replies": [ { "id": 365003, "author_name": "Eduard Tyantov", "author_url": "", "post_date": "2018-08-01T17:47:30.760000", "content": "Especially this part of the video is nicely explained:\n\"IoU not to be confused with I owe you that much money\"
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "364156": "Fro anyone new to this type of competitions that did not now the Jaccard index metric (like myself), quite neat explanation with code on [stack overflow][1] and on [wikipedia][2]\n\n\n [1]: https://stackoverflow.com/questions/28723670/intersection-over-union-between-two-detections\n [2]: https://en.wikipedia.org/wiki/Jaccard_index", "364162": "The evaluation page https://www.kaggle.com/c/airbus-ship-detection#evaluation look very similar to evaluation page in other competition (TGS Salt Identification Challenge) https://www.kaggle.com/c/tgs-salt-identification-challenge#evaluation . Even there is similar string \" Each row in your submission represents a single predicted salt segmentation for the given image.\"", "366575": "That doesn't explain why empty submissions give a quite high score >0.8. The intersection of anything to None is 0, isn't it?\nOr are they using the intersection on the 0's in case there's no boat in the image?", "383921": "Thanks for the links jaccard or IOU seems to be fairly easy to understand. Yet I have another question: how to match the predictions to ground truth objects ? Thanks\n", "364394": "Andrew Ng also has quite a nice explanation [here][1]!\n\n\n [1]: https://www.youtube.com/watch?v=ANIzQ5G-XPE" } }