tensorflow's model detection zoo has faster_rcnn_inception_resnet_v2_atrous_oid mAP on open images dataset listed as 0.37, but running it here only gives a score of 0.00006.
\n\nI know that the test images in this challenge are more densely annotated, but 0.00006? That's like not detecting anything.
\n\nAny ideas?
\n\n[resolved, bug when creating submission file]
", "messages": [ { "id": "371657", "postDate": "08/17/2018 10:44:10", "content": "tensorflow's model detection zoo has faster_rcnn_inception_resnet_v2_atrous_oid mAP on open images dataset listed as 0.37, but running it here only gives a score of 0.00006.
\n\nI know that the test images in this challenge are more densely annotated, but 0.00006? That's like not detecting anything.
\n\nAny ideas?
\n\n[resolved, bug when creating submission file]
", "rawMarkdown": "tensorflow's model detection zoo has faster_rcnn_inception_resnet_v2_atrous_oid mAP on open images dataset listed as 0.37, but running it here only gives a score of 0.00006.\n\nI know that the test images in this challenge are more densely annotated, but 0.00006? That's like not detecting anything.\n\nAny ideas?\n\n[resolved, bug when creating submission file]", "votes": null }, { "id": "371660", "postDate": "08/17/2018 10:54:06", "content": "It usually has to do with either wrong pre processing (rgb instead of the other thing, can't remember the order of the letters), resizing when there is no need to, grayscale.\nOr... Just a bug in creating the submission...\nBeen there, done that
", "rawMarkdown": "It usually has to do with either wrong pre processing (rgb instead of the other thing, can't remember the order of the letters), resizing when there is no need to, grayscale.\nOr... Just a bug in creating the submission...\nBeen there, done that", "votes": null }, { "id": "371746", "postDate": "08/17/2018 15:01:01", "content": "As moshel said you might be doing something wrong. Using the faster_rcnn_inception_resnet_v2_atrous_oid model i could get a LB score of .18. The correct pre-processing of images is RGB instead of OpenCV BGR, you might want to check format of submission, the correct prediction format is <label confidence x_min y_min x_max y_max>
Hi @Master.\nI tried to fine-tune faster_rcnn_inception_resnet_v2_atrous_oid model without any luck. I am just wondering if you can share the process a bit. Thanks in advance.
", "rawMarkdown": "Hi @Master.\nI tried to fine-tune faster_rcnn_inception_resnet_v2_atrous_oid model without any luck. I am just wondering if you can share the process a bit. Thanks in advance.", "votes": null }, { "id": "372109", "postDate": "08/18/2018 10:47:37", "content": "I preprocess with PIL, so it's RGB. Did you get 0.18 with finetuning or straight out of the box?
", "rawMarkdown": "I preprocess with PIL, so it's RGB. Did you get 0.18 with finetuning or straight out of the box?", "votes": null }, { "id": "372110", "postDate": "08/18/2018 10:49:39", "content": "I just retrain it on this dataset. I'm getting only 0.07 on LB, so it's not really working for me either.
", "rawMarkdown": "I just retrain it on this dataset. I'm getting only 0.07 on LB, so it's not really working for me either.", "votes": null }, { "id": "372242", "postDate": "08/18/2018 18:08:07", "content": "straight out of the box
", "rawMarkdown": "straight out of the box", "votes": null }, { "id": "373845", "postDate": "08/22/2018 02:37:24", "content": "Hello @Master\n> [resolved, bug when creating submission file]\nI am just wondering if you can share what kind of bug it was and how to solve it. I wasted a lot of time to figure out low score issue, but yet to find the hole. Thank you very much.
", "rawMarkdown": "Hello @Master\n> [resolved, bug when creating submission file]\nI am just wondering if you can share what kind of bug it was and how to solve it. I wasted a lot of time to figure out low score issue, but yet to find the hole. Thank you very much.", "votes": null }, { "id": "373899", "postDate": "08/22/2018 04:16:20", "content": "casting detection classes to np.uint8 when there are 500 classes. np.uint8 has max of 255.
", "rawMarkdown": "casting detection classes to np.uint8 when there are 500 classes. np.uint8 has max of 255.", "votes": null }, { "id": "374273", "postDate": "08/22/2018 18:09:52", "content": "Thanks a lot for this great help.
", "rawMarkdown": "Thanks a lot for this great help.", "votes": null }, { "id": "374442", "postDate": "08/23/2018 03:42:25", "content": "Thank you very much, that seems to explain all the low scores I've been getting.....
", "rawMarkdown": "Thank you very much, that seems to explain all the low scores I've been getting.....", "votes": null }, { "id": "374938", "postDate": "08/24/2018 08:26:20", "content": "Hi @rupraze,\nHave you processed single image/batch of images during inference? I am still stuck at low scored ~0.04.
", "rawMarkdown": "Hi @rupraze,\nHave you processed single image/batch of images during inference? I am still stuck at low scored ~0.04.", "votes": null }, { "id": "374939", "postDate": "08/24/2018 08:26:42", "content": "Hi @Master,\nChanging np.uint8 increased the model performance a little only. Is there any more minor change to be made?
", "rawMarkdown": "Hi @Master,\nChanging np.uint8 increased the model performance a little only. Is there any more minor change to be made?", "votes": null }, { "id": "374953", "postDate": "08/24/2018 08:51:30", "content": "Train your model for longer and lower the learning rate.
", "rawMarkdown": "Train your model for longer and lower the learning rate.", "votes": null }, { "id": "374957", "postDate": "08/24/2018 08:56:38", "content": "Single image during inference
", "rawMarkdown": "Single image during inference", "votes": null } ], "comments": [ { "id": 371660, "author_name": "moshel", "author_url": "", "post_date": "08/17/2018 10:54:06", "content": "It usually has to do with either wrong pre processing (rgb instead of the other thing, can't remember the order of the letters), resizing when there is no need to, grayscale.\nOr... Just a bug in creating the submission...\nBeen there, done that
", "votes": null, "replies": [] }, { "id": 371746, "author_name": "rupraze", "author_url": "", "post_date": "08/17/2018 15:01:01", "content": "As moshel said you might be doing something wrong. Using the faster_rcnn_inception_resnet_v2_atrous_oid model i could get a LB score of .18. The correct pre-processing of images is RGB instead of OpenCV BGR, you might want to check format of submission, the correct prediction format is <label confidence x_min y_min x_max y_max>
I preprocess with PIL, so it's RGB. Did you get 0.18 with finetuning or straight out of the box?
", "votes": null, "replies": [] }, { "id": 372242, "author_name": "rupraze", "author_url": "", "post_date": "08/18/2018 18:08:07", "content": "straight out of the box
", "votes": null, "replies": [] }, { "id": 374938, "author_name": "muhammedazamkhan", "author_url": "", "post_date": "08/24/2018 08:26:20", "content": "Hi @rupraze,\nHave you processed single image/batch of images during inference? I am still stuck at low scored ~0.04.
", "votes": null, "replies": [] }, { "id": 374957, "author_name": "rupraze", "author_url": "", "post_date": "08/24/2018 08:56:38", "content": "Single image during inference
", "votes": null, "replies": [] } ] }, { "id": 372078, "author_name": "muhammedazamkhan", "author_url": "", "post_date": "08/18/2018 08:52:27", "content": "Hi @Master.\nI tried to fine-tune faster_rcnn_inception_resnet_v2_atrous_oid model without any luck. I am just wondering if you can share the process a bit. Thanks in advance.
", "votes": null, "replies": [ { "id": 372110, "author_name": "nzabcd", "author_url": "", "post_date": "08/18/2018 10:49:39", "content": "I just retrain it on this dataset. I'm getting only 0.07 on LB, so it's not really working for me either.
", "votes": null, "replies": [] } ] }, { "id": 373845, "author_name": "muhammedazamkhan", "author_url": "", "post_date": "08/22/2018 02:37:24", "content": "Hello @Master\n> [resolved, bug when creating submission file]\nI am just wondering if you can share what kind of bug it was and how to solve it. I wasted a lot of time to figure out low score issue, but yet to find the hole. Thank you very much.
", "votes": null, "replies": [ { "id": 373899, "author_name": "nzabcd", "author_url": "", "post_date": "08/22/2018 04:16:20", "content": "casting detection classes to np.uint8 when there are 500 classes. np.uint8 has max of 255.
", "votes": null, "replies": [] }, { "id": 374273, "author_name": "muhammedazamkhan", "author_url": "", "post_date": "08/22/2018 18:09:52", "content": "Thanks a lot for this great help.
", "votes": null, "replies": [] }, { "id": 374442, "author_name": "alexanderliao", "author_url": "", "post_date": "08/23/2018 03:42:25", "content": "Thank you very much, that seems to explain all the low scores I've been getting.....
", "votes": null, "replies": [] }, { "id": 374939, "author_name": "muhammedazamkhan", "author_url": "", "post_date": "08/24/2018 08:26:42", "content": "Hi @Master,\nChanging np.uint8 increased the model performance a little only. Is there any more minor change to be made?
", "votes": null, "replies": [] }, { "id": 374953, "author_name": "nzabcd", "author_url": "", "post_date": "08/24/2018 08:51:30", "content": "Train your model for longer and lower the learning rate.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "371657": "tensorflow's model detection zoo has faster_rcnn_inception_resnet_v2_atrous_oid mAP on open images dataset listed as 0.37, but running it here only gives a score of 0.00006.\n\nI know that the test images in this challenge are more densely annotated, but 0.00006? That's like not detecting anything.\n\nAny ideas?\n\n[resolved, bug when creating submission file]", "371660": "It usually has to do with either wrong pre processing (rgb instead of the other thing, can't remember the order of the letters), resizing when there is no need to, grayscale.\nOr... Just a bug in creating the submission...\nBeen there, done that", "371746": "As moshel said you might be doing something wrong. Using the faster_rcnn_inception_resnet_v2_atrous_oid model i could get a LB score of .18. The correct pre-processing of images is RGB instead of OpenCV BGR, you might want to check format of submission, the correct prediction format is `", "372078": "Hi @Master.\nI tried to fine-tune faster_rcnn_inception_resnet_v2_atrous_oid model without any luck. I am just wondering if you can share the process a bit. Thanks in advance.", "372109": "I preprocess with PIL, so it's RGB. Did you get 0.18 with finetuning or straight out of the box?", "372110": "I just retrain it on this dataset. I'm getting only 0.07 on LB, so it's not really working for me either.", "372242": "straight out of the box", "373845": "Hello @Master\n> [resolved, bug when creating submission file]\nI am just wondering if you can share what kind of bug it was and how to solve it. I wasted a lot of time to figure out low score issue, but yet to find the hole. Thank you very much.", "373899": "casting detection classes to np.uint8 when there are 500 classes. np.uint8 has max of 255.", "374273": "Thanks a lot for this great help.", "374442": "Thank you very much, that seems to explain all the low scores I've been getting.....", "374938": "Hi @rupraze,\nHave you processed single image/batch of images during inference? I am still stuck at low scored ~0.04.", "374939": "Hi @Master,\nChanging np.uint8 increased the model performance a little only. Is there any more minor change to be made?", "374953": "Train your model for longer and lower the learning rate.", "374957": "Single image during inference" }, "source": "meta" }