![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3233311%2F27f94cc41152abd20607921fd6586b06%2F2de5778a61a87f01.jpg?generation=1567004844934990&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3233311%2F01f517c5ab8f893586a89ec81386a2b7%2F74362934ae874b7d.jpg?generation=1567004893100008&alt=media\")
Hey guys,
\n\nI visualized the ground truth for the following images (see attachments):\n\"Openimages/train_06/2de5778a61a87f01.jpg'' and \"Openimages/train_04/74362934ae874b7d.jpg\". Can somebody confirm that the ground truth masks are actually not so accurate or did I make a mistake?
\n\nThanks in advance.\nBest,\nWouter
", "messages": [ { "id": "608551", "postDate": "08/26/2019 23:27:49", "content": "Hey guys,
\n\nI visualized the ground truth for the following images (see attachments):\n\"Openimages/train_06/2de5778a61a87f01.jpg'' and \"Openimages/train_04/74362934ae874b7d.jpg\". Can somebody confirm that the ground truth masks are actually not so accurate or did I make a mistake?
\n\nThanks in advance.\nBest,\nWouter
", "rawMarkdown": "Hey guys,\n\nI visualized the ground truth for the following images (see attachments):\n\"Openimages/train_06/2de5778a61a87f01.jpg'' and \"Openimages/train_04/74362934ae874b7d.jpg\". Can somebody confirm that the ground truth masks are actually not so accurate or did I make a mistake?\n\nThanks in advance.\nBest,\nWouter", "votes": null }, { "id": "610236", "postDate": "08/28/2019 15:10:09", "content": "Here are some visualisations obtained with a script of mine, which confirm your observation.\n(Beware that if there are multiple masks covering the same pixels, one at random would be overwritten)\nTo be fair, 2de5778a61a87f01 seems really tough...\n\n
Those examples look about right.\nYou can inspect more annotation examples at http://g.co/dataset/open-images -> \"Explore\" .\nMake sure \"Type: Segmentation\", and notice you can switch the \"Subset\" from train to validation+test.\nNote that for the train set, the visualizer shows \"the top 500 images ranked by estimated quality\". The quality is automatically estimated as explained in section 4.2 of the paper.
The annotation process is explained [here] (https://storage.googleapis.com/openimages/web/factsfigures.html)
\n\n\n\n\nThe segmentation masks on the training set have been produced by a state-of-the-art interactive segmentation process [4], where professional human annotators iteratively correct the output of a segmentation neural network.\n [...]\n For the validation and test splits we created 99k masks spread over 54k images. These have been annotated with a purely manual free-painting tool and with a strong focus on quality. They are near-perfect (self-consistency 90% mIoU [4]) and capture even fine details of complex object boundaries (e.g. spiky flowers and thin structures in man-made objects).
\n
The annotations used for the Kaggle challenge test set are of the same type as val+test (fully manual).
In the supplementary material of the paper, we provide more details (Fig. 11) regarding the fraction of training samples that result in \"not so great\" quality. In general most of them are good (may I so myself), and a few are bad or even possibly very bad.
\n\nNote that the download files include the PredictedIoU value (same used for ranking in visualizer). This can be used to weight the training samples based on the algorithmically predicted quality, or to prune subsets of the training data.
Thanks for the quick reponse.
", "rawMarkdown": "Thanks for the quick reponse.", "votes": null } ], "comments": [ { "id": 610236, "author_name": "lntsmn", "author_url": "", "post_date": "08/28/2019 15:10:09", "content": "Here are some visualisations obtained with a script of mine, which confirm your observation.\n(Beware that if there are multiple masks covering the same pixels, one at random would be overwritten)\nTo be fair, 2de5778a61a87f01 seems really tough...\n\n
Those examples look about right.\nYou can inspect more annotation examples at http://g.co/dataset/open-images -> \"Explore\" .\nMake sure \"Type: Segmentation\", and notice you can switch the \"Subset\" from train to validation+test.\nNote that for the train set, the visualizer shows \"the top 500 images ranked by estimated quality\". The quality is automatically estimated as explained in section 4.2 of the paper.
The annotation process is explained [here] (https://storage.googleapis.com/openimages/web/factsfigures.html)
\n\n\n\n\nThe segmentation masks on the training set have been produced by a state-of-the-art interactive segmentation process [4], where professional human annotators iteratively correct the output of a segmentation neural network.\n [...]\n For the validation and test splits we created 99k masks spread over 54k images. These have been annotated with a purely manual free-painting tool and with a strong focus on quality. They are near-perfect (self-consistency 90% mIoU [4]) and capture even fine details of complex object boundaries (e.g. spiky flowers and thin structures in man-made objects).
\n
The annotations used for the Kaggle challenge test set are of the same type as val+test (fully manual).
In the supplementary material of the paper, we provide more details (Fig. 11) regarding the fraction of training samples that result in \"not so great\" quality. In general most of them are good (may I so myself), and a few are bad or even possibly very bad.
\n\nNote that the download files include the PredictedIoU value (same used for ranking in visualizer). This can be used to weight the training samples based on the algorithmically predicted quality, or to prune subsets of the training data.
Thanks for the quick reponse.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "608551": "Hey guys,\n\nI visualized the ground truth for the following images (see attachments):\n\"Openimages/train_06/2de5778a61a87f01.jpg'' and \"Openimages/train_04/74362934ae874b7d.jpg\". Can somebody confirm that the ground truth masks are actually not so accurate or did I make a mistake?\n\nThanks in advance.\nBest,\nWouter", "610236": "Here are some visualisations obtained with a script of mine, which confirm your observation.\n(Beware that if there are multiple masks covering the same pixels, one at random would be overwritten)\nTo be fair, 2de5778a61a87f01 seems really tough...\n\n", "611776": "Those examples look about right.\nYou can inspect more annotation examples at http://g.co/dataset/open-images -> \"Explore\" .\nMake sure \"Type: Segmentation\", and notice you can switch the \"Subset\" from `train` to `validation+test`.\nNote that for the train set, the visualizer shows \"the top 500 images ranked by estimated quality\". The quality is automatically estimated as explained in section 4.2 of the [paper](https://arxiv.org/pdf/1903.10830.pdf).\n\nThe annotation process is explained [here] (https://storage.googleapis.com/openimages/web/factsfigures.html)\n \n> The segmentation masks on the training set have been produced by a state-of-the-art interactive segmentation process [[4]](https://arxiv.org/pdf/1903.10830.pdf), where professional human annotators iteratively correct the output of a segmentation neural network.\n[...]\n> For the validation and test splits we created 99k masks spread over 54k images. These have been annotated with a purely manual free-painting tool and with a strong focus on quality. They are near-perfect (self-consistency 90% mIoU [[4]](https://arxiv.org/pdf/1903.10830.pdf)) and capture even fine details of complex object boundaries (e.g. spiky flowers and thin structures in man-made objects).\n\nThe annotations used for the Kaggle challenge test set are of the same type as `val+test` (fully manual).\n\nIn the [supplementary material](http://openaccess.thecvf.com/content_CVPR_2019/supplemental/Benenson_Large-Scale_Interactive_Object_CVPR_2019_supplemental.pdf) of the paper, we provide more details (Fig. 11) regarding the fraction of training samples that result in \"not so great\" quality. In general most of them are good (may I so myself), and a few are bad or even possibly very bad.\n\nNote that the [download files](https://storage.googleapis.com/openimages/web/download.html) include the `PredictedIoU` value (same used for ranking in visualizer). This can be used to weight the training samples based on the algorithmically predicted quality, or to prune subsets of the training data.", "614386": "Thanks for the quick reponse." }, "source": "meta" }