![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6498106%2F2e3736737aaff737e0354dea9808c03f%2FScreenshot%20from%202025-10-29%2012-17-32.png?generation=1761736681041566&alt=media\")
Hi everyone,
\nWhile exploring the dataset, I noticed a potential labeling issue. I computed a mask by comparing the authentic and forged images (pixels that differ). This computed mask should match the ground truth mask in terms of the number and location of forged regions.
\nHowever, in some examples I checked, the results do not align:
\n
The computed mask identifies one red object, while the ground truth mask shows three. However, only two should be present — the red object in the middle should not be labeled.
\nSince the ground truth is expected to represent all forged regions and copied objects, this inconsistency suggests that some annotations may be incorrect or incomplete.
\nCould the organizers confirm whether this labeling issue will be corrected? Additionally, if similar errors exist in the test set, they could impact evaluation fairness — is there a plan to mitigate this risk?
\nThanks!
", "messages": [ { "id": 3308254, "postDate": "2025-10-29T00:53:59.833Z", "content": "Hi everyone,
\nWhile exploring the dataset, I noticed a potential labeling issue. I computed a mask by comparing the authentic and forged images (pixels that differ). This computed mask should match the ground truth mask in terms of the number and location of forged regions.
\nHowever, in some examples I checked, the results do not align:
\n
The computed mask identifies one red object, while the ground truth mask shows three. However, only two should be present — the red object in the middle should not be labeled.
\nSince the ground truth is expected to represent all forged regions and copied objects, this inconsistency suggests that some annotations may be incorrect or incomplete.
\nCould the organizers confirm whether this labeling issue will be corrected? Additionally, if similar errors exist in the test set, they could impact evaluation fairness — is there a plan to mitigate this risk?
\nThanks!
", "rawMarkdown": "Hi everyone,\n\nWhile exploring the dataset, I noticed a potential labeling issue. I computed a mask by comparing the authentic and forged images (pixels that differ). This computed mask should match the ground truth mask in terms of the number and location of forged regions.\n\nHowever, in some examples I checked, the results do not align:\n\n\n\nThe computed mask identifies one red object, while the ground truth mask shows three. However, only two should be present — the red object in the middle should not be labeled.\n\nSince the ground truth is expected to represent all forged regions and copied objects, this inconsistency suggests that some annotations may be incorrect or incomplete.\n\nCould the organizers confirm whether this labeling issue will be corrected? Additionally, if similar errors exist in the test set, they could impact evaluation fairness — is there a plan to mitigate this risk?\n\nThanks!\n", "votes": 11 }, { "id": 3308430, "postDate": "2025-10-29T12:08:54.883Z", "content": "Thanks for pointing this out!
\nI can confirm that this is an extremely rare case in the training set, and it does not occur in the test set. The test set was annotated from actual forgeries found in research papers, while the training set was created by our team.
\nIn the sample you highlighted, the red areas from the two top cells (left and middle) were copied together as a single object. After being rotated, this new piece was pasted to create the forgery on the right. The red cell on the right only contains a partial view of the combined elements because the pasted piece was placed partially outside the image boundary.
\nThanks again for the catch! I hope that clarification helps.
", "rawMarkdown": "Thanks for pointing this out!\n\nI can confirm that this is an extremely rare case in the training set, and it does not occur in the test set. The test set was annotated from actual forgeries found in research papers, while the training set was created by our team.\n\nIn the sample you highlighted, the red areas from the two top cells (left and middle) were copied together as a single object. After being rotated, this new piece was pasted to create the forgery on the right. The red cell on the right only contains a partial view of the combined elements because the pasted piece was placed partially outside the image boundary.\n\nThanks again for the catch! I hope that clarification helps.", "votes": 3, "replies": [ { "id": 3311595, "postDate": "2025-11-05T11:23:59.193Z", "content": "mdam… that's what I thought. I've already submitted 70+ submissions- I've used R-CNN, YOLO, DeepLab, and all the models don't predict even at high conf. This means that test does not compete with train at all.
", "rawMarkdown": "mdam... that's what I thought. I've already submitted 70+ submissions- I've used R-CNN, YOLO, DeepLab, and all the models don't predict even at high conf. This means that test does not compete with train at all." } ] }, { "id": 3308345, "postDate": "2025-10-29T06:58:02.990Z", "content": "Did you notice that mask.shape is (2, 712, 414) for image 10070 ?
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F214989%2F5d50533d330783150061e9829e0f11e2%2Fa.png?generation=1761721063184691&alt=media\")
Thanks, I updated the visualization for the 2-channel mask. The annotation issue remains though , the GT shows three red regions, but only two are valid (the middle one shouldn’t be labeled).
", "rawMarkdown": "Thanks, I updated the visualization for the 2-channel mask. The annotation issue remains though , the GT shows three red regions, but only two are valid (the middle one shouldn’t be labeled)." } ] }, { "id": 3308333, "postDate": "2025-10-29T05:59:10.370Z", "content": "This is a good catch thanks for raising this
", "rawMarkdown": "This is a good catch thanks for raising this\n" }, { "id": 3311410, "postDate": "2025-11-05T00:31:30.850Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 3308430, "author_name": "João Phillipe Cardenuto", "author_url": "", "post_date": "2025-10-29T12:08:54.883000", "content": "Thanks for pointing this out!
\nI can confirm that this is an extremely rare case in the training set, and it does not occur in the test set. The test set was annotated from actual forgeries found in research papers, while the training set was created by our team.
\nIn the sample you highlighted, the red areas from the two top cells (left and middle) were copied together as a single object. After being rotated, this new piece was pasted to create the forgery on the right. The red cell on the right only contains a partial view of the combined elements because the pasted piece was placed partially outside the image boundary.
\nThanks again for the catch! I hope that clarification helps.
", "votes": 3, "replies": [ { "id": 3311595, "author_name": "Antonoof", "author_url": "", "post_date": "2025-11-05T11:23:59.193000", "content": "mdam… that's what I thought. I've already submitted 70+ submissions- I've used R-CNN, YOLO, DeepLab, and all the models don't predict even at high conf. This means that test does not compete with train at all.
", "votes": 0, "replies": [] } ] }, { "id": 3308345, "author_name": "steubk", "author_url": "", "post_date": "2025-10-29T06:58:02.990000", "content": "Did you notice that mask.shape is (2, 712, 414) for image 10070 ?
\n
Thanks, I updated the visualization for the 2-channel mask. The annotation issue remains though , the GT shows three red regions, but only two are valid (the middle one shouldn’t be labeled).
", "votes": 0, "replies": [] } ] }, { "id": 3308333, "author_name": "Manel ALOUI", "author_url": "", "post_date": "2025-10-29T05:59:10.370000", "content": "This is a good catch thanks for raising this
", "votes": 0, "replies": [] }, { "id": 3311410, "author_name": "", "author_url": "", "post_date": "2025-11-05T00:31:30.850000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3308254": "Hi everyone,\n\nWhile exploring the dataset, I noticed a potential labeling issue. I computed a mask by comparing the authentic and forged images (pixels that differ). This computed mask should match the ground truth mask in terms of the number and location of forged regions.\n\nHowever, in some examples I checked, the results do not align:\n\n\n\nThe computed mask identifies one red object, while the ground truth mask shows three. However, only two should be present — the red object in the middle should not be labeled.\n\nSince the ground truth is expected to represent all forged regions and copied objects, this inconsistency suggests that some annotations may be incorrect or incomplete.\n\nCould the organizers confirm whether this labeling issue will be corrected? Additionally, if similar errors exist in the test set, they could impact evaluation fairness — is there a plan to mitigate this risk?\n\nThanks!\n", "3308430": "Thanks for pointing this out!\n\nI can confirm that this is an extremely rare case in the training set, and it does not occur in the test set. The test set was annotated from actual forgeries found in research papers, while the training set was created by our team.\n\nIn the sample you highlighted, the red areas from the two top cells (left and middle) were copied together as a single object. After being rotated, this new piece was pasted to create the forgery on the right. The red cell on the right only contains a partial view of the combined elements because the pasted piece was placed partially outside the image boundary.\n\nThanks again for the catch! I hope that clarification helps.", "3308345": "Did you notice that mask.shape is (2, 712, 414) for image 10070 ?\n\n", "3308333": "This is a good catch thanks for raising this\n", "3311410": "" } }