I recently joined this competition, and I've been facing a scoring error for the past two days. I've tried replacing 1 0 with 1 1 and overwriting the predicted data into the sample submission file, but it hasn't resolved the issue. Is there any solution to this problem?
\nand my inference notebook book shows below
\nhttps://www.kaggle.com/code/seeingtimes/sennet-hoa-inference?kernelSessionId=156263252
", "messages": [ { "id": "2572204", "postDate": "12/24/2023 02:15:02", "content": "I recently joined this competition, and I've been facing a scoring error for the past two days. I've tried replacing 1 0 with 1 1 and overwriting the predicted data into the sample submission file, but it hasn't resolved the issue. Is there any solution to this problem?
\nand my inference notebook book shows below
\nhttps://www.kaggle.com/code/seeingtimes/sennet-hoa-inference?kernelSessionId=156263252
", "rawMarkdown": "I recently joined this competition, and I've been facing a scoring error for the past two days. I've tried replacing 1 0 with 1 1 and overwriting the predicted data into the sample submission file, but it hasn't resolved the issue. Is there any solution to this problem?\n\nand my inference notebook book shows below\n\nhttps://www.kaggle.com/code/seeingtimes/sennet-hoa-inference?kernelSessionId=156263252", "votes": null }, { "id": "2572213", "postDate": "12/24/2023 02:37:19", "content": "Hmmm, not sure, but might help:
\n1) use rle implementation provided in one of the public notebooks, as it handles empty-mask case
\n2) I can see that problem may occure if your mask is not the same size as of gt mask [you should resize the mask to match original image before encoding it]
I‘ll try it later ，thanks for your comment！
", "rawMarkdown": "I‘ll try it later ，thanks for your comment！", "votes": null }, { "id": "2572451", "postDate": "12/24/2023 08:54:13", "content": "I can't see the notebook. The link seems to be broken.
\nJust in case, don't assume that slices in the test set start at 0.
I just updated the link 🥲
", "rawMarkdown": "I just updated the link 🥲", "votes": null }, { "id": "2573414", "postDate": "12/25/2023 02:53:12", "content": "update， I solved this problem by resize mask with original size！but I got 0.021🥲
", "rawMarkdown": "update， I solved this problem by resize mask with original size！but I got 0.021🥲", "votes": null }, { "id": "2573484", "postDate": "12/25/2023 05:38:09", "content": "Did you resize it to the original original image shape? (the vanilla image, with no transformation)
", "rawMarkdown": "Did you resize it to the original original image shape? (the vanilla image, with no transformation)", "votes": null }, { "id": "2573609", "postDate": "12/25/2023 07:58:51", "content": "`for _, (images, image_ids,orig_sizes) in pbar:
\n images = images.to(CFG.device, dtype=torch.float32)
\n # pdb.set_trace()
\n with torch.no_grad():
\n masks = model(images)
\n masks = (nn.Sigmoid()(masks) > 0.5).double()
for , image_id, orig_size in (masks, image_ids, orig_sizes):\n #!\n mask = mask.().().(np.uint8)\n\n p = ((mask > ) * ).(np.uint8)\n p = cv2.(p, (orig_size[].(), orig_size[].()), cv2.INTER_NEAREST)\n\n rle_mask = (p,)\n (rle_mask.shape)\n rle = (rle_mask)\n\n rles.(rle)\n ids.(image_id)`\njust like this
", "rawMarkdown": "`for _, (images, image_ids,orig_sizes) in pbar:\n images = images.to(CFG.device, dtype=torch.float32)\n # pdb.set_trace()\n with torch.no_grad():\n masks = model(images)\n masks = (nn.Sigmoid()(masks) > 0.5).double()\n\n for mask, image_id, orig_size in zip(masks, image_ids, orig_sizes):\n #!\n mask = mask.cpu().numpy().astype(np.uint8)\n\n p = ((mask > 0.4) * 255).astype(np.uint8)\n p = cv2.resize(p, (orig_size[1].item(), orig_size[0].item()), cv2.INTER_NEAREST)\n\n rle_mask = remove_small_objects(p,10)\n print(rle_mask.shape)\n rle = rle_encode(rle_mask)\n\n rles.append(rle)\n ids.append(image_id)`\njust like this", "votes": null }, { "id": "2574214", "postDate": "12/25/2023 18:37:00", "content": "I did something similar:
\n`
\nimport numpy as np
\nimport matplotlib.pyplot as plt
\nimport nibabel as nib
\nimport glob
\nimport os
\nfrom PIL import Image
\ndef resize_mask(mask, target_shape):
mask_image = Image()\nresized_mask_image = mask_image(target_shape) \nreturn np(resized_mask_image)\ndef remove_small_objects(mask, min_size=30):
\nstructure = .ones((, ), dtype=int) # -connectivity\nlabeled, ncomponents = ndimage.(mask, structure)\n, counts = .(labeled, return_counts=True)\n\n unique_label, count zip(, counts):\n count < min_size:\n mask[labeled == unique_label] = \n mask`\n`import numpy as np
\nimport matplotlib.pyplot as plt
\nimport nibabel as nib
\nimport glob
\nimport os
\nfrom PIL import Image
\nfrom scipy import ndimage
output_folder = '/kaggle/working/'
\nsubmission = []
for p_img in(ls_images):
\n path_ = p_img.split(os.path.sep)
\n dataset = path_[-3]
\n slice_id, _ = os.path.splitext(path_[-1])
\n img = plt.imread(p_img)
\n img_shape = img.shape[:2]
mask_path = sorted(glob.glob(os.path.join(output_folder, slice_id, '*.nii.gz')))[0]\nmask_nii = nib.load(mask_path)\nmask = mask_nii.get_fdata()\n\n\nresized_mask = resize_mask(mask, img_shape)\nbinary_mask = resized_mask > 0.5 \ncleaned_mask = remove_small_objects(binary_mask)\n\n\nsubmission.append({\n : f,\n : rle_encode(cleaned_mask)\n})`\nbut suffered the same error
", "rawMarkdown": "I did something similar:\n\n`\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport nibabel as nib\nimport glob\nimport os\nfrom PIL import Image\ndef resize_mask(mask, target_shape):\n \n mask_image = Image.fromarray(mask)\n resized_mask_image = mask_image.resize(target_shape[::-1]) \n return np.array(resized_mask_image)\n\ndef remove_small_objects(mask, min_size=30):\n \n structure = np.ones((3, 3), dtype=int) # 8-connectivity\n labeled, ncomponents = ndimage.label(mask, structure)\n unique, counts = np.unique(labeled, return_counts=True)\n \n for unique_label, count in zip(unique, counts):\n if count < min_size:\n mask[labeled == unique_label] = 0\n return mask`\n\n`import numpy as np\nimport matplotlib.pyplot as plt\nimport nibabel as nib\nimport glob\nimport os\nfrom PIL import Image\nfrom scipy import ndimage\n\n\noutput_folder = '/kaggle/working/'\nsubmission = []\n\nfor p_img in(ls_images):\n path_ = p_img.split(os.path.sep)\n dataset = path_[-3]\n slice_id, _ = os.path.splitext(path_[-1])\n img = plt.imread(p_img)\n img_shape = img.shape[:2]\n\n mask_path = sorted(glob.glob(os.path.join(output_folder, slice_id, '*.nii.gz')))[0]\n mask_nii = nib.load(mask_path)\n mask = mask_nii.get_fdata()\n\n # Resize and adjust the mask\n resized_mask = resize_mask(mask, img_shape)\n binary_mask = resized_mask > 0.5 # Convert to binary if necessary\n cleaned_mask = remove_small_objects(binary_mask)\n\n # Submission entry\n submission.append({\n \"id\": f\"{dataset}_{slice_id}\",\n \"rle\": rle_encode(cleaned_mask)\n })`\n\nbut suffered the same error", "votes": null }, { "id": "2574435", "postDate": "12/25/2023 23:54:01", "content": "I replaced a new model it appeared again😭
", "rawMarkdown": "I replaced a new model it appeared again😭", "votes": null }, { "id": "2574459", "postDate": "12/26/2023 01:20:01", "content": "Yeah I can't seem to shake this error off
", "rawMarkdown": "Yeah I can't seem to shake this error off", "votes": null } ], "comments": [ { "id": 2572213, "author_name": "martynoveduard", "author_url": "", "post_date": "12/24/2023 02:37:19", "content": "Hmmm, not sure, but might help:
\n1) use rle implementation provided in one of the public notebooks, as it handles empty-mask case
\n2) I can see that problem may occure if your mask is not the same size as of gt mask [you should resize the mask to match original image before encoding it]
I‘ll try it later ，thanks for your comment！
", "votes": null, "replies": [] }, { "id": 2573414, "author_name": "seeingtimes", "author_url": "", "post_date": "12/25/2023 02:53:12", "content": "update， I solved this problem by resize mask with original size！but I got 0.021🥲
", "votes": null, "replies": [ { "id": 2573484, "author_name": "zuni8789798", "author_url": "", "post_date": "12/25/2023 05:38:09", "content": "Did you resize it to the original original image shape? (the vanilla image, with no transformation)
", "votes": null, "replies": [ { "id": 2573609, "author_name": "seeingtimes", "author_url": "", "post_date": "12/25/2023 07:58:51", "content": "`for _, (images, image_ids,orig_sizes) in pbar:
\n images = images.to(CFG.device, dtype=torch.float32)
\n # pdb.set_trace()
\n with torch.no_grad():
\n masks = model(images)
\n masks = (nn.Sigmoid()(masks) > 0.5).double()
for , image_id, orig_size in (masks, image_ids, orig_sizes):\n #!\n mask = mask.().().(np.uint8)\n\n p = ((mask > ) * ).(np.uint8)\n p = cv2.(p, (orig_size[].(), orig_size[].()), cv2.INTER_NEAREST)\n\n rle_mask = (p,)\n (rle_mask.shape)\n rle = (rle_mask)\n\n rles.(rle)\n ids.(image_id)`\njust like this
", "votes": null, "replies": [ { "id": 2574214, "author_name": "zuni8789798", "author_url": "", "post_date": "12/25/2023 18:37:00", "content": "I did something similar:
\n`
\nimport numpy as np
\nimport matplotlib.pyplot as plt
\nimport nibabel as nib
\nimport glob
\nimport os
\nfrom PIL import Image
\ndef resize_mask(mask, target_shape):
mask_image = Image()\nresized_mask_image = mask_image(target_shape) \nreturn np(resized_mask_image)\ndef remove_small_objects(mask, min_size=30):
\nstructure = .ones((, ), dtype=int) # -connectivity\nlabeled, ncomponents = ndimage.(mask, structure)\n, counts = .(labeled, return_counts=True)\n\n unique_label, count zip(, counts):\n count < min_size:\n mask[labeled == unique_label] = \n mask`\n`import numpy as np
\nimport matplotlib.pyplot as plt
\nimport nibabel as nib
\nimport glob
\nimport os
\nfrom PIL import Image
\nfrom scipy import ndimage
output_folder = '/kaggle/working/'
\nsubmission = []
for p_img in(ls_images):
\n path_ = p_img.split(os.path.sep)
\n dataset = path_[-3]
\n slice_id, _ = os.path.splitext(path_[-1])
\n img = plt.imread(p_img)
\n img_shape = img.shape[:2]
mask_path = sorted(glob.glob(os.path.join(output_folder, slice_id, '*.nii.gz')))[0]\nmask_nii = nib.load(mask_path)\nmask = mask_nii.get_fdata()\n\n\nresized_mask = resize_mask(mask, img_shape)\nbinary_mask = resized_mask > 0.5 \ncleaned_mask = remove_small_objects(binary_mask)\n\n\nsubmission.append({\n : f,\n : rle_encode(cleaned_mask)\n})`\nbut suffered the same error
", "votes": null, "replies": [ { "id": 2574435, "author_name": "seeingtimes", "author_url": "", "post_date": "12/25/2023 23:54:01", "content": "I replaced a new model it appeared again😭
", "votes": null, "replies": [ { "id": 2574459, "author_name": "zuni8789798", "author_url": "", "post_date": "12/26/2023 01:20:01", "content": "Yeah I can't seem to shake this error off
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] }, { "id": 2572451, "author_name": "sakvaua", "author_url": "", "post_date": "12/24/2023 08:54:13", "content": "I can't see the notebook. The link seems to be broken.
\nJust in case, don't assume that slices in the test set start at 0.
I just updated the link 🥲
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2572204": "I recently joined this competition, and I've been facing a scoring error for the past two days. I've tried replacing 1 0 with 1 1 and overwriting the predicted data into the sample submission file, but it hasn't resolved the issue. Is there any solution to this problem?\n\nand my inference notebook book shows below\n\nhttps://www.kaggle.com/code/seeingtimes/sennet-hoa-inference?kernelSessionId=156263252", "2572213": "Hmmm, not sure, but might help:\n1) use [rle implementation](https://www.kaggle.com/code/hengck23/lb0-808-resnet50-2d-unet-xy-zy-zx-cc3d) provided in one of the public notebooks, as it handles empty-mask case \n2) I can see that problem may occure if your mask is not the same size as of gt mask [you should resize the mask to match original image before encoding it]", "2572224": "I‘ll try it later ，thanks for your comment！", "2572451": "I can't see the notebook. The link seems to be broken.\nJust in case, don't assume that slices in the test set start at 0.", "2572486": "I just updated the link 🥲", "2573414": "update， I solved this problem by resize mask with original size！but I got 0.021🥲", "2573484": "Did you resize it to the original original image shape? (the vanilla image, with no transformation)", "2573609": "`for _, (images, image_ids,orig_sizes) in pbar:\n images = images.to(CFG.device, dtype=torch.float32)\n # pdb.set_trace()\n with torch.no_grad():\n masks = model(images)\n masks = (nn.Sigmoid()(masks) > 0.5).double()\n\n for mask, image_id, orig_size in zip(masks, image_ids, orig_sizes):\n #!\n mask = mask.cpu().numpy().astype(np.uint8)\n\n p = ((mask > 0.4) * 255).astype(np.uint8)\n p = cv2.resize(p, (orig_size[1].item(), orig_size[0].item()), cv2.INTER_NEAREST)\n\n rle_mask = remove_small_objects(p,10)\n print(rle_mask.shape)\n rle = rle_encode(rle_mask)\n\n rles.append(rle)\n ids.append(image_id)`\njust like this", "2574214": "I did something similar:\n\n`\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport nibabel as nib\nimport glob\nimport os\nfrom PIL import Image\ndef resize_mask(mask, target_shape):\n \n mask_image = Image.fromarray(mask)\n resized_mask_image = mask_image.resize(target_shape[::-1]) \n return np.array(resized_mask_image)\n\ndef remove_small_objects(mask, min_size=30):\n \n structure = np.ones((3, 3), dtype=int) # 8-connectivity\n labeled, ncomponents = ndimage.label(mask, structure)\n unique, counts = np.unique(labeled, return_counts=True)\n \n for unique_label, count in zip(unique, counts):\n if count < min_size:\n mask[labeled == unique_label] = 0\n return mask`\n\n`import numpy as np\nimport matplotlib.pyplot as plt\nimport nibabel as nib\nimport glob\nimport os\nfrom PIL import Image\nfrom scipy import ndimage\n\n\noutput_folder = '/kaggle/working/'\nsubmission = []\n\nfor p_img in(ls_images):\n path_ = p_img.split(os.path.sep)\n dataset = path_[-3]\n slice_id, _ = os.path.splitext(path_[-1])\n img = plt.imread(p_img)\n img_shape = img.shape[:2]\n\n mask_path = sorted(glob.glob(os.path.join(output_folder, slice_id, '*.nii.gz')))[0]\n mask_nii = nib.load(mask_path)\n mask = mask_nii.get_fdata()\n\n # Resize and adjust the mask\n resized_mask = resize_mask(mask, img_shape)\n binary_mask = resized_mask > 0.5 # Convert to binary if necessary\n cleaned_mask = remove_small_objects(binary_mask)\n\n # Submission entry\n submission.append({\n \"id\": f\"{dataset}_{slice_id}\",\n \"rle\": rle_encode(cleaned_mask)\n })`\n\nbut suffered the same error", "2574435": "I replaced a new model it appeared again😭", "2574459": "Yeah I can't seem to shake this error off" }, "source": "meta" }