![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F651363%2F766bb0efb9b45644276e0a9488d1e604%2Fsample_mask.png?generation=1589396758634812&alt=media\")
Just submitted the csv for this competition and got 0.0 score.\nWhen I run the model on my local notebook and look at the visualizations the look fine and very convincing. Might not go to the top of the leaderboard but certainly no way they should get a zero score. \nThe issue is now I don't know what I am doing wrong? \nSample image masks:\n
Some possibilities:-\n1. Do the classIds in evaluation start from 0 or 1? My predictions are from 0 to 45, if all predictions are shifted by one then this might explain the zero score.\n2. I have used cv2 instead of PIL to resize images, does that matter?
\n\nAny ideas on the why I might be getting a zero score would be great!
\n\nTo competition host: Please give some more clear instructions on submission and make submission as less complicated as possible. It's really disheartening to make something that works and get a low score because your submission format is not right.
", "messages": [ { "id": "846444", "postDate": "05/13/2020 19:06:35", "content": "Just submitted the csv for this competition and got 0.0 score.\nWhen I run the model on my local notebook and look at the visualizations the look fine and very convincing. Might not go to the top of the leaderboard but certainly no way they should get a zero score. \nThe issue is now I don't know what I am doing wrong? \nSample image masks:\n
Some possibilities:-\n1. Do the classIds in evaluation start from 0 or 1? My predictions are from 0 to 45, if all predictions are shifted by one then this might explain the zero score.\n2. I have used cv2 instead of PIL to resize images, does that matter?
\n\nAny ideas on the why I might be getting a zero score would be great!
\n\nTo competition host: Please give some more clear instructions on submission and make submission as less complicated as possible. It's really disheartening to make something that works and get a low score because your submission format is not right.
", "rawMarkdown": "Just submitted the csv for this competition and got 0.0 score.\nWhen I run the model on my local notebook and look at the visualizations the look fine and very convincing. Might not go to the top of the leaderboard but certainly no way they should get a zero score. \nThe issue is now I don't know what I am doing wrong? \nSample image masks:\n\n\n\nSome possibilities:-\n1. Do the classIds in evaluation start from 0 or 1? My predictions are from 0 to 45, if all predictions are shifted by one then this might explain the zero score.\n2. I have used cv2 instead of PIL to resize images, does that matter?\n\n\nAny ideas on the why I might be getting a zero score would be great!\n\nTo competition host: Please give some more clear instructions on submission and make submission as less complicated as possible. It's really disheartening to make something that works and get a low score because your submission format is not right.", "votes": null }, { "id": "846750", "postDate": "05/14/2020 01:41:24", "content": "my score is zero too, is your classification right? Is same as the train.csv?
", "rawMarkdown": "my score is zero too, is your classification right? Is same as the train.csv?", "votes": null }, { "id": "846756", "postDate": "05/14/2020 01:54:46", "content": "Yes, I checked if ids start from 0 to 45 I looked up the results and they are correct. Yes I am using train.csv.\nDo you know how I can contact the host of this competition to get some clarification?
", "rawMarkdown": "Yes, I checked if ids start from 0 to 45 I looked up the results and they are correct. Yes I am using train.csv.\nDo you know how I can contact the host of this competition to get some clarification?", "votes": null }, { "id": "846787", "postDate": "05/14/2020 02:07:33", "content": "", "rawMarkdown": "makeitworkjml", "votes": null }, { "id": "847090", "postDate": "05/14/2020 06:56:56", "content": "no , i @ her too, but no work.
", "rawMarkdown": "no , i @ her too, but no work.", "votes": null }, { "id": "847118", "postDate": "05/14/2020 07:25:18", "content": "hi, does this example's attributes match ground-truth? F1 should be bigger than 0.5 .
", "rawMarkdown": "hi, does this example's attributes match ground-truth? F1 should be bigger than 0.5 .", "votes": null }, { "id": "847391", "postDate": "05/14/2020 11:22:55", "content": "I got a zero score once as well, but I simply forgot to convert 1-indexed class IDs to 0-indexed, but I see that's not your case.
\n\nIt doesn't matter which library you're using for image resizing, but you need to make sure that:\n1. You resize an image to fit into the square 1024x1024 and keep image aspect ratio\n2. You not just downscale large images, but also upscale small images to the same size
\n\nThen I see 2 potential cases how you could get a zero score:\n1. RLE encoding. What code do you use to do encoding? Did you try to re-encode some masks from the training dataset to make sure it works correctly?\n2. Attribute IDs. Do you predict attribute IDs for the segments? If not, you'll get zero scores for the segments with classes 0-12, even if the segmentation is 100% accurate. So if most of your predictions are classes 0-12 that would explain your final zero score. Check the distribution of the classes that you have in your predictions.
", "rawMarkdown": "I got a zero score once as well, but I simply forgot to convert 1-indexed class IDs to 0-indexed, but I see that's not your case.\n\nIt doesn't matter which library you're using for image resizing, but you need to make sure that:\n1. You resize an image to fit into the square 1024x1024 and keep image aspect ratio\n2. You not just downscale large images, but also upscale small images to the same size\n\nThen I see 2 potential cases how you could get a zero score:\n1. **RLE encoding**. What code do you use to do encoding? Did you try to re-encode some masks from the training dataset to make sure it works correctly?\n2. **Attribute IDs**. Do you predict attribute IDs for the segments? If not, you'll get zero scores for the segments with classes 0-12, even if the segmentation is 100% accurate. So if most of your predictions are classes 0-12 that would explain your final zero score. Check the distribution of the classes that you have in your predictions.", "votes": null }, { "id": "847931", "postDate": "05/14/2020 17:00:41", "content": "Hi @polosin ,
\n\nAre you sure I should scale all of them to 1024x1024?
\n\nI checked the rle encoding and decoding and decoding gives the masks back correctly.
I haven't checked the distribution of classes that I have predicted so I will definitely have look into that too. You mean to say that for segments with \"ClassIds\" 0-12 if I get a Nan for attribute Ids (and ground truth is not Nan) I will get a 0 even if the classIds and masks are correct?\nAlso the predictions for classIds should be from 0-45 right? Not 1-46?
Thanks again! Will give you an update on how it goes!
", "rawMarkdown": "Hi @polosin ,\n\n1. Thanks for the suggestions. I scaled all the images to 1024,1024 but that gives this error-\n4 Exceptions:\nOne or more submitted values exceeded the image dimensions.\nOne or more submitted values exceeded the image dimensions.\nOne or more submitted values exceeded the image dimensions.\nOne or more submitted values exceeded the image dimensions.\n\nAre you sure I should scale all of them to 1024x1024?\n\n2. I checked the rle encoding and decoding and decoding gives the masks back correctly.\n\n3. I haven't checked the distribution of classes that I have predicted so I will definitely have look into that too. You mean to say that for segments with \"ClassIds\" 0-12 if I get a Nan for attribute Ids (and ground truth is not Nan) I will get a 0 even if the classIds and masks are correct?\nAlso the predictions for classIds should be from 0-45 right? Not 1-46?\n\nThanks again! Will give you an update on how it goes!", "votes": null }, { "id": "847936", "postDate": "05/14/2020 17:04:05", "content": "Hi @fanhaobei \nYa the ClassIds match the ground truth as in for this image it predicted says that classes dress, sleeve (2) and neckline exist in the image. It return NaN for the attributeIds though.
", "rawMarkdown": "Hi @fanhaobei \nYa the ClassIds match the ground truth as in for this image it predicted says that classes dress, sleeve (2) and neckline exist in the image. It return NaN for the attributeIds though.", "votes": null }, { "id": "848268", "postDate": "05/14/2020 20:38:49", "content": "\n\n\nI scaled all the images to 1024,1024 but that gives this error-
\n
I experienced this exception before as well. They didn't mention it on the Evaluation page, but when you resize an image you should floor the value of the smaller side, not round it. For example, if your smaller side of the image is 736.83, you should resize it to 736, not 737.
\n\n\n\n\nAre you sure I should scale all of them to 1024x1024?
\n
Yes, I checked both variants
\n\n\n\n\nYou mean to say that for segments with \"ClassIds\" 0-12 if I get a Nan for attribute Ids (and ground truth is not Nan) I will get a 0 even if the classIds and masks are correct?
\n
Yes, you can read more about the metric on the Evaluation page
\n\n\n\n\nAlso the predictions for classIds should be from 0-45 right? Not 1-46?
\n
Yes, exactly as it's in the training data
", "rawMarkdown": "> I scaled all the images to 1024,1024 but that gives this error-\n\nI experienced this exception before as well. They didn't mention it on the Evaluation page, but when you resize an image you should floor the value of the smaller side, not round it. For example, if your smaller side of the image is 736.83, you should resize it to 736, not 737.\n\n> Are you sure I should scale all of them to 1024x1024?\n\nYes, I checked both variants\n\n> You mean to say that for segments with \"ClassIds\" 0-12 if I get a Nan for attribute Ids (and ground truth is not Nan) I will get a 0 even if the classIds and masks are correct?\n\nYes, you can read more about the metric on the [Evaluation page](https://www.kaggle.com/c/imaterialist-fashion-2020-fgvc7/overview/evaluation)\n\n> Also the predictions for classIds should be from 0-45 right? Not 1-46?\n\nYes, exactly as it's in the training data", "votes": null }, { "id": "848275", "postDate": "05/14/2020 20:49:38", "content": "I might be understanding this wrong but how can the height or width of an image be a floating point number? It has to be an integer right?\n\"... .For example, if your smaller side of the image is 736.83, you should resize it to 736, not 737\"
", "rawMarkdown": "I might be understanding this wrong but how can the height or width of an image be a floating point number? It has to be an integer right?\n\"... .For example, if your smaller side of the image is **736.83**, you should resize it to 736, not 737\"", "votes": null }, { "id": "848295", "postDate": "05/14/2020 21:16:55", "content": "For example, you have an image of size 1067x1600, you need to resize to 1024 for the longest side. So the new size before converting it to an integer is 682.88 x 1024. You should floor 682.88 to 682, not round it to 683.
", "rawMarkdown": "For example, you have an image of size 1067x1600, you need to resize to 1024 for the longest side. So the new size before converting it to an integer is 682.88 x 1024. You should floor 682.88 to 682, not round it to 683.", "votes": null }, { "id": "848412", "postDate": "05/15/2020 00:29:49", "content": "Oh okay got it, makes sense
", "rawMarkdown": "Oh okay got it, makes sense", "votes": null }, { "id": "849294", "postDate": "05/15/2020 16:17:54", "content": "@polosin \nJust another clarification.\nSay in an image if the classId prediction is [10 31 31 33] then in the out there should two separate run length encodings for both the 31 classes right? \nOr is it okay if we combine the RLE for both 31 predictions?
", "rawMarkdown": "polosin \nJust another clarification.\nSay in an image if the classId prediction is [10 31 31 33] then in the out there should two separate run length encodings for both the 31 classes right? \nOr is it okay if we combine the RLE for both 31 predictions?", "votes": null }, { "id": "849367", "postDate": "05/15/2020 17:41:24", "content": "Yes, if you found several instances of the same class on the image - you should submit them as separate RLEs. Otherwise, you may significantly drop IOU for the merged regions (assuming your model very accurate at detecting separate instances of the same class).
", "rawMarkdown": "Yes, if you found several instances of the same class on the image - you should submit them as separate RLEs. Otherwise, you may significantly drop IOU for the merged regions (assuming your model very accurate at detecting separate instances of the same class).", "votes": null }, { "id": "849451", "postDate": "05/15/2020 19:07:33", "content": "Okay! Thanks for confirming
", "rawMarkdown": "Okay! Thanks for confirming", "votes": null } ], "comments": [ { "id": 846750, "author_name": "hensonwells", "author_url": "", "post_date": "05/14/2020 01:41:24", "content": "my score is zero too, is your classification right? Is same as the train.csv?
", "votes": null, "replies": [] }, { "id": 846756, "author_name": "mihirmavalankar", "author_url": "", "post_date": "05/14/2020 01:54:46", "content": "Yes, I checked if ids start from 0 to 45 I looked up the results and they are correct. Yes I am using train.csv.\nDo you know how I can contact the host of this competition to get some clarification?
", "votes": null, "replies": [] }, { "id": 846787, "author_name": "mihirmavalankar", "author_url": "", "post_date": "05/14/2020 02:07:33", "content": "", "votes": null, "replies": [] }, { "id": 847090, "author_name": "hensonwells", "author_url": "", "post_date": "05/14/2020 06:56:56", "content": "no , i @ her too, but no work.
", "votes": null, "replies": [] }, { "id": 847118, "author_name": "fanhaobei", "author_url": "", "post_date": "05/14/2020 07:25:18", "content": "hi, does this example's attributes match ground-truth? F1 should be bigger than 0.5 .
", "votes": null, "replies": [ { "id": 847936, "author_name": "mihirmavalankar", "author_url": "", "post_date": "05/14/2020 17:04:05", "content": "Hi @fanhaobei \nYa the ClassIds match the ground truth as in for this image it predicted says that classes dress, sleeve (2) and neckline exist in the image. It return NaN for the attributeIds though.
", "votes": null, "replies": [] } ] }, { "id": 847391, "author_name": "polosin", "author_url": "", "post_date": "05/14/2020 11:22:55", "content": "I got a zero score once as well, but I simply forgot to convert 1-indexed class IDs to 0-indexed, but I see that's not your case.
\n\nIt doesn't matter which library you're using for image resizing, but you need to make sure that:\n1. You resize an image to fit into the square 1024x1024 and keep image aspect ratio\n2. You not just downscale large images, but also upscale small images to the same size
\n\nThen I see 2 potential cases how you could get a zero score:\n1. RLE encoding. What code do you use to do encoding? Did you try to re-encode some masks from the training dataset to make sure it works correctly?\n2. Attribute IDs. Do you predict attribute IDs for the segments? If not, you'll get zero scores for the segments with classes 0-12, even if the segmentation is 100% accurate. So if most of your predictions are classes 0-12 that would explain your final zero score. Check the distribution of the classes that you have in your predictions.
", "votes": null, "replies": [] }, { "id": 847931, "author_name": "mihirmavalankar", "author_url": "", "post_date": "05/14/2020 17:00:41", "content": "Hi @polosin ,
\n\nAre you sure I should scale all of them to 1024x1024?
\n\nI checked the rle encoding and decoding and decoding gives the masks back correctly.
I haven't checked the distribution of classes that I have predicted so I will definitely have look into that too. You mean to say that for segments with \"ClassIds\" 0-12 if I get a Nan for attribute Ids (and ground truth is not Nan) I will get a 0 even if the classIds and masks are correct?\nAlso the predictions for classIds should be from 0-45 right? Not 1-46?
Thanks again! Will give you an update on how it goes!
", "votes": null, "replies": [ { "id": 848268, "author_name": "polosin", "author_url": "", "post_date": "05/14/2020 20:38:49", "content": "\n\n\nI scaled all the images to 1024,1024 but that gives this error-
\n
I experienced this exception before as well. They didn't mention it on the Evaluation page, but when you resize an image you should floor the value of the smaller side, not round it. For example, if your smaller side of the image is 736.83, you should resize it to 736, not 737.
\n\n\n\n\nAre you sure I should scale all of them to 1024x1024?
\n
Yes, I checked both variants
\n\n\n\n\nYou mean to say that for segments with \"ClassIds\" 0-12 if I get a Nan for attribute Ids (and ground truth is not Nan) I will get a 0 even if the classIds and masks are correct?
\n
Yes, you can read more about the metric on the Evaluation page
\n\n\n\n\nAlso the predictions for classIds should be from 0-45 right? Not 1-46?
\n
Yes, exactly as it's in the training data
", "votes": null, "replies": [] }, { "id": 848275, "author_name": "mihirmavalankar", "author_url": "", "post_date": "05/14/2020 20:49:38", "content": "I might be understanding this wrong but how can the height or width of an image be a floating point number? It has to be an integer right?\n\"... .For example, if your smaller side of the image is 736.83, you should resize it to 736, not 737\"
", "votes": null, "replies": [] }, { "id": 848295, "author_name": "polosin", "author_url": "", "post_date": "05/14/2020 21:16:55", "content": "For example, you have an image of size 1067x1600, you need to resize to 1024 for the longest side. So the new size before converting it to an integer is 682.88 x 1024. You should floor 682.88 to 682, not round it to 683.
", "votes": null, "replies": [] }, { "id": 848412, "author_name": "mihirmavalankar", "author_url": "", "post_date": "05/15/2020 00:29:49", "content": "Oh okay got it, makes sense
", "votes": null, "replies": [] } ] }, { "id": 849294, "author_name": "mihirmavalankar", "author_url": "", "post_date": "05/15/2020 16:17:54", "content": "@polosin \nJust another clarification.\nSay in an image if the classId prediction is [10 31 31 33] then in the out there should two separate run length encodings for both the 31 classes right? \nOr is it okay if we combine the RLE for both 31 predictions?
", "votes": null, "replies": [ { "id": 849367, "author_name": "polosin", "author_url": "", "post_date": "05/15/2020 17:41:24", "content": "Yes, if you found several instances of the same class on the image - you should submit them as separate RLEs. Otherwise, you may significantly drop IOU for the merged regions (assuming your model very accurate at detecting separate instances of the same class).
", "votes": null, "replies": [] }, { "id": 849451, "author_name": "mihirmavalankar", "author_url": "", "post_date": "05/15/2020 19:07:33", "content": "Okay! Thanks for confirming
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "846444": "Just submitted the csv for this competition and got 0.0 score.\nWhen I run the model on my local notebook and look at the visualizations the look fine and very convincing. Might not go to the top of the leaderboard but certainly no way they should get a zero score. \nThe issue is now I don't know what I am doing wrong? \nSample image masks:\n\n\n\nSome possibilities:-\n1. Do the classIds in evaluation start from 0 or 1? My predictions are from 0 to 45, if all predictions are shifted by one then this might explain the zero score.\n2. I have used cv2 instead of PIL to resize images, does that matter?\n\n\nAny ideas on the why I might be getting a zero score would be great!\n\nTo competition host: Please give some more clear instructions on submission and make submission as less complicated as possible. It's really disheartening to make something that works and get a low score because your submission format is not right.", "846750": "my score is zero too, is your classification right? Is same as the train.csv?", "846756": "Yes, I checked if ids start from 0 to 45 I looked up the results and they are correct. Yes I am using train.csv.\nDo you know how I can contact the host of this competition to get some clarification?", "846787": "makeitworkjml", "847090": "no , i @ her too, but no work.", "847118": "hi, does this example's attributes match ground-truth? F1 should be bigger than 0.5 .", "847391": "I got a zero score once as well, but I simply forgot to convert 1-indexed class IDs to 0-indexed, but I see that's not your case.\n\nIt doesn't matter which library you're using for image resizing, but you need to make sure that:\n1. You resize an image to fit into the square 1024x1024 and keep image aspect ratio\n2. You not just downscale large images, but also upscale small images to the same size\n\nThen I see 2 potential cases how you could get a zero score:\n1. **RLE encoding**. What code do you use to do encoding? Did you try to re-encode some masks from the training dataset to make sure it works correctly?\n2. **Attribute IDs**. Do you predict attribute IDs for the segments? If not, you'll get zero scores for the segments with classes 0-12, even if the segmentation is 100% accurate. So if most of your predictions are classes 0-12 that would explain your final zero score. Check the distribution of the classes that you have in your predictions.", "847931": "Hi @polosin ,\n\n1. Thanks for the suggestions. I scaled all the images to 1024,1024 but that gives this error-\n4 Exceptions:\nOne or more submitted values exceeded the image dimensions.\nOne or more submitted values exceeded the image dimensions.\nOne or more submitted values exceeded the image dimensions.\nOne or more submitted values exceeded the image dimensions.\n\nAre you sure I should scale all of them to 1024x1024?\n\n2. I checked the rle encoding and decoding and decoding gives the masks back correctly.\n\n3. I haven't checked the distribution of classes that I have predicted so I will definitely have look into that too. You mean to say that for segments with \"ClassIds\" 0-12 if I get a Nan for attribute Ids (and ground truth is not Nan) I will get a 0 even if the classIds and masks are correct?\nAlso the predictions for classIds should be from 0-45 right? Not 1-46?\n\nThanks again! Will give you an update on how it goes!", "847936": "Hi @fanhaobei \nYa the ClassIds match the ground truth as in for this image it predicted says that classes dress, sleeve (2) and neckline exist in the image. It return NaN for the attributeIds though.", "848268": "> I scaled all the images to 1024,1024 but that gives this error-\n\nI experienced this exception before as well. They didn't mention it on the Evaluation page, but when you resize an image you should floor the value of the smaller side, not round it. For example, if your smaller side of the image is 736.83, you should resize it to 736, not 737.\n\n> Are you sure I should scale all of them to 1024x1024?\n\nYes, I checked both variants\n\n> You mean to say that for segments with \"ClassIds\" 0-12 if I get a Nan for attribute Ids (and ground truth is not Nan) I will get a 0 even if the classIds and masks are correct?\n\nYes, you can read more about the metric on the [Evaluation page](https://www.kaggle.com/c/imaterialist-fashion-2020-fgvc7/overview/evaluation)\n\n> Also the predictions for classIds should be from 0-45 right? Not 1-46?\n\nYes, exactly as it's in the training data", "848275": "I might be understanding this wrong but how can the height or width of an image be a floating point number? It has to be an integer right?\n\"... .For example, if your smaller side of the image is **736.83**, you should resize it to 736, not 737\"", "848295": "For example, you have an image of size 1067x1600, you need to resize to 1024 for the longest side. So the new size before converting it to an integer is 682.88 x 1024. You should floor 682.88 to 682, not round it to 683.", "848412": "Oh okay got it, makes sense", "849294": "polosin \nJust another clarification.\nSay in an image if the classId prediction is [10 31 31 33] then in the out there should two separate run length encodings for both the 31 classes right? \nOr is it okay if we combine the RLE for both 31 predictions?", "849367": "Yes, if you found several instances of the same class on the image - you should submit them as separate RLEs. Otherwise, you may significantly drop IOU for the merged regions (assuming your model very accurate at detecting separate instances of the same class).", "849451": "Okay! Thanks for confirming" }, "source": "meta" }