After having the first look at the data . If I were a Unet , I would have picked a Fish easily . (There could be a bias here , I am from a place where fishes are abundant) . However , to my surprise , Unet picks flower way better than fish . The dice is .72 to .74 for Flower, whereas Fish ,Gravel and Sugar it is around .60 to .62 . \nAre others seeing the same ?
", "messages": [ { "id": "660701", "postDate": "10/29/2019 13:50:25", "content": "After having the first look at the data . If I were a Unet , I would have picked a Fish easily . (There could be a bias here , I am from a place where fishes are abundant) . However , to my surprise , Unet picks flower way better than fish . The dice is .72 to .74 for Flower, whereas Fish ,Gravel and Sugar it is around .60 to .62 . \nAre others seeing the same ?
", "rawMarkdown": "After having the first look at the data . If I were a Unet , I would have picked a Fish easily . (There could be a bias here , I am from a place where fishes are abundant) . However , to my surprise , Unet picks flower way better than fish . The dice is .72 to .74 for Flower, whereas Fish ,Gravel and Sugar it is around .60 to .62 . \nAre others seeing the same ?", "votes": null }, { "id": "660717", "postDate": "10/29/2019 14:17:26", "content": "Yes, same here.
", "rawMarkdown": "Yes, same here.", "votes": null }, { "id": "660719", "postDate": "10/29/2019 14:19:26", "content": "My best model in the LB is 0.666 the dice value of my test data is as follows:\nFish: 0.631\nFlower: 0.777\nGravel: 0.647\nSugar: 0.627
", "rawMarkdown": "My best model in the LB is 0.666 the dice value of my test data is as follows:\nFish: 0.631\nFlower: 0.777\nGravel: 0.647\nSugar: 0.627", "votes": null }, { "id": "660724", "postDate": "10/29/2019 14:22:54", "content": "I think it's a nice thing that the CV and LB correlates to certain extent . \nI get around .646 single fold single model with stats as \n.62 \n.74\n.60\n.61
", "rawMarkdown": "I think it's a nice thing that the CV and LB correlates to certain extent . \nI get around .646 single fold single model with stats as \n.62 \n.74\n.60\n.61", "votes": null }, { "id": "660729", "postDate": "10/29/2019 14:24:43", "content": "Question is why ? Any theories ? \nGravel and flowers could be similar to my eyes . But Fish is quite distinct shape and orientation .
", "rawMarkdown": "Question is why ? Any theories ? \nGravel and flowers could be similar to my eyes . But Fish is quite distinct shape and orientation .", "votes": null }, { "id": "660765", "postDate": "10/29/2019 15:08:06", "content": "After I checked some images and labels, I think the labels of flower seems not that noisy.
", "rawMarkdown": "After I checked some images and labels, I think the labels of flower seems not that noisy.", "votes": null }, { "id": "660837", "postDate": "10/29/2019 17:23:49", "content": "I just posted my first model here. It uses bounding boxes and no segmentation. It scores dice 0.567, 0.704, 0.559, 0.498 respectively for Fish, Flower, Gravel, Sugar. And the classifier accuracy is 0.683, 0.818, 0.688, 0.748 respectively. So yes, it does better with Flowers.
\n\nWhen I look with my eyes, Flowers appear to be the easiest and less subjective to detect.
", "rawMarkdown": "I just posted my first model [here][1]. It uses bounding boxes and no segmentation. It scores dice 0.567, 0.704, 0.559, 0.498 respectively for Fish, Flower, Gravel, Sugar. And the classifier accuracy is 0.683, 0.818, 0.688, 0.748 respectively. So yes, it does better with Flowers.\n\nWhen I look with my eyes, Flowers appear to be the easiest and less subjective to detect.\n\n[1]: https://www.kaggle.com/cdeotte/cloud-bounding-boxes-cv-0-58", "votes": null }, { "id": "660878", "postDate": "10/29/2019 18:23:27", "content": "Wow .. That is amazing ! I never knew of such algorithm . And it did score great ! Is it Bounding Box Regression Algorithm ? What exactly have you used to determine the bounding boxes ?
", "rawMarkdown": "Wow .. That is amazing ! I never knew of such algorithm . And it did score great ! Is it Bounding Box Regression Algorithm ? What exactly have you used to determine the bounding boxes ?", "votes": null }, { "id": "660886", "postDate": "10/29/2019 18:30:41", "content": "Yes it's regression. It's how vision recognition locates objects. A bounding box is just 4 numbers (x, y, width, height) where (x,y) is top left corner of bounding box. Build a model to predict those 4 numbers and use mean_squared_error loss. That kernel is only a starter. It can score much better than that.
You need to determine how you will change the given training masks into bounding boxes. In that kernel I just naively find the smallest box that contains the entire training mask. I also only train with the highest quality training image bounding boxes.
", "rawMarkdown": "Yes it's regression. It's how vision recognition locates objects. A bounding box is just 4 numbers `(x, y, width, height)` where `(x,y)` is top left corner of bounding box. Build a model to predict those 4 numbers and use `mean_squared_error` loss. That kernel is only a starter. It can score much better than that.\n\nYou need to determine how you will change the given training masks into bounding boxes. In that kernel I just naively find the smallest box that contains the entire training mask. I also only train with the highest quality training image bounding boxes.", "votes": null }, { "id": "662305", "postDate": "10/31/2019 11:59:32", "content": "UPDATE: My second model here uses segmentation is achieves Dice 0.574, 0.721, 0.590, 0.612 on Fish, Flower, Gravel, Sugar respectively.
\n\nIt's interesting to observe that segmentation increased Sugar the most significantly compared with bounding boxes.
", "rawMarkdown": "UPDATE: My second model [here][1] uses segmentation is achieves Dice 0.574, 0.721, 0.590, 0.612 on Fish, Flower, Gravel, Sugar respectively.\n\nIt's interesting to observe that segmentation increased Sugar the most significantly compared with bounding boxes.\n\n[1]: https://www.kaggle.com/cdeotte/train-with-crops-cv-0-60", "votes": null }, { "id": "662308", "postDate": "10/31/2019 12:06:28", "content": "I can not confirm right now(Got lost in not maintaining the proper experiment log) , but i remember seeing Sugar - CV increasing when the size was 256x256 . But , when the size increases then other CV increases , sugar reduces . Not sure of the reason.
", "rawMarkdown": "I can not confirm right now(Got lost in not maintaining the proper experiment log) , but i remember seeing Sugar - CV increasing when the size was 256x256 . But , when the size increases then other CV increases , sugar reduces . Not sure of the reason.", "votes": null }, { "id": "662326", "postDate": "10/31/2019 12:33:36", "content": "That makes sense. If you restrict your vision to a 256x256 crop, you cannot tell if you are looking at a Fish because you don't have the context of it's Fish head and Fish tail. However even with a small crop, you can recognize Sugar (i.e. Sugar doesn't need global context whereas the others benefit from global context).
\n\nMy referenced model above only trains with crops. I should try fine-tuning training with larger crops and/or full images and see how it affects the Dice score of Fish, Flower, and Gravel.
", "rawMarkdown": "That makes sense. If you restrict your vision to a 256x256 crop, you cannot tell if you are looking at a Fish because you don't have the context of it's Fish head and Fish tail. However even with a small crop, you can recognize Sugar (i.e. Sugar doesn't need global context whereas the others benefit from global context). \n \nMy referenced model above only trains with crops. I should try fine-tuning training with larger crops and/or full images and see how it affects the Dice score of Fish, Flower, and Gravel.", "votes": null }, { "id": "662434", "postDate": "10/31/2019 14:52:13", "content": "The organiser’s paper says that Sugar and Gravel are often confused by labellers. They seem to be a question of scale. So crops vs full may hurt performance?
\n\nI’ve tried various attempts at combining sugar and gravel masks/predictions to no huge effect.
", "rawMarkdown": "The organiser’s paper says that Sugar and Gravel are often confused by labellers. They seem to be a question of scale. So crops vs full may hurt performance?\n\nI’ve tried various attempts at combining sugar and gravel masks/predictions to no huge effect.", "votes": null }, { "id": "662449", "postDate": "10/31/2019 15:16:25", "content": "\n\n\nSo crops vs full may hurt performance?
\n
When I say \"crops\" I mean cut rectangles out of the original images. I am not referring to resizing. If you randomly resize images, then yes you will make Sugar look like Gravel and make Gravel look like Sugar.
\n\nBut even though I am careful with my crops, humans may use the context of the entire image to determine which clouds are small versus very small (Gravel versus Sugar), so perhaps removing global context hurts classification of Gravel and Sugar.
", "rawMarkdown": "> So crops vs full may hurt performance?\n\nWhen I say \"crops\" I mean cut rectangles out of the original images. I am not referring to resizing. If you randomly resize images, then yes you will make Sugar look like Gravel and make Gravel look like Sugar. \n\nBut even though I am careful with my crops, humans may use the context of the entire image to determine which clouds are small versus very small (Gravel versus Sugar), so perhaps removing global context hurts classification of Gravel and Sugar.", "votes": null } ], "comments": [ { "id": 660717, "author_name": "igormunizims", "author_url": "", "post_date": "10/29/2019 14:17:26", "content": "Yes, same here.
", "votes": null, "replies": [ { "id": 660729, "author_name": "phoenix9032", "author_url": "", "post_date": "10/29/2019 14:24:43", "content": "Question is why ? Any theories ? \nGravel and flowers could be similar to my eyes . But Fish is quite distinct shape and orientation .
", "votes": null, "replies": [] } ] }, { "id": 660719, "author_name": "shentaowang", "author_url": "", "post_date": "10/29/2019 14:19:26", "content": "My best model in the LB is 0.666 the dice value of my test data is as follows:\nFish: 0.631\nFlower: 0.777\nGravel: 0.647\nSugar: 0.627
", "votes": null, "replies": [ { "id": 660724, "author_name": "phoenix9032", "author_url": "", "post_date": "10/29/2019 14:22:54", "content": "I think it's a nice thing that the CV and LB correlates to certain extent . \nI get around .646 single fold single model with stats as \n.62 \n.74\n.60\n.61
", "votes": null, "replies": [] } ] }, { "id": 660765, "author_name": "xiejialun", "author_url": "", "post_date": "10/29/2019 15:08:06", "content": "After I checked some images and labels, I think the labels of flower seems not that noisy.
", "votes": null, "replies": [] }, { "id": 660837, "author_name": "cdeotte", "author_url": "", "post_date": "10/29/2019 17:23:49", "content": "I just posted my first model here. It uses bounding boxes and no segmentation. It scores dice 0.567, 0.704, 0.559, 0.498 respectively for Fish, Flower, Gravel, Sugar. And the classifier accuracy is 0.683, 0.818, 0.688, 0.748 respectively. So yes, it does better with Flowers.
\n\nWhen I look with my eyes, Flowers appear to be the easiest and less subjective to detect.
", "votes": null, "replies": [ { "id": 660878, "author_name": "phoenix9032", "author_url": "", "post_date": "10/29/2019 18:23:27", "content": "Wow .. That is amazing ! I never knew of such algorithm . And it did score great ! Is it Bounding Box Regression Algorithm ? What exactly have you used to determine the bounding boxes ?
", "votes": null, "replies": [] }, { "id": 660886, "author_name": "cdeotte", "author_url": "", "post_date": "10/29/2019 18:30:41", "content": "Yes it's regression. It's how vision recognition locates objects. A bounding box is just 4 numbers (x, y, width, height) where (x,y) is top left corner of bounding box. Build a model to predict those 4 numbers and use mean_squared_error loss. That kernel is only a starter. It can score much better than that.
You need to determine how you will change the given training masks into bounding boxes. In that kernel I just naively find the smallest box that contains the entire training mask. I also only train with the highest quality training image bounding boxes.
", "votes": null, "replies": [] }, { "id": 662305, "author_name": "cdeotte", "author_url": "", "post_date": "10/31/2019 11:59:32", "content": "UPDATE: My second model here uses segmentation is achieves Dice 0.574, 0.721, 0.590, 0.612 on Fish, Flower, Gravel, Sugar respectively.
\n\nIt's interesting to observe that segmentation increased Sugar the most significantly compared with bounding boxes.
", "votes": null, "replies": [] }, { "id": 662308, "author_name": "phoenix9032", "author_url": "", "post_date": "10/31/2019 12:06:28", "content": "I can not confirm right now(Got lost in not maintaining the proper experiment log) , but i remember seeing Sugar - CV increasing when the size was 256x256 . But , when the size increases then other CV increases , sugar reduces . Not sure of the reason.
", "votes": null, "replies": [] }, { "id": 662326, "author_name": "cdeotte", "author_url": "", "post_date": "10/31/2019 12:33:36", "content": "That makes sense. If you restrict your vision to a 256x256 crop, you cannot tell if you are looking at a Fish because you don't have the context of it's Fish head and Fish tail. However even with a small crop, you can recognize Sugar (i.e. Sugar doesn't need global context whereas the others benefit from global context).
\n\nMy referenced model above only trains with crops. I should try fine-tuning training with larger crops and/or full images and see how it affects the Dice score of Fish, Flower, and Gravel.
", "votes": null, "replies": [] }, { "id": 662434, "author_name": "robga", "author_url": "", "post_date": "10/31/2019 14:52:13", "content": "The organiser’s paper says that Sugar and Gravel are often confused by labellers. They seem to be a question of scale. So crops vs full may hurt performance?
\n\nI’ve tried various attempts at combining sugar and gravel masks/predictions to no huge effect.
", "votes": null, "replies": [] }, { "id": 662449, "author_name": "cdeotte", "author_url": "", "post_date": "10/31/2019 15:16:25", "content": "\n\n\nSo crops vs full may hurt performance?
\n
When I say \"crops\" I mean cut rectangles out of the original images. I am not referring to resizing. If you randomly resize images, then yes you will make Sugar look like Gravel and make Gravel look like Sugar.
\n\nBut even though I am careful with my crops, humans may use the context of the entire image to determine which clouds are small versus very small (Gravel versus Sugar), so perhaps removing global context hurts classification of Gravel and Sugar.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "660701": "After having the first look at the data . If I were a Unet , I would have picked a Fish easily . (There could be a bias here , I am from a place where fishes are abundant) . However , to my surprise , Unet picks flower way better than fish . The dice is .72 to .74 for Flower, whereas Fish ,Gravel and Sugar it is around .60 to .62 . \nAre others seeing the same ?", "660717": "Yes, same here.", "660719": "My best model in the LB is 0.666 the dice value of my test data is as follows:\nFish: 0.631\nFlower: 0.777\nGravel: 0.647\nSugar: 0.627", "660724": "I think it's a nice thing that the CV and LB correlates to certain extent . \nI get around .646 single fold single model with stats as \n.62 \n.74\n.60\n.61", "660729": "Question is why ? Any theories ? \nGravel and flowers could be similar to my eyes . But Fish is quite distinct shape and orientation .", "660765": "After I checked some images and labels, I think the labels of flower seems not that noisy.", "660837": "I just posted my first model [here][1]. It uses bounding boxes and no segmentation. It scores dice 0.567, 0.704, 0.559, 0.498 respectively for Fish, Flower, Gravel, Sugar. And the classifier accuracy is 0.683, 0.818, 0.688, 0.748 respectively. So yes, it does better with Flowers.\n\nWhen I look with my eyes, Flowers appear to be the easiest and less subjective to detect.\n\n[1]: https://www.kaggle.com/cdeotte/cloud-bounding-boxes-cv-0-58", "660878": "Wow .. That is amazing ! I never knew of such algorithm . And it did score great ! Is it Bounding Box Regression Algorithm ? What exactly have you used to determine the bounding boxes ?", "660886": "Yes it's regression. It's how vision recognition locates objects. A bounding box is just 4 numbers `(x, y, width, height)` where `(x,y)` is top left corner of bounding box. Build a model to predict those 4 numbers and use `mean_squared_error` loss. That kernel is only a starter. It can score much better than that.\n\nYou need to determine how you will change the given training masks into bounding boxes. In that kernel I just naively find the smallest box that contains the entire training mask. I also only train with the highest quality training image bounding boxes.", "662305": "UPDATE: My second model [here][1] uses segmentation is achieves Dice 0.574, 0.721, 0.590, 0.612 on Fish, Flower, Gravel, Sugar respectively.\n\nIt's interesting to observe that segmentation increased Sugar the most significantly compared with bounding boxes.\n\n[1]: https://www.kaggle.com/cdeotte/train-with-crops-cv-0-60", "662308": "I can not confirm right now(Got lost in not maintaining the proper experiment log) , but i remember seeing Sugar - CV increasing when the size was 256x256 . But , when the size increases then other CV increases , sugar reduces . Not sure of the reason.", "662326": "That makes sense. If you restrict your vision to a 256x256 crop, you cannot tell if you are looking at a Fish because you don't have the context of it's Fish head and Fish tail. However even with a small crop, you can recognize Sugar (i.e. Sugar doesn't need global context whereas the others benefit from global context). \n \nMy referenced model above only trains with crops. I should try fine-tuning training with larger crops and/or full images and see how it affects the Dice score of Fish, Flower, and Gravel.", "662434": "The organiser’s paper says that Sugar and Gravel are often confused by labellers. They seem to be a question of scale. So crops vs full may hurt performance?\n\nI’ve tried various attempts at combining sugar and gravel masks/predictions to no huge effect.", "662449": "> So crops vs full may hurt performance?\n\nWhen I say \"crops\" I mean cut rectangles out of the original images. I am not referring to resizing. If you randomly resize images, then yes you will make Sugar look like Gravel and make Gravel look like Sugar. \n\nBut even though I am careful with my crops, humans may use the context of the entire image to determine which clouds are small versus very small (Gravel versus Sugar), so perhaps removing global context hurts classification of Gravel and Sugar." }, "source": "meta" }