![\"test\"](\"https://i.imgur.com/JES6Zuf.gif\")
I planned to make synthetic dataset with rendered 3d whale tails on water background and masks for them. Whale tail has some basic procedural textures, rig for animation, trailing edge randomizer and mesh cage for changing tail's shape. I think that could help make more accurate segmentation (also one could make masks not for entire tail, but separately also for scratches or trailing edge). I'm not sure that this approach is actually useful, but I decided to try. Unfortunately, I realized that this will take little bit more time than I expected, so now I decided to share .blend (Blender 2.8 needed) file as it is with a short video explanation. Maybe that will be useful for someone. If you're somewhere in top of leaderboard, feel free to contact me - maybe I'll find some time for finishing this for you.
\n\nVideo: https://youtu.be/lkISn9g6q9E
", "messages": [ { "id": 459383, "postDate": "2019-01-21T16:31:53.713Z", "content": "
I planned to make synthetic dataset with rendered 3d whale tails on water background and masks for them. Whale tail has some basic procedural textures, rig for animation, trailing edge randomizer and mesh cage for changing tail's shape. I think that could help make more accurate segmentation (also one could make masks not for entire tail, but separately also for scratches or trailing edge). I'm not sure that this approach is actually useful, but I decided to try. Unfortunately, I realized that this will take little bit more time than I expected, so now I decided to share .blend (Blender 2.8 needed) file as it is with a short video explanation. Maybe that will be useful for someone. If you're somewhere in top of leaderboard, feel free to contact me - maybe I'll find some time for finishing this for you.
\n\nVideo: https://youtu.be/lkISn9g6q9E
", "rawMarkdown": "![test][1]\n\nI planned to make synthetic dataset with rendered 3d whale tails on water background and masks for them. Whale tail has some basic procedural textures, rig for animation, trailing edge randomizer and mesh cage for changing tail's shape. I think that could help make more accurate segmentation (also one could make masks not for entire tail, but separately also for scratches or trailing edge). I'm not sure that this approach is actually useful, but I decided to try. Unfortunately, I realized that this will take little bit more time than I expected, so now I decided to share .blend (Blender 2.8 needed) file as it is with a short video explanation. Maybe that will be useful for someone. If you're somewhere in top of leaderboard, feel free to contact me - maybe I'll find some time for finishing this for you.\n\nVideo: https://youtu.be/lkISn9g6q9E\n\n\n [1]: https://i.imgur.com/JES6Zuf.gif", "votes": 59 }, { "id": 459518, "postDate": "2019-01-21T22:40:53.317Z", "content": "Fun concept. Not sure if beneficial to the ID algorithm. But it is relevant to note that the flukes are ~ 30-40cm thick at the base and 5cm thick at the trailing edge, so, as far as rotation and distortion are concerned, they are not completely planar
", "rawMarkdown": "Fun concept. Not sure if beneficial to the ID algorithm. But it is relevant to note that the flukes are ~ 30-40cm thick at the base and 5cm thick at the trailing edge, so, as far as rotation and distortion are concerned, they are not completely planar", "votes": 6, "replies": [ { "id": 460528, "postDate": "2019-01-23T22:38:33.633Z", "content": "I think it may very well be useful. Unfortunately I haven't had time to compete in this event but I had an idea to train a preprocessing net (similar to martin's bounding box network) that identified key anchor points on a whale's fluke.
\n\nUsing these anchor points, i could adjust the perspective to make each image look more like a perfect face-on view of the fluke (like Dene's rendered image below). Of course this couldn't correct all issues because a whale's tail doesn't have bones so it is pretty deform-able.
\n\nI hand-annotated 1000 of the training dataset with ~10 points per whale, but didn't have time to use them.
\n\nDoing this preprocessing I believe that a much more accurate network could be trained. Similar to how some face identification systems do face alignment and scaling before comparison.
\n\nIn addition, (free ideas here to those who read!) with these key points, you could actually split a whale's fluke into 2 halves. This could allow you to do 2 independent IDs -- one on each half. Occasionally part of a whale's fluke is deformed, obscured, or even has changed from a previous picture due to scarring/environmental factors.
\n\nEDIT:
\n\nWith Dene's dataset, you could train a network with synthetic data, you could generate segmentation masks, anchor points all automatically. Then with some small amount of hand-labeled real life data, utilize transfer learning to adapt the model to the humpback whale dataset.
", "rawMarkdown": "I think it may very well be useful. Unfortunately I haven't had time to compete in this event but I had an idea to train a preprocessing net (similar to martin's bounding box network) that identified key anchor points on a whale's fluke.\n\nUsing these anchor points, i could adjust the perspective to make each image look more like a perfect face-on view of the fluke (like Dene's rendered image below). Of course this couldn't correct all issues because a whale's tail doesn't have bones so it is pretty deform-able.\n\nI hand-annotated 1000 of the training dataset with ~10 points per whale, but didn't have time to use them.\n\nDoing this preprocessing I believe that a much more accurate network could be trained. Similar to how some face identification systems do face alignment and scaling before comparison.\n\nIn addition, (free ideas here to those who read!) with these key points, you could actually split a whale's fluke into 2 halves. This could allow you to do 2 independent IDs -- one on each half. Occasionally part of a whale's fluke is deformed, obscured, or even has changed from a previous picture due to scarring/environmental factors.\n\nEDIT:\n\nWith Dene's dataset, you could train a network with synthetic data, you could generate segmentation masks, anchor points all automatically. Then with some small amount of hand-labeled real life data, utilize transfer learning to adapt the model to the humpback whale dataset.\n", "votes": 1 }, { "id": 460530, "postDate": "2019-01-23T22:45:33.793Z", "content": "Yup, I had almost the same ideas when I started doing this. It would be nice to know if somebody tried one of suggested approaches actually. :)
", "rawMarkdown": "Yup, I had almost the same ideas when I started doing this. It would be nice to know if somebody tried one of suggested approaches actually. :)" }, { "id": 460541, "postDate": "2019-01-23T23:10:38.733Z", "content": "Paul, thanks for your ideas! Are you considering sharing your annotations with the community?
", "rawMarkdown": "Paul, thanks for your ideas! Are you considering sharing your annotations with the community?", "votes": 1 }, { "id": 461414, "postDate": "2019-01-26T00:41:28.657Z", "content": "I will see if i can find the time to convert it to a kaggle dataset and provide a simple EDA kernel.
", "rawMarkdown": "I will see if i can find the time to convert it to a kaggle dataset and provide a simple EDA kernel.", "votes": 1 }, { "id": 461818, "postDate": "2019-01-27T04:04:57.860Z", "content": "I've created a new post that shares the data I have collected and provides a very simple starter kernel. Check it out in the discussion for this competition!
", "rawMarkdown": "I've created a new post that shares the data I have collected and provides a very simple starter kernel. Check it out in the discussion for this competition!", "votes": 3 } ] }, { "id": 465775, "postDate": "2019-02-04T01:28:37.297Z", "content": "An algorithm that fits 3d model to image will be interesting. I recall i read about it in some paper. It is like fitting 3d shape and uv map to automatic generate a 3d model.
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/465775/11164/046304E7-63F9-4225-B259-860A76707A21.png\")
found one paper:
\n\nhttps://github.com/akanazawa/cmr
", "rawMarkdown": "found one paper:\n\nhttps://github.com/akanazawa/cmr" } ] }, { "id": 462365, "postDate": "2019-01-28T07:33:50.077Z", "content": "impressive idea!
", "rawMarkdown": "impressive idea!" }, { "id": 459440, "postDate": "2019-01-21T18:32:39.840Z", "content": "Very nice! Do you already have any examples how the final train image would look like?
", "rawMarkdown": "Very nice! Do you already have any examples how the final train image would look like?", "replies": [ { "id": 459458, "postDate": "2019-01-21T19:15:44.770Z", "content": "I planned to write a little script for Blender to render tail from different angles and also generating texture, trailing edge and tail's form variations. But I have little time now, so not sure if I'll actually make it. That's not hard actually but will take time.
\n\nFor now tail looks like that (default):\n![\"default\"](\"https://i.imgur.com/98Ne8g4.png\")
Custom pose example:\n![\"custom\"](\"https://i.imgur.com/q3ToRIg.png\")
Very nice! Thanks! \nI'm not strong in Blender. Could you please generate 10.000-100.000 of these (10-50 images per id) and upload here? I think our community would much appreciate it. Even without the background
", "rawMarkdown": "Very nice! Thanks! \nI'm not strong in Blender. Could you please generate 10.000-100.000 of these (10-50 images per id) and upload here? I think our community would much appreciate it. Even without the background", "votes": -1 } ] }, { "id": 461762, "postDate": "2019-01-27T00:00:35.400Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 459518, "author_name": "Ted Cheeseman", "author_url": "", "post_date": "2019-01-21T22:40:53.317000", "content": "Fun concept. Not sure if beneficial to the ID algorithm. But it is relevant to note that the flukes are ~ 30-40cm thick at the base and 5cm thick at the trailing edge, so, as far as rotation and distortion are concerned, they are not completely planar
", "votes": 6, "replies": [ { "id": 460528, "author_name": "Paul Johnson", "author_url": "", "post_date": "2019-01-23T22:38:33.633000", "content": "I think it may very well be useful. Unfortunately I haven't had time to compete in this event but I had an idea to train a preprocessing net (similar to martin's bounding box network) that identified key anchor points on a whale's fluke.
\n\nUsing these anchor points, i could adjust the perspective to make each image look more like a perfect face-on view of the fluke (like Dene's rendered image below). Of course this couldn't correct all issues because a whale's tail doesn't have bones so it is pretty deform-able.
\n\nI hand-annotated 1000 of the training dataset with ~10 points per whale, but didn't have time to use them.
\n\nDoing this preprocessing I believe that a much more accurate network could be trained. Similar to how some face identification systems do face alignment and scaling before comparison.
\n\nIn addition, (free ideas here to those who read!) with these key points, you could actually split a whale's fluke into 2 halves. This could allow you to do 2 independent IDs -- one on each half. Occasionally part of a whale's fluke is deformed, obscured, or even has changed from a previous picture due to scarring/environmental factors.
\n\nEDIT:
\n\nWith Dene's dataset, you could train a network with synthetic data, you could generate segmentation masks, anchor points all automatically. Then with some small amount of hand-labeled real life data, utilize transfer learning to adapt the model to the humpback whale dataset.
", "votes": 1, "replies": [] }, { "id": 460530, "author_name": "Dene", "author_url": "", "post_date": "2019-01-23T22:45:33.793000", "content": "Yup, I had almost the same ideas when I started doing this. It would be nice to know if somebody tried one of suggested approaches actually. :)
", "votes": 0, "replies": [] }, { "id": 460541, "author_name": "Vladislav Shakhrai", "author_url": "", "post_date": "2019-01-23T23:10:38.733000", "content": "Paul, thanks for your ideas! Are you considering sharing your annotations with the community?
", "votes": 1, "replies": [] }, { "id": 461414, "author_name": "Paul Johnson", "author_url": "", "post_date": "2019-01-26T00:41:28.657000", "content": "I will see if i can find the time to convert it to a kaggle dataset and provide a simple EDA kernel.
", "votes": 1, "replies": [] }, { "id": 461818, "author_name": "Paul Johnson", "author_url": "", "post_date": "2019-01-27T04:04:57.860000", "content": "I've created a new post that shares the data I have collected and provides a very simple starter kernel. Check it out in the discussion for this competition!
", "votes": 3, "replies": [] } ] }, { "id": 465775, "author_name": "hengck23", "author_url": "", "post_date": "2019-02-04T01:28:37.297000", "content": "An algorithm that fits 3d model to image will be interesting. I recall i read about it in some paper. It is like fitting 3d shape and uv map to automatic generate a 3d model.
\n\n
found one paper:
\n\nhttps://github.com/akanazawa/cmr
", "votes": 0, "replies": [] } ] }, { "id": 462365, "author_name": "Hansen Zhao", "author_url": "", "post_date": "2019-01-28T07:33:50.077000", "content": "impressive idea!
", "votes": 0, "replies": [] }, { "id": 459440, "author_name": "Artem.Sanakoev", "author_url": "", "post_date": "2019-01-21T18:32:39.840000", "content": "Very nice! Do you already have any examples how the final train image would look like?
", "votes": 0, "replies": [ { "id": 459458, "author_name": "Dene", "author_url": "", "post_date": "2019-01-21T19:15:44.770000", "content": "I planned to write a little script for Blender to render tail from different angles and also generating texture, trailing edge and tail's form variations. But I have little time now, so not sure if I'll actually make it. That's not hard actually but will take time.
\n\nFor now tail looks like that (default):\n
Custom pose example:\n
Very nice! Thanks! \nI'm not strong in Blender. Could you please generate 10.000-100.000 of these (10-50 images per id) and upload here? I think our community would much appreciate it. Even without the background
", "votes": -1, "replies": [] } ] }, { "id": 461762, "author_name": "", "author_url": "", "post_date": "2019-01-27T00:00:35.400000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "459383": "![test][1]\n\nI planned to make synthetic dataset with rendered 3d whale tails on water background and masks for them. Whale tail has some basic procedural textures, rig for animation, trailing edge randomizer and mesh cage for changing tail's shape. I think that could help make more accurate segmentation (also one could make masks not for entire tail, but separately also for scratches or trailing edge). I'm not sure that this approach is actually useful, but I decided to try. Unfortunately, I realized that this will take little bit more time than I expected, so now I decided to share .blend (Blender 2.8 needed) file as it is with a short video explanation. Maybe that will be useful for someone. If you're somewhere in top of leaderboard, feel free to contact me - maybe I'll find some time for finishing this for you.\n\nVideo: https://youtu.be/lkISn9g6q9E\n\n\n [1]: https://i.imgur.com/JES6Zuf.gif", "459518": "Fun concept. Not sure if beneficial to the ID algorithm. But it is relevant to note that the flukes are ~ 30-40cm thick at the base and 5cm thick at the trailing edge, so, as far as rotation and distortion are concerned, they are not completely planar", "465775": "An algorithm that fits 3d model to image will be interesting. I recall i read about it in some paper. It is like fitting 3d shape and uv map to automatic generate a 3d model. \n\n![enter image description here][1]\n\n\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/465775/11164/046304E7-63F9-4225-B259-860A76707A21.png", "462365": "impressive idea!", "459440": "Very nice! Do you already have any examples how the final train image would look like?", "461762": "" } }