\n
![\"\"](\"https://www.kaggleusercontent.com/kf/89909522/eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2In0..1DIpqdtOqhxX_PF4imgHnA.MI_7kUKiNsGOcIj7lXDUZrHxZaTPjSIADC8hEDRkHQvPMRw7yh32YLbcj2xmJMqOyNdbbKtl9nZOhRqb8nHymvdUL42mStLypRVr0LOhzqV_S23BBk-No3I4vf4jc2bXO2kn8Qv34r1KnC1HID2EYs6W_HuoVIY_UQS7x0ccCsQ_Pp_zHaXR45fXcxSqzS2mGsdLR4Xpgt3qVVUB9BhJ60LvbnLeLAyt_n7sh2mqatCRzUvb356AL-vRbdKhE3EAbYi31uRfKiTqofmBZiKEDNKosZdKZDYEwvLQGgkHmYrJF1G5aoyGogEKc0OXuIwdvxJ-2TKhVKaZ-50nN38jFpCMkOrSYzejQfROQr6l_gFgBVbz2eabjQcDMdXvNyRsCzNbBzzi5c5PiMA8olId9dhPFh0LpjEdqkGMvoCTKLRjIFJuxsoBi9aEETW8V40rWol0A-ugJV7sp10lhi08yHinRVcbwReBZuVBV5SzMPfC80GgrJe5--E9NokFhYyraBaDCpSdV5Hh4ZYveTQoT-kx5nwcHb1paZfaMbuSylAmoTp8e5Kjm4PLSBDE1dPRUqcBK8NSt5BnI6UlE1UGuPtfCajvnTFG7nzDSRNfAK9E6iXc-blwpfdcVn4C1DYca4cM_MC9DHJVZEDs7KVU1LR0OEG3jivHPx6ievNjoa8SOeMTN42JHezMAo1rmsIz.5UxFif_e8ZLSi__rC8zPAw/__results___files/__results___9_0.png\")
Created a dataset using the current State-of-the-Art Salient Object Detector - TRACER
\nConsidering the fact that the images presented in this dataset have a lot of variability in them, not just the fins and other body parts of marine life presented here, but also when it comes to background scenes.
\nWith this insight, I lingered with the Idea that segmenting out the marine life from their background might prove to be a helpful step for the deep learning model when performing classification.
\nThat's when I started searching out for the current best-open-sourced implementation of Salient Object Detector and came across this awesome repo.
\nMade a few changes to generate output for my use case in my forked repo.
And generated this Dataset
\n
The code I've used for generating this dataset can be found in this Notebook
\nThough still not perfect I hope this proves to be a starting point for someone to use and work on while I figure out ways to further improve this dataset.
\nSimilar work by @remekkinas : background removal using U2-Net
", "messages": [ { "id": "1720055", "postDate": "03/12/2022 12:20:07", "content": "Created a dataset using the current State-of-the-Art Salient Object Detector - TRACER
\nConsidering the fact that the images presented in this dataset have a lot of variability in them, not just the fins and other body parts of marine life presented here, but also when it comes to background scenes.
\nWith this insight, I lingered with the Idea that segmenting out the marine life from their background might prove to be a helpful step for the deep learning model when performing classification.
\nThat's when I started searching out for the current best-open-sourced implementation of Salient Object Detector and came across this awesome repo.
\nMade a few changes to generate output for my use case in my forked repo.
And generated this Dataset
\n
The code I've used for generating this dataset can be found in this Notebook
\nThough still not perfect I hope this proves to be a starting point for someone to use and work on while I figure out ways to further improve this dataset.
\nSimilar work by @remekkinas : background removal using U2-Net
", "rawMarkdown": "**Created a dataset using the current State-of-the-Art Salient Object Detector - TRACER**\n\nConsidering the fact that the images presented in this dataset have a lot of variability in them, not just the fins and other body parts of marine life presented here, but also when it comes to background scenes.\n\nWith this insight, I lingered with the Idea that segmenting out the marine life from their background might prove to be a helpful step for the deep learning model when performing classification.\n\nThat's when I started searching out for the current best-open-sourced implementation of Salient Object Detector and came across [this awesome repo.](https://github.com/Karel911/TRACER)\nMade a few changes to generate output for my use case in [my forked repo.](https://github.com/adnan119/TRACER)\n\nAnd generated this [Dataset](https://www.kaggle.com/adnanpen/background-removed-happywhale-dataset)\n\n\nThe code I've used for generating this dataset can be found in this [Notebook](https://www.kaggle.com/adnanpen/background-removal-using-tracer-sota-sod)\n\n***Though still not perfect I hope this proves to be a starting point for someone to use and work on while I figure out ways to further improve this dataset.***\n\nSimilar work by @remekkinas : [background removal using U2-Net](https://www.kaggle.com/remekkinas/remove-background-salient-object-detection)", "votes": null }, { "id": "1720417", "postDate": "03/12/2022 19:09:29", "content": "thanks for sharing. upvote. Have you create TFrecord format dataset?
", "rawMarkdown": "thanks for sharing. upvote. Have you create TFrecord format dataset?", "votes": null }, { "id": "1720507", "postDate": "03/12/2022 21:26:51", "content": "Nice! Thank you for mentioning my work!
\nLooks great. Yes, I had some problem with quality as well. We can improve it by:
or
\nI think that is possible to generate over (from my experiment) 90% of \"corrected\" maps for this dataset.
", "rawMarkdown": "Nice! Thank you for mentioning my work! \nLooks great. Yes, I had some problem with quality as well. We can improve it by:\n- better model - to be done in future - we can spend some time annotating data and creating specific model - I assume that ... there is no need to create big datset for this task\n\nor \n\n- heuristic for dealing with bad sailency maps ... it is possible as well (some combination of SoD and object detector)\n\nI think that is possible to generate over (from my experiment) 90% of \"corrected\" maps for this dataset.", "votes": null }, { "id": "1720816", "postDate": "03/13/2022 07:05:28", "content": "Not yet, but will do so once I'm able to produce even more fine-grained segmentation maps.
", "rawMarkdown": "Not yet, but will do so once I'm able to produce even more fine-grained segmentation maps.", "votes": null }, { "id": "1720817", "postDate": "03/13/2022 07:11:44", "content": "Thanks for your insights!
\nDid your accuracy for classification improve when using such a segmented version of the dataset instead of the raw dataset?
", "rawMarkdown": "Thanks for your insights!\n\nDid your accuracy for classification improve when using such a segmented version of the dataset instead of the raw dataset?", "votes": null }, { "id": "1720878", "postDate": "03/13/2022 08:23:40", "content": "I will test this week.
\nCurrently (now) I am generating new dataset based on new strategy. We will look if they works. If yes .. I will try with removing background. Still looking for optimal dataset baseline.
Looking forward to it!
\nAlways being a fan of your workπ
Thank you! Your work and support is great as well. πππ
", "rawMarkdown": "Thank you! Your work and support is great as well. πππ", "votes": null }, { "id": "1721135", "postDate": "03/13/2022 12:50:12", "content": "Thanks for creating and sharing this, Adnan!
\nI'm curious if it provides a boost-have you had the chance to try this in your pipeline?
", "rawMarkdown": "Thanks for creating and sharing this, Adnan! \n\nI'm curious if it provides a boost-have you had the chance to try this in your pipeline?", "votes": null }, { "id": "1721447", "postDate": "03/13/2022 17:32:09", "content": "I converted it and trained a model with small image size, LB only 0.49.
\nTry tuning model to see if It could be much better.
This is not model problem but β¦. data problem. Look into dataset and try to understand data.
", "rawMarkdown": "This is not model problem but .... data problem. Look into dataset and try to understand data.", "votes": null }, { "id": "1721800", "postDate": "03/14/2022 03:11:35", "content": "new one 0.569, still not good. val loss decreases much slower.
", "rawMarkdown": "new one 0.569, still not good. val loss decreases much slower.", "votes": null }, { "id": "1723026", "postDate": "03/15/2022 04:04:01", "content": "When I increase the img size, got NaN in loss and accuracy, still don't know why it's happens.
", "rawMarkdown": "When I increase the img size, got NaN in loss and accuracy, still don't know why it's happens.", "votes": null }, { "id": "1723422", "postDate": "03/15/2022 12:09:35", "content": "@dragonzhang I checked your notebook - this is dataset problem. I can see many problems but main is β¦ not resized ROI and β¦ there is no ROI on some samples.
", "rawMarkdown": "dragonzhang I checked your notebook - this is dataset problem. I can see many problems but main is ... not resized ROI and ... there is no ROI on some samples.", "votes": null }, { "id": "1723448", "postDate": "03/15/2022 12:28:42", "content": "Thanks for your reply. I haven't checked dataset yet.
", "rawMarkdown": "Thanks for your reply. I haven't checked dataset yet.", "votes": null }, { "id": "1723523", "postDate": "03/15/2022 13:48:16", "content": "That's true indeed @remekkinas @dragonzhang
\nSince some of the images from the source dataset are opaque, blurred and much noisy, The SOD didn't produce any tangible output leading to a dataset containing images that are completely blank.
This is why I wrote about heuristic in previous post. You can deal with both - resizing (I show how to deal with this in my SOD notebook - resize area of SOD) and then find blank (or small) areas.
", "rawMarkdown": "This is why I wrote about heuristic in previous post. You can deal with both - resizing (I show how to deal with this in my SOD notebook - resize area of SOD) and then find blank (or small) areas.", "votes": null }, { "id": "1729430", "postDate": "03/20/2022 04:44:45", "content": "great workπ
", "rawMarkdown": "great workπ", "votes": null }, { "id": "1730515", "postDate": "03/21/2022 11:31:12", "content": "Thankyou @tigerhuli
", "rawMarkdown": "Thankyou @tigerhuli", "votes": null } ], "comments": [ { "id": 1720417, "author_name": "dragonzhang", "author_url": "", "post_date": "03/12/2022 19:09:29", "content": "thanks for sharing. upvote. Have you create TFrecord format dataset?
", "votes": null, "replies": [ { "id": 1720816, "author_name": "adnanpen", "author_url": "", "post_date": "03/13/2022 07:05:28", "content": "Not yet, but will do so once I'm able to produce even more fine-grained segmentation maps.
", "votes": null, "replies": [] }, { "id": 1721447, "author_name": "dragonzhang", "author_url": "", "post_date": "03/13/2022 17:32:09", "content": "I converted it and trained a model with small image size, LB only 0.49.
\nTry tuning model to see if It could be much better.
This is not model problem but β¦. data problem. Look into dataset and try to understand data.
", "votes": null, "replies": [] }, { "id": 1721800, "author_name": "dragonzhang", "author_url": "", "post_date": "03/14/2022 03:11:35", "content": "new one 0.569, still not good. val loss decreases much slower.
", "votes": null, "replies": [] }, { "id": 1723026, "author_name": "locbaop", "author_url": "", "post_date": "03/15/2022 04:04:01", "content": "When I increase the img size, got NaN in loss and accuracy, still don't know why it's happens.
", "votes": null, "replies": [] }, { "id": 1723422, "author_name": "remekkinas", "author_url": "", "post_date": "03/15/2022 12:09:35", "content": "@dragonzhang I checked your notebook - this is dataset problem. I can see many problems but main is β¦ not resized ROI and β¦ there is no ROI on some samples.
", "votes": null, "replies": [] }, { "id": 1723448, "author_name": "dragonzhang", "author_url": "", "post_date": "03/15/2022 12:28:42", "content": "Thanks for your reply. I haven't checked dataset yet.
", "votes": null, "replies": [] }, { "id": 1723523, "author_name": "adnanpen", "author_url": "", "post_date": "03/15/2022 13:48:16", "content": "That's true indeed @remekkinas @dragonzhang
\nSince some of the images from the source dataset are opaque, blurred and much noisy, The SOD didn't produce any tangible output leading to a dataset containing images that are completely blank.
This is why I wrote about heuristic in previous post. You can deal with both - resizing (I show how to deal with this in my SOD notebook - resize area of SOD) and then find blank (or small) areas.
", "votes": null, "replies": [] } ] }, { "id": 1720507, "author_name": "remekkinas", "author_url": "", "post_date": "03/12/2022 21:26:51", "content": "Nice! Thank you for mentioning my work!
\nLooks great. Yes, I had some problem with quality as well. We can improve it by:
or
\nI think that is possible to generate over (from my experiment) 90% of \"corrected\" maps for this dataset.
", "votes": null, "replies": [ { "id": 1720817, "author_name": "adnanpen", "author_url": "", "post_date": "03/13/2022 07:11:44", "content": "Thanks for your insights!
\nDid your accuracy for classification improve when using such a segmented version of the dataset instead of the raw dataset?
", "votes": null, "replies": [] }, { "id": 1720878, "author_name": "remekkinas", "author_url": "", "post_date": "03/13/2022 08:23:40", "content": "I will test this week.
\nCurrently (now) I am generating new dataset based on new strategy. We will look if they works. If yes .. I will try with removing background. Still looking for optimal dataset baseline.
Looking forward to it!
\nAlways being a fan of your workπ
Thank you! Your work and support is great as well. πππ
", "votes": null, "replies": [] } ] }, { "id": 1721135, "author_name": "init27", "author_url": "", "post_date": "03/13/2022 12:50:12", "content": "Thanks for creating and sharing this, Adnan!
\nI'm curious if it provides a boost-have you had the chance to try this in your pipeline?
", "votes": null, "replies": [] }, { "id": 1729430, "author_name": "tigerhuli", "author_url": "", "post_date": "03/20/2022 04:44:45", "content": "great workπ
", "votes": null, "replies": [ { "id": 1730515, "author_name": "adnanpen", "author_url": "", "post_date": "03/21/2022 11:31:12", "content": "Thankyou @tigerhuli
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1720055": "**Created a dataset using the current State-of-the-Art Salient Object Detector - TRACER**\n\nConsidering the fact that the images presented in this dataset have a lot of variability in them, not just the fins and other body parts of marine life presented here, but also when it comes to background scenes.\n\nWith this insight, I lingered with the Idea that segmenting out the marine life from their background might prove to be a helpful step for the deep learning model when performing classification.\n\nThat's when I started searching out for the current best-open-sourced implementation of Salient Object Detector and came across [this awesome repo.](https://github.com/Karel911/TRACER)\nMade a few changes to generate output for my use case in [my forked repo.](https://github.com/adnan119/TRACER)\n\nAnd generated this [Dataset](https://www.kaggle.com/adnanpen/background-removed-happywhale-dataset)\n\n\nThe code I've used for generating this dataset can be found in this [Notebook](https://www.kaggle.com/adnanpen/background-removal-using-tracer-sota-sod)\n\n***Though still not perfect I hope this proves to be a starting point for someone to use and work on while I figure out ways to further improve this dataset.***\n\nSimilar work by @remekkinas : [background removal using U2-Net](https://www.kaggle.com/remekkinas/remove-background-salient-object-detection)", "1720417": "thanks for sharing. upvote. Have you create TFrecord format dataset?", "1720507": "Nice! Thank you for mentioning my work! \nLooks great. Yes, I had some problem with quality as well. We can improve it by:\n- better model - to be done in future - we can spend some time annotating data and creating specific model - I assume that ... there is no need to create big datset for this task\n\nor \n\n- heuristic for dealing with bad sailency maps ... it is possible as well (some combination of SoD and object detector)\n\nI think that is possible to generate over (from my experiment) 90% of \"corrected\" maps for this dataset.", "1720816": "Not yet, but will do so once I'm able to produce even more fine-grained segmentation maps.", "1720817": "Thanks for your insights!\n\nDid your accuracy for classification improve when using such a segmented version of the dataset instead of the raw dataset?", "1720878": "I will test this week.\nCurrently (now) I am generating new dataset based on new strategy. We will look if they works. If yes .. I will try with removing background. Still looking for optimal dataset baseline.", "1720880": "Looking forward to it!\nAlways being a fan of your workπ", "1720887": "Thank you! Your work and support is great as well. πππ", "1721135": "Thanks for creating and sharing this, Adnan! \n\nI'm curious if it provides a boost-have you had the chance to try this in your pipeline?", "1721447": "I converted it and trained a model with small image size, LB only 0.49.\nTry tuning model to see if It could be much better.", "1721473": "This is not model problem but .... data problem. Look into dataset and try to understand data.", "1721800": "new one 0.569, still not good. val loss decreases much slower.", "1723026": "When I increase the img size, got NaN in loss and accuracy, still don't know why it's happens.", "1723422": "dragonzhang I checked your notebook - this is dataset problem. I can see many problems but main is ... not resized ROI and ... there is no ROI on some samples.", "1723448": "Thanks for your reply. I haven't checked dataset yet.", "1723523": "That's true indeed @remekkinas @dragonzhang \nSince some of the images from the source dataset are opaque, blurred and much noisy, The SOD didn't produce any tangible output leading to a dataset containing images that are completely blank.", "1723533": "This is why I wrote about heuristic in previous post. You can deal with both - resizing (I show how to deal with this in my SOD notebook - resize area of SOD) and then find blank (or small) areas.", "1729430": "great workπ", "1730515": "Thankyou @tigerhuli" }, "source": "meta" }