Physics based unsupervised whale detector
\n\nHere is a small contribution to discussion on automated whale detection. Earlier posting on this topic included Lam Dang using R/B and R/G, and eduardofv using HSV histogram. Both are attempting to exploit the three color camera as a poor-man’s multi-spectral camera. I do also....
\n\nI start with some basic physics. The animal skin reflects red more than blue and green. Furthermore, the animal blocks blue and green upwelling radiance, which leads to a deficit of blue and green in pixels with animal below – and that makes animal pixel still redder.
\n\nHere are two papers for background, math, and further insights.
\n\nDennis Silva and Ron Abileah, Two Algorithms For Removing Ocean Surface Clutter in Multispectral and Hyperspectral Images. 1999. Download pdf from here: http://www.watercolumncorrection.com/documents/SilvaandAbileah1999_TWO_ALGORITHMS_FOR_REMOVING_OCEAN_SURFACE_CLUTTER.pdf
\n\nRon Abileah, Marine mammal census using space satellite imagery. U.S. Navy Journal of Underwater Acoustics 52 (3) July 2002. Download pdf from here: http://abileah.com/DigitalOcean/Publications/Password_protected_publications/Abileah%202005%20JUA%20paper%20-%20marine%20mammals.pdf
\n\nThe algorithm is: (1) compute a likelihood score based on blue and green deficit in each pixel; (2) threshold into b&w; (3) cluster b&w regions with Matlab regionprops function; (4) find region with largest Area; (5) Use regionpropos properties major/minor axes, centroid, and orientation to draw an ellipse around the animal.
\n\nIn attachments are half dozen examples. So far I have run the algorithm on about 100 images. Success rate is 99%. (One miss was a fluke, case No. 284, included in attachments) with trash and surfactants floating on ocean surface. This would have blocked upwelling radiance just as a whale would.)
\n\nThe earlier posts with R/B and HSV histograms are probably getting at the same physics indirectly. My approach is more rigorous but not necessarily better. If any team wishes to incorporate this approach... go for it.
", "messages": [ { "id": "101462", "postDate": "12/15/2015 14:15:19", "content": "Physics based unsupervised whale detector
\n\nHere is a small contribution to discussion on automated whale detection. Earlier posting on this topic included Lam Dang using R/B and R/G, and eduardofv using HSV histogram. Both are attempting to exploit the three color camera as a poor-man’s multi-spectral camera. I do also....
\n\nI start with some basic physics. The animal skin reflects red more than blue and green. Furthermore, the animal blocks blue and green upwelling radiance, which leads to a deficit of blue and green in pixels with animal below – and that makes animal pixel still redder.
\n\nHere are two papers for background, math, and further insights.
\n\nDennis Silva and Ron Abileah, Two Algorithms For Removing Ocean Surface Clutter in Multispectral and Hyperspectral Images. 1999. Download pdf from here: http://www.watercolumncorrection.com/documents/SilvaandAbileah1999_TWO_ALGORITHMS_FOR_REMOVING_OCEAN_SURFACE_CLUTTER.pdf
\n\nRon Abileah, Marine mammal census using space satellite imagery. U.S. Navy Journal of Underwater Acoustics 52 (3) July 2002. Download pdf from here: http://abileah.com/DigitalOcean/Publications/Password_protected_publications/Abileah%202005%20JUA%20paper%20-%20marine%20mammals.pdf
\n\nThe algorithm is: (1) compute a likelihood score based on blue and green deficit in each pixel; (2) threshold into b&w; (3) cluster b&w regions with Matlab regionprops function; (4) find region with largest Area; (5) Use regionpropos properties major/minor axes, centroid, and orientation to draw an ellipse around the animal.
\n\nIn attachments are half dozen examples. So far I have run the algorithm on about 100 images. Success rate is 99%. (One miss was a fluke, case No. 284, included in attachments) with trash and surfactants floating on ocean surface. This would have blocked upwelling radiance just as a whale would.)
\n\nThe earlier posts with R/B and HSV histograms are probably getting at the same physics indirectly. My approach is more rigorous but not necessarily better. If any team wishes to incorporate this approach... go for it.
", "rawMarkdown": "Physics based unsupervised whale detector\r\n\r\nHere is a small contribution to discussion on automated whale detection. Earlier posting on this topic included Lam Dang using R/B and R/G, and eduardofv using HSV histogram. Both are attempting to exploit the three color camera as a poor-man’s multi-spectral camera. I do also....\r\n\r\nI start with some basic physics. The animal skin reflects red more than blue and green. Furthermore, the animal blocks blue and green upwelling radiance, which leads to a deficit of blue and green in pixels with animal below – and that makes animal pixel still redder.\r\n\r\nHere are two papers for background, math, and further insights. \r\n\r\nDennis Silva and Ron Abileah, Two Algorithms For Removing Ocean Surface Clutter in Multispectral and Hyperspectral Images. 1999. Download pdf from here: http://www.watercolumncorrection.com/documents/SilvaandAbileah1999_TWO_ALGORITHMS_FOR_REMOVING_OCEAN_SURFACE_CLUTTER.pdf\r\n\r\nRon Abileah, Marine mammal census using space satellite imagery. U.S. Navy Journal of Underwater Acoustics 52 (3) July 2002. Download pdf from here: http://abileah.com/DigitalOcean/Publications/Password_protected_publications/Abileah%202005%20JUA%20paper%20-%20marine%20mammals.pdf\r\n\r\nThe algorithm is: (1) compute a likelihood score based on blue and green deficit in each pixel; (2) threshold into b&w; (3) cluster b&w regions with Matlab regionprops function; (4) find region with largest Area; (5) Use regionpropos properties major/minor axes, centroid, and orientation to draw an ellipse around the animal.\r\n\r\nIn attachments are half dozen examples. So far I have run the algorithm on about 100 images. Success rate is 99%. (One miss was a fluke, case No. 284, included in attachments) with trash and surfactants floating on ocean surface. This would have blocked upwelling radiance just as a whale would.)\r\n\r\nThe earlier posts with R/B and HSV histograms are probably getting at the same physics indirectly. My approach is more rigorous but not necessarily better. If any team wishes to incorporate this approach... go for it.", "votes": null }, { "id": "102337", "postDate": "12/21/2015 17:40:34", "content": "Great to learn a different approach! Thnx!
", "rawMarkdown": "Great to learn a different approach! Thnx!", "votes": null }, { "id": "102743", "postDate": "12/25/2015 08:13:54", "content": "Hi Ron, \nthanks for sharing your method and the papers. \nI was wondering if you would like to team up for this competition ? \nI can work on feature extraction and the machine learning aspects. \nIf so, my email address is: mpsampat@gmail.com. I am based in sunnyvale. ca). \nRegards\nMehul
", "rawMarkdown": "Hi Ron, \r\nthanks for sharing your method and the papers. \r\nI was wondering if you would like to team up for this competition ? \r\nI can work on feature extraction and the machine learning aspects. \r\nIf so, my email address is: mpsampat@gmail.com. I am based in sunnyvale. ca). \r\nRegards\r\nMehul", "votes": null }, { "id": "103563", "postDate": "01/04/2016 11:12:02", "content": "Hi,
\n\nInteresting approach! I took a glimpse at the papers. I also work with satellite data. There are some pitfalls, like: the camera have only 3 bands; the bandwidth is large for the camera while for the satellite is narrow; the data products usually contain measurements in some physical units while the photo camera applies some transformations like white balance, giving some numerical values that can't be easily converted in physical units.
\n\nI would like to see how well these methods distinguish between the sun glare/water foam and the whale head spots. This would be a true non trivial jump forward!
\n\nKeep up the good work!
", "rawMarkdown": "Hi, \r\n\r\nInteresting approach! I took a glimpse at the papers. I also work with satellite data. There are some pitfalls, like: the camera have only 3 bands; the bandwidth is large for the camera while for the satellite is narrow; the data products usually contain measurements in some physical units while the photo camera applies some transformations like white balance, giving some numerical values that can't be easily converted in physical units.\r\n\r\nI would like to see how well these methods distinguish between the sun glare/water foam and the whale head spots. This would be a true non trivial jump forward!\r\n\r\nKeep up the good work!", "votes": null }, { "id": "103641", "postDate": "01/05/2016 04:39:00", "content": "Hello visoft,
\n\nI am in complete agreement. RGB camera is a (very) poor-man's multi-spectral sensor. There are all the limitation you mention and more.
\n\nI wish they had collected those images in raw mode. That would have helped in three ways: no white balance; no jpeg compression artifacts; and maybe 11 bit in dynamic range instead of 8 bits.
\n\nBut that said there is still useful color information in the images they supplied for the Kaggle challenge. I put together a simple whale detector based on just red excess - no smarts about shape. See attached w_185.jpg for example. The method seems to be 95%+ accurate. If anyone wants to check that claim, there are 288 more examples in attached zip file.
\n\nRon
", "rawMarkdown": "Hello visoft,\r\n\r\nI am in complete agreement. RGB camera is a (very) poor-man's multi-spectral sensor. There are all the limitation you mention and more. \r\n\r\nI wish they had collected those images in raw mode. That would have helped in three ways: no white balance; no jpeg compression artifacts; and maybe 11 bit in dynamic range instead of 8 bits.\r\n\r\nBut that said there is still useful color information in the images they supplied for the Kaggle challenge. I put together a simple whale detector based on just red excess - no smarts about shape. See attached w_185.jpg for example. The method seems to be 95%+ accurate. If anyone wants to check that claim, there are 288 more examples in attached zip file. \r\n\r\nRon", "votes": null } ], "comments": [ { "id": 102337, "author_name": "eduardofv", "author_url": "", "post_date": "12/21/2015 17:40:34", "content": "Great to learn a different approach! Thnx!
", "votes": null, "replies": [] }, { "id": 102743, "author_name": "mpsampat", "author_url": "", "post_date": "12/25/2015 08:13:54", "content": "Hi Ron, \nthanks for sharing your method and the papers. \nI was wondering if you would like to team up for this competition ? \nI can work on feature extraction and the machine learning aspects. \nIf so, my email address is: mpsampat@gmail.com. I am based in sunnyvale. ca). \nRegards\nMehul
", "votes": null, "replies": [] }, { "id": 103563, "author_name": "visoft", "author_url": "", "post_date": "01/04/2016 11:12:02", "content": "Hi,
\n\nInteresting approach! I took a glimpse at the papers. I also work with satellite data. There are some pitfalls, like: the camera have only 3 bands; the bandwidth is large for the camera while for the satellite is narrow; the data products usually contain measurements in some physical units while the photo camera applies some transformations like white balance, giving some numerical values that can't be easily converted in physical units.
\n\nI would like to see how well these methods distinguish between the sun glare/water foam and the whale head spots. This would be a true non trivial jump forward!
\n\nKeep up the good work!
", "votes": null, "replies": [] }, { "id": 103641, "author_name": "abileah", "author_url": "", "post_date": "01/05/2016 04:39:00", "content": "Hello visoft,
\n\nI am in complete agreement. RGB camera is a (very) poor-man's multi-spectral sensor. There are all the limitation you mention and more.
\n\nI wish they had collected those images in raw mode. That would have helped in three ways: no white balance; no jpeg compression artifacts; and maybe 11 bit in dynamic range instead of 8 bits.
\n\nBut that said there is still useful color information in the images they supplied for the Kaggle challenge. I put together a simple whale detector based on just red excess - no smarts about shape. See attached w_185.jpg for example. The method seems to be 95%+ accurate. If anyone wants to check that claim, there are 288 more examples in attached zip file.
\n\nRon
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "101462": "Physics based unsupervised whale detector\r\n\r\nHere is a small contribution to discussion on automated whale detection. Earlier posting on this topic included Lam Dang using R/B and R/G, and eduardofv using HSV histogram. Both are attempting to exploit the three color camera as a poor-man’s multi-spectral camera. I do also....\r\n\r\nI start with some basic physics. The animal skin reflects red more than blue and green. Furthermore, the animal blocks blue and green upwelling radiance, which leads to a deficit of blue and green in pixels with animal below – and that makes animal pixel still redder.\r\n\r\nHere are two papers for background, math, and further insights. \r\n\r\nDennis Silva and Ron Abileah, Two Algorithms For Removing Ocean Surface Clutter in Multispectral and Hyperspectral Images. 1999. Download pdf from here: http://www.watercolumncorrection.com/documents/SilvaandAbileah1999_TWO_ALGORITHMS_FOR_REMOVING_OCEAN_SURFACE_CLUTTER.pdf\r\n\r\nRon Abileah, Marine mammal census using space satellite imagery. U.S. Navy Journal of Underwater Acoustics 52 (3) July 2002. Download pdf from here: http://abileah.com/DigitalOcean/Publications/Password_protected_publications/Abileah%202005%20JUA%20paper%20-%20marine%20mammals.pdf\r\n\r\nThe algorithm is: (1) compute a likelihood score based on blue and green deficit in each pixel; (2) threshold into b&w; (3) cluster b&w regions with Matlab regionprops function; (4) find region with largest Area; (5) Use regionpropos properties major/minor axes, centroid, and orientation to draw an ellipse around the animal.\r\n\r\nIn attachments are half dozen examples. So far I have run the algorithm on about 100 images. Success rate is 99%. (One miss was a fluke, case No. 284, included in attachments) with trash and surfactants floating on ocean surface. This would have blocked upwelling radiance just as a whale would.)\r\n\r\nThe earlier posts with R/B and HSV histograms are probably getting at the same physics indirectly. My approach is more rigorous but not necessarily better. If any team wishes to incorporate this approach... go for it.", "102337": "Great to learn a different approach! Thnx!", "102743": "Hi Ron, \r\nthanks for sharing your method and the papers. \r\nI was wondering if you would like to team up for this competition ? \r\nI can work on feature extraction and the machine learning aspects. \r\nIf so, my email address is: mpsampat@gmail.com. I am based in sunnyvale. ca). \r\nRegards\r\nMehul", "103563": "Hi, \r\n\r\nInteresting approach! I took a glimpse at the papers. I also work with satellite data. There are some pitfalls, like: the camera have only 3 bands; the bandwidth is large for the camera while for the satellite is narrow; the data products usually contain measurements in some physical units while the photo camera applies some transformations like white balance, giving some numerical values that can't be easily converted in physical units.\r\n\r\nI would like to see how well these methods distinguish between the sun glare/water foam and the whale head spots. This would be a true non trivial jump forward!\r\n\r\nKeep up the good work!", "103641": "Hello visoft,\r\n\r\nI am in complete agreement. RGB camera is a (very) poor-man's multi-spectral sensor. There are all the limitation you mention and more. \r\n\r\nI wish they had collected those images in raw mode. That would have helped in three ways: no white balance; no jpeg compression artifacts; and maybe 11 bit in dynamic range instead of 8 bits.\r\n\r\nBut that said there is still useful color information in the images they supplied for the Kaggle challenge. I put together a simple whale detector based on just red excess - no smarts about shape. See attached w_185.jpg for example. The method seems to be 95%+ accurate. If anyone wants to check that claim, there are 288 more examples in attached zip file. \r\n\r\nRon" }, "source": "meta" }