Welcome to the competition! My name is Christin Khan and I am a fishery biologist at NOAA working on right whale issues. If you have any general questions about right whales, I am happy to answer them here.
\n\nThank you for your participation, this project has real conservation value for the species!
", "messages": [ { "id": "90645", "postDate": "08/28/2015 14:14:26", "content": "Welcome to the competition! My name is Christin Khan and I am a fishery biologist at NOAA working on right whale issues. If you have any general questions about right whales, I am happy to answer them here.
\n\nThank you for your participation, this project has real conservation value for the species!
", "rawMarkdown": "Welcome to the competition! My name is Christin Khan and I am a fishery biologist at NOAA working on right whale issues. If you have any general questions about right whales, I am happy to answer them here. \r\n\r\nThank you for your participation, this project has real conservation value for the species!", "votes": null }, { "id": "91208", "postDate": "09/01/2015 16:13:19", "content": "Really nice competition. Also in the future for better performance results maybe an infrared camera/thermal cameras would be more useful or add features for the models. But in general, very nice competition. Thanks for this chance.
", "rawMarkdown": "Really nice competition. Also in the future for better performance results maybe an infrared camera/thermal cameras would be more useful or add features for the models. But in general, very nice competition. Thanks for this chance.", "votes": null }, { "id": "91236", "postDate": "09/01/2015 17:54:17", "content": "And now we have a better than random guess result on the leaderboard...sooner than I expected.
", "rawMarkdown": "And now we have a better than random guess result on the leaderboard...sooner than I expected.", "votes": null }, { "id": "91245", "postDate": "09/01/2015 18:30:04", "content": "[quote=James King;91236]
\n\nAnd now we have a better than random guess result on the leaderboard...sooner than I expected.
\n\n[/quote]
\n\nI always hate when my algorithm does worse than a random guess. :-)
", "rawMarkdown": "[quote=James King;91236]\r\n\r\nAnd now we have a better than random guess result on the leaderboard...sooner than I expected.\r\n\r\n[/quote]\r\n\r\nI always hate when my algorithm does worse than a random guess. :-)", "votes": null }, { "id": "91262", "postDate": "09/01/2015 20:02:58", "content": "I suspect it takes some data prep to beat random guessing. I plan to start with whale_95370 and try to distinguish it from all other whales. His facial markings are quite distinctive.
\n\nBut I think I need to first discard the images where the face cannot be seen.
", "rawMarkdown": "I suspect it takes some data prep to beat random guessing. I plan to start with whale_95370 and try to distinguish it from all other whales. His facial markings are quite distinctive.\r\n\r\n![enter image description here][1]\r\n\r\n\r\n [1]: https://mail.google.com/mail/u/0/?ui=2&ik=24cefad02d&view=fimg&th=14f8601a77d96c94&attid=0.1&disp=emb&realattid=ii_14f8600e943bba3e&attbid=ANGjdJ8GxwCU9Em4jkr8r7ZI_82DPKgAjlFsGvynBc0zlAp9ayhnIxrwTKKqlOs9wLJYOqgA6XDaOkYZ5VE6oYhiX6iiA8KWzIwW5mNctvUqZYL81vdFW19jyx44nxU&sz=w460-h254&ats=1441137585574&rm=14f8601a77d96c94&zw&atsh=1\r\n\r\nBut I think I need to first discard the images where the face cannot be seen.", "votes": null }, { "id": "91273", "postDate": "09/01/2015 21:24:36", "content": "Doesn't look like the same whale (95370) to me. Do those white thinger-doodles change over time, e.g., so that one of the images was from a younger whale?
\n\nw_3076
\n\n![\"enter](\"http://i.imgur.com/ocAN22n.png\")
w_3085
\n\n![\"enter](\"http://i.imgur.com/m1RhU22.png\")
Very good question inversion, it looks as if the bottom one is from a younger whale before the line was fully connected. Or maybe it had a fight and created issues with his/her lines.
", "rawMarkdown": "Very good question inversion, it looks as if the bottom one is from a younger whale before the line was fully connected. Or maybe it had a fight and created issues with his/her lines.", "votes": null }, { "id": "91379", "postDate": "09/02/2015 17:59:53", "content": "The white markings on top of the whale's head are called callosities and they do change very slightly over time, particularly when the whale is still young, so it is reasonable to expect slight deviations such as this from one photograph to another. On the other hand, there are some whales that look very very similar to each other. For more background on whale matching, visit these websites:
\n\n\n\nand
\n\nhttp://rwcatalog.neaq.org/Default.aspx
", "rawMarkdown": "The white markings on top of the whale's head are called callosities and they do change very slightly over time, particularly when the whale is still young, so it is reasonable to expect slight deviations such as this from one photograph to another. On the other hand, there are some whales that look very very similar to each other. For more background on whale matching, visit these websites:\r\n\r\nhttp://www.neaq.org/education_and_activities/games_and_activities/online_games/right_whale_identification_games.php\r\n\r\nand \r\n\r\nhttp://rwcatalog.neaq.org/Default.aspx", "votes": null }, { "id": "91380", "postDate": "09/02/2015 18:05:26", "content": "It may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head. Right whales are first divided into two broad categories to describe the overall appearance of the main section of the callosity- continuous and broken. Next we usually discuss how many islands or peninsulas there are and on which side of the head. More subtle but important cues are the presence and size of lip callosities, and how many post blowhole and chin callosities there are.
\n\n![\"Sketch](\"https://teacheratsea.files.wordpress.com/2015/05/img_2292.jpg\")
[quote=Christin B. Khan;91380]\nIt may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head.\n[/quote]
\n\nSo, you're saying "thinger-doodles" isn't part of the marine biology vernacular? :-)
", "rawMarkdown": "[quote=Christin B. Khan;91380]\r\nIt may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head.\r\n[/quote]\r\n\r\nSo, you're saying \"thinger-doodles\" isn't part of the marine biology vernacular? :-)", "votes": null }, { "id": "91396", "postDate": "09/02/2015 19:04:47", "content": ""thinger-doodles" gave me quite a smile :-)
", "rawMarkdown": "\"thinger-doodles\" gave me quite a smile :-)", "votes": null }, { "id": "91417", "postDate": "09/02/2015 22:08:14", "content": "[quote=Christin B. Khan;91380]
\n\nIt may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head.
\n\n[/quote]
\n\nSo, Christin, are the callosity patterns the main feature used to distinguish right whales from other whale species? Are there other features that are useful in distinguishing a right whale from another type of whale?
", "rawMarkdown": "[quote=Christin B. Khan;91380]\r\n\r\nIt may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head. \r\n\r\n[/quote]\r\n\r\nSo, Christin, are the callosity patterns the main feature used to distinguish right whales from other whale species? Are there other features that are useful in distinguishing a right whale from another type of whale?", "votes": null }, { "id": "91432", "postDate": "09/03/2015 03:21:06", "content": "Well they have the twin blowholes (sort of a blow-nostril) as you see in the photos, and no dorsal fin. Also the male right whale's whalehood is without peer in the animal kingdom, although you usually would not be able to notice this from above the water.
\n\nhttps://en.wikipedia.org/wiki/Right_whale
", "rawMarkdown": "Well they have the twin blowholes (sort of a blow-nostril) as you see in the photos, and no dorsal fin. Also the male right whale's whalehood is without peer in the animal kingdom, although you usually would not be able to notice this from above the water.\r\n\r\nhttps://en.wikipedia.org/wiki/Right_whale", "votes": null }, { "id": "91439", "postDate": "09/03/2015 04:28:50", "content": "Is the whale in w_8026.jpg pooping?
", "rawMarkdown": "Is the whale in w_8026.jpg pooping?", "votes": null }, { "id": "91455", "postDate": "09/03/2015 07:22:03", "content": "@KA hahaha, I cannot control my laugh :) Out of all these data, you're focused on this :)
", "rawMarkdown": "KA hahaha, I cannot control my laugh :) Out of all these data, you're focused on this :)", "votes": null }, { "id": "91479", "postDate": "09/03/2015 12:43:36", "content": "@Matt Wenger - Here are some pages that talk about how to identify the whales. It is mainly based on the callosity pattern, but at the bottom of the page is a link to other identifying features.
", "rawMarkdown": "Matt Wenger - [Here](http://www.neaq.org/conservation_and_research/projects/endangered_species_habitats/right_whale_research/right_whale_projects/monitoring_individuals_and_family_trees/identifying_with_photographs/how_it_works/callosity_patterns.php) are some pages that talk about how to identify the whales. It is mainly based on the callosity pattern, but at the bottom of the page is a link to other identifying features.", "votes": null }, { "id": "91504", "postDate": "09/03/2015 18:26:22", "content": "@Kevin Burnham - Thanks! This is very useful.
", "rawMarkdown": "Kevin Burnham - Thanks! This is very useful.", "votes": null }, { "id": "91518", "postDate": "09/03/2015 21:11:35", "content": "@Matt Wenger - right whale are identified from other whale species primarily by:
\n\n"Is the whale in w_8026.jpg pooping?"
\n\n@KA - yes! nice find! We marine biologists refer to it as defecation
", "rawMarkdown": "\"Is the whale in w_8026.jpg pooping?\"\r\n\r\n@KA - yes! nice find! We marine biologists refer to it as defecation", "votes": null }, { "id": "91522", "postDate": "09/03/2015 21:15:49", "content": "[quote=KA;91439]
\n\nIs the whale in w_8026.jpg pooping?
\n\n[/quote]
\n\nhahahahahhahaha
", "rawMarkdown": "[quote=KA;91439]\r\n\r\nIs the whale in w_8026.jpg pooping?\r\n\r\n[/quote]\r\n\r\nhahahahahhahaha", "votes": null }, { "id": "91523", "postDate": "09/03/2015 21:16:22", "content": "[quote=Christin B. Khan;91519]
\n\n"Is the whale in w_8026.jpg pooping?"
\n\n@KA - yes! nice find! We marine biologists refer to it as defecation
\n\n[/quote]
\n\neven a bigger hahahahahahahah
", "rawMarkdown": "[quote=Christin B. Khan;91519]\r\n\r\n\"Is the whale in w_8026.jpg pooping?\"\r\n\r\n@KA - yes! nice find! We marine biologists refer to it as defecation\r\n\r\n[/quote]\r\n\r\neven a bigger hahahahahahahah", "votes": null }, { "id": "91647", "postDate": "09/05/2015 13:18:50", "content": "I have a question.
\n\nYou provide MATLAB license to students for this competition.
\nhttp://jp.mathworks.com/academia/student-competitions/kaggle/
Is this license only used by students ?
\nI'm not a student.
\nCan't I use MATLAB without buying business license ?
@toshi_k - check out this post. It says that ALL participants, without exception, are able to get a free copy of the software.
", "rawMarkdown": "toshi_k - check out [this post](https://www.kaggle.com/c/noaa-right-whale-recognition/forums/t/16328/python-alternative-to-labelling-tool/91625#post91625). It says that ALL participants, without exception, are able to get a free copy of the software.", "votes": null }, { "id": "91649", "postDate": "09/05/2015 13:33:34", "content": "Thank you !!
\nI'll request the license for more improvement :-)
With the images, has a person looked at each image independently and been able to identify the whale, or has the whale been identified in one of a given set and then all the photos from the set been given the label. This would help establish the upper-bound accuracy we might expect for the task given these scientists are our expert level annotators :)
\n\nIf it's the former, people are scarily good at identifying whales. During annotation I sometimes have trouble working out where the head is amongst the blowhole's spray!
", "rawMarkdown": "With the images, has a person looked at each image independently and been able to identify the whale, or has the whale been identified in one of a given set and then all the photos from the set been given the label. This would help establish the upper-bound accuracy we might expect for the task given these scientists are our expert level annotators :)\r\n\r\nIf it's the former, people are scarily good at identifying whales. During annotation I sometimes have trouble working out where the head is amongst the blowhole's spray!", "votes": null }, { "id": "92901", "postDate": "09/18/2015 14:43:16", "content": "[quote=Smerity;92826]
\n\nWith the images, has a person looked at each image independently and been able to identify the whale, or has the whale been identified in one of a given set and then all the photos from the set been given the label. This would help establish the upper-bound accuracy we might expect for the task given these scientists are our expert level annotators :)
\n\nIf it's the former, people are scarily good at identifying whales. During annotation I sometimes have trouble working out where the head is amongst the blowhole's spray!
\n\n[/quote]
\n\nSmerity, it is somewhere in between. Each image is independently inspected and the matched to the particular whale or whales in the photo (for the sake of the competition, we have provided images with only one whale but reality is usually a bit more messy). However, the context of a series of photographs is definitely taken into account, so when you are circling over one whale and take lots of photographs, they will be checked to verify that it's all the same whale but it's not like each image is separately and matched independently. More often we are circling over a group of whales and take a bunch of images, so that when you look at the next image, you are looking to see who in the group that you have already identified and who is new.
\n\nHowever, taken out of context such as you have in the competition, researchers can and do match each whale. For some experienced researchers and distinctive whales, this is done on sight; for other less experienced folks and/or whales that look more generic, the process can take quite a bit of time and painstaking comparison to other photographs.
", "rawMarkdown": "[quote=Smerity;92826]\r\n\r\nWith the images, has a person looked at each image independently and been able to identify the whale, or has the whale been identified in one of a given set and then all the photos from the set been given the label. This would help establish the upper-bound accuracy we might expect for the task given these scientists are our expert level annotators :)\r\n\r\nIf it's the former, people are scarily good at identifying whales. During annotation I sometimes have trouble working out where the head is amongst the blowhole's spray!\r\n\r\n[/quote]\r\n\r\nSmerity, it is somewhere in between. Each image is independently inspected and the matched to the particular whale or whales in the photo (for the sake of the competition, we have provided images with only one whale but reality is usually a bit more messy). However, the context of a series of photographs is definitely taken into account, so when you are circling over one whale and take lots of photographs, they will be checked to verify that it's all the same whale but it's not like each image is separately and matched independently. More often we are circling over a group of whales and take a bunch of images, so that when you look at the next image, you are looking to see who in the group that you have already identified and who is new. \r\n\r\nHowever, taken out of context such as you have in the competition, researchers can and do match each whale. For some experienced researchers and distinctive whales, this is done on sight; for other less experienced folks and/or whales that look more generic, the process can take quite a bit of time and painstaking comparison to other photographs.", "votes": null }, { "id": "93011", "postDate": "09/20/2015 02:40:17", "content": "Hi.
\n\nI am a high school student interested in using the data you have provided for this competition for a research project to submit to a high school science fair (ISTS); I will only submit my research/analysis and will not disseminate the actual data in my research paper. Is this allowed? My project will be a machine learning/image recognition system (basically what the competition is also asking for), and my research paper/materials will only talk about my system and the results I gleaned. I will not use the data for any other purposes.
\n\nI would really appreciate your letting me use the data; I really would love to enter the science fair and contribute to the whale image recognition sphere.
\n\nThank you.
", "rawMarkdown": "Hi.\r\n\r\nI am a high school student interested in using the data you have provided for this competition for a research project to submit to a high school science fair (ISTS); I will only submit my research/analysis and will not disseminate the actual data in my research paper. Is this allowed? My project will be a machine learning/image recognition system (basically what the competition is also asking for), and my research paper/materials will only talk about my system and the results I gleaned. I will not use the data for any other purposes.\r\n\r\nI would really appreciate your letting me use the data; I really would love to enter the science fair and contribute to the whale image recognition sphere.\r\n\r\nThank you.", "votes": null }, { "id": "93013", "postDate": "09/20/2015 04:18:29", "content": "[quote=Tanzim Hassan;93011]
\n\nHi.
\n\nI am a high school student interested in using the data you have provided for this competition for a research project to submit to a high school science fair (ISTS); I will only submit my research/analysis and will not disseminate the actual data in my research paper. Is this allowed? My project will be a machine learning/image recognition system (basically what the competition is also asking for), and my research paper/materials will only talk about my system and the results I gleaned. I will not use the data for any other purposes.
\n\nI would really appreciate your letting me use the data; I really would love to enter the science fair and contribute to the whale image recognition sphere.
\n\nThank you.
\n\n[/quote]
\n\nTanzim,
\n\nFrom the perspective of NOAA Fisheries, it should be fine, although it would be helpful if you could email me with a brief description of your project so that we can confirm permission for you - christin.khan@noaa.gov. However, you should confirm permissions with the Kaggle folks. If you can email me, we'll get it sorted on Monday or Tuesday for you.
", "rawMarkdown": "[quote=Tanzim Hassan;93011]\r\n\r\nHi.\r\n\r\nI am a high school student interested in using the data you have provided for this competition for a research project to submit to a high school science fair (ISTS); I will only submit my research/analysis and will not disseminate the actual data in my research paper. Is this allowed? My project will be a machine learning/image recognition system (basically what the competition is also asking for), and my research paper/materials will only talk about my system and the results I gleaned. I will not use the data for any other purposes.\r\n\r\nI would really appreciate your letting me use the data; I really would love to enter the science fair and contribute to the whale image recognition sphere.\r\n\r\nThank you.\r\n\r\n[/quote]\r\n\r\nTanzim,\r\n\r\nFrom the perspective of NOAA Fisheries, it should be fine, although it would be helpful if you could email me with a brief description of your project so that we can confirm permission for you - christin.khan@noaa.gov. However, you should confirm permissions with the Kaggle folks. If you can email me, we'll get it sorted on Monday or Tuesday for you.", "votes": null }, { "id": "93076", "postDate": "09/21/2015 07:56:38", "content": "Hi,
\n\nSo I wanted to ask a question about how the organizers would tackle the problem of "over annotating" the whale pictures. By this I mean that it is possible to literally annotate all of the images (it's only about 12k images and I can annotate about 1k per hour, so roughly 2 days for this). And this includes the evaluation images as well. Thus than one can proceed with a model with high enough capacity which to learn to detect the whale heads correct on the images provided (not necessarily on new images though). Even it is possible to just manually crop them and directly train on the cropped images. Do you care about this, or it is not so important part of the problem. How would you "judge" this part in the final submition (if at all) are you gona run the code (and potentially not allow to use more than X annotated images).
\n\nBest regards
", "rawMarkdown": "Hi,\r\n\r\nSo I wanted to ask a question about how the organizers would tackle the problem of \"over annotating\" the whale pictures. By this I mean that it is possible to literally annotate all of the images (it's only about 12k images and I can annotate about 1k per hour, so roughly 2 days for this). And this includes the evaluation images as well. Thus than one can proceed with a model with high enough capacity which to learn to detect the whale heads correct on the images provided (not necessarily on new images though). Even it is possible to just manually crop them and directly train on the cropped images. Do you care about this, or it is not so important part of the problem. How would you \"judge\" this part in the final submition (if at all) are you gona run the code (and potentially not allow to use more than X annotated images). \r\n\r\nBest regards", "votes": null }, { "id": "93078", "postDate": "09/21/2015 08:33:17", "content": "[quote=Christin B. Khan;92901]
\n\nmerity, it is somewhere in between. Each image is independently inspected and the matched to the particular whale or whales in the photo (for the sake of the competition, we have provided images with only one whale but reality is usually a bit more messy).
\n[/quote]\nJust want to point out that could actually have some importance to the competition, as I guess that some whale species may have more gregarious behaviours than others. Some individuals, too.
Hi all!
\n\n1) I have a question about w_154: The spots on the back are a feature or a bug? (I.e. I can recognize this whale by those spots OR they are just some temporary artifacts?)
\n\n2) The head is always nicely visible? Ex: you tried to include only "nice" pictures in the set, or you randomly sampled all your whale database?
\n\n3) Did I understand correctly? This is about recognizing some specific whales in the pictures? Ex: In w_154 is whale ID 3342, but we call it John! In the next picture is Joseph! So on so forth . . .
\n\n4) It seems that other features, like fins or tails or whale dimensions are harder see/get. Is it true or I didn't look at enough samples?
\n\n5) If 4 is correct, why is the discussion about the whale species relevant?
\n\nThanks!
", "rawMarkdown": "Hi all!\r\n\r\n1) I have a question about w_154: The spots on the back are a feature or a bug? (I.e. I can recognize this whale by those spots OR they are just some temporary artifacts?)\r\n\r\n2) The head is always nicely visible? Ex: you tried to include only \"nice\" pictures in the set, or you randomly sampled all your whale database?\r\n\r\n3) Did I understand correctly? This is about recognizing some specific whales in the pictures? Ex: In w_154 is whale ID 3342, but we call it John! In the next picture is Joseph! So on so forth . . .\r\n\r\n4) It seems that other features, like fins or tails or whale dimensions are harder see/get. Is it true or I didn't look at enough samples?\r\n\r\n5) If 4 is correct, why is the discussion about the whale species relevant?\r\n\r\nThanks!", "votes": null }, { "id": "93125", "postDate": "09/21/2015 19:10:49", "content": "[quote=WhizWilde;93078]
\n\n[quote=Christin B. Khan;92901]
\n\nmerity, it is somewhere in between. Each image is independently inspected and the matched to the particular whale or whales in the photo (for the sake of the competition, we have provided images with only one whale but reality is usually a bit more messy).
\n[/quote]\nJust want to point out that could actually have some importance to the competition, as I guess that some whale species may have more gregarious behaviours than others. Some individuals, too.
[/quote]
\n\nFor this competition, it is only images with one right whale in the photo
", "rawMarkdown": "[quote=WhizWilde;93078]\r\n\r\n[quote=Christin B. Khan;92901]\r\n\r\nmerity, it is somewhere in between. Each image is independently inspected and the matched to the particular whale or whales in the photo (for the sake of the competition, we have provided images with only one whale but reality is usually a bit more messy). \r\n[/quote]\r\nJust want to point out that could actually have some importance to the competition, as I guess that some whale species may have more gregarious behaviours than others. Some individuals, too.\r\n\r\n[/quote]\r\n\r\nFor this competition, it is only images with one right whale in the photo", "votes": null }, { "id": "93126", "postDate": "09/21/2015 19:17:23", "content": "[quote=visoft;93093]
\n\nHi all!
\n\n1) I have a question about w_154: The spots on the back are a feature or a bug? (I.e. I can recognize this whale by those spots OR they are just some temporary artifacts?)
\n\n2) The head is always nicely visible? Ex: you tried to include only "nice" pictures in the set, or you randomly sampled all your whale database?
\n\n3) Did I understand correctly? This is about recognizing some specific whales in the pictures? Ex: In w_154 is whale ID 3342, but we call it John! In the next picture is Joseph! So on so forth . . .
\n\n4) It seems that other features, like fins or tails or whale dimensions are harder see/get. Is it true or I didn't look at enough samples?
\n\n5) If 4 is correct, why is the discussion about the whale species relevant?
\n\nThanks!
\n\n[/quote]
\n\n1) Those spots are real features of the whale but they change more rapidly over time than the callosity so they are not typically used in photo id unless the two sightings were very close in time. Best not to use them and focus on the callosity pattern and scars.
\n\n2) These images are a very clean subsample of what photographs are taken in the field - these images all contain only one whale, contain a clear shot of the head, and the whale was individually identifiable based on that photograph.
\n\n3.) I'm not sure that I am clear on your question here but the goal of the competition is to create an algorithm that correctly identifies which individual whale is in each photo.
\n\n4.) Other dimensions are less predictable, partially because researchers are primarily targeting a good photo of the head region as that is the most commonly used area for photo identification in this species.
\n\n5.) Species identification is not relevant to this competition - all photographs are of North Atlantic right whales.
", "rawMarkdown": "[quote=visoft;93093]\r\n\r\nHi all!\r\n\r\n1) I have a question about w_154: The spots on the back are a feature or a bug? (I.e. I can recognize this whale by those spots OR they are just some temporary artifacts?)\r\n\r\n2) The head is always nicely visible? Ex: you tried to include only \"nice\" pictures in the set, or you randomly sampled all your whale database?\r\n\r\n3) Did I understand correctly? This is about recognizing some specific whales in the pictures? Ex: In w_154 is whale ID 3342, but we call it John! In the next picture is Joseph! So on so forth . . .\r\n\r\n4) It seems that other features, like fins or tails or whale dimensions are harder see/get. Is it true or I didn't look at enough samples?\r\n\r\n5) If 4 is correct, why is the discussion about the whale species relevant?\r\n\r\nThanks!\r\n\r\n[/quote]\r\n\r\n\r\n1) Those spots are real features of the whale but they change more rapidly over time than the callosity so they are not typically used in photo id unless the two sightings were very close in time. Best not to use them and focus on the callosity pattern and scars.\r\n\r\n2) These images are a very clean subsample of what photographs are taken in the field - these images all contain only one whale, contain a clear shot of the head, and the whale was individually identifiable based on that photograph.\r\n\r\n3.) I'm not sure that I am clear on your question here but the goal of the competition is to create an algorithm that correctly identifies which individual whale is in each photo.\r\n\r\n4.) Other dimensions are less predictable, partially because researchers are primarily targeting a good photo of the head region as that is the most commonly used area for photo identification in this species.\r\n\r\n5.) Species identification is not relevant to this competition - all photographs are of North Atlantic right whales.", "votes": null }, { "id": "93189", "postDate": "09/22/2015 17:42:31", "content": "Thanx a lot!
\n\nI am a bit "old school" and domain knowledge is vital for me. I would use image processing to model this knowledge rather than jamming everything on some CNN and hope for the best!
", "rawMarkdown": "Thanx a lot!\r\n\r\nI am a bit \"old school\" and domain knowledge is vital for me. I would use image processing to model this knowledge rather than jamming everything on some CNN and hope for the best!", "votes": null }, { "id": "93202", "postDate": "09/22/2015 19:24:11", "content": "Hi Christin,
\n\nCan you tell me what is going on in this image?
\n\nkb
", "rawMarkdown": "Hi Christin, \r\n\r\nCan you tell me what is going on in this image?\r\n\r\nkb", "votes": null }, { "id": "93203", "postDate": "09/22/2015 19:33:11", "content": "[quote=Kevin Burnham;93202]
\n\nHi Christin,
\n\nCan you tell me what is going on in this image?
\n\nkb
\n\n[/quote]
\n\nImage w_3313 is showing two right whales in a SAG or "surface active group" where the whale on the right side of the image is belly up.
\n\nDefinition of SAG from New England Aquarium right whale research team website:
\n\n"Surface active groups (SAGs) are characterized by two or more whales socializing at the surface. Frequent body contact is observed, often with whales rolling on their sides or backs. They generally involve a single female that vocalizes to attract males. The female typically spends most of the time on her back with her genitals out of the water while males clasp her from either side with their flippers. Penises are often observed and SAGs are believed to play a role in mating. However, many SAGs do not lead directly to calving because: 1) SAGs have been seen during almost all months, whereas calving occurs primarily from December through March; 2) some SAGs comprise only males and/or juveniles; and 3) many of the focal females are already pregnant."
", "rawMarkdown": "[quote=Kevin Burnham;93202]\r\n\r\nHi Christin, \r\n\r\nCan you tell me what is going on in this image?\r\n\r\nkb\r\n\r\n[/quote]\r\n\r\nImage w_3313 is showing two right whales in a SAG or \"surface active group\" where the whale on the right side of the image is belly up.\r\n\r\nDefinition of SAG from New England Aquarium right whale research team website:\r\n\r\n\"Surface active groups (SAGs) are characterized by two or more whales socializing at the surface. Frequent body contact is observed, often with whales rolling on their sides or backs. They generally involve a single female that vocalizes to attract males. The female typically spends most of the time on her back with her genitals out of the water while males clasp her from either side with their flippers. Penises are often observed and SAGs are believed to play a role in mating. However, many SAGs do not lead directly to calving because: 1) SAGs have been seen during almost all months, whereas calving occurs primarily from December through March; 2) some SAGs comprise only males and/or juveniles; and 3) many of the focal females are already pregnant.\"", "votes": null }, { "id": "93491", "postDate": "09/26/2015 16:15:39", "content": "Hi Christian! Thanx for the answer! Very useful!
\n\n1) I noted that there are many (~20) whales with one picture only. Is this by intention or by fate (i.e. "hard" to catch whales?)
\n\n2) What is the whale_25715 doing in w_7219 doing? And how to read the image?
\n\nThank you!
", "rawMarkdown": "Hi Christian! Thanx for the answer! Very useful!\r\n\r\n1) I noted that there are many (~20) whales with one picture only. Is this by intention or by fate (i.e. \"hard\" to catch whales?)\r\n\r\n\r\n2) What is the whale_25715 doing in w_7219 doing? And how to read the image? \r\n\r\nThank you!", "votes": null }, { "id": "93518", "postDate": "09/27/2015 02:24:06", "content": "[quote=visoft;93491]
\n\nHi Christian! Thanx for the answer! Very useful!
\n\n1) I noted that there are many (~20) whales with one picture only. Is this by intention or by fate (i.e. "hard" to catch whales?)
\n\n2) What is the whale_25715 doing in w_7219 doing? And how to read the image?
\n\nThank you!
\n\n[/quote]
\n\n1) This is just coincidence, we narrowed the images only by a.) that the image was matched to a whale in the catalog, and b.) that the image contained only one whale.
\n\n2.) The whale in image w_7219 is skim feeding which means swimming along at the surface of the water with mouth open and only upper jaw visible. It is a very challenging situation for whale matching and greatly complicates matters for whale researchers as well as we have less cues to work with (frustrating when you can't use whether or not the whale has lip callosities or chin callosities as a cue).
", "rawMarkdown": "[quote=visoft;93491]\r\n\r\nHi Christian! Thanx for the answer! Very useful!\r\n\r\n1) I noted that there are many (~20) whales with one picture only. Is this by intention or by fate (i.e. \"hard\" to catch whales?)\r\n\r\n\r\n2) What is the whale_25715 doing in w_7219 doing? And how to read the image? \r\n\r\nThank you!\r\n\r\n[/quote]\r\n\r\n1) This is just coincidence, we narrowed the images only by a.) that the image was matched to a whale in the catalog, and b.) that the image contained only one whale.\r\n\r\n2.) The whale in image w_7219 is skim feeding which means swimming along at the surface of the water with mouth open and only upper jaw visible. It is a very challenging situation for whale matching and greatly complicates matters for whale researchers as well as we have less cues to work with (frustrating when you can't use whether or not the whale has lip callosities or chin callosities as a cue).", "votes": null }, { "id": "93599", "postDate": "09/28/2015 17:22:32", "content": "Following up on the question about the "representativeness" of the photos, am I right in assuming that a whale with many "portraits" might have several photos taken in rapid succession?
\n\nFor instance, in the photos of whale 33961 I'm willing to bet that that photos 9327 and 1045 were taken fairly close together in time - you can almost see the water on the left and right come together to form the burst of foam in 1054 as 33961 rolls.
\n\nSimilarly, photos 4827 and 8409 might have been snapped as the helicopter changed course onto the track of 33961 as he or she fed close to the surface.
\n\nFrom a competitor standpoint, this suggests that it might be useful to attempt to group similar test images and make sure that they all get the same label. To make this work you'd need some easy to compute indexing mechanism (not whale identity!) like water color and sea state to quickly find possible matches.
\n\nFrom a whale researcher standpoint, it suggests that a better accuracy measure on the test set could be had by presenting images separated by at least 10 minutes or something.
\n\nThanks for having this competition. I'm learning a lot about whales!
", "rawMarkdown": "Following up on the question about the \"representativeness\" of the photos, am I right in assuming that a whale with many \"portraits\" might have several photos taken in rapid succession?\r\n\r\nFor instance, in the photos of whale 33961 I'm willing to bet that that photos 9327 and 1045 were taken fairly close together in time - you can almost see the water on the left and right come together to form the burst of foam in 1054 as 33961 rolls.\r\n\r\nSimilarly, photos 4827 and 8409 might have been snapped as the helicopter changed course onto the track of 33961 as he or she fed close to the surface.\r\n\r\nFrom a competitor standpoint, this suggests that it might be useful to attempt to group similar test images and make sure that they all get the same label. To make this work you'd need some easy to compute indexing mechanism (not whale identity!) like water color and sea state to quickly find possible matches. \r\n\r\nFrom a whale researcher standpoint, it suggests that a better accuracy measure on the test set could be had by presenting images separated by at least 10 minutes or something.\r\n\r\nThanks for having this competition. I'm learning a lot about whales!", "votes": null }, { "id": "93605", "postDate": "09/28/2015 19:22:26", "content": "Hi Christian!
\n\nAbout one whale per picture. :-) Could anybody guess which of two whales on the w_751.jpg is whale_95091 ?\nSeems that the right one, but I'm not sure.
I'm agree with @John Blankenbaker, there are a lot of similar entries in training subset where one whale was captured two times within few seconds/minutes.
\n\nThanks in advance.
", "rawMarkdown": "Hi Christian!\r\n\r\nAbout one whale per picture. :-) Could anybody guess which of two whales on the **`w_751.jpg`** is **`whale_95091`** ?\r\nSeems that the right one, but I'm not sure.\r\n\r\nI'm agree with @John Blankenbaker, there are a lot of similar entries in training subset where one whale was captured two times within few seconds/minutes. \r\n\r\nThanks in advance.", "votes": null }, { "id": "93680", "postDate": "09/30/2015 06:56:21", "content": "Hi,
\n\nI am new to Kaggle and this would be my first competition. I am trying to understand the data provided and I realize that in the train.csv file provided each image is labeled with an ID. I also get that in the submission file, each of the images provided need to be recognized as one or more of the whale IDs with varying probabilities. So my question is, which of the whale IDs is the Right Whale?
\n\nThanks in advance!
", "rawMarkdown": "Hi,\r\n\r\nI am new to Kaggle and this would be my first competition. I am trying to understand the data provided and I realize that in the train.csv file provided each image is labeled with an ID. I also get that in the submission file, each of the images provided need to be recognized as one or more of the whale IDs with varying probabilities. So my question is, which of the whale IDs is the Right Whale?\r\n\r\nThanks in advance!", "votes": null }, { "id": "93894", "postDate": "10/02/2015 14:54:16", "content": "[quote=John Blankenbaker;93599]
\n\nFollowing up on the question about the "representativeness" of the photos, am I right in assuming that a whale with many "portraits" might have several photos taken in rapid succession?
\n\nFor instance, in the photos of whale 33961 I'm willing to bet that that photos 9327 and 1045 were taken fairly close together in time - you can almost see the water on the left and right come together to form the burst of foam in 1054 as 33961 rolls.
\n\nSimilarly, photos 4827 and 8409 might have been snapped as the helicopter changed course onto the track of 33961 as he or she fed close to the surface.
\n\nFrom a competitor standpoint, this suggests that it might be useful to attempt to group similar test images and make sure that they all get the same label. To make this work you'd need some easy to compute indexing mechanism (not whale identity!) like water color and sea state to quickly find possible matches.
\n\nFrom a whale researcher standpoint, it suggests that a better accuracy measure on the test set could be had by presenting images separated by at least 10 minutes or something.
\n\nThanks for having this competition. I'm learning a lot about whales!
\n\n[/quote]
\n\nYes, there are sometimes a bunch of photos taken of the same whale on the same day in rapid succession, and although I can't confirm, it does look pretty likely that is the case with both sets of images you mentioned (9327 + 1045, and again with 4827 + 8409). Both pairs look to me as if they were taken from different vantage points as the aircraft circled overhead.
\n\nI don't have good perspective from a competitor standpoint, but from a researcher standpoint we would like the algorithm to stand alone when given only one image as we would most likely implement it as a software program that matches each image independently. Matching is particularly time-critical for us when we are out in the field (airplane, vessel) and in those scenarios we would likely photograph until we have a really good image and then feed that one image to the software program / algorithm.
\n\nGood point about measuring accuracy with images separated by 10 minutes, although I think that train has already left the station for this competition :-)
\n\nI am happy to hear you are learning a lot about whales. I have put an attachment on here that is a great summary of what I would like everyone to learn about right whales.
\n\n[quote=Arrchana;93680]
\n\nHi,
\n\nI am new to Kaggle and this would be my first competition. I am trying to understand the data provided and I realize that in the train.csv file provided each image is labeled with an ID. I also get that in the submission file, each of the images provided need to be recognized as one or more of the whale IDs with varying probabilities. So my question is, which of the whale IDs is the Right Whale?
\n\nThanks in advance!
\n\n[/quote]
\n\nHi,
\n\nAll of the images are of right whales (species name North Atlantic Right Whale, Eubalaena glacialis).
\n\nThe trick is to identify which individual right whale is in each photograph (analogous to saying that these are all humans but which one is Mary Jane and which one is John Smith, except that for the whales, we give them a number instead of a name).
\n\nHope that helps and good luck with your first try at competing!
", "rawMarkdown": "[quote=Arrchana;93680]\r\n\r\nHi,\r\n\r\nI am new to Kaggle and this would be my first competition. I am trying to understand the data provided and I realize that in the train.csv file provided each image is labeled with an ID. I also get that in the submission file, each of the images provided need to be recognized as one or more of the whale IDs with varying probabilities. So my question is, which of the whale IDs is the Right Whale?\r\n\r\nThanks in advance!\r\n\r\n[/quote]\r\n\r\nHi, \r\n\r\nAll of the images are of right whales (species name North Atlantic Right Whale, *Eubalaena glacialis*).\r\n\r\nThe trick is to identify which individual right whale is in each photograph (analogous to saying that these are all humans but which one is Mary Jane and which one is John Smith, except that for the whales, we give them a number instead of a name).\r\n\r\nHope that helps and good luck with your first try at competing!", "votes": null }, { "id": "95120", "postDate": "10/05/2015 11:04:31", "content": "Hi! I am a MS student. I have a question. My research area is Machine Learning for Computer Vision and this task fits exactly to my research interest. My question is can I publish my results if I participate in this competition?
", "rawMarkdown": "Hi! I am a MS student. I have a question. My research area is Machine Learning for Computer Vision and this task fits exactly to my research interest. My question is can I publish my results if I participate in this competition?", "votes": null }, { "id": "95121", "postDate": "10/05/2015 11:06:41", "content": "When I say publish I mean publishing in some scholarly proceedings or journals.
", "rawMarkdown": "When I say publish I mean publishing in some scholarly proceedings or journals.", "votes": null }, { "id": "95166", "postDate": "10/05/2015 16:35:39", "content": "[quote=JumabekAlihanov;95121]
\n\nWhen I say publish I mean publishing in some scholarly proceedings or journals.
\n\n[/quote]
\n\nYes, I don't see why not. Just please send me an email that details your "data request" in a bit of detail so I can get confirmation from my supervisors (christin.khan@noaa.gov). I do not foresee any issues with that though.
", "rawMarkdown": "[quote=JumabekAlihanov;95121]\r\n\r\nWhen I say publish I mean publishing in some scholarly proceedings or journals.\r\n\r\n[/quote]\r\n\r\n\r\nYes, I don't see why not. Just please send me an email that details your \"data request\" in a bit of detail so I can get confirmation from my supervisors (christin.khan@noaa.gov). I do not foresee any issues with that though.", "votes": null }, { "id": "96648", "postDate": "10/19/2015 13:21:30", "content": "I want to ask how the main approach used for detection of right whale with the help of machine learning ?
", "rawMarkdown": "I want to ask how the main approach used for detection of right whale with the help of machine learning ?", "votes": null }, { "id": "96652", "postDate": "10/19/2015 13:44:49", "content": "Hello mouzmeen,
\n\nI am not sure if there is a main approach. However, based the posts in this forum, here are some things that are being tried:
\n\nI would imagine there are a number of other approaches that could be tried as well. You can learn a lot just by browsing through the forum posts and seeing where that leads you.
\n\nGood luck!
\n\nKevin
", "rawMarkdown": "Hello mouzmeen,\r\n\r\nI am not sure if there is a main approach. However, based the posts in this forum, here are some things that are being tried:\r\n\r\n- haar cascade (you can do this with a program called [OpenCV](http://opencv.org/))\r\n- linear binary patterns (also possible with OpenCV)\r\n- a sliding window with convnets (see example [here](http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/))\r\n- HOG (histogram of oriented gradients) - I think you can do this with [dlib](http://blog.dlib.net/2014/02/dlib-186-released-make-your-own-object.html)\r\n- color ratios (see [this post](https://www.kaggle.com/c/noaa-right-whale-recognition/forums/t/16681/unsupervised-whale-detection/93498#post93498))\r\n- edge detection (I think OpenCV can help with this too)\r\n- using the free version of MatLab that you can get as part of this competition. See [here](https://www.kaggle.com/c/noaa-right-whale-recognition/details/creating-a-face-detector-for-whales).\r\n\r\nI would imagine there are a number of other approaches that could be tried as well. You can learn a lot just by browsing through the forum posts and seeing where that leads you.\r\n\r\nGood luck!\r\n\r\nKevin", "votes": null }, { "id": "98135", "postDate": "11/04/2015 07:13:19", "content": "I just did a quick analysis of all the images and the resolutions for the images range from a 1590x1059 (w_9162.jpg) and 7010x4674 (w_5049.jpg). The most frequent resolution being 3072x2048 @ 3600 with the next frequent resolutions being: 5184x3456 @ 809, 2464x1648 @ 110, 3504x2336 @ 30 with the remaining resolution frequencies dropping below 20; with numerous unique image resolutions. Actually, there are 2,941 unique image resolutions in the image entire 11,469 image data set.
\n\nWith that being said, can someone...
\n\n![\"right](\"https://static1.squarespace.com/static/54480937e4b0a872525f0757/t/5539189de4b0a710f69dd4c6/1429805245089/Northeast+region+right+whale+sign+NOAA+Christin+Khan?format=500w\")
1) ...explain why there is such a large range of image resolutions?\n2) ...provide us with both an upper and lower limit on the image resolutions that a solution will need to support?\nOn a slightly different topic, but a topic that has been raised previously, there are several whale IDs that have only one image. Ideally we would have several images for each whale like 4+, but is there a possibility of at least getting two images per whale ID?
\n\n -- Manny\nHehe, my bet is that the image size is somehow linked with the date when that image was taken (different cameras, different technologies).
\n\nAlso, afaik, the images are manually checked and cropped so the head is clearly visible and there is only one head in the image. Maybe another source of variation is this cropping.
\n\np.s. Can you [manually] check if the whale spots evolve as the image size grows? :)
", "rawMarkdown": "Hehe, my bet is that the image size is somehow linked with the date when that image was taken (different cameras, different technologies).\r\n\r\nAlso, afaik, the images are manually checked and cropped so the head is clearly visible and there is only one head in the image. Maybe another source of variation is this cropping.\r\n\r\np.s. Can you [manually] check if the whale spots evolve as the image size grows? :)", "votes": null }, { "id": "98788", "postDate": "11/13/2015 16:46:38", "content": "It is allowed to use bbox-labeling in this competition.
\nhttps://www.kaggle.com/c/noaa-right-whale-recognition/details/creating-a-face-detector-for-whales
By the way, silhouette based labeling is also allowed ?\n![\"silhouette-labeling\"](\"https://dl.dropboxusercontent.com/u/69374823/kaggle/Right_Whale_Recognition/labeling.png\")
[quote=toshi_k;98788]
\n\nIt is allowed to use bbox-labeling in this competition.
\n\nBy the way, silhouette based labeling is also allowed ?
\n\n[/quote]
\n\ntoshi_k:
\nThe images you just posted are brilliant. How did you produce them? bbox-labeling and/or silhouette based labeling, perhaps? Whatever you did, please let me know what it was.
Hi George,
\n\nThis image display silhouette based labeling. I draw mask path for each training images in Adobe Software.
\nAlthough, bbox-labeling can be put into practice in MATLAB Labeler,
\nsilhouette based labeling can not be put into practice in it.
When you say I draw mask path for each training images in Adobe Software, what do you mean? I am familiar with Photoshop, Illustrator and Adobe but none of those would seem suitable. Did you draw these silhouettes programmatically or by hand?
", "rawMarkdown": "When you say **I draw mask path for each training images in Adobe Software**, what do you mean? I am familiar with Photoshop, Illustrator and Adobe but none of those would seem suitable. Did you draw these silhouettes programmatically or by hand?", "votes": null }, { "id": "98812", "postDate": "11/14/2015 02:51:39", "content": "The idea of silhouette based labeling is as follows.
\nFirst, draw silhouette by hand for each training images.
\nSecond, learn model which can generate silhouette images from original images (machine learning).
\nFinally, predict silhouette for each test images by learned model.
Imho this should be allowed. As long as you don't touch the test images. I mean, you will not be packed with the software so you will not be able to delineate fresh pictures taken ovre the high seas.
\n\nI think everybody uses some kind of manual input for training.
", "rawMarkdown": "Imho this should be allowed. As long as you don't touch the test images. I mean, you will not be packed with the software so you will not be able to delineate fresh pictures taken ovre the high seas.\r\n\r\nI think everybody uses some kind of manual input for training.", "votes": null }, { "id": "98821", "postDate": "11/14/2015 07:25:39", "content": "Hi visoft,
\n\nThank you for your opinion !
\nI'll try this approach.
Apologies for the slow reply, but yes, variation in image size and resolution is because these photographs are taken by a variety of people with all different cameras and under a wide variety of circumstances. We curated the images in this competition so that they are all high quality photographs with only one whale in the frame which was of sufficient quality to identify manually, but there are many different cameras, lenses, and photographers represented.
\n\n[quote=pdx_manny;98135]
\n\nI just did a quick analysis of all the images and the resolutions for the images range from a 1590x1059 (w_9162.jpg) and 7010x4674 (w_5049.jpg). The most frequent resolution being 3072x2048 @ 3600 with the next frequent resolutions being: 5184x3456 @ 809, 2464x1648 @ 110, 3504x2336 @ 30 with the remaining resolution frequencies dropping below 20; with numerous unique image resolutions. Actually, there are 2,941 unique image resolutions in the image entire 11,469 image data set.
\n\nWith that being said, can someone...
\n\n1) ...explain why there is such a large range of image resolutions?\n2) ...provide us with both an upper and lower limit on the image resolutions that a solution will need to support?\nOn a slightly different topic, but a topic that has been raised previously, there are several whale IDs that have only one image. Ideally we would have several images for each whale like 4+, but is there a possibility of at least getting two images per whale ID?
\n\n -- Manny\n[/quote]
", "rawMarkdown": "Apologies for the slow reply, but yes, variation in image size and resolution is because these photographs are taken by a variety of people with all different cameras and under a wide variety of circumstances. We curated the images in this competition so that they are all high quality photographs with only one whale in the frame which was of sufficient quality to identify manually, but there are many different cameras, lenses, and photographers represented.\r\n\r\n[quote=pdx_manny;98135]\r\n\r\nI just did a quick analysis of all the images and the resolutions for the images range from a 1590x1059 (w_9162.jpg) and 7010x4674 (w_5049.jpg). The most frequent resolution being 3072x2048 @ 3600 with the next frequent resolutions being: 5184x3456 @ 809, 2464x1648 @ 110, 3504x2336 @ 30 with the remaining resolution frequencies dropping below 20; with numerous unique image resolutions. Actually, there are 2,941 unique image resolutions in the image entire 11,469 image data set.\r\n\r\nWith that being said, can someone...\r\n\r\n 1) ...explain why there is such a large range of image resolutions?\r\n 2) ...provide us with both an upper and lower limit on the image resolutions that a solution will need to support?\r\n\r\nOn a slightly different topic, but a topic that has been raised previously, there are several whale IDs that have only one image. Ideally we would have several images for each whale like 4+, but is there a possibility of at least getting two images per whale ID?\r\n\r\n -- Manny\r\n\r\n[/quote]", "votes": null }, { "id": "99509", "postDate": "11/25/2015 21:25:55", "content": "Hi Christin,
\n\nThanks for the reply on the image resolution topic, but is there a chance that we can get a range of camera resolutions...particularly the upper resolution range as the lower resolutions can be handled by filing in the unused pixels with zeros.\nAnd is there any chance on getting at least two images per whale ID? ;-)
\n\n- Manny\nUnfortunately I do not know the upper and lower bounds of image resolutions, and it's best not to depend on that knowledge anyhow as in the future, some folks will be using cell phones and some folks will be using really high end high resolution cameras.
\n\nI will think more on the other question but I think it is too late to make changes like that to the competition.
\n\n[quote=pdx_manny;99509]
\n\nHi Christin,
\n\nThanks for the reply on the image resolution topic, but is there a chance that we can get a range of camera resolutions...particularly the upper resolution range as the lower resolutions can be handled by filing in the unused pixels with zeros.\nAnd is there any chance on getting at least two images per whale ID? ;-)
\n\n- Manny\n[/quote]
", "rawMarkdown": "Unfortunately I do not know the upper and lower bounds of image resolutions, and it's best not to depend on that knowledge anyhow as in the future, some folks will be using cell phones and some folks will be using really high end high resolution cameras.\r\n\r\nI will think more on the other question but I think it is too late to make changes like that to the competition.\r\n\r\n\r\n[quote=pdx_manny;99509]\r\n\r\nHi Christin,\r\n\r\n Thanks for the reply on the image resolution topic, but is there a chance that we can get a range of camera resolutions...particularly the upper resolution range as the lower resolutions can be handled by filing in the unused pixels with zeros.\r\n\r\n And is there any chance on getting at least two images per whale ID? ;-)\r\n\r\n - Manny\r\n\r\n[/quote]", "votes": null }, { "id": "100730", "postDate": "12/09/2015 23:16:08", "content": "Does the color information help you professionals identify the whales? what would have been lost if you had only greyscale photographs?
\n\nThanks for this extremely challenging competition :)
", "rawMarkdown": "Does the color information help you professionals identify the whales? what would have been lost if you had only greyscale photographs?\r\n\r\nThanks for this extremely challenging competition :)", "votes": null }, { "id": "100831", "postDate": "12/10/2015 22:16:19", "content": "No - we primarily rely on non-color information so I think black and white images would work just as well
\n\n[quote=Hrant Khachatrian;100730]
\n\nDoes the color information help you professionals identify the whales? what would have been lost if you had only greyscale photographs?
\n\nThanks for this extremely challenging competition :)
\n\n[/quote]
", "rawMarkdown": "No - we primarily rely on non-color information so I think black and white images would work just as well\r\n\r\n[quote=Hrant Khachatrian;100730]\r\n\r\nDoes the color information help you professionals identify the whales? what would have been lost if you had only greyscale photographs?\r\n\r\nThanks for this extremely challenging competition :)\r\n\r\n[/quote]", "votes": null }, { "id": "103567", "postDate": "01/04/2016 11:53:42", "content": "Hi Christin,\nHi Kaggle administrators!
\n\nThis was an interesting competition and the domain (ocean, whales) is a fresh breath of air in computer vision field.
\n\nI would like to implore you to make the data publicly available or to transform this competition into a yearly runner like Titanic or digit recognizer (101 competitions) I think that this will be a good thing for computer vision field, for NOAA and for kaggle:
\n\nIn the hope that I captured your interest, I wish you a Happy New Year!
\n\nCristian Vicas
", "rawMarkdown": "Hi Christin,\r\nHi Kaggle administrators!\r\n\r\nThis was an interesting competition and the domain (ocean, whales) is a fresh breath of air in computer vision field. \r\n\r\nI would like to implore you to make the data publicly available or to transform this competition into a yearly runner like Titanic or digit recognizer (101 competitions) I think that this will be a good thing for computer vision field, for NOAA and for kaggle:\r\n\r\n* The CV field will have another, fresh domain of natural images. Existing ones focus on human habitat only.\r\n* 3 months is a very short period of time for most of us. A larger time span will gather more attention and there are chances that new, faster and better solutions will emerge.\r\n* When the whale database will gain interest outside this competition, kaggle will aslo benefit from the exposure.\r\n\r\nIn the hope that I captured your interest, I wish you a Happy New Year!\r\n\r\nCristian Vicas", "votes": null }, { "id": "103789", "postDate": "01/06/2016 14:59:38", "content": "Cristian,
\n\nThanks so much for your active participation in this competition, I can't tell you how much I appreciate all the enthusiasm!
\n\nThe data will certainly be publicly available - just email me to request the data (Christin.Khan@noaa.gov).
\n\nI will certainly discuss your ideas with the folks at Kaggle as I of course would like to see this field advance and to support the marine science community. There are many other species that would also benefit from a similar competition so we will do our best to keep the ball rolling!
\n\n[quote=visoft;103567]
\n\nHi Christin,\nHi Kaggle administrators!
\n\nThis was an interesting competition and the domain (ocean, whales) is a fresh breath of air in computer vision field.
\n\nI would like to implore you to make the data publicly available or to transform this competition into a yearly runner like Titanic or digit recognizer (101 competitions) I think that this will be a good thing for computer vision field, for NOAA and for kaggle:
\n\nIn the hope that I captured your interest, I wish you a Happy New Year!
\n\nCristian Vicas
\n\n[/quote]
", "rawMarkdown": "Cristian,\r\n\r\nThanks so much for your active participation in this competition, I can't tell you how much I appreciate all the enthusiasm!\r\n\r\nThe data will certainly be publicly available - just email me to request the data (Christin.Khan@noaa.gov).\r\n\r\nI will certainly discuss your ideas with the folks at Kaggle as I of course would like to see this field advance and to support the marine science community. There are many other species that would also benefit from a similar competition so we will do our best to keep the ball rolling!\r\n\r\n[quote=visoft;103567]\r\n\r\nHi Christin,\r\nHi Kaggle administrators!\r\n\r\nThis was an interesting competition and the domain (ocean, whales) is a fresh breath of air in computer vision field. \r\n\r\nI would like to implore you to make the data publicly available or to transform this competition into a yearly runner like Titanic or digit recognizer (101 competitions) I think that this will be a good thing for computer vision field, for NOAA and for kaggle:\r\n\r\n* The CV field will have another, fresh domain of natural images. Existing ones focus on human habitat only.\r\n* 3 months is a very short period of time for most of us. A larger time span will gather more attention and there are chances that new, faster and better solutions will emerge.\r\n* When the whale database will gain interest outside this competition, kaggle will aslo benefit from the exposure.\r\n\r\nIn the hope that I captured your interest, I wish you a Happy New Year!\r\n\r\nCristian Vicas\r\n\r\n[/quote]", "votes": null }, { "id": "104282", "postDate": "01/11/2016 11:33:21", "content": "Thank you very much!
\n\nI will keep an eye on the kaggle forums for updates!
", "rawMarkdown": "Thank you very much! \r\n\r\n\r\nI will keep an eye on the kaggle forums for updates!", "votes": null }, { "id": "171861", "postDate": "03/31/2017 18:29:46", "content": "What is the circumference of the outermost (skin) part of the peduncle? I've been curious how big that area has to be to support everything going into the flukes.
", "rawMarkdown": "What is the circumference of the outermost (skin) part of the peduncle? I've been curious how big that area has to be to support everything going into the flukes.", "votes": null } ], "comments": [ { "id": 91208, "author_name": "antreasantoniou", "author_url": "", "post_date": "09/01/2015 16:13:19", "content": "Really nice competition. Also in the future for better performance results maybe an infrared camera/thermal cameras would be more useful or add features for the models. But in general, very nice competition. Thanks for this chance.
", "votes": null, "replies": [] }, { "id": 91236, "author_name": "jfkingiii", "author_url": "", "post_date": "09/01/2015 17:54:17", "content": "And now we have a better than random guess result on the leaderboard...sooner than I expected.
", "votes": null, "replies": [] }, { "id": 91245, "author_name": "inversion", "author_url": "", "post_date": "09/01/2015 18:30:04", "content": "[quote=James King;91236]
\n\nAnd now we have a better than random guess result on the leaderboard...sooner than I expected.
\n\n[/quote]
\n\nI always hate when my algorithm does worse than a random guess. :-)
", "votes": null, "replies": [] }, { "id": 91262, "author_name": "jfkingiii", "author_url": "", "post_date": "09/01/2015 20:02:58", "content": "I suspect it takes some data prep to beat random guessing. I plan to start with whale_95370 and try to distinguish it from all other whales. His facial markings are quite distinctive.
\n\nBut I think I need to first discard the images where the face cannot be seen.
", "votes": null, "replies": [] }, { "id": 91273, "author_name": "inversion", "author_url": "", "post_date": "09/01/2015 21:24:36", "content": "Doesn't look like the same whale (95370) to me. Do those white thinger-doodles change over time, e.g., so that one of the images was from a younger whale?
\n\nw_3076
\n\n
w_3085
\n\n
Very good question inversion, it looks as if the bottom one is from a younger whale before the line was fully connected. Or maybe it had a fight and created issues with his/her lines.
", "votes": null, "replies": [] }, { "id": 91379, "author_name": "cbkhan", "author_url": "", "post_date": "09/02/2015 17:59:53", "content": "The white markings on top of the whale's head are called callosities and they do change very slightly over time, particularly when the whale is still young, so it is reasonable to expect slight deviations such as this from one photograph to another. On the other hand, there are some whales that look very very similar to each other. For more background on whale matching, visit these websites:
\n\n\n\nand
\n\nhttp://rwcatalog.neaq.org/Default.aspx
", "votes": null, "replies": [] }, { "id": 91380, "author_name": "cbkhan", "author_url": "", "post_date": "09/02/2015 18:05:26", "content": "It may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head. Right whales are first divided into two broad categories to describe the overall appearance of the main section of the callosity- continuous and broken. Next we usually discuss how many islands or peninsulas there are and on which side of the head. More subtle but important cues are the presence and size of lip callosities, and how many post blowhole and chin callosities there are.
\n\n
[quote=Christin B. Khan;91380]\nIt may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head.\n[/quote]
\n\nSo, you're saying "thinger-doodles" isn't part of the marine biology vernacular? :-)
", "votes": null, "replies": [] }, { "id": 91396, "author_name": "cbkhan", "author_url": "", "post_date": "09/02/2015 19:04:47", "content": ""thinger-doodles" gave me quite a smile :-)
", "votes": null, "replies": [] }, { "id": 91417, "author_name": "mattwenger", "author_url": "", "post_date": "09/02/2015 22:08:14", "content": "[quote=Christin B. Khan;91380]
\n\nIt may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head.
\n\n[/quote]
\n\nSo, Christin, are the callosity patterns the main feature used to distinguish right whales from other whale species? Are there other features that are useful in distinguishing a right whale from another type of whale?
", "votes": null, "replies": [] }, { "id": 91432, "author_name": "jfkingiii", "author_url": "", "post_date": "09/03/2015 03:21:06", "content": "Well they have the twin blowholes (sort of a blow-nostril) as you see in the photos, and no dorsal fin. Also the male right whale's whalehood is without peer in the animal kingdom, although you usually would not be able to notice this from above the water.
\n\nhttps://en.wikipedia.org/wiki/Right_whale
", "votes": null, "replies": [] }, { "id": 91439, "author_name": "", "author_url": "", "post_date": "09/03/2015 04:28:50", "content": "Is the whale in w_8026.jpg pooping?
", "votes": null, "replies": [] }, { "id": 91455, "author_name": "deepcnn", "author_url": "", "post_date": "09/03/2015 07:22:03", "content": "@KA hahaha, I cannot control my laugh :) Out of all these data, you're focused on this :)
", "votes": null, "replies": [] }, { "id": 91479, "author_name": "kburnham", "author_url": "", "post_date": "09/03/2015 12:43:36", "content": "@Matt Wenger - Here are some pages that talk about how to identify the whales. It is mainly based on the callosity pattern, but at the bottom of the page is a link to other identifying features.
", "votes": null, "replies": [] }, { "id": 91504, "author_name": "mattwenger", "author_url": "", "post_date": "09/03/2015 18:26:22", "content": "@Kevin Burnham - Thanks! This is very useful.
", "votes": null, "replies": [] }, { "id": 91518, "author_name": "cbkhan", "author_url": "", "post_date": "09/03/2015 21:11:35", "content": "@Matt Wenger - right whale are identified from other whale species primarily by:
\n\n"Is the whale in w_8026.jpg pooping?"
\n\n@KA - yes! nice find! We marine biologists refer to it as defecation
", "votes": null, "replies": [] }, { "id": 91522, "author_name": "domcastro", "author_url": "", "post_date": "09/03/2015 21:15:49", "content": "[quote=KA;91439]
\n\nIs the whale in w_8026.jpg pooping?
\n\n[/quote]
\n\nhahahahahhahaha
", "votes": null, "replies": [] }, { "id": 91523, "author_name": "domcastro", "author_url": "", "post_date": "09/03/2015 21:16:22", "content": "[quote=Christin B. Khan;91519]
\n\n"Is the whale in w_8026.jpg pooping?"
\n\n@KA - yes! nice find! We marine biologists refer to it as defecation
\n\n[/quote]
\n\neven a bigger hahahahahahahah
", "votes": null, "replies": [] }, { "id": 91647, "author_name": "toshik", "author_url": "", "post_date": "09/05/2015 13:18:50", "content": "I have a question.
\n\nYou provide MATLAB license to students for this competition.
\nhttp://jp.mathworks.com/academia/student-competitions/kaggle/
Is this license only used by students ?
\nI'm not a student.
\nCan't I use MATLAB without buying business license ?
@toshi_k - check out this post. It says that ALL participants, without exception, are able to get a free copy of the software.
", "votes": null, "replies": [] }, { "id": 91649, "author_name": "toshik", "author_url": "", "post_date": "09/05/2015 13:33:34", "content": "Thank you !!
\nI'll request the license for more improvement :-)
With the images, has a person looked at each image independently and been able to identify the whale, or has the whale been identified in one of a given set and then all the photos from the set been given the label. This would help establish the upper-bound accuracy we might expect for the task given these scientists are our expert level annotators :)
\n\nIf it's the former, people are scarily good at identifying whales. During annotation I sometimes have trouble working out where the head is amongst the blowhole's spray!
", "votes": null, "replies": [] }, { "id": 92901, "author_name": "cbkhan", "author_url": "", "post_date": "09/18/2015 14:43:16", "content": "[quote=Smerity;92826]
\n\nWith the images, has a person looked at each image independently and been able to identify the whale, or has the whale been identified in one of a given set and then all the photos from the set been given the label. This would help establish the upper-bound accuracy we might expect for the task given these scientists are our expert level annotators :)
\n\nIf it's the former, people are scarily good at identifying whales. During annotation I sometimes have trouble working out where the head is amongst the blowhole's spray!
\n\n[/quote]
\n\nSmerity, it is somewhere in between. Each image is independently inspected and the matched to the particular whale or whales in the photo (for the sake of the competition, we have provided images with only one whale but reality is usually a bit more messy). However, the context of a series of photographs is definitely taken into account, so when you are circling over one whale and take lots of photographs, they will be checked to verify that it's all the same whale but it's not like each image is separately and matched independently. More often we are circling over a group of whales and take a bunch of images, so that when you look at the next image, you are looking to see who in the group that you have already identified and who is new.
\n\nHowever, taken out of context such as you have in the competition, researchers can and do match each whale. For some experienced researchers and distinctive whales, this is done on sight; for other less experienced folks and/or whales that look more generic, the process can take quite a bit of time and painstaking comparison to other photographs.
", "votes": null, "replies": [] }, { "id": 93011, "author_name": "", "author_url": "", "post_date": "09/20/2015 02:40:17", "content": "Hi.
\n\nI am a high school student interested in using the data you have provided for this competition for a research project to submit to a high school science fair (ISTS); I will only submit my research/analysis and will not disseminate the actual data in my research paper. Is this allowed? My project will be a machine learning/image recognition system (basically what the competition is also asking for), and my research paper/materials will only talk about my system and the results I gleaned. I will not use the data for any other purposes.
\n\nI would really appreciate your letting me use the data; I really would love to enter the science fair and contribute to the whale image recognition sphere.
\n\nThank you.
", "votes": null, "replies": [] }, { "id": 93013, "author_name": "cbkhan", "author_url": "", "post_date": "09/20/2015 04:18:29", "content": "[quote=Tanzim Hassan;93011]
\n\nHi.
\n\nI am a high school student interested in using the data you have provided for this competition for a research project to submit to a high school science fair (ISTS); I will only submit my research/analysis and will not disseminate the actual data in my research paper. Is this allowed? My project will be a machine learning/image recognition system (basically what the competition is also asking for), and my research paper/materials will only talk about my system and the results I gleaned. I will not use the data for any other purposes.
\n\nI would really appreciate your letting me use the data; I really would love to enter the science fair and contribute to the whale image recognition sphere.
\n\nThank you.
\n\n[/quote]
\n\nTanzim,
\n\nFrom the perspective of NOAA Fisheries, it should be fine, although it would be helpful if you could email me with a brief description of your project so that we can confirm permission for you - christin.khan@noaa.gov. However, you should confirm permissions with the Kaggle folks. If you can email me, we'll get it sorted on Monday or Tuesday for you.
", "votes": null, "replies": [] }, { "id": 93076, "author_name": "alexanderbotev", "author_url": "", "post_date": "09/21/2015 07:56:38", "content": "Hi,
\n\nSo I wanted to ask a question about how the organizers would tackle the problem of "over annotating" the whale pictures. By this I mean that it is possible to literally annotate all of the images (it's only about 12k images and I can annotate about 1k per hour, so roughly 2 days for this). And this includes the evaluation images as well. Thus than one can proceed with a model with high enough capacity which to learn to detect the whale heads correct on the images provided (not necessarily on new images though). Even it is possible to just manually crop them and directly train on the cropped images. Do you care about this, or it is not so important part of the problem. How would you "judge" this part in the final submition (if at all) are you gona run the code (and potentially not allow to use more than X annotated images).
\n\nBest regards
", "votes": null, "replies": [] }, { "id": 93078, "author_name": "wildwizard", "author_url": "", "post_date": "09/21/2015 08:33:17", "content": "[quote=Christin B. Khan;92901]
\n\nmerity, it is somewhere in between. Each image is independently inspected and the matched to the particular whale or whales in the photo (for the sake of the competition, we have provided images with only one whale but reality is usually a bit more messy).
\n[/quote]\nJust want to point out that could actually have some importance to the competition, as I guess that some whale species may have more gregarious behaviours than others. Some individuals, too.
Hi all!
\n\n1) I have a question about w_154: The spots on the back are a feature or a bug? (I.e. I can recognize this whale by those spots OR they are just some temporary artifacts?)
\n\n2) The head is always nicely visible? Ex: you tried to include only "nice" pictures in the set, or you randomly sampled all your whale database?
\n\n3) Did I understand correctly? This is about recognizing some specific whales in the pictures? Ex: In w_154 is whale ID 3342, but we call it John! In the next picture is Joseph! So on so forth . . .
\n\n4) It seems that other features, like fins or tails or whale dimensions are harder see/get. Is it true or I didn't look at enough samples?
\n\n5) If 4 is correct, why is the discussion about the whale species relevant?
\n\nThanks!
", "votes": null, "replies": [] }, { "id": 93125, "author_name": "cbkhan", "author_url": "", "post_date": "09/21/2015 19:10:49", "content": "[quote=WhizWilde;93078]
\n\n[quote=Christin B. Khan;92901]
\n\nmerity, it is somewhere in between. Each image is independently inspected and the matched to the particular whale or whales in the photo (for the sake of the competition, we have provided images with only one whale but reality is usually a bit more messy).
\n[/quote]\nJust want to point out that could actually have some importance to the competition, as I guess that some whale species may have more gregarious behaviours than others. Some individuals, too.
[/quote]
\n\nFor this competition, it is only images with one right whale in the photo
", "votes": null, "replies": [] }, { "id": 93126, "author_name": "cbkhan", "author_url": "", "post_date": "09/21/2015 19:17:23", "content": "[quote=visoft;93093]
\n\nHi all!
\n\n1) I have a question about w_154: The spots on the back are a feature or a bug? (I.e. I can recognize this whale by those spots OR they are just some temporary artifacts?)
\n\n2) The head is always nicely visible? Ex: you tried to include only "nice" pictures in the set, or you randomly sampled all your whale database?
\n\n3) Did I understand correctly? This is about recognizing some specific whales in the pictures? Ex: In w_154 is whale ID 3342, but we call it John! In the next picture is Joseph! So on so forth . . .
\n\n4) It seems that other features, like fins or tails or whale dimensions are harder see/get. Is it true or I didn't look at enough samples?
\n\n5) If 4 is correct, why is the discussion about the whale species relevant?
\n\nThanks!
\n\n[/quote]
\n\n1) Those spots are real features of the whale but they change more rapidly over time than the callosity so they are not typically used in photo id unless the two sightings were very close in time. Best not to use them and focus on the callosity pattern and scars.
\n\n2) These images are a very clean subsample of what photographs are taken in the field - these images all contain only one whale, contain a clear shot of the head, and the whale was individually identifiable based on that photograph.
\n\n3.) I'm not sure that I am clear on your question here but the goal of the competition is to create an algorithm that correctly identifies which individual whale is in each photo.
\n\n4.) Other dimensions are less predictable, partially because researchers are primarily targeting a good photo of the head region as that is the most commonly used area for photo identification in this species.
\n\n5.) Species identification is not relevant to this competition - all photographs are of North Atlantic right whales.
", "votes": null, "replies": [] }, { "id": 93189, "author_name": "visoft", "author_url": "", "post_date": "09/22/2015 17:42:31", "content": "Thanx a lot!
\n\nI am a bit "old school" and domain knowledge is vital for me. I would use image processing to model this knowledge rather than jamming everything on some CNN and hope for the best!
", "votes": null, "replies": [] }, { "id": 93202, "author_name": "kburnham", "author_url": "", "post_date": "09/22/2015 19:24:11", "content": "Hi Christin,
\n\nCan you tell me what is going on in this image?
\n\nkb
", "votes": null, "replies": [] }, { "id": 93203, "author_name": "cbkhan", "author_url": "", "post_date": "09/22/2015 19:33:11", "content": "[quote=Kevin Burnham;93202]
\n\nHi Christin,
\n\nCan you tell me what is going on in this image?
\n\nkb
\n\n[/quote]
\n\nImage w_3313 is showing two right whales in a SAG or "surface active group" where the whale on the right side of the image is belly up.
\n\nDefinition of SAG from New England Aquarium right whale research team website:
\n\n"Surface active groups (SAGs) are characterized by two or more whales socializing at the surface. Frequent body contact is observed, often with whales rolling on their sides or backs. They generally involve a single female that vocalizes to attract males. The female typically spends most of the time on her back with her genitals out of the water while males clasp her from either side with their flippers. Penises are often observed and SAGs are believed to play a role in mating. However, many SAGs do not lead directly to calving because: 1) SAGs have been seen during almost all months, whereas calving occurs primarily from December through March; 2) some SAGs comprise only males and/or juveniles; and 3) many of the focal females are already pregnant."
", "votes": null, "replies": [] }, { "id": 93491, "author_name": "visoft", "author_url": "", "post_date": "09/26/2015 16:15:39", "content": "Hi Christian! Thanx for the answer! Very useful!
\n\n1) I noted that there are many (~20) whales with one picture only. Is this by intention or by fate (i.e. "hard" to catch whales?)
\n\n2) What is the whale_25715 doing in w_7219 doing? And how to read the image?
\n\nThank you!
", "votes": null, "replies": [] }, { "id": 93518, "author_name": "cbkhan", "author_url": "", "post_date": "09/27/2015 02:24:06", "content": "[quote=visoft;93491]
\n\nHi Christian! Thanx for the answer! Very useful!
\n\n1) I noted that there are many (~20) whales with one picture only. Is this by intention or by fate (i.e. "hard" to catch whales?)
\n\n2) What is the whale_25715 doing in w_7219 doing? And how to read the image?
\n\nThank you!
\n\n[/quote]
\n\n1) This is just coincidence, we narrowed the images only by a.) that the image was matched to a whale in the catalog, and b.) that the image contained only one whale.
\n\n2.) The whale in image w_7219 is skim feeding which means swimming along at the surface of the water with mouth open and only upper jaw visible. It is a very challenging situation for whale matching and greatly complicates matters for whale researchers as well as we have less cues to work with (frustrating when you can't use whether or not the whale has lip callosities or chin callosities as a cue).
", "votes": null, "replies": [] }, { "id": 93599, "author_name": "johnblankenbaker", "author_url": "", "post_date": "09/28/2015 17:22:32", "content": "Following up on the question about the "representativeness" of the photos, am I right in assuming that a whale with many "portraits" might have several photos taken in rapid succession?
\n\nFor instance, in the photos of whale 33961 I'm willing to bet that that photos 9327 and 1045 were taken fairly close together in time - you can almost see the water on the left and right come together to form the burst of foam in 1054 as 33961 rolls.
\n\nSimilarly, photos 4827 and 8409 might have been snapped as the helicopter changed course onto the track of 33961 as he or she fed close to the surface.
\n\nFrom a competitor standpoint, this suggests that it might be useful to attempt to group similar test images and make sure that they all get the same label. To make this work you'd need some easy to compute indexing mechanism (not whale identity!) like water color and sea state to quickly find possible matches.
\n\nFrom a whale researcher standpoint, it suggests that a better accuracy measure on the test set could be had by presenting images separated by at least 10 minutes or something.
\n\nThanks for having this competition. I'm learning a lot about whales!
", "votes": null, "replies": [] }, { "id": 93605, "author_name": "rasimakhunzyanov", "author_url": "", "post_date": "09/28/2015 19:22:26", "content": "Hi Christian!
\n\nAbout one whale per picture. :-) Could anybody guess which of two whales on the w_751.jpg is whale_95091 ?\nSeems that the right one, but I'm not sure.
I'm agree with @John Blankenbaker, there are a lot of similar entries in training subset where one whale was captured two times within few seconds/minutes.
\n\nThanks in advance.
", "votes": null, "replies": [] }, { "id": 93680, "author_name": "arrchana", "author_url": "", "post_date": "09/30/2015 06:56:21", "content": "Hi,
\n\nI am new to Kaggle and this would be my first competition. I am trying to understand the data provided and I realize that in the train.csv file provided each image is labeled with an ID. I also get that in the submission file, each of the images provided need to be recognized as one or more of the whale IDs with varying probabilities. So my question is, which of the whale IDs is the Right Whale?
\n\nThanks in advance!
", "votes": null, "replies": [] }, { "id": 93894, "author_name": "cbkhan", "author_url": "", "post_date": "10/02/2015 14:54:16", "content": "[quote=John Blankenbaker;93599]
\n\nFollowing up on the question about the "representativeness" of the photos, am I right in assuming that a whale with many "portraits" might have several photos taken in rapid succession?
\n\nFor instance, in the photos of whale 33961 I'm willing to bet that that photos 9327 and 1045 were taken fairly close together in time - you can almost see the water on the left and right come together to form the burst of foam in 1054 as 33961 rolls.
\n\nSimilarly, photos 4827 and 8409 might have been snapped as the helicopter changed course onto the track of 33961 as he or she fed close to the surface.
\n\nFrom a competitor standpoint, this suggests that it might be useful to attempt to group similar test images and make sure that they all get the same label. To make this work you'd need some easy to compute indexing mechanism (not whale identity!) like water color and sea state to quickly find possible matches.
\n\nFrom a whale researcher standpoint, it suggests that a better accuracy measure on the test set could be had by presenting images separated by at least 10 minutes or something.
\n\nThanks for having this competition. I'm learning a lot about whales!
\n\n[/quote]
\n\nYes, there are sometimes a bunch of photos taken of the same whale on the same day in rapid succession, and although I can't confirm, it does look pretty likely that is the case with both sets of images you mentioned (9327 + 1045, and again with 4827 + 8409). Both pairs look to me as if they were taken from different vantage points as the aircraft circled overhead.
\n\nI don't have good perspective from a competitor standpoint, but from a researcher standpoint we would like the algorithm to stand alone when given only one image as we would most likely implement it as a software program that matches each image independently. Matching is particularly time-critical for us when we are out in the field (airplane, vessel) and in those scenarios we would likely photograph until we have a really good image and then feed that one image to the software program / algorithm.
\n\nGood point about measuring accuracy with images separated by 10 minutes, although I think that train has already left the station for this competition :-)
\n\nI am happy to hear you are learning a lot about whales. I have put an attachment on here that is a great summary of what I would like everyone to learn about right whales.
\n\n[quote=Arrchana;93680]
\n\nHi,
\n\nI am new to Kaggle and this would be my first competition. I am trying to understand the data provided and I realize that in the train.csv file provided each image is labeled with an ID. I also get that in the submission file, each of the images provided need to be recognized as one or more of the whale IDs with varying probabilities. So my question is, which of the whale IDs is the Right Whale?
\n\nThanks in advance!
\n\n[/quote]
\n\nHi,
\n\nAll of the images are of right whales (species name North Atlantic Right Whale, Eubalaena glacialis).
\n\nThe trick is to identify which individual right whale is in each photograph (analogous to saying that these are all humans but which one is Mary Jane and which one is John Smith, except that for the whales, we give them a number instead of a name).
\n\nHope that helps and good luck with your first try at competing!
", "votes": null, "replies": [] }, { "id": 95120, "author_name": "jumabek", "author_url": "", "post_date": "10/05/2015 11:04:31", "content": "Hi! I am a MS student. I have a question. My research area is Machine Learning for Computer Vision and this task fits exactly to my research interest. My question is can I publish my results if I participate in this competition?
", "votes": null, "replies": [] }, { "id": 95121, "author_name": "jumabek", "author_url": "", "post_date": "10/05/2015 11:06:41", "content": "When I say publish I mean publishing in some scholarly proceedings or journals.
", "votes": null, "replies": [] }, { "id": 95166, "author_name": "cbkhan", "author_url": "", "post_date": "10/05/2015 16:35:39", "content": "[quote=JumabekAlihanov;95121]
\n\nWhen I say publish I mean publishing in some scholarly proceedings or journals.
\n\n[/quote]
\n\nYes, I don't see why not. Just please send me an email that details your "data request" in a bit of detail so I can get confirmation from my supervisors (christin.khan@noaa.gov). I do not foresee any issues with that though.
", "votes": null, "replies": [] }, { "id": 96648, "author_name": "mouzmeensiraj", "author_url": "", "post_date": "10/19/2015 13:21:30", "content": "I want to ask how the main approach used for detection of right whale with the help of machine learning ?
", "votes": null, "replies": [] }, { "id": 96652, "author_name": "kburnham", "author_url": "", "post_date": "10/19/2015 13:44:49", "content": "Hello mouzmeen,
\n\nI am not sure if there is a main approach. However, based the posts in this forum, here are some things that are being tried:
\n\nI would imagine there are a number of other approaches that could be tried as well. You can learn a lot just by browsing through the forum posts and seeing where that leads you.
\n\nGood luck!
\n\nKevin
", "votes": null, "replies": [] }, { "id": 98135, "author_name": "pdxmanny", "author_url": "", "post_date": "11/04/2015 07:13:19", "content": "I just did a quick analysis of all the images and the resolutions for the images range from a 1590x1059 (w_9162.jpg) and 7010x4674 (w_5049.jpg). The most frequent resolution being 3072x2048 @ 3600 with the next frequent resolutions being: 5184x3456 @ 809, 2464x1648 @ 110, 3504x2336 @ 30 with the remaining resolution frequencies dropping below 20; with numerous unique image resolutions. Actually, there are 2,941 unique image resolutions in the image entire 11,469 image data set.
\n\nWith that being said, can someone...
\n\n1) ...explain why there is such a large range of image resolutions?\n2) ...provide us with both an upper and lower limit on the image resolutions that a solution will need to support?\nOn a slightly different topic, but a topic that has been raised previously, there are several whale IDs that have only one image. Ideally we would have several images for each whale like 4+, but is there a possibility of at least getting two images per whale ID?
\n\n -- Manny\nHehe, my bet is that the image size is somehow linked with the date when that image was taken (different cameras, different technologies).
\n\nAlso, afaik, the images are manually checked and cropped so the head is clearly visible and there is only one head in the image. Maybe another source of variation is this cropping.
\n\np.s. Can you [manually] check if the whale spots evolve as the image size grows? :)
", "votes": null, "replies": [] }, { "id": 98788, "author_name": "toshik", "author_url": "", "post_date": "11/13/2015 16:46:38", "content": "It is allowed to use bbox-labeling in this competition.
\nhttps://www.kaggle.com/c/noaa-right-whale-recognition/details/creating-a-face-detector-for-whales
By the way, silhouette based labeling is also allowed ?\n
[quote=toshi_k;98788]
\n\nIt is allowed to use bbox-labeling in this competition.
\n\nBy the way, silhouette based labeling is also allowed ?
\n\n[/quote]
\n\ntoshi_k:
\nThe images you just posted are brilliant. How did you produce them? bbox-labeling and/or silhouette based labeling, perhaps? Whatever you did, please let me know what it was.
Hi George,
\n\nThis image display silhouette based labeling. I draw mask path for each training images in Adobe Software.
\nAlthough, bbox-labeling can be put into practice in MATLAB Labeler,
\nsilhouette based labeling can not be put into practice in it.
When you say I draw mask path for each training images in Adobe Software, what do you mean? I am familiar with Photoshop, Illustrator and Adobe but none of those would seem suitable. Did you draw these silhouettes programmatically or by hand?
", "votes": null, "replies": [] }, { "id": 98812, "author_name": "toshik", "author_url": "", "post_date": "11/14/2015 02:51:39", "content": "The idea of silhouette based labeling is as follows.
\nFirst, draw silhouette by hand for each training images.
\nSecond, learn model which can generate silhouette images from original images (machine learning).
\nFinally, predict silhouette for each test images by learned model.
Imho this should be allowed. As long as you don't touch the test images. I mean, you will not be packed with the software so you will not be able to delineate fresh pictures taken ovre the high seas.
\n\nI think everybody uses some kind of manual input for training.
", "votes": null, "replies": [] }, { "id": 98821, "author_name": "toshik", "author_url": "", "post_date": "11/14/2015 07:25:39", "content": "Hi visoft,
\n\nThank you for your opinion !
\nI'll try this approach.
Apologies for the slow reply, but yes, variation in image size and resolution is because these photographs are taken by a variety of people with all different cameras and under a wide variety of circumstances. We curated the images in this competition so that they are all high quality photographs with only one whale in the frame which was of sufficient quality to identify manually, but there are many different cameras, lenses, and photographers represented.
\n\n[quote=pdx_manny;98135]
\n\nI just did a quick analysis of all the images and the resolutions for the images range from a 1590x1059 (w_9162.jpg) and 7010x4674 (w_5049.jpg). The most frequent resolution being 3072x2048 @ 3600 with the next frequent resolutions being: 5184x3456 @ 809, 2464x1648 @ 110, 3504x2336 @ 30 with the remaining resolution frequencies dropping below 20; with numerous unique image resolutions. Actually, there are 2,941 unique image resolutions in the image entire 11,469 image data set.
\n\nWith that being said, can someone...
\n\n1) ...explain why there is such a large range of image resolutions?\n2) ...provide us with both an upper and lower limit on the image resolutions that a solution will need to support?\nOn a slightly different topic, but a topic that has been raised previously, there are several whale IDs that have only one image. Ideally we would have several images for each whale like 4+, but is there a possibility of at least getting two images per whale ID?
\n\n -- Manny\n[/quote]
", "votes": null, "replies": [] }, { "id": 99509, "author_name": "pdxmanny", "author_url": "", "post_date": "11/25/2015 21:25:55", "content": "Hi Christin,
\n\nThanks for the reply on the image resolution topic, but is there a chance that we can get a range of camera resolutions...particularly the upper resolution range as the lower resolutions can be handled by filing in the unused pixels with zeros.\nAnd is there any chance on getting at least two images per whale ID? ;-)
\n\n- Manny\nUnfortunately I do not know the upper and lower bounds of image resolutions, and it's best not to depend on that knowledge anyhow as in the future, some folks will be using cell phones and some folks will be using really high end high resolution cameras.
\n\nI will think more on the other question but I think it is too late to make changes like that to the competition.
\n\n[quote=pdx_manny;99509]
\n\nHi Christin,
\n\nThanks for the reply on the image resolution topic, but is there a chance that we can get a range of camera resolutions...particularly the upper resolution range as the lower resolutions can be handled by filing in the unused pixels with zeros.\nAnd is there any chance on getting at least two images per whale ID? ;-)
\n\n- Manny\n[/quote]
", "votes": null, "replies": [] }, { "id": 100730, "author_name": "hrantkhachatrian", "author_url": "", "post_date": "12/09/2015 23:16:08", "content": "Does the color information help you professionals identify the whales? what would have been lost if you had only greyscale photographs?
\n\nThanks for this extremely challenging competition :)
", "votes": null, "replies": [] }, { "id": 100831, "author_name": "cbkhan", "author_url": "", "post_date": "12/10/2015 22:16:19", "content": "No - we primarily rely on non-color information so I think black and white images would work just as well
\n\n[quote=Hrant Khachatrian;100730]
\n\nDoes the color information help you professionals identify the whales? what would have been lost if you had only greyscale photographs?
\n\nThanks for this extremely challenging competition :)
\n\n[/quote]
", "votes": null, "replies": [] }, { "id": 103567, "author_name": "visoft", "author_url": "", "post_date": "01/04/2016 11:53:42", "content": "Hi Christin,\nHi Kaggle administrators!
\n\nThis was an interesting competition and the domain (ocean, whales) is a fresh breath of air in computer vision field.
\n\nI would like to implore you to make the data publicly available or to transform this competition into a yearly runner like Titanic or digit recognizer (101 competitions) I think that this will be a good thing for computer vision field, for NOAA and for kaggle:
\n\nIn the hope that I captured your interest, I wish you a Happy New Year!
\n\nCristian Vicas
", "votes": null, "replies": [] }, { "id": 103789, "author_name": "cbkhan", "author_url": "", "post_date": "01/06/2016 14:59:38", "content": "Cristian,
\n\nThanks so much for your active participation in this competition, I can't tell you how much I appreciate all the enthusiasm!
\n\nThe data will certainly be publicly available - just email me to request the data (Christin.Khan@noaa.gov).
\n\nI will certainly discuss your ideas with the folks at Kaggle as I of course would like to see this field advance and to support the marine science community. There are many other species that would also benefit from a similar competition so we will do our best to keep the ball rolling!
\n\n[quote=visoft;103567]
\n\nHi Christin,\nHi Kaggle administrators!
\n\nThis was an interesting competition and the domain (ocean, whales) is a fresh breath of air in computer vision field.
\n\nI would like to implore you to make the data publicly available or to transform this competition into a yearly runner like Titanic or digit recognizer (101 competitions) I think that this will be a good thing for computer vision field, for NOAA and for kaggle:
\n\nIn the hope that I captured your interest, I wish you a Happy New Year!
\n\nCristian Vicas
\n\n[/quote]
", "votes": null, "replies": [] }, { "id": 104282, "author_name": "visoft", "author_url": "", "post_date": "01/11/2016 11:33:21", "content": "Thank you very much!
\n\nI will keep an eye on the kaggle forums for updates!
", "votes": null, "replies": [] }, { "id": 171861, "author_name": "thepodthebook", "author_url": "", "post_date": "03/31/2017 18:29:46", "content": "What is the circumference of the outermost (skin) part of the peduncle? I've been curious how big that area has to be to support everything going into the flukes.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "90645": "Welcome to the competition! My name is Christin Khan and I am a fishery biologist at NOAA working on right whale issues. If you have any general questions about right whales, I am happy to answer them here. \r\n\r\nThank you for your participation, this project has real conservation value for the species!", "91208": "Really nice competition. Also in the future for better performance results maybe an infrared camera/thermal cameras would be more useful or add features for the models. But in general, very nice competition. Thanks for this chance.", "91236": "And now we have a better than random guess result on the leaderboard...sooner than I expected.", "91245": "[quote=James King;91236]\r\n\r\nAnd now we have a better than random guess result on the leaderboard...sooner than I expected.\r\n\r\n[/quote]\r\n\r\nI always hate when my algorithm does worse than a random guess. :-)", "91262": "I suspect it takes some data prep to beat random guessing. I plan to start with whale_95370 and try to distinguish it from all other whales. His facial markings are quite distinctive.\r\n\r\n![enter image description here][1]\r\n\r\n\r\n [1]: https://mail.google.com/mail/u/0/?ui=2&ik=24cefad02d&view=fimg&th=14f8601a77d96c94&attid=0.1&disp=emb&realattid=ii_14f8600e943bba3e&attbid=ANGjdJ8GxwCU9Em4jkr8r7ZI_82DPKgAjlFsGvynBc0zlAp9ayhnIxrwTKKqlOs9wLJYOqgA6XDaOkYZ5VE6oYhiX6iiA8KWzIwW5mNctvUqZYL81vdFW19jyx44nxU&sz=w460-h254&ats=1441137585574&rm=14f8601a77d96c94&zw&atsh=1\r\n\r\nBut I think I need to first discard the images where the face cannot be seen.", "91273": "Doesn't look like the same whale (95370) to me. Do those white thinger-doodles change over time, e.g., so that one of the images was from a younger whale?\r\n\r\n**w_3076**\r\n\r\n![enter image description here][1]\r\n\r\n**w_3085**\r\n\r\n![enter image description here][2]\r\n\r\n\r\n [1]: http://i.imgur.com/ocAN22n.png\r\n [2]: http://i.imgur.com/m1RhU22.png", "91275": "Very good question inversion, it looks as if the bottom one is from a younger whale before the line was fully connected. Or maybe it had a fight and created issues with his/her lines.", "91379": "The white markings on top of the whale's head are called callosities and they do change very slightly over time, particularly when the whale is still young, so it is reasonable to expect slight deviations such as this from one photograph to another. On the other hand, there are some whales that look very very similar to each other. For more background on whale matching, visit these websites:\r\n\r\nhttp://www.neaq.org/education_and_activities/games_and_activities/online_games/right_whale_identification_games.php\r\n\r\nand \r\n\r\nhttp://rwcatalog.neaq.org/Default.aspx", "91380": "It may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head. Right whales are first divided into two broad categories to describe the overall appearance of the main section of the callosity- continuous and broken. Next we usually discuss how many islands or peninsulas there are and on which side of the head. More subtle but important cues are the presence and size of lip callosities, and how many post blowhole and chin callosities there are.\r\n\r\n![Sketch of right whale callosities and names of features by Kelly Dilliard][1]\r\n\r\n\r\n [1]: https://teacheratsea.files.wordpress.com/2015/05/img_2292.jpg", "91381": "[quote=Christin B. Khan;91380]\r\nIt may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head.\r\n[/quote]\r\n\r\nSo, you're saying \"thinger-doodles\" isn't part of the marine biology vernacular? :-)", "91396": "\"thinger-doodles\" gave me quite a smile :-)", "91417": "[quote=Christin B. Khan;91380]\r\n\r\nIt may be helpful folks to be aware of some of the common terminology we use when discussing the callosity pattern on top of a whale's head. \r\n\r\n[/quote]\r\n\r\nSo, Christin, are the callosity patterns the main feature used to distinguish right whales from other whale species? Are there other features that are useful in distinguishing a right whale from another type of whale?", "91432": "Well they have the twin blowholes (sort of a blow-nostril) as you see in the photos, and no dorsal fin. Also the male right whale's whalehood is without peer in the animal kingdom, although you usually would not be able to notice this from above the water.\r\n\r\nhttps://en.wikipedia.org/wiki/Right_whale", "91439": "Is the whale in w_8026.jpg pooping?", "91455": "KA hahaha, I cannot control my laugh :) Out of all these data, you're focused on this :)", "91479": "Matt Wenger - [Here](http://www.neaq.org/conservation_and_research/projects/endangered_species_habitats/right_whale_research/right_whale_projects/monitoring_individuals_and_family_trees/identifying_with_photographs/how_it_works/callosity_patterns.php) are some pages that talk about how to identify the whales. It is mainly based on the callosity pattern, but at the bottom of the page is a link to other identifying features.", "91504": "Kevin Burnham - Thanks! This is very useful.", "91518": "Matt Wenger - right whale are identified from other whale species primarily by:\r\n\r\n - 50 feet long\r\n - v-shaped blow\r\n - lack of a dorsal fin\r\n - callosities on their head\r\n - square shaped dark flippers", "91519": "\"Is the whale in w_8026.jpg pooping?\"\r\n\r\n@KA - yes! nice find! We marine biologists refer to it as defecation", "91522": "[quote=KA;91439]\r\n\r\nIs the whale in w_8026.jpg pooping?\r\n\r\n[/quote]\r\n\r\nhahahahahhahaha", "91523": "[quote=Christin B. Khan;91519]\r\n\r\n\"Is the whale in w_8026.jpg pooping?\"\r\n\r\n@KA - yes! nice find! We marine biologists refer to it as defecation\r\n\r\n[/quote]\r\n\r\neven a bigger hahahahahahahah", "91647": "I have a question.\r\n\r\nYou provide MATLAB license to students for this competition.