After reading some discussions and past competition solutions, I thought that one approach to this competition would be to use 2-stage model prediction. (This may be obvious to many of the participants).
\nI don't know if this approach will work for this competition, but if anyone has already tried it, I would appreciate it if you could share your results.
Image classification: Using general image classification (CNN, vit, etc.), classify 30 types (actually, 26 species).
\nIndividual prediction: Identify individuals using current mainstream baseline methods such as Arcface.
\nIn addition to the above, as shown in this discussion, there are a lot of photos of individuals with only one sample, so it is important to select pre/post processing and models that can cope with this challenge.
", "messages": [ { "id": "1688454", "postDate": "02/13/2022 16:22:12", "content": "After reading some discussions and past competition solutions, I thought that one approach to this competition would be to use 2-stage model prediction. (This may be obvious to many of the participants).
\nI don't know if this approach will work for this competition, but if anyone has already tried it, I would appreciate it if you could share your results.
Image classification: Using general image classification (CNN, vit, etc.), classify 30 types (actually, 26 species).
\nIndividual prediction: Identify individuals using current mainstream baseline methods such as Arcface.
\nIn addition to the above, as shown in this discussion, there are a lot of photos of individuals with only one sample, so it is important to select pre/post processing and models that can cope with this challenge.
", "rawMarkdown": "## Overview\nAfter reading some discussions and past competition solutions, I thought that one approach to this competition would be to use 2-stage model prediction. (This may be obvious to many of the participants).\nI don't know if this approach will work for this competition, but if anyone has already tried it, I would appreciate it if you could share your results.\n\n\n## 2 stage model prediction\n#### stage1: 'Species' prediction\nImage classification: Using general image classification (CNN, vit, etc.), classify 30 types (actually, [26 species](https://www.kaggle.com/c/happy-whale-and-dolphin/discussion/305341)).\n\n#### stage2:'Individual' prediction\nIndividual prediction: Identify individuals using current mainstream baseline methods such as Arcface.\n\nIn addition to the above, as shown in this discussion, there are a lot of photos of individuals with [only one sample](https://www.kaggle.com/c/happy-whale-and-dolphin/discussion/305428), so it is important to select pre/post processing and models that can cope with this challenge.", "votes": null }, { "id": "1689089", "postDate": "02/14/2022 03:11:04", "content": "Thanks for sharing your thoughts @mujrush ! Y this is definitely a tough one. it might be even more complex as there is extreme difference in the views, thus very different looking images for.the same instance.
\nIt would benifit a lot if we can someone be able to identify between the different views. To either treat them as different individuals during training so their embedding location could be separate.
\nYou might be interested in this paper that first predicts the view and have multiple pair view loss based on the views. And they establish that making the model view invariant helps contrastive learning significantly. But it requires labels that say if two postive images are from the same view or different.
\npaper
Thank you for your comments and for sharing paper!
\nI also read your paper and found it very interesting.
\nIt was a bit difficult for me to understand😅
I think you are right, we need to deal with not only the one sample problem, but also the problem of recognizing the same individual in different views.
\nAnd when the above two are combined (i.e., when there is only one sample in the train data and the same individual in different views in the test data), it becomes even more complicated😂
I'm pondering three models (two in parallel):
\nFeed those two + source image into :
\nIt's great if you could just get the main idea of what they were trying to solve and how.
\nYes indeed, we got to make sure that our encoder is smart enough to understand that. 😅
", "rawMarkdown": "It's great if you could just get the main idea of what they were trying to solve and how. \n\nYes indeed, we got to make sure that our encoder is smart enough to understand that. 😅", "votes": null }, { "id": "1690709", "postDate": "02/15/2022 04:18:34", "content": "That sounds creative, curious to know how you would identify these views. Thanks for sharing your thoughts @lobrien
", "rawMarkdown": "That sounds creative, curious to know how you would identify these views. Thanks for sharing your thoughts @lobrien", "votes": null }, { "id": "1691602", "postDate": "02/15/2022 13:52:12", "content": "@lobrien
\nThree models… It's so cool!
\nEspecially this part.(I've already made good progress on this.)
Thanks for sharing your advanced approach.
\nIt's going to take time and patience😆
I was thinking about the same approach. But I am afraid of a mistake of species classifier. The overall error is equal to the multiplication of every stage error.
\nI came up with the idea to attach species classification logits/embedding to some kind of ArcFace embedding. I hope it will allow the model to decide that is more important in a particular cases - species or raw embedding. What do you think about this idea?
", "rawMarkdown": "I was thinking about the same approach. But I am afraid of a mistake of species classifier. The overall error is equal to the multiplication of every stage error. \n\nI came up with the idea to attach species classification logits/embedding to some kind of ArcFace embedding. I hope it will allow the model to decide that is more important in a particular cases - species or raw embedding. What do you think about this idea?", "votes": null }, { "id": "1696472", "postDate": "02/18/2022 21:29:09", "content": "Haha, I was thinking about 3 stage approach 😆😆
\n1) Whale Vs Dolphin
\n2) Species
\n3) Individual
But 2 stage is sufficient as we have 28 species
", "rawMarkdown": "Haha, I was thinking about 3 stage approach 😆😆\n1) Whale Vs Dolphin\n2) Species\n3) Individual\n\nBut 2 stage is sufficient as we have 28 species", "votes": null }, { "id": "1696491", "postDate": "02/18/2022 22:28:36", "content": "Do you expect any performace boost by splitting species recognition into whale/dolphin -> species prediction? Some whales are visually similar to dolphins. It add more complexity to task becase whale/dolphin error case prior error in final species, isn't it?
", "rawMarkdown": "Do you expect any performace boost by splitting species recognition into whale/dolphin -> species prediction? Some whales are visually similar to dolphins. It add more complexity to task becase whale/dolphin error case prior error in final species, isn't it?", "votes": null }, { "id": "1698420", "postDate": "02/20/2022 11:25:08", "content": "@meowmeowmeowmeowmeow
\nIs it something like stacking?
\nThat's interesting!
Since I don't understand the details about arcface, I'm sorry I don't know the theoretical behavior of entering logits into arcface, but I think you need to do the kaggler's rule, 'do everything' in such cases😅
", "rawMarkdown": "meowmeowmeowmeowmeow \nIs it something like stacking?\nThat's interesting!\n\nSince I don't understand the details about arcface, I'm sorry I don't know the theoretical behavior of entering logits into arcface, but I think you need to do the kaggler's rule, 'do everything' in such cases😅", "votes": null }, { "id": "1698576", "postDate": "02/20/2022 13:46:21", "content": "I am not familiar with ArcFace yet too (:
\nThe approach I've described isn't about stacking. It is about incorporating additional features to the arc face rather than stacking models.
\nI will start with kaggler's rule :)
\"Kaggler's rule\"-love it!
\nI have just returned to Kaggle after my Chai break and this is the third time I'm quoting @cpmpml in a week.
\nI hope its not a bad thing to quote people you respect? 🙏
\nCPMP mentioned this in reply to
\nQ: \"What have you learned in your Kaggle journey?\"
\nA: Try your ideas instead of asking if they would work :)
", "rawMarkdown": "\"Kaggler's rule\"-love it! \n\nI have just returned to Kaggle after my Chai break and this is the third time I'm quoting @cpmpml in a week. \n\nI hope its not a bad thing to quote people you respect? 🙏\n\nCPMP mentioned [this](https://twitter.com/JFPuget/status/1443902657630855170?s=20&t=ggjUxiA0LHqqiYhdsCxXQg) in reply to\n\nQ: \"What have you learned in your Kaggle journey?\"\n\nA: Try your ideas instead of asking if they would work :)", "votes": null }, { "id": "1701618", "postDate": "02/22/2022 23:43:50", "content": "Without experiment, you never know.
", "rawMarkdown": "Without experiment, you never know.", "votes": null }, { "id": "1701759", "postDate": "02/23/2022 03:56:24", "content": "\nA: Try your ideas instead of asking if they would work :)
\nThat's a nice answer!
I want to have the coding skill to try many ideas 😆
", "rawMarkdown": "init27 \n\n`A: Try your ideas instead of asking if they would work :)`\nThat's a nice answer!\n\nI want to have the coding skill to try many ideas 😆", "votes": null }, { "id": "1701796", "postDate": "02/23/2022 04:29:45", "content": "I feel the same way 🙏
", "rawMarkdown": "I feel the same way 🙏", "votes": null }, { "id": "1701808", "postDate": "02/23/2022 04:43:53", "content": "Yes, I refer to this as…
\n\"Suck it and see\"
\nIt's how science is done.
", "rawMarkdown": "Yes, I refer to this as...\n\n\"Suck it and see\"\n\nIt's how science is done.", "votes": null } ], "comments": [ { "id": 1689089, "author_name": "bsridatta", "author_url": "", "post_date": "02/14/2022 03:11:04", "content": "Thanks for sharing your thoughts @mujrush ! Y this is definitely a tough one. it might be even more complex as there is extreme difference in the views, thus very different looking images for.the same instance.
\nIt would benifit a lot if we can someone be able to identify between the different views. To either treat them as different individuals during training so their embedding location could be separate.
\nYou might be interested in this paper that first predicts the view and have multiple pair view loss based on the views. And they establish that making the model view invariant helps contrastive learning significantly. But it requires labels that say if two postive images are from the same view or different.
\npaper
Thank you for your comments and for sharing paper!
\nI also read your paper and found it very interesting.
\nIt was a bit difficult for me to understand😅
I think you are right, we need to deal with not only the one sample problem, but also the problem of recognizing the same individual in different views.
\nAnd when the above two are combined (i.e., when there is only one sample in the train data and the same individual in different views in the test data), it becomes even more complicated😂
It's great if you could just get the main idea of what they were trying to solve and how.
\nYes indeed, we got to make sure that our encoder is smart enough to understand that. 😅
", "votes": null, "replies": [] } ] }, { "id": 1690154, "author_name": "lobrien", "author_url": "", "post_date": "02/14/2022 18:28:09", "content": "I'm pondering three models (two in parallel):
\nFeed those two + source image into :
\nThat sounds creative, curious to know how you would identify these views. Thanks for sharing your thoughts @lobrien
", "votes": null, "replies": [] }, { "id": 1691602, "author_name": "mujrush", "author_url": "", "post_date": "02/15/2022 13:52:12", "content": "@lobrien
\nThree models… It's so cool!
\nEspecially this part.(I've already made good progress on this.)
Thanks for sharing your advanced approach.
\nIt's going to take time and patience😆
I was thinking about the same approach. But I am afraid of a mistake of species classifier. The overall error is equal to the multiplication of every stage error.
\nI came up with the idea to attach species classification logits/embedding to some kind of ArcFace embedding. I hope it will allow the model to decide that is more important in a particular cases - species or raw embedding. What do you think about this idea?
", "votes": null, "replies": [ { "id": 1698420, "author_name": "mujrush", "author_url": "", "post_date": "02/20/2022 11:25:08", "content": "@meowmeowmeowmeowmeow
\nIs it something like stacking?
\nThat's interesting!
Since I don't understand the details about arcface, I'm sorry I don't know the theoretical behavior of entering logits into arcface, but I think you need to do the kaggler's rule, 'do everything' in such cases😅
", "votes": null, "replies": [] }, { "id": 1698576, "author_name": "meowmeowmeowmeowmeow", "author_url": "", "post_date": "02/20/2022 13:46:21", "content": "I am not familiar with ArcFace yet too (:
\nThe approach I've described isn't about stacking. It is about incorporating additional features to the arc face rather than stacking models.
\nI will start with kaggler's rule :)
\"Kaggler's rule\"-love it!
\nI have just returned to Kaggle after my Chai break and this is the third time I'm quoting @cpmpml in a week.
\nI hope its not a bad thing to quote people you respect? 🙏
\nCPMP mentioned this in reply to
\nQ: \"What have you learned in your Kaggle journey?\"
\nA: Try your ideas instead of asking if they would work :)
", "votes": null, "replies": [] }, { "id": 1701759, "author_name": "mujrush", "author_url": "", "post_date": "02/23/2022 03:56:24", "content": "\nA: Try your ideas instead of asking if they would work :)
\nThat's a nice answer!
I want to have the coding skill to try many ideas 😆
", "votes": null, "replies": [] }, { "id": 1701796, "author_name": "init27", "author_url": "", "post_date": "02/23/2022 04:29:45", "content": "I feel the same way 🙏
", "votes": null, "replies": [] }, { "id": 1701808, "author_name": "mutantspore", "author_url": "", "post_date": "02/23/2022 04:43:53", "content": "Yes, I refer to this as…
\n\"Suck it and see\"
\nIt's how science is done.
", "votes": null, "replies": [] } ] }, { "id": 1696472, "author_name": "rajborgohain", "author_url": "", "post_date": "02/18/2022 21:29:09", "content": "Haha, I was thinking about 3 stage approach 😆😆
\n1) Whale Vs Dolphin
\n2) Species
\n3) Individual
But 2 stage is sufficient as we have 28 species
", "votes": null, "replies": [ { "id": 1696491, "author_name": "meowmeowmeowmeowmeow", "author_url": "", "post_date": "02/18/2022 22:28:36", "content": "Do you expect any performace boost by splitting species recognition into whale/dolphin -> species prediction? Some whales are visually similar to dolphins. It add more complexity to task becase whale/dolphin error case prior error in final species, isn't it?
", "votes": null, "replies": [] }, { "id": 1701618, "author_name": "rajborgohain", "author_url": "", "post_date": "02/22/2022 23:43:50", "content": "Without experiment, you never know.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1688454": "## Overview\nAfter reading some discussions and past competition solutions, I thought that one approach to this competition would be to use 2-stage model prediction. (This may be obvious to many of the participants).\nI don't know if this approach will work for this competition, but if anyone has already tried it, I would appreciate it if you could share your results.\n\n\n## 2 stage model prediction\n#### stage1: 'Species' prediction\nImage classification: Using general image classification (CNN, vit, etc.), classify 30 types (actually, [26 species](https://www.kaggle.com/c/happy-whale-and-dolphin/discussion/305341)).\n\n#### stage2:'Individual' prediction\nIndividual prediction: Identify individuals using current mainstream baseline methods such as Arcface.\n\nIn addition to the above, as shown in this discussion, there are a lot of photos of individuals with [only one sample](https://www.kaggle.com/c/happy-whale-and-dolphin/discussion/305428), so it is important to select pre/post processing and models that can cope with this challenge.", "1689089": "Thanks for sharing your thoughts @mujrush ! Y this is definitely a tough one. it might be even more complex as there is extreme difference in the views, thus very different looking images for.the same instance. \n\nIt would benifit a lot if we can someone be able to identify between the different views. To either treat them as different individuals during training so their embedding location could be separate.\n\n You might be interested in this paper that first predicts the view and have multiple pair view loss based on the views. And they establish that making the model view invariant helps contrastive learning significantly. But it requires labels that say if two postive images are from the same view or different. \n[paper](https://www.google.com/url?sa=t&source=web&rct=j&url=https://arxiv.org/abs/1910.04104&ved=2ahUKEwit96btm_71AhV5Q_EDHVeQBK4QFnoECAUQAQ&usg=AOvVaw2cnujaPLLNmyUkD9R0i3A_)", "1689819": "bsridatta \n\nThank you for your comments and for sharing paper!\nI also read your paper and found it very interesting.\nIt was a bit difficult for me to understand😅\n\nI think you are right, we need to deal with not only the one sample problem, but also the problem of recognizing the same individual in different views.\nAnd when the above two are combined (i.e., when there is only one sample in the train data and the same individual in different views in the test data), it becomes even more complicated😂", "1690154": "I'm pondering _three_ models (two in parallel):\n\n- Species identification\n- Camera viewpoint (port full body, starboard full body, port dorsal fin crop, starboard dorsal fin crop, ...) etc. (I've already made good progress on this.)\n\nFeed those two + source image into : \n\n- Twin network(s) (One can well imagine \"dorsal fin geometry\" and \"head callosities\" problems are different enough that they'd be better solved by multiple networks, not just throwing everything into a single model.)", "1690705": "It's great if you could just get the main idea of what they were trying to solve and how. \n\nYes indeed, we got to make sure that our encoder is smart enough to understand that. 😅", "1690709": "That sounds creative, curious to know how you would identify these views. Thanks for sharing your thoughts @lobrien", "1691602": "lobrien \nThree models... It's so cool!\nEspecially this part.` (I've already made good progress on this.)`\n\nThanks for sharing your advanced approach.\nIt's going to take time and patience😆", "1696182": "I was thinking about the same approach. But I am afraid of a mistake of species classifier. The overall error is equal to the multiplication of every stage error. \n\nI came up with the idea to attach species classification logits/embedding to some kind of ArcFace embedding. I hope it will allow the model to decide that is more important in a particular cases - species or raw embedding. What do you think about this idea?", "1696472": "Haha, I was thinking about 3 stage approach 😆😆\n1) Whale Vs Dolphin\n2) Species\n3) Individual\n\nBut 2 stage is sufficient as we have 28 species", "1696491": "Do you expect any performace boost by splitting species recognition into whale/dolphin -> species prediction? Some whales are visually similar to dolphins. It add more complexity to task becase whale/dolphin error case prior error in final species, isn't it?", "1698420": "meowmeowmeowmeowmeow \nIs it something like stacking?\nThat's interesting!\n\nSince I don't understand the details about arcface, I'm sorry I don't know the theoretical behavior of entering logits into arcface, but I think you need to do the kaggler's rule, 'do everything' in such cases😅", "1698576": "I am not familiar with ArcFace yet too (:\nThe approach I've described isn't about stacking. It is about incorporating additional features to the arc face rather than stacking models.\nI will start with kaggler's rule :)", "1698749": "\"Kaggler's rule\"-love it! \n\nI have just returned to Kaggle after my Chai break and this is the third time I'm quoting @cpmpml in a week. \n\nI hope its not a bad thing to quote people you respect? 🙏\n\nCPMP mentioned [this](https://twitter.com/JFPuget/status/1443902657630855170?s=20&t=ggjUxiA0LHqqiYhdsCxXQg) in reply to\n\nQ: \"What have you learned in your Kaggle journey?\"\n\nA: Try your ideas instead of asking if they would work :)", "1701618": "Without experiment, you never know.", "1701759": "init27 \n\n`A: Try your ideas instead of asking if they would work :)`\nThat's a nice answer!\n\nI want to have the coding skill to try many ideas 😆", "1701796": "I feel the same way 🙏", "1701808": "Yes, I refer to this as...\n\n\"Suck it and see\"\n\nIt's how science is done." }, "source": "meta" }