{
  "id": 309214,
  "title": "🔥 DATASET - dorsal fins for all IDs without background   🔥 ",
  "url": "/competitions/happy-whale-and-dolphin/discussion/309214",
  "author_name": "",
  "post_date": "2022-02-22T11:42:09.764495100Z",
  "votes": 97,
  "comment_count": 34,
  "views": 0,
  "content": "<p>We have perfomed some test with new dataset generation - it is automatically generated from Happywhale dataset. Do you like it?</p>\n<p>Full implementation - removing background from images: <a href=\"https://www.kaggle.com/remekkinas/remove-background-salient-object-detection\" target=\"_blank\">https://www.kaggle.com/remekkinas/remove-background-salient-object-detection</a></p>\n<p>Below you can see dorsal fins dataset collected from 50k train images.<br>\n<img src=\"https://i.ibb.co/r2GMZ6h/grid.jpg\" alt=\"dataset\"></p>\n<p><img src=\"https://i.ibb.co/qy5TmJ7/set.jpg\" alt=\"data\"></p>",
  "messages": [
    {
      "id": "1700921",
      "postDate": "02/22/2022 11:42:09",
      "content": "<p>We have perfomed some test with new dataset generation - it is automatically generated from Happywhale dataset. Do you like it?</p>\n<p>Full implementation - removing background from images: <a href=\"https://www.kaggle.com/remekkinas/remove-background-salient-object-detection\" target=\"_blank\">https://www.kaggle.com/remekkinas/remove-background-salient-object-detection</a></p>\n<p>Below you can see dorsal fins dataset collected from 50k train images.<br>\n<img src=\"https://i.ibb.co/r2GMZ6h/grid.jpg\" alt=\"dataset\"></p>\n<p><img src=\"https://i.ibb.co/qy5TmJ7/set.jpg\" alt=\"data\"></p>",
      "rawMarkdown": "We have perfomed some test with new dataset generation - it is automatically generated from Happywhale dataset. Do you like it?\n\nFull implementation - removing background from images: https://www.kaggle.com/remekkinas/remove-background-salient-object-detection\n\nBelow you can see dorsal fins dataset collected from 50k train images.\n![dataset](https://i.ibb.co/r2GMZ6h/grid.jpg)\n\n![data](https://i.ibb.co/qy5TmJ7/set.jpg)",
      "votes": null
    },
    {
      "id": "1701177",
      "postDate": "02/22/2022 15:09:30",
      "content": "<p>Is the FIN unique biologically?  like finger print?</p>",
      "rawMarkdown": "Is the FIN unique biologically?  like finger print?",
      "votes": null
    },
    {
      "id": "1701188",
      "postDate": "02/22/2022 15:17:32",
      "content": "<p>it is species dependent but in general yes that is one way to identify an individual animal.</p>",
      "rawMarkdown": "it is species dependent but in general yes that is one way to identify an individual animal.",
      "votes": null
    },
    {
      "id": "1701243",
      "postDate": "02/22/2022 15:53:46",
      "content": "<p>I think when you want to recognize people …. you have to act on face not whole person. <br>\nMost reserches I went through describe dorsal fin as a finger print (in most cases). Then you can use additional features (as a helpers or in in special cases as a primary feature - some signs are seasonal).</p>\n<p>When you use whole picture … you search similarity between one landscape and another one. 😂</p>",
      "rawMarkdown": "I think when you want to recognize people .... you have to act on face not whole person. \nMost reserches I went through describe dorsal fin as a finger print (in most cases). Then you can use additional features (as a helpers or in in special cases as a primary feature - some signs are seasonal).\n\nWhen you use whole picture ... you search similarity between one landscape and another one. 😂",
      "votes": null
    },
    {
      "id": "1701640",
      "postDate": "02/23/2022 00:42:26",
      "content": "<p><a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> did you segmentation masks to create this ? If so can you share the dataset ?</p>",
      "rawMarkdown": "remekkinas did you segmentation masks to create this ? If so can you share the dataset ?",
      "votes": null
    },
    {
      "id": "1701835",
      "postDate": "02/23/2022 05:12:48",
      "content": "<p>Awesome, are you planning to share how you were able to do it?</p>",
      "rawMarkdown": "Awesome, are you planning to share how you were able to do it?",
      "votes": null
    },
    {
      "id": "1701839",
      "postDate": "02/23/2022 05:15:12",
      "content": "<p>thanks for reply.   I am worry about the image quality if good enough to distinguish them?</p>",
      "rawMarkdown": "thanks for reply.   I am worry about the image quality if good enough to distinguish them?",
      "votes": null
    },
    {
      "id": "1701844",
      "postDate": "02/23/2022 05:23:38",
      "content": "<p>You are absolutely right but a bit more complicated when you only get certain views of a person and the sample size of training data is small. Need to incorporate a lot of different unique characteristics.</p>",
      "rawMarkdown": "You are absolutely right but a bit more complicated when you only get certain views of a person and the sample size of training data is small. Need to incorporate a lot of different unique characteristics.",
      "votes": null
    },
    {
      "id": "1701894",
      "postDate": "02/23/2022 06:34:40",
      "content": "<p>We are working on solution quality and dataset cleaning (it is one of the most important part of this competition - and not anly in this one). You can see first model result. Now we have 3rd edition …. training now.</p>",
      "rawMarkdown": "We are working on solution quality and dataset cleaning (it is one of the most important part of this competition - and not anly in this one). You can see first model result. Now we have 3rd edition .... training now.",
      "votes": null
    },
    {
      "id": "1701895",
      "postDate": "02/23/2022 06:36:25",
      "content": "<p><a href=\"https://www.kaggle.com/uheheu\" target=\"_blank\">@uheheu</a> you are right. To create good solution here inference pipeline will be important. There will be a lot of \"ifs\" I thnink. I went through almost 50-60% of dataset and see some groups now where we should propose different inference / recognition pipelines.</p>",
      "rawMarkdown": "uheheu you are right. To create good solution here inference pipeline will be important. There will be a lot of \"ifs\" I thnink. I went through almost 50-60% of dataset and see some groups now where we should propose different inference / recognition pipelines.",
      "votes": null
    },
    {
      "id": "1701896",
      "postDate": "02/23/2022 06:37:55",
      "content": "<p>This is kind of segmentation … but more advanced.<br>\nSo far we are working on dataset quality. Some dorsal fins are blury. Then we decide about publishing. </p>",
      "rawMarkdown": "This is kind of segmentation ... but more advanced.\nSo far we are working on dataset quality. Some dorsal fins are blury. Then we decide about publishing.",
      "votes": null
    },
    {
      "id": "1701903",
      "postDate": "02/23/2022 06:44:03",
      "content": "<p>We will see. For sure as a final soultion. Now we have to deal with quality. Some fins are still blurred and as I can see we loose important information in image (edges are blurred for some fins). But so far we managed to build full automated pipeline to create new dataset - without background and normalized crops (for all cases in dataset). Model plays additional role - cleaner - is trying to recognize quality of source image. It is quite complex/tricky pipeline now but works. </p>",
      "rawMarkdown": "We will see. For sure as a final soultion. Now we have to deal with quality. Some fins are still blurred and as I can see we loose important information in image (edges are blurred for some fins). But so far we managed to build full automated pipeline to create new dataset - without background and normalized crops (for all cases in dataset). Model plays additional role - cleaner - is trying to recognize quality of source image. It is quite complex/tricky pipeline now but works.",
      "votes": null
    },
    {
      "id": "1701928",
      "postDate": "02/23/2022 07:10:20",
      "content": "<p><a href=\"https://www.kaggle.com/dragonzhang\" target=\"_blank\">@dragonzhang</a> image of source quality is quite good ore even very good. Certainly you can find low quality photos but … this happens in all computer vision cases (nothing new). You have 2 months still to work on this topic :) </p>",
      "rawMarkdown": "dragonzhang image of source quality is quite good ore even very good. Certainly you can find low quality photos but ... this happens in all computer vision cases (nothing new). You have 2 months still to work on this topic :)",
      "votes": null
    },
    {
      "id": "1702020",
      "postDate": "02/23/2022 09:04:14",
      "content": "<p>thanks for your reply.  Due to time consuming hardware,  I prefer more just to think about it. :)  </p>",
      "rawMarkdown": "thanks for your reply.  Due to time consuming hardware,  I prefer more just to think about it. :)",
      "votes": null
    },
    {
      "id": "1702027",
      "postDate": "02/23/2022 09:09:58",
      "content": "<p>For sure you do not need information from whole image - factories, boats, coast, forest are not required for whale similarity check :)</p>",
      "rawMarkdown": "For sure you do not need information from whole image - factories, boats, coast, forest are not required for whale similarity check :)",
      "votes": null
    },
    {
      "id": "1702349",
      "postDate": "02/23/2022 14:37:40",
      "content": "<p>Nice, thanks for the insights!</p>",
      "rawMarkdown": "Nice, thanks for the insights!",
      "votes": null
    },
    {
      "id": "1702362",
      "postDate": "02/23/2022 14:49:05",
      "content": "<p>Today I created … 4 edition of model …. now it works almost perfect - from ~73.000 photos only do not understand ~500 … (can't find dorsal fin or whole whale). Most of problems are with whale where I have tail fin or … really small object. </p>",
      "rawMarkdown": "Today I created ... 4 edition of model .... now it works almost perfect - from ~73.000 photos only do not understand ~500 ... (can't find dorsal fin or whole whale). Most of problems are with whale where I have tail fin or ... really small object.",
      "votes": null
    },
    {
      "id": "1702585",
      "postDate": "02/23/2022 18:15:34",
      "content": "<p>Congrats for reaching GM tier position Remek.</p>\n<p>By the way, nice dorsal fins topic.</p>",
      "rawMarkdown": "Congrats for reaching GM tier position Remek.\n\nBy the way, nice dorsal fins topic.",
      "votes": null
    },
    {
      "id": "1702760",
      "postDate": "02/23/2022 21:58:09",
      "content": "<p>Thank you very much <a href=\"https://www.kaggle.com/mpwolke\" target=\"_blank\">@mpwolke</a>. 🙏🙏🙏</p>\n<p>Yes, I am still working on this. 3 NN (2 models) works on such effect. I am very pleased with the effect of the first one, but the next two still require a lot of work to make it work well. However, I see great potential in the solution because you can create a system that will separate the fins or whales from the environment.  </p>",
      "rawMarkdown": "Thank you very much @mpwolke. 🙏🙏🙏\n\nYes, I am still working on this. 3 NN (2 models) works on such effect. I am very pleased with the effect of the first one, but the next two still require a lot of work to make it work well. However, I see great potential in the solution because you can create a system that will separate the fins or whales from the environment.",
      "votes": null
    },
    {
      "id": "1702912",
      "postDate": "02/24/2022 04:00:03",
      "content": "<p>Since a generation network can capture any information of whales in the image, if it is based on stylegan, it can linearly divide these whales by using his latent code.</p>\n<p>Using gan to generate is very contradictory. Since it can be generated, the information that distinguishes these targets should be encode. If the generation distribution is different from the original distribution, then the information will be lost, which is not good news for classification.</p>\n<p>By the way, are the generated pictures helpful for classification?</p>",
      "rawMarkdown": "Since a generation network can capture any information of whales in the image, if it is based on stylegan, it can linearly divide these whales by using his latent code.\n\nUsing gan to generate is very contradictory. Since it can be generated, the information that distinguishes these targets should be encode. If the generation distribution is different from the original distribution, then the information will be lost, which is not good news for classification.\n\nBy the way, are the generated pictures helpful for classification?",
      "votes": null
    },
    {
      "id": "1703219",
      "postDate": "02/24/2022 09:59:21",
      "content": "<p>when will you share the dataset or the generation notebook?<br>\nI search and find that there are papers about FIN detection, but not paper with code .</p>",
      "rawMarkdown": "when will you share the dataset or the generation notebook?\nI search and find that there are papers about FIN detection, but not paper with code .",
      "votes": null
    },
    {
      "id": "1705032",
      "postDate": "02/26/2022 04:49:56",
      "content": "<p>Thanks for sharing👍</p>",
      "rawMarkdown": "Thanks for sharing👍",
      "votes": null
    },
    {
      "id": "1705182",
      "postDate": "02/26/2022 08:05:54",
      "content": "<p>Very cool, thanks for sharing and keep us updated!</p>",
      "rawMarkdown": "Very cool, thanks for sharing and keep us updated!",
      "votes": null
    },
    {
      "id": "1705185",
      "postDate": "02/26/2022 08:09:55",
      "content": "<p><a href=\"https://www.kaggle.com/biglafe\" target=\"_blank\">@biglafe</a> this is not synthetic dataset generated by GAN. This is competition dataset. I created solution (detector + extractors) which extracts object I want (dorsal fin, tail fin, whole whale) from background. </p>",
      "rawMarkdown": "biglafe this is not synthetic dataset generated by GAN. This is competition dataset. I created solution (detector + extractors) which extracts object I want (dorsal fin, tail fin, whole whale) from background.",
      "votes": null
    },
    {
      "id": "1705186",
      "postDate": "02/26/2022 08:10:31",
      "content": "<p>Working on solution. As I said this is not easy but making progress everyday :)</p>",
      "rawMarkdown": "Working on solution. As I said this is not easy but making progress everyday :)",
      "votes": null
    },
    {
      "id": "1705262",
      "postDate": "02/26/2022 10:08:04",
      "content": "<p>Great work <a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> <br>\nThank you very much and good competition!</p>",
      "rawMarkdown": "Great work @remekkinas \nThank you very much and good competition!",
      "votes": null
    },
    {
      "id": "1706194",
      "postDate": "02/27/2022 09:41:23",
      "content": "<p>Do you use segmentation for your idea ? </p>",
      "rawMarkdown": "Do you use segmentation for your idea ?",
      "votes": null
    },
    {
      "id": "1706209",
      "postDate": "02/27/2022 09:51:52",
      "content": "<p>Yes. This is kind of segmentation. I will show solution in this competition. Still working on improving background separation.</p>",
      "rawMarkdown": "Yes. This is kind of segmentation. I will show solution in this competition. Still working on improving background separation.",
      "votes": null
    },
    {
      "id": "1710430",
      "postDate": "03/03/2022 03:22:25",
      "content": "<p>are you using unsupervised mechanism like DeepLabV3 for this ? if no then what precisely</p>",
      "rawMarkdown": "are you using unsupervised mechanism like DeepLabV3 for this ? if no then what precisely",
      "votes": null
    },
    {
      "id": "1711048",
      "postDate": "03/03/2022 15:36:14",
      "content": "<p>I am finishing working on solution. Works quite good :) and will be published soon for research reason - hope somebody generate great dataset we can use.</p>",
      "rawMarkdown": "I am finishing working on solution. Works quite good :) and will be published soon for research reason - hope somebody generate great dataset we can use.",
      "votes": null
    },
    {
      "id": "1711061",
      "postDate": "03/03/2022 15:57:30",
      "content": "<p>In fact, I'm curious about the improvement of segmentation. Assuming that the scaling degree of the same image is different, when calculating the feature similarity, their features will be as similar as possible, which will weaken the influence of background and pay attention to the target area. This can be observed with cam.</p>",
      "rawMarkdown": "In fact, I'm curious about the improvement of segmentation. Assuming that the scaling degree of the same image is different, when calculating the feature similarity, their features will be as similar as possible, which will weaken the influence of background and pay attention to the target area. This can be observed with cam.",
      "votes": null
    },
    {
      "id": "1711113",
      "postDate": "03/03/2022 16:27:24",
      "content": "<p>It is great now. I can separate object from background very well using simple trick (it is as a result of my researches in this area). You will see it in weekend (I will finish notebook and publish). I have not created dataset yet but think that could improve socre a little bit. We will test this soon.  </p>\n<p><img src=\"https://i.ibb.co/QK4PRVm/s-m.jpg\" alt=\"s\"></p>\n<p><img src=\"https://i.ibb.co/C6cRJb1/m-2.jpg\" alt=\"c\"></p>",
      "rawMarkdown": "It is great now. I can separate object from background very well using simple trick (it is as a result of my researches in this area). You will see it in weekend (I will finish notebook and publish). I have not created dataset yet but think that could improve socre a little bit. We will test this soon.  \n\n![s](https://i.ibb.co/QK4PRVm/s-m.jpg)\n\n![c](https://i.ibb.co/C6cRJb1/m-2.jpg)",
      "votes": null
    },
    {
      "id": "1711778",
      "postDate": "03/04/2022 10:00:47",
      "content": "<p>This is amazing, congratulations !</p>",
      "rawMarkdown": "This is amazing, congratulations !",
      "votes": null
    },
    {
      "id": "1714300",
      "postDate": "03/06/2022 19:05:16",
      "content": "<p>I shared my notebook: <a href=\"https://www.kaggle.com/remekkinas/remove-background-salient-object-detection\" target=\"_blank\">https://www.kaggle.com/remekkinas/remove-background-salient-object-detection</a>  Now you can play with background removal.</p>",
      "rawMarkdown": "I shared my notebook: https://www.kaggle.com/remekkinas/remove-background-salient-object-detection  Now you can play with background removal.",
      "votes": null
    },
    {
      "id": "1747301",
      "postDate": "04/06/2022 14:50:45",
      "content": "<p><a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> How do I train my model on these inputs as some part of it is 0 and it creates vanishing gradient problem for me. Is there an other way to do so?</p>",
      "rawMarkdown": "remekkinas How do I train my model on these inputs as some part of it is 0 and it creates vanishing gradient problem for me. Is there an other way to do so?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1701177,
      "author_name": "dragonzhang",
      "author_url": "",
      "post_date": "02/22/2022 15:09:30",
      "content": "<p>Is the FIN unique biologically?  like finger print?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1701188,
          "author_name": "uheheu",
          "author_url": "",
          "post_date": "02/22/2022 15:17:32",
          "content": "<p>it is species dependent but in general yes that is one way to identify an individual animal.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1701243,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/22/2022 15:53:46",
          "content": "<p>I think when you want to recognize people …. you have to act on face not whole person. <br>\nMost reserches I went through describe dorsal fin as a finger print (in most cases). Then you can use additional features (as a helpers or in in special cases as a primary feature - some signs are seasonal).</p>\n<p>When you use whole picture … you search similarity between one landscape and another one. 😂</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1701839,
          "author_name": "dragonzhang",
          "author_url": "",
          "post_date": "02/23/2022 05:15:12",
          "content": "<p>thanks for reply.   I am worry about the image quality if good enough to distinguish them?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1701844,
          "author_name": "uheheu",
          "author_url": "",
          "post_date": "02/23/2022 05:23:38",
          "content": "<p>You are absolutely right but a bit more complicated when you only get certain views of a person and the sample size of training data is small. Need to incorporate a lot of different unique characteristics.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1701894,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/23/2022 06:34:40",
          "content": "<p>We are working on solution quality and dataset cleaning (it is one of the most important part of this competition - and not anly in this one). You can see first model result. Now we have 3rd edition …. training now.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1701895,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/23/2022 06:36:25",
          "content": "<p><a href=\"https://www.kaggle.com/uheheu\" target=\"_blank\">@uheheu</a> you are right. To create good solution here inference pipeline will be important. There will be a lot of \"ifs\" I thnink. I went through almost 50-60% of dataset and see some groups now where we should propose different inference / recognition pipelines.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1701928,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/23/2022 07:10:20",
          "content": "<p><a href=\"https://www.kaggle.com/dragonzhang\" target=\"_blank\">@dragonzhang</a> image of source quality is quite good ore even very good. Certainly you can find low quality photos but … this happens in all computer vision cases (nothing new). You have 2 months still to work on this topic :) </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1702020,
          "author_name": "dragonzhang",
          "author_url": "",
          "post_date": "02/23/2022 09:04:14",
          "content": "<p>thanks for your reply.  Due to time consuming hardware,  I prefer more just to think about it. :)  </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1702027,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/23/2022 09:09:58",
          "content": "<p>For sure you do not need information from whole image - factories, boats, coast, forest are not required for whale similarity check :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1701640,
      "author_name": "atharvap329",
      "author_url": "",
      "post_date": "02/23/2022 00:42:26",
      "content": "<p><a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> did you segmentation masks to create this ? If so can you share the dataset ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1701896,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/23/2022 06:37:55",
          "content": "<p>This is kind of segmentation … but more advanced.<br>\nSo far we are working on dataset quality. Some dorsal fins are blury. Then we decide about publishing. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1710430,
          "author_name": "harshris21",
          "author_url": "",
          "post_date": "03/03/2022 03:22:25",
          "content": "<p>are you using unsupervised mechanism like DeepLabV3 for this ? if no then what precisely</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1701835,
      "author_name": "bsridatta",
      "author_url": "",
      "post_date": "02/23/2022 05:12:48",
      "content": "<p>Awesome, are you planning to share how you were able to do it?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1701903,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/23/2022 06:44:03",
          "content": "<p>We will see. For sure as a final soultion. Now we have to deal with quality. Some fins are still blurred and as I can see we loose important information in image (edges are blurred for some fins). But so far we managed to build full automated pipeline to create new dataset - without background and normalized crops (for all cases in dataset). Model plays additional role - cleaner - is trying to recognize quality of source image. It is quite complex/tricky pipeline now but works. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1702349,
          "author_name": "bsridatta",
          "author_url": "",
          "post_date": "02/23/2022 14:37:40",
          "content": "<p>Nice, thanks for the insights!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1702362,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/23/2022 14:49:05",
          "content": "<p>Today I created … 4 edition of model …. now it works almost perfect - from ~73.000 photos only do not understand ~500 … (can't find dorsal fin or whole whale). Most of problems are with whale where I have tail fin or … really small object. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1702585,
      "author_name": "mpwolke",
      "author_url": "",
      "post_date": "02/23/2022 18:15:34",
      "content": "<p>Congrats for reaching GM tier position Remek.</p>\n<p>By the way, nice dorsal fins topic.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1702760,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/23/2022 21:58:09",
          "content": "<p>Thank you very much <a href=\"https://www.kaggle.com/mpwolke\" target=\"_blank\">@mpwolke</a>. 🙏🙏🙏</p>\n<p>Yes, I am still working on this. 3 NN (2 models) works on such effect. I am very pleased with the effect of the first one, but the next two still require a lot of work to make it work well. However, I see great potential in the solution because you can create a system that will separate the fins or whales from the environment.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1702912,
      "author_name": "biglafe",
      "author_url": "",
      "post_date": "02/24/2022 04:00:03",
      "content": "<p>Since a generation network can capture any information of whales in the image, if it is based on stylegan, it can linearly divide these whales by using his latent code.</p>\n<p>Using gan to generate is very contradictory. Since it can be generated, the information that distinguishes these targets should be encode. If the generation distribution is different from the original distribution, then the information will be lost, which is not good news for classification.</p>\n<p>By the way, are the generated pictures helpful for classification?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1705185,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/26/2022 08:09:55",
          "content": "<p><a href=\"https://www.kaggle.com/biglafe\" target=\"_blank\">@biglafe</a> this is not synthetic dataset generated by GAN. This is competition dataset. I created solution (detector + extractors) which extracts object I want (dorsal fin, tail fin, whole whale) from background. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1703219,
      "author_name": "dragonzhang",
      "author_url": "",
      "post_date": "02/24/2022 09:59:21",
      "content": "<p>when will you share the dataset or the generation notebook?<br>\nI search and find that there are papers about FIN detection, but not paper with code .</p>",
      "votes": null,
      "replies": [
        {
          "id": 1705186,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/26/2022 08:10:31",
          "content": "<p>Working on solution. As I said this is not easy but making progress everyday :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1705032,
      "author_name": "imnoob",
      "author_url": "",
      "post_date": "02/26/2022 04:49:56",
      "content": "<p>Thanks for sharing👍</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1705182,
      "author_name": "stdcout42",
      "author_url": "",
      "post_date": "02/26/2022 08:05:54",
      "content": "<p>Very cool, thanks for sharing and keep us updated!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1705262,
      "author_name": "datascientistfp",
      "author_url": "",
      "post_date": "02/26/2022 10:08:04",
      "content": "<p>Great work <a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> <br>\nThank you very much and good competition!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1706194,
      "author_name": "nghiahoangtrung",
      "author_url": "",
      "post_date": "02/27/2022 09:41:23",
      "content": "<p>Do you use segmentation for your idea ? </p>",
      "votes": null,
      "replies": [
        {
          "id": 1706209,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "02/27/2022 09:51:52",
          "content": "<p>Yes. This is kind of segmentation. I will show solution in this competition. Still working on improving background separation.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1711048,
      "author_name": "remekkinas",
      "author_url": "",
      "post_date": "03/03/2022 15:36:14",
      "content": "<p>I am finishing working on solution. Works quite good :) and will be published soon for research reason - hope somebody generate great dataset we can use.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1711061,
          "author_name": "biglafe",
          "author_url": "",
          "post_date": "03/03/2022 15:57:30",
          "content": "<p>In fact, I'm curious about the improvement of segmentation. Assuming that the scaling degree of the same image is different, when calculating the feature similarity, their features will be as similar as possible, which will weaken the influence of background and pay attention to the target area. This can be observed with cam.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1711113,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "03/03/2022 16:27:24",
          "content": "<p>It is great now. I can separate object from background very well using simple trick (it is as a result of my researches in this area). You will see it in weekend (I will finish notebook and publish). I have not created dataset yet but think that could improve socre a little bit. We will test this soon.  </p>\n<p><img src=\"https://i.ibb.co/QK4PRVm/s-m.jpg\" alt=\"s\"></p>\n<p><img src=\"https://i.ibb.co/C6cRJb1/m-2.jpg\" alt=\"c\"></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1714300,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "03/06/2022 19:05:16",
          "content": "<p>I shared my notebook: <a href=\"https://www.kaggle.com/remekkinas/remove-background-salient-object-detection\" target=\"_blank\">https://www.kaggle.com/remekkinas/remove-background-salient-object-detection</a>  Now you can play with background removal.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1711778,
      "author_name": "wolfy73",
      "author_url": "",
      "post_date": "03/04/2022 10:00:47",
      "content": "<p>This is amazing, congratulations !</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1747301,
      "author_name": "jainishsavalia",
      "author_url": "",
      "post_date": "04/06/2022 14:50:45",
      "content": "<p><a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> How do I train my model on these inputs as some part of it is 0 and it creates vanishing gradient problem for me. Is there an other way to do so?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1700921": "We have perfomed some test with new dataset generation - it is automatically generated from Happywhale dataset. Do you like it?\n\nFull implementation - removing background from images: https://www.kaggle.com/remekkinas/remove-background-salient-object-detection\n\nBelow you can see dorsal fins dataset collected from 50k train images.\n![dataset](https://i.ibb.co/r2GMZ6h/grid.jpg)\n\n![data](https://i.ibb.co/qy5TmJ7/set.jpg)",
    "1701177": "Is the FIN unique biologically?  like finger print?",
    "1701188": "it is species dependent but in general yes that is one way to identify an individual animal.",
    "1701243": "I think when you want to recognize people .... you have to act on face not whole person. \nMost reserches I went through describe dorsal fin as a finger print (in most cases). Then you can use additional features (as a helpers or in in special cases as a primary feature - some signs are seasonal).\n\nWhen you use whole picture ... you search similarity between one landscape and another one. 😂",
    "1701640": "remekkinas did you segmentation masks to create this ? If so can you share the dataset ?",
    "1701835": "Awesome, are you planning to share how you were able to do it?",
    "1701839": "thanks for reply.   I am worry about the image quality if good enough to distinguish them?",
    "1701844": "You are absolutely right but a bit more complicated when you only get certain views of a person and the sample size of training data is small. Need to incorporate a lot of different unique characteristics.",
    "1701894": "We are working on solution quality and dataset cleaning (it is one of the most important part of this competition - and not anly in this one). You can see first model result. Now we have 3rd edition .... training now.",
    "1701895": "uheheu you are right. To create good solution here inference pipeline will be important. There will be a lot of \"ifs\" I thnink. I went through almost 50-60% of dataset and see some groups now where we should propose different inference / recognition pipelines.",
    "1701896": "This is kind of segmentation ... but more advanced.\nSo far we are working on dataset quality. Some dorsal fins are blury. Then we decide about publishing.",
    "1701903": "We will see. For sure as a final soultion. Now we have to deal with quality. Some fins are still blurred and as I can see we loose important information in image (edges are blurred for some fins). But so far we managed to build full automated pipeline to create new dataset - without background and normalized crops (for all cases in dataset). Model plays additional role - cleaner - is trying to recognize quality of source image. It is quite complex/tricky pipeline now but works.",
    "1701928": "dragonzhang image of source quality is quite good ore even very good. Certainly you can find low quality photos but ... this happens in all computer vision cases (nothing new). You have 2 months still to work on this topic :)",
    "1702020": "thanks for your reply.  Due to time consuming hardware,  I prefer more just to think about it. :)",
    "1702027": "For sure you do not need information from whole image - factories, boats, coast, forest are not required for whale similarity check :)",
    "1702349": "Nice, thanks for the insights!",
    "1702362": "Today I created ... 4 edition of model .... now it works almost perfect - from ~73.000 photos only do not understand ~500 ... (can't find dorsal fin or whole whale). Most of problems are with whale where I have tail fin or ... really small object.",
    "1702585": "Congrats for reaching GM tier position Remek.\n\nBy the way, nice dorsal fins topic.",
    "1702760": "Thank you very much @mpwolke. 🙏🙏🙏\n\nYes, I am still working on this. 3 NN (2 models) works on such effect. I am very pleased with the effect of the first one, but the next two still require a lot of work to make it work well. However, I see great potential in the solution because you can create a system that will separate the fins or whales from the environment.",
    "1702912": "Since a generation network can capture any information of whales in the image, if it is based on stylegan, it can linearly divide these whales by using his latent code.\n\nUsing gan to generate is very contradictory. Since it can be generated, the information that distinguishes these targets should be encode. If the generation distribution is different from the original distribution, then the information will be lost, which is not good news for classification.\n\nBy the way, are the generated pictures helpful for classification?",
    "1703219": "when will you share the dataset or the generation notebook?\nI search and find that there are papers about FIN detection, but not paper with code .",
    "1705032": "Thanks for sharing👍",
    "1705182": "Very cool, thanks for sharing and keep us updated!",
    "1705185": "biglafe this is not synthetic dataset generated by GAN. This is competition dataset. I created solution (detector + extractors) which extracts object I want (dorsal fin, tail fin, whole whale) from background.",
    "1705186": "Working on solution. As I said this is not easy but making progress everyday :)",
    "1705262": "Great work @remekkinas \nThank you very much and good competition!",
    "1706194": "Do you use segmentation for your idea ?",
    "1706209": "Yes. This is kind of segmentation. I will show solution in this competition. Still working on improving background separation.",
    "1710430": "are you using unsupervised mechanism like DeepLabV3 for this ? if no then what precisely",
    "1711048": "I am finishing working on solution. Works quite good :) and will be published soon for research reason - hope somebody generate great dataset we can use.",
    "1711061": "In fact, I'm curious about the improvement of segmentation. Assuming that the scaling degree of the same image is different, when calculating the feature similarity, their features will be as similar as possible, which will weaken the influence of background and pay attention to the target area. This can be observed with cam.",
    "1711113": "It is great now. I can separate object from background very well using simple trick (it is as a result of my researches in this area). You will see it in weekend (I will finish notebook and publish). I have not created dataset yet but think that could improve socre a little bit. We will test this soon.  \n\n![s](https://i.ibb.co/QK4PRVm/s-m.jpg)\n\n![c](https://i.ibb.co/C6cRJb1/m-2.jpg)",
    "1711778": "This is amazing, congratulations !",
    "1714300": "I shared my notebook: https://www.kaggle.com/remekkinas/remove-background-salient-object-detection  Now you can play with background removal.",
    "1747301": "remekkinas How do I train my model on these inputs as some part of it is 0 and it creates vanishing gradient problem for me. Is there an other way to do so?"
  },
  "source": "meta"
}