{
  "id": 301210,
  "title": "These COTS do not exist",
  "url": "/competitions/tensorflow-great-barrier-reef/discussion/301210",
  "author_name": "",
  "post_date": "2022-01-16T11:48:54.729075200Z",
  "votes": 43,
  "comment_count": 26,
  "views": 0,
  "content": "<p>GAN (generative adversarial net) results<br>\n(from <a href=\"https://github.com/lucidrains/lightweight-gan\" target=\"_blank\">https://github.com/lucidrains/lightweight-gan</a>)</p>\n<p><img src=\"https://i.ibb.co/wzxL4jN/19-ema.jpg\" alt=\"https://i.ibb.co/wzxL4jN/19-ema.jpg\"></p>\n<p>However, i want to do in painting like this:<br>\n<img src=\"https://i.ibb.co/wWYTnpR/Selection-999-727.png\" alt=\"https://i.ibb.co/wWYTnpR/Selection-999-727.png\"></p>\n<p>maybe this application is suitable for GAN. the COTS objects are quite small, so we only need to generate  20x20 ~ 60x60 region. Is there any fast GAN  (e.g. train within 24hrs on one GPU), especially condition GAN that does inpainting, that anyone can recommend?</p>",
  "messages": [
    {
      "id": "1652167",
      "postDate": "01/16/2022 11:48:54",
      "content": "<p>GAN (generative adversarial net) results<br>\n(from <a href=\"https://github.com/lucidrains/lightweight-gan\" target=\"_blank\">https://github.com/lucidrains/lightweight-gan</a>)</p>\n<p><img src=\"https://i.ibb.co/wzxL4jN/19-ema.jpg\" alt=\"https://i.ibb.co/wzxL4jN/19-ema.jpg\"></p>\n<p>However, i want to do in painting like this:<br>\n<img src=\"https://i.ibb.co/wWYTnpR/Selection-999-727.png\" alt=\"https://i.ibb.co/wWYTnpR/Selection-999-727.png\"></p>\n<p>maybe this application is suitable for GAN. the COTS objects are quite small, so we only need to generate  20x20 ~ 60x60 region. Is there any fast GAN  (e.g. train within 24hrs on one GPU), especially condition GAN that does inpainting, that anyone can recommend?</p>",
      "rawMarkdown": "GAN (generative adversarial net) results\n(from https://github.com/lucidrains/lightweight-gan)\n\n![https://i.ibb.co/wzxL4jN/19-ema.jpg](https://i.ibb.co/wzxL4jN/19-ema.jpg)\n\nHowever, i want to do in painting like this:\n![https://i.ibb.co/wWYTnpR/Selection-999-727.png](https://i.ibb.co/wWYTnpR/Selection-999-727.png)\n\n\nmaybe this application is suitable for GAN. the COTS objects are quite small, so we only need to generate  20x20 ~ 60x60 region. Is there any fast GAN  (e.g. train within 24hrs on one GPU), especially condition GAN that does inpainting, that anyone can recommend?",
      "votes": null
    },
    {
      "id": "1652185",
      "postDate": "01/16/2022 12:01:11",
      "content": "<p>It’s not recommend, but I remembered what I tried in early time of competition.<br>\n<em>Pedestrian GAN</em> seemed to suits this competition, but the public official code contains plenty of bug.<br>\nI managed to make it run, but the output is far from realistic…</p>\n<p><a href=\"https://www.kaggle.com/tatamikenn/generating-fake-starfish\" target=\"_blank\">https://www.kaggle.com/tatamikenn/generating-fake-starfish</a></p>",
      "rawMarkdown": "It’s not recommend, but I remembered what I tried in early time of competition.\n*Pedestrian GAN* seemed to suits this competition, but the public official code contains plenty of bug.\nI managed to make it run, but the output is far from realistic…\n\nhttps://www.kaggle.com/tatamikenn/generating-fake-starfish",
      "votes": null
    },
    {
      "id": "1652193",
      "postDate": "01/16/2022 12:10:56",
      "content": "<p>I think the approach below might be more promising (no official implementation though).<br>\n<a href=\"https://arxiv.org/abs/1910.07169\" target=\"_blank\">https://arxiv.org/abs/1910.07169</a></p>",
      "rawMarkdown": "I think the approach below might be more promising (no official implementation though).\nhttps://arxiv.org/abs/1910.07169",
      "votes": null
    },
    {
      "id": "1652207",
      "postDate": "01/16/2022 12:21:58",
      "content": "<p>\". We observe that DetectorGAN significantly improves the average precision. Compared to training on real data only, the AP nearly doubles from 0.124 to<br>\n0.236, and recall over triples from 0.184 to 0.649\"</p>\n<p>recall seems good</p>",
      "rawMarkdown": "\". We observe that DetectorGAN significantly improves the average precision. Compared to training on real data only, the AP nearly doubles from 0.124 to\n0.236, and recall over triples from 0.184 to 0.649\"\n\nrecall seems good",
      "votes": null
    },
    {
      "id": "1652208",
      "postDate": "01/16/2022 12:22:04",
      "content": "<p>What are result of your experiment on this dataset? How much have you improved? </p>",
      "rawMarkdown": "What are result of your experiment on this dataset? How much have you improved?",
      "votes": null
    },
    {
      "id": "1652223",
      "postDate": "01/16/2022 12:37:54",
      "content": "<p>Yes. That’s why I thought it promising.</p>\n<p>Considering that It’s designed for finding tumors which is simpler than generating COTS, I don’t know whether it also effective in this competition.</p>\n<p>If only we had official code.. The code seems hard to implement for me.</p>",
      "rawMarkdown": "Yes. That’s why I thought it promising.\n\nConsidering that It’s designed for finding tumors which is simpler than generating COTS, I don’t know whether it also effective in this competition.\n\nIf only we had official code.. The code seems hard to implement for me.",
      "votes": null
    },
    {
      "id": "1652242",
      "postDate": "01/16/2022 13:01:30",
      "content": "<p>The patch size is small and the bbox dataset are not large, most GAN model might fit the 24hr-per gpu requirement.<br>\nHowerver,<br>\nI haved tried FUnIE-GAN(both using  pretrained weight and retrained)  <br>\n                     cyclegan(unpaired training with GBR data)<br>\n                     UGATIT (unpaired training with GBR data)<br>\n                     NiceGAN (unpaired training with GBR data)<br>\n                     ESRGAN (paired training with FUnIE data and apply in GBR data ) <br>\nall scores(MAP) go down a little  </p>\n<p>(ps: I apply the function to whole image but not patch . might it be different ? )</p>",
      "rawMarkdown": "The patch size is small and the bbox dataset are not large, most GAN model might fit the 24hr-per gpu requirement.\nHowerver,\nI haved tried FUnIE-GAN(both using  pretrained weight and retrained)  \n                     cyclegan(unpaired training with GBR data)\n                     UGATIT (unpaired training with GBR data)\n                     NiceGAN (unpaired training with GBR data)\n                     ESRGAN (paired training with FUnIE data and apply in GBR data ) \nall scores(MAP) go down a little  \n\n(ps: I apply the function to whole image but not patch . might it be different ? )",
      "votes": null
    },
    {
      "id": "1652253",
      "postDate": "01/16/2022 13:16:20",
      "content": "<p>let me read and think about it.<br>\nthe rolling step is similar to the one in \"DARTS: DIFFERENTIABLE ARCHITECTURE SEARCH\" or \"UNROLLED GENERATIVE ADVERSARIAL NETWORKS\", which I have implemented before. you can read that paper and check their code</p>\n<p>I will try this</p>\n<pre><code>x = get data from dataloader\nx_fake = generator( ...)\n\n#copy weights from detector for unrolling later\nw = copy from detector\n\npredict = detector(torch.cat[x,x_fake])\nloss = loss_function(predict, truth)\nloss.backward()\noptimizer_d.step()\n\ndw = copy from optimizer_d\n\n#reset detector weight\nreset(w+dw, detector)\n\npredict_fake = detector(x_fake)\nloss = loss_function(predict_fake , truth_as_real) #adversial loss\nloss.backward()\noptimizer_g.step() #upate generator\n</code></pre>",
      "rawMarkdown": "let me read and think about it.\nthe rolling step is similar to the one in \"DARTS: DIFFERENTIABLE ARCHITECTURE SEARCH\" or \"UNROLLED GENERATIVE ADVERSARIAL NETWORKS\", which I have implemented before. you can read that paper and check their code\n\nI will try this\n```\nx = get data from dataloader\nx_fake = generator( ...)\n\n#copy weights from detector for unrolling later\nw = copy from detector\n\npredict = detector(torch.cat[x,x_fake])\nloss = loss_function(predict, truth)\nloss.backward()\noptimizer_d.step()\n\ndw = copy from optimizer_d\n\n#reset detector weight\nreset(w+dw, detector)\n\npredict_fake = detector(x_fake)\nloss = loss_function(predict_fake , truth_as_real) #adversial loss\nloss.backward()\noptimizer_g.step() #upate generator\n\n```",
      "votes": null
    },
    {
      "id": "1652254",
      "postDate": "01/16/2022 13:19:36",
      "content": "<p>how about treating the gan images as unlabelled and relabeling them again (assuming the gan quality is not good)<br>\ni.e. it becomes semi-supervised.</p>",
      "rawMarkdown": "how about treating the gan images as unlabelled and relabeling them again (assuming the gan quality is not good)\ni.e. it becomes semi-supervised.",
      "votes": null
    },
    {
      "id": "1652307",
      "postDate": "01/16/2022 13:59:04",
      "content": "<p>Thank you for sharing. I will check them.</p>\n<p>I don’t know it’s correct, but first time I read the paper, I imagined something like meta-learning:<br>\n<a href=\"https://github.com/facebookresearch/higher\" target=\"_blank\">https://github.com/facebookresearch/higher</a></p>\n<p>I never implemented those kind of code, and maybe too much for me considering one month left. But the reference implementation will definitely help because I have no idea right now.</p>",
      "rawMarkdown": "Thank you for sharing. I will check them.\n\nI don’t know it’s correct, but first time I read the paper, I imagined something like meta-learning:\nhttps://github.com/facebookresearch/higher\n\nI never implemented those kind of code, and maybe too much for me considering one month left. But the reference implementation will definitely help because I have no idea right now.",
      "votes": null
    },
    {
      "id": "1652311",
      "postDate": "01/16/2022 14:01:30",
      "content": "<p>meta learning is correct</p>",
      "rawMarkdown": "meta learning is correct",
      "votes": null
    },
    {
      "id": "1652337",
      "postDate": "01/16/2022 14:20:00",
      "content": "<p>The gan-generated images are from GBR ,we already have annotation data. <br>\n While we use them to train, we might treat  it as a kind of preprocessing  .<br>\n If the model trained with low quality image can not recognize the bad patch, it fail.</p>\n<p>For the bad patch, we might see the GBR as a super resolution problem.<br>\nit might be help if we directly \"blur\" all the annotation bbox and train a super resolution model.</p>",
      "rawMarkdown": "The gan-generated images are from GBR ,we already have annotation data. \n While we use them to train, we might treat  it as a kind of preprocessing  .\n If the model trained with low quality image can not recognize the bad patch, it fail.\n\nFor the bad patch, we might see the GBR as a super resolution problem.\nit might be help if we directly \"blur\" all the annotation bbox and train a super resolution model.",
      "votes": null
    },
    {
      "id": "1652353",
      "postDate": "01/16/2022 14:32:00",
      "content": "<p>I see. Thank you for clarification.</p>",
      "rawMarkdown": "I see. Thank you for clarification.",
      "votes": null
    },
    {
      "id": "1652389",
      "postDate": "01/16/2022 15:04:01",
      "content": "<p>I think he/she is trying to do inpainting, so he/she’ve not get result from this yet.</p>",
      "rawMarkdown": "I think he/she is trying to do inpainting, so he/she’ve not get result from this yet.",
      "votes": null
    },
    {
      "id": "1654136",
      "postDate": "01/18/2022 07:32:44",
      "content": "<p>this is a shortcut</p>\n<p><img src=\"https://i.ibb.co/y69s6Qm/Selection-004.png\" alt=\"https://i.ibb.co/y69s6Qm/Selection-004.png\"></p>\n<p>since we are given video, it is sufficient to cut and paste (merge the boundary using soft mask).<br>\nusing this we can remove or add cots.</p>\n<p>tips 1:<br>\nwhen moving patches across different frames, try to limited x,y displacement if you don't  post process to keep object size and image  quality similar</p>\n<hr>\n<p>else you would need to to do some style transfer (e.g. color, blurriness, etc) which is also not too difficult using GAN</p>\n<p>-- <br>\nsee<br>\nPatchMatch<br>\n<a href=\"https://en.wikipedia.org/wiki/PatchMatch\" target=\"_blank\">https://en.wikipedia.org/wiki/PatchMatch</a><br>\n<a href=\"https://gfx.cs.princeton.edu/pubs/Barnes_2009_PAR/\" target=\"_blank\">https://gfx.cs.princeton.edu/pubs/Barnes_2009_PAR/</a></p>\n<p><img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/PatchMatch.jpg/1920px-PatchMatch.jpg\" alt=\"https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/PatchMatch.jpg/1920px-PatchMatch.jpg\"></p>",
      "rawMarkdown": "this is a shortcut\n\n![https://i.ibb.co/y69s6Qm/Selection-004.png](https://i.ibb.co/y69s6Qm/Selection-004.png)\n\nsince we are given video, it is sufficient to cut and paste (merge the boundary using soft mask).\nusing this we can remove or add cots.\n\ntips 1:\nwhen moving patches across different frames, try to limited x,y displacement if you don't  post process to keep object size and image  quality similar\n\n----\n\nelse you would need to to do some style transfer (e.g. color, blurriness, etc) which is also not too difficult using GAN\n\n\n-- \nsee\nPatchMatch\nhttps://en.wikipedia.org/wiki/PatchMatch\nhttps://gfx.cs.princeton.edu/pubs/Barnes_2009_PAR/\n\n![https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/PatchMatch.jpg/1920px-PatchMatch.jpg](https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/PatchMatch.jpg/1920px-PatchMatch.jpg)",
      "votes": null
    },
    {
      "id": "1656023",
      "postDate": "01/19/2022 03:43:21",
      "content": "<p>image harmonization will help</p>",
      "rawMarkdown": "image harmonization will help",
      "votes": null
    },
    {
      "id": "1657192",
      "postDate": "01/20/2022 02:24:51",
      "content": "<p>amazing, tkx</p>",
      "rawMarkdown": "amazing, tkx",
      "votes": null
    },
    {
      "id": "1659390",
      "postDate": "01/21/2022 18:57:38",
      "content": "<p><br>\n<a href=\"https://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955\" target=\"_blank\">https://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955</a></p>\n<p><img src=\"https://i.ibb.co/58wSyX9/Selection-019.png\" alt=\"https://i.ibb.co/58wSyX9/Selection-019.png\"></p>",
      "rawMarkdown": "~~code coming soon!~~\nhttps://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955\n\n![https://i.ibb.co/58wSyX9/Selection-019.png](https://i.ibb.co/58wSyX9/Selection-019.png)",
      "votes": null
    },
    {
      "id": "1659609",
      "postDate": "01/22/2022 02:09:44",
      "content": "<p>Cool. But is it safe to rely on google’s algorithm to filter license?</p>",
      "rawMarkdown": "Cool. But is it safe to rely on google’s algorithm to filter license?",
      "votes": null
    },
    {
      "id": "1659617",
      "postDate": "01/22/2022 02:19:47",
      "content": "<p>What they say:</p>\n<blockquote>\n  <p>Google filters images by license based on information provided by the sites that host those images, or the image provider.</p>\n</blockquote>\n<p><a href=\"https://support.google.com/websearch/answer/29508\" target=\"_blank\">https://support.google.com/websearch/answer/29508</a></p>\n<p>I wonder if the algorithm is perfect or not.</p>",
      "rawMarkdown": "What they say:\n\n> Google filters images by license based on information provided by the sites that host those images, or the image provider.\n\nhttps://support.google.com/websearch/answer/29508\n\nI wonder if the algorithm is perfect or not.",
      "votes": null
    },
    {
      "id": "1659619",
      "postDate": "01/22/2022 02:23:36",
      "content": "<p>It seems we have to double check the license manually. I takes some cost, but I’m looking forward to the code.</p>",
      "rawMarkdown": "It seems we have to double check the license manually. I takes some cost, but I’m looking forward to the code.",
      "votes": null
    },
    {
      "id": "1659645",
      "postDate": "01/22/2022 03:03:45",
      "content": "<p>i have to check the license manually.</p>\n<p>also you can cut and paste from different frames of the kaggle train data.</p>",
      "rawMarkdown": "i have to check the license manually.\n\nalso you can cut and paste from different frames of the kaggle train data.",
      "votes": null
    },
    {
      "id": "1659742",
      "postDate": "01/22/2022 05:00:05",
      "content": "<p>we wait )))</p>",
      "rawMarkdown": "we wait )))",
      "votes": null
    },
    {
      "id": "1660072",
      "postDate": "01/22/2022 11:27:08",
      "content": "<p>code is up:<br>\n<a href=\"https://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955\" target=\"_blank\">https://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955</a></p>",
      "rawMarkdown": "code is up:\nhttps://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955",
      "votes": null
    },
    {
      "id": "1666031",
      "postDate": "01/27/2022 10:45:05",
      "content": "<p>check thsi paper!<br>\nScale-aware Automatic Augmentation for Object Detection<br>\n<a href=\"https://openaccess.thecvf.com/content/CVPR2021/papers/Chen_Scale-Aware_Automatic_Augmentation_for_Object_Detection_CVPR_2021_paper.pdf\" target=\"_blank\">https://openaccess.thecvf.com/content/CVPR2021/papers/Chen_Scale-Aware_Automatic_Augmentation_for_Object_Detection_CVPR_2021_paper.pdf</a></p>\n<p>Learning Data Augmentation Strategies for Object Detection<br>\n<a href=\"https://paperswithcode.com/paper/learning-data-augmentation-strategies-for\" target=\"_blank\">https://paperswithcode.com/paper/learning-data-augmentation-strategies-for</a></p>",
      "rawMarkdown": "check thsi paper!\nScale-aware Automatic Augmentation for Object Detection\nhttps://openaccess.thecvf.com/content/CVPR2021/papers/Chen_Scale-Aware_Automatic_Augmentation_for_Object_Detection_CVPR_2021_paper.pdf\n\n\nLearning Data Augmentation Strategies for Object Detection\nhttps://paperswithcode.com/paper/learning-data-augmentation-strategies-for",
      "votes": null
    },
    {
      "id": "1673304",
      "postDate": "02/02/2022 16:30:19",
      "content": "<p>while blending works well, but we need to find a way to cut and paste to and from the correct region!<br>\nthere are two important image characteristics:</p>\n<ol>\n<li>blurness/shaprness : a blur COTS must be pasted into a blur background</li>\n<li>size: the reef and COTS must match in size<br>\nafter several experiments, i think this works best<br>\nresults of automatic COTS placement estimation:<br>\n<img src=\"https://i.ibb.co/G7JFbp0/Selection-024.png\" alt=\"https://i.ibb.co/G7JFbp0/Selection-024.png\"><br>\nwe neet a patch smiliarty matcher. the easiest way to train one (or download one from github) is the contrastive<br>\nnetwork:<br>\ntrain data <br>\none patch + rotated90/flip augment = same pair (i.e. smiliar patch texture)<br>\none patch + another patch (far in spatial and temporal distance) = different pair</li>\n</ol>",
      "rawMarkdown": "while blending works well, but we need to find a way to cut and paste to and from the correct region!\n\nthere are two important image characteristics:\n1. blurness/shaprness : a blur COTS must be pasted into a blur background\n2. size: the reef and COTS must match in size\n\nafter several experiments, i think this works best\n\nresults of automatic COTS placement estimation:\n![https://i.ibb.co/G7JFbp0/Selection-024.png](https://i.ibb.co/G7JFbp0/Selection-024.png)\n\n\nwe neet a patch smiliarty matcher. the easiest way to train one (or download one from github) is the contrastive\nnetwork:\n\ntrain data \n\none patch + rotated90/flip augment = same pair (i.e. smiliar patch texture)\none patch + another patch (far in spatial and temporal distance) = different pair",
      "votes": null
    },
    {
      "id": "1673611",
      "postDate": "02/02/2022 20:48:38",
      "content": "<p>Check <a href=\"https://www.kaggle.com/outwrest/augmentation-using-image-blending-cot-masks\" target=\"_blank\">this</a> out: Maybe with other types of blending and style transfers it can be improved. I used a cot-masks dataset that someone else had posted before and it was helpful to create many more images (you can download 10k unique images of 5 cots. I didn't see many results improvements, I'm looking into doing pretraining on it once I get the data to be better. This can be something to look into.</p>",
      "rawMarkdown": "Check [this](https://www.kaggle.com/outwrest/augmentation-using-image-blending-cot-masks) out: Maybe with other types of blending and style transfers it can be improved. I used a cot-masks dataset that someone else had posted before and it was helpful to create many more images (you can download 10k unique images of 5 cots. I didn't see many results improvements, I'm looking into doing pretraining on it once I get the data to be better. This can be something to look into.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1652185,
      "author_name": "tatamikenn",
      "author_url": "",
      "post_date": "01/16/2022 12:01:11",
      "content": "<p>It’s not recommend, but I remembered what I tried in early time of competition.<br>\n<em>Pedestrian GAN</em> seemed to suits this competition, but the public official code contains plenty of bug.<br>\nI managed to make it run, but the output is far from realistic…</p>\n<p><a href=\"https://www.kaggle.com/tatamikenn/generating-fake-starfish\" target=\"_blank\">https://www.kaggle.com/tatamikenn/generating-fake-starfish</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1652193,
          "author_name": "tatamikenn",
          "author_url": "",
          "post_date": "01/16/2022 12:10:56",
          "content": "<p>I think the approach below might be more promising (no official implementation though).<br>\n<a href=\"https://arxiv.org/abs/1910.07169\" target=\"_blank\">https://arxiv.org/abs/1910.07169</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1652207,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "01/16/2022 12:21:58",
          "content": "<p>\". We observe that DetectorGAN significantly improves the average precision. Compared to training on real data only, the AP nearly doubles from 0.124 to<br>\n0.236, and recall over triples from 0.184 to 0.649\"</p>\n<p>recall seems good</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1652223,
          "author_name": "tatamikenn",
          "author_url": "",
          "post_date": "01/16/2022 12:37:54",
          "content": "<p>Yes. That’s why I thought it promising.</p>\n<p>Considering that It’s designed for finding tumors which is simpler than generating COTS, I don’t know whether it also effective in this competition.</p>\n<p>If only we had official code.. The code seems hard to implement for me.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1652253,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "01/16/2022 13:16:20",
          "content": "<p>let me read and think about it.<br>\nthe rolling step is similar to the one in \"DARTS: DIFFERENTIABLE ARCHITECTURE SEARCH\" or \"UNROLLED GENERATIVE ADVERSARIAL NETWORKS\", which I have implemented before. you can read that paper and check their code</p>\n<p>I will try this</p>\n<pre><code>x = get data from dataloader\nx_fake = generator( ...)\n\n#copy weights from detector for unrolling later\nw = copy from detector\n\npredict = detector(torch.cat[x,x_fake])\nloss = loss_function(predict, truth)\nloss.backward()\noptimizer_d.step()\n\ndw = copy from optimizer_d\n\n#reset detector weight\nreset(w+dw, detector)\n\npredict_fake = detector(x_fake)\nloss = loss_function(predict_fake , truth_as_real) #adversial loss\nloss.backward()\noptimizer_g.step() #upate generator\n</code></pre>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1652307,
          "author_name": "tatamikenn",
          "author_url": "",
          "post_date": "01/16/2022 13:59:04",
          "content": "<p>Thank you for sharing. I will check them.</p>\n<p>I don’t know it’s correct, but first time I read the paper, I imagined something like meta-learning:<br>\n<a href=\"https://github.com/facebookresearch/higher\" target=\"_blank\">https://github.com/facebookresearch/higher</a></p>\n<p>I never implemented those kind of code, and maybe too much for me considering one month left. But the reference implementation will definitely help because I have no idea right now.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1652311,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "01/16/2022 14:01:30",
          "content": "<p>meta learning is correct</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1652353,
          "author_name": "tatamikenn",
          "author_url": "",
          "post_date": "01/16/2022 14:32:00",
          "content": "<p>I see. Thank you for clarification.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1652208,
      "author_name": "remekkinas",
      "author_url": "",
      "post_date": "01/16/2022 12:22:04",
      "content": "<p>What are result of your experiment on this dataset? How much have you improved? </p>",
      "votes": null,
      "replies": [
        {
          "id": 1652389,
          "author_name": "tatamikenn",
          "author_url": "",
          "post_date": "01/16/2022 15:04:01",
          "content": "<p>I think he/she is trying to do inpainting, so he/she’ve not get result from this yet.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1652242,
      "author_name": "atom1231",
      "author_url": "",
      "post_date": "01/16/2022 13:01:30",
      "content": "<p>The patch size is small and the bbox dataset are not large, most GAN model might fit the 24hr-per gpu requirement.<br>\nHowerver,<br>\nI haved tried FUnIE-GAN(both using  pretrained weight and retrained)  <br>\n                     cyclegan(unpaired training with GBR data)<br>\n                     UGATIT (unpaired training with GBR data)<br>\n                     NiceGAN (unpaired training with GBR data)<br>\n                     ESRGAN (paired training with FUnIE data and apply in GBR data ) <br>\nall scores(MAP) go down a little  </p>\n<p>(ps: I apply the function to whole image but not patch . might it be different ? )</p>",
      "votes": null,
      "replies": [
        {
          "id": 1652254,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "01/16/2022 13:19:36",
          "content": "<p>how about treating the gan images as unlabelled and relabeling them again (assuming the gan quality is not good)<br>\ni.e. it becomes semi-supervised.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1652337,
          "author_name": "atom1231",
          "author_url": "",
          "post_date": "01/16/2022 14:20:00",
          "content": "<p>The gan-generated images are from GBR ,we already have annotation data. <br>\n While we use them to train, we might treat  it as a kind of preprocessing  .<br>\n If the model trained with low quality image can not recognize the bad patch, it fail.</p>\n<p>For the bad patch, we might see the GBR as a super resolution problem.<br>\nit might be help if we directly \"blur\" all the annotation bbox and train a super resolution model.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1654136,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "01/18/2022 07:32:44",
      "content": "<p>this is a shortcut</p>\n<p><img src=\"https://i.ibb.co/y69s6Qm/Selection-004.png\" alt=\"https://i.ibb.co/y69s6Qm/Selection-004.png\"></p>\n<p>since we are given video, it is sufficient to cut and paste (merge the boundary using soft mask).<br>\nusing this we can remove or add cots.</p>\n<p>tips 1:<br>\nwhen moving patches across different frames, try to limited x,y displacement if you don't  post process to keep object size and image  quality similar</p>\n<hr>\n<p>else you would need to to do some style transfer (e.g. color, blurriness, etc) which is also not too difficult using GAN</p>\n<p>-- <br>\nsee<br>\nPatchMatch<br>\n<a href=\"https://en.wikipedia.org/wiki/PatchMatch\" target=\"_blank\">https://en.wikipedia.org/wiki/PatchMatch</a><br>\n<a href=\"https://gfx.cs.princeton.edu/pubs/Barnes_2009_PAR/\" target=\"_blank\">https://gfx.cs.princeton.edu/pubs/Barnes_2009_PAR/</a></p>\n<p><img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/PatchMatch.jpg/1920px-PatchMatch.jpg\" alt=\"https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/PatchMatch.jpg/1920px-PatchMatch.jpg\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1656023,
      "author_name": "gjzhongdf163com",
      "author_url": "",
      "post_date": "01/19/2022 03:43:21",
      "content": "<p>image harmonization will help</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1657192,
      "author_name": "herbertbert",
      "author_url": "",
      "post_date": "01/20/2022 02:24:51",
      "content": "<p>amazing, tkx</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1659390,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "01/21/2022 18:57:38",
      "content": "<p><br>\n<a href=\"https://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955\" target=\"_blank\">https://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955</a></p>\n<p><img src=\"https://i.ibb.co/58wSyX9/Selection-019.png\" alt=\"https://i.ibb.co/58wSyX9/Selection-019.png\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 1659609,
          "author_name": "tatamikenn",
          "author_url": "",
          "post_date": "01/22/2022 02:09:44",
          "content": "<p>Cool. But is it safe to rely on google’s algorithm to filter license?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1659617,
          "author_name": "tatamikenn",
          "author_url": "",
          "post_date": "01/22/2022 02:19:47",
          "content": "<p>What they say:</p>\n<blockquote>\n  <p>Google filters images by license based on information provided by the sites that host those images, or the image provider.</p>\n</blockquote>\n<p><a href=\"https://support.google.com/websearch/answer/29508\" target=\"_blank\">https://support.google.com/websearch/answer/29508</a></p>\n<p>I wonder if the algorithm is perfect or not.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1659619,
          "author_name": "tatamikenn",
          "author_url": "",
          "post_date": "01/22/2022 02:23:36",
          "content": "<p>It seems we have to double check the license manually. I takes some cost, but I’m looking forward to the code.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1659645,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "01/22/2022 03:03:45",
          "content": "<p>i have to check the license manually.</p>\n<p>also you can cut and paste from different frames of the kaggle train data.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1659742,
          "author_name": "aleksandrkruchinin",
          "author_url": "",
          "post_date": "01/22/2022 05:00:05",
          "content": "<p>we wait )))</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1660072,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "01/22/2022 11:27:08",
          "content": "<p>code is up:<br>\n<a href=\"https://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955\" target=\"_blank\">https://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1666031,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "01/27/2022 10:45:05",
      "content": "<p>check thsi paper!<br>\nScale-aware Automatic Augmentation for Object Detection<br>\n<a href=\"https://openaccess.thecvf.com/content/CVPR2021/papers/Chen_Scale-Aware_Automatic_Augmentation_for_Object_Detection_CVPR_2021_paper.pdf\" target=\"_blank\">https://openaccess.thecvf.com/content/CVPR2021/papers/Chen_Scale-Aware_Automatic_Augmentation_for_Object_Detection_CVPR_2021_paper.pdf</a></p>\n<p>Learning Data Augmentation Strategies for Object Detection<br>\n<a href=\"https://paperswithcode.com/paper/learning-data-augmentation-strategies-for\" target=\"_blank\">https://paperswithcode.com/paper/learning-data-augmentation-strategies-for</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1673304,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/02/2022 16:30:19",
      "content": "<p>while blending works well, but we need to find a way to cut and paste to and from the correct region!<br>\nthere are two important image characteristics:</p>\n<ol>\n<li>blurness/shaprness : a blur COTS must be pasted into a blur background</li>\n<li>size: the reef and COTS must match in size<br>\nafter several experiments, i think this works best<br>\nresults of automatic COTS placement estimation:<br>\n<img src=\"https://i.ibb.co/G7JFbp0/Selection-024.png\" alt=\"https://i.ibb.co/G7JFbp0/Selection-024.png\"><br>\nwe neet a patch smiliarty matcher. the easiest way to train one (or download one from github) is the contrastive<br>\nnetwork:<br>\ntrain data <br>\none patch + rotated90/flip augment = same pair (i.e. smiliar patch texture)<br>\none patch + another patch (far in spatial and temporal distance) = different pair</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 1673611,
          "author_name": "outwrest",
          "author_url": "",
          "post_date": "02/02/2022 20:48:38",
          "content": "<p>Check <a href=\"https://www.kaggle.com/outwrest/augmentation-using-image-blending-cot-masks\" target=\"_blank\">this</a> out: Maybe with other types of blending and style transfers it can be improved. I used a cot-masks dataset that someone else had posted before and it was helpful to create many more images (you can download 10k unique images of 5 cots. I didn't see many results improvements, I'm looking into doing pretraining on it once I get the data to be better. This can be something to look into.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1652167": "GAN (generative adversarial net) results\n(from https://github.com/lucidrains/lightweight-gan)\n\n![https://i.ibb.co/wzxL4jN/19-ema.jpg](https://i.ibb.co/wzxL4jN/19-ema.jpg)\n\nHowever, i want to do in painting like this:\n![https://i.ibb.co/wWYTnpR/Selection-999-727.png](https://i.ibb.co/wWYTnpR/Selection-999-727.png)\n\n\nmaybe this application is suitable for GAN. the COTS objects are quite small, so we only need to generate  20x20 ~ 60x60 region. Is there any fast GAN  (e.g. train within 24hrs on one GPU), especially condition GAN that does inpainting, that anyone can recommend?",
    "1652185": "It’s not recommend, but I remembered what I tried in early time of competition.\n*Pedestrian GAN* seemed to suits this competition, but the public official code contains plenty of bug.\nI managed to make it run, but the output is far from realistic…\n\nhttps://www.kaggle.com/tatamikenn/generating-fake-starfish",
    "1652193": "I think the approach below might be more promising (no official implementation though).\nhttps://arxiv.org/abs/1910.07169",
    "1652207": "\". We observe that DetectorGAN significantly improves the average precision. Compared to training on real data only, the AP nearly doubles from 0.124 to\n0.236, and recall over triples from 0.184 to 0.649\"\n\nrecall seems good",
    "1652208": "What are result of your experiment on this dataset? How much have you improved?",
    "1652223": "Yes. That’s why I thought it promising.\n\nConsidering that It’s designed for finding tumors which is simpler than generating COTS, I don’t know whether it also effective in this competition.\n\nIf only we had official code.. The code seems hard to implement for me.",
    "1652242": "The patch size is small and the bbox dataset are not large, most GAN model might fit the 24hr-per gpu requirement.\nHowerver,\nI haved tried FUnIE-GAN(both using  pretrained weight and retrained)  \n                     cyclegan(unpaired training with GBR data)\n                     UGATIT (unpaired training with GBR data)\n                     NiceGAN (unpaired training with GBR data)\n                     ESRGAN (paired training with FUnIE data and apply in GBR data ) \nall scores(MAP) go down a little  \n\n(ps: I apply the function to whole image but not patch . might it be different ? )",
    "1652253": "let me read and think about it.\nthe rolling step is similar to the one in \"DARTS: DIFFERENTIABLE ARCHITECTURE SEARCH\" or \"UNROLLED GENERATIVE ADVERSARIAL NETWORKS\", which I have implemented before. you can read that paper and check their code\n\nI will try this\n```\nx = get data from dataloader\nx_fake = generator( ...)\n\n#copy weights from detector for unrolling later\nw = copy from detector\n\npredict = detector(torch.cat[x,x_fake])\nloss = loss_function(predict, truth)\nloss.backward()\noptimizer_d.step()\n\ndw = copy from optimizer_d\n\n#reset detector weight\nreset(w+dw, detector)\n\npredict_fake = detector(x_fake)\nloss = loss_function(predict_fake , truth_as_real) #adversial loss\nloss.backward()\noptimizer_g.step() #upate generator\n\n```",
    "1652254": "how about treating the gan images as unlabelled and relabeling them again (assuming the gan quality is not good)\ni.e. it becomes semi-supervised.",
    "1652307": "Thank you for sharing. I will check them.\n\nI don’t know it’s correct, but first time I read the paper, I imagined something like meta-learning:\nhttps://github.com/facebookresearch/higher\n\nI never implemented those kind of code, and maybe too much for me considering one month left. But the reference implementation will definitely help because I have no idea right now.",
    "1652311": "meta learning is correct",
    "1652337": "The gan-generated images are from GBR ,we already have annotation data. \n While we use them to train, we might treat  it as a kind of preprocessing  .\n If the model trained with low quality image can not recognize the bad patch, it fail.\n\nFor the bad patch, we might see the GBR as a super resolution problem.\nit might be help if we directly \"blur\" all the annotation bbox and train a super resolution model.",
    "1652353": "I see. Thank you for clarification.",
    "1652389": "I think he/she is trying to do inpainting, so he/she’ve not get result from this yet.",
    "1654136": "this is a shortcut\n\n![https://i.ibb.co/y69s6Qm/Selection-004.png](https://i.ibb.co/y69s6Qm/Selection-004.png)\n\nsince we are given video, it is sufficient to cut and paste (merge the boundary using soft mask).\nusing this we can remove or add cots.\n\ntips 1:\nwhen moving patches across different frames, try to limited x,y displacement if you don't  post process to keep object size and image  quality similar\n\n----\n\nelse you would need to to do some style transfer (e.g. color, blurriness, etc) which is also not too difficult using GAN\n\n\n-- \nsee\nPatchMatch\nhttps://en.wikipedia.org/wiki/PatchMatch\nhttps://gfx.cs.princeton.edu/pubs/Barnes_2009_PAR/\n\n![https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/PatchMatch.jpg/1920px-PatchMatch.jpg](https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/PatchMatch.jpg/1920px-PatchMatch.jpg)",
    "1656023": "image harmonization will help",
    "1657192": "amazing, tkx",
    "1659390": "~~code coming soon!~~\nhttps://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955\n\n![https://i.ibb.co/58wSyX9/Selection-019.png](https://i.ibb.co/58wSyX9/Selection-019.png)",
    "1659609": "Cool. But is it safe to rely on google’s algorithm to filter license?",
    "1659617": "What they say:\n\n> Google filters images by license based on information provided by the sites that host those images, or the image provider.\n\nhttps://support.google.com/websearch/answer/29508\n\nI wonder if the algorithm is perfect or not.",
    "1659619": "It seems we have to double check the license manually. I takes some cost, but I’m looking forward to the code.",
    "1659645": "i have to check the license manually.\n\nalso you can cut and paste from different frames of the kaggle train data.",
    "1659742": "we wait )))",
    "1660072": "code is up:\nhttps://www.kaggle.com/hengck23/augmentation-using-image-blending?scriptVersionId=85859955",
    "1666031": "check thsi paper!\nScale-aware Automatic Augmentation for Object Detection\nhttps://openaccess.thecvf.com/content/CVPR2021/papers/Chen_Scale-Aware_Automatic_Augmentation_for_Object_Detection_CVPR_2021_paper.pdf\n\n\nLearning Data Augmentation Strategies for Object Detection\nhttps://paperswithcode.com/paper/learning-data-augmentation-strategies-for",
    "1673304": "while blending works well, but we need to find a way to cut and paste to and from the correct region!\n\nthere are two important image characteristics:\n1. blurness/shaprness : a blur COTS must be pasted into a blur background\n2. size: the reef and COTS must match in size\n\nafter several experiments, i think this works best\n\nresults of automatic COTS placement estimation:\n![https://i.ibb.co/G7JFbp0/Selection-024.png](https://i.ibb.co/G7JFbp0/Selection-024.png)\n\n\nwe neet a patch smiliarty matcher. the easiest way to train one (or download one from github) is the contrastive\nnetwork:\n\ntrain data \n\none patch + rotated90/flip augment = same pair (i.e. smiliar patch texture)\none patch + another patch (far in spatial and temporal distance) = different pair",
    "1673611": "Check [this](https://www.kaggle.com/outwrest/augmentation-using-image-blending-cot-masks) out: Maybe with other types of blending and style transfers it can be improved. I used a cot-masks dataset that someone else had posted before and it was helpful to create many more images (you can download 10k unique images of 5 cots. I didn't see many results improvements, I'm looking into doing pretraining on it once I get the data to be better. This can be something to look into."
  },
  "source": "meta"
}