\nWe can generate more star fishes and put them in different under-sea images.
\n
![\"poisson.png\"](\"https://i.imgur.com/eRzAF1s.png\")
Congrats to all the winners, and thanks to organizers.
\nThe following ideas and methods helped me survive from this challenging competition.
\n1.Copy COTS box and paste into background image, apply poisson blending.
\n2.Several detection models training and inference with different image sizes.
\n3.Appending boxes to detection results by finding homography matrix.
\n4.Trust Local CV.
As the dataset is relative small, if we have more samples we can benefit from them.
\nWe can generate more star fishes and put them in different under-sea images.
\n
The simplest way is to crop the star fishes in train-set and paste it to other images, but we can find two obvious problems.
\n1.The boundary and color is not real.
\n2.The COTS is not on reasonable places.
I solved these two problems by:
\n1.Poisson blending
\n2.Training a classification model to predict whether a box with COTS is real or fake.
This idea helped me to improve the score.
\nI did not stop my experiments, I tried to train GAN models to generate starfishes, although the generated samples were quite real, the score did not improve.
\nI think it’s limited by the count of unique star fishes are small, if we can use more COTS downloaded from internet, I guess we could improve the score. I am not sure whether an image is allowed to use or not, so I did not use any of them.
\nOther methods such as image harmonization did not bring improvement.
\nAnyway, despite quite a lot of time spent without too many gains in this competition, I quite enjoyed it.
Validation Strategy
\nSplit 5 folds by sequence
Models
\nYOLOv5-S6, YOLOv5-M6, YOLOv5-L6, YOLOX-L, YOLOR-P6 and HRNetV2P-W18
Training details
\nYOLOv5:
\nNetwork: YOLOv5-S6, YOLOv5-M6, YOLOv5-L6
\nTraining-size: 3600
\nInference-Size: 4800
\nOptimizer: SGD
\nScheduler: Warm Up + Linear LR + lr=0.01 + 15 epochs
\nAugmentation: hsv, translate, scale, flipud, fliplr, mosaic, mixup, water-augment, transpose
YOLOX:
\nNetwork: YOLOX-L
\nTraining-size: 1280
\nInference-Size: 1600
\nOptimizer: SGD
\nScheduler: yoloxwarmcos + 20 epochs
\nAugmentation: hsv, flip, degrees, translate, shear, mosaic, mixup, no_aug_epochs=5
YOLOR:
\nNetwork: YOLOR-P6
\nTraining-size: 2560
\nInference-Size: 2560
\nOptimizer: SGD
\nScheduler: Warm Up + Linear LR + lr=0.005 + 15 epochs
\nAugmentation: hsv, translate, scale, flipud, fliplr, mosaic, mixup, water-augment, transpose
HRNet:
\nNetwork: HRNetV2P-W18
\nTraining-size: 3600
\nInference-Size: 3600
\nOptimizer: SGD
\nScheduler: Warm Up + linear LR + lr=0.02 + 10 epochs
\nAugmentation: Resize, RandomFlip
If the detection model has found a box in previous frames and we can predict the box in the current frame by the following way:
\n1.Kalman Filter
\n2.Optic Flow
\n3.Finding homography matrix and then apply the matrix to get the box in the current frame.
I find the third method is the best for this dataset.
\nWe can get keypoint descriptors by using SuperPoint/SuperGlue and then find homography matrix.
The Tracking pipeline is:
\n1.If there is a box detected a frame t-10, then append a box using the homography matrix to frame t-9 unless there already a box is there(IOU greater than 0.4). Then repeat this procedure from t-9 until current frame.
\n2.Apply DeepSort to get a track.
\n3.Determining a box on a track is kept or not by the ratio of the model predicted boxes. If the ratio is too low, the track may be False Positive.
Calculating IOU between the boxes which predicted by different models in an image.
\nIf max IOU with other boxes of a box is less than 0.55,then drop this box.
\nThe remaining boxes are ensembled using WBF.
![\"result1.png\"](\"https://i.imgur.com/p2iHvpi.png\")
\n![\"result2.png\"](\"https://i.imgur.com/6Hm3GRQ.png\")
Congrats to all the winners, and thanks to organizers.
\nThe following ideas and methods helped me survive from this challenging competition.
\n1.Copy COTS box and paste into background image, apply poisson blending.
\n2.Several detection models training and inference with different image sizes.
\n3.Appending boxes to detection results by finding homography matrix.
\n4.Trust Local CV.
As the dataset is relative small, if we have more samples we can benefit from them.
\nWe can generate more star fishes and put them in different under-sea images.
\n
The simplest way is to crop the star fishes in train-set and paste it to other images, but we can find two obvious problems.
\n1.The boundary and color is not real.
\n2.The COTS is not on reasonable places.
I solved these two problems by:
\n1.Poisson blending
\n2.Training a classification model to predict whether a box with COTS is real or fake.
This idea helped me to improve the score.
\nI did not stop my experiments, I tried to train GAN models to generate starfishes, although the generated samples were quite real, the score did not improve.
\nI think it’s limited by the count of unique star fishes are small, if we can use more COTS downloaded from internet, I guess we could improve the score. I am not sure whether an image is allowed to use or not, so I did not use any of them.
\nOther methods such as image harmonization did not bring improvement.
\nAnyway, despite quite a lot of time spent without too many gains in this competition, I quite enjoyed it.
Validation Strategy
\nSplit 5 folds by sequence
Models
\nYOLOv5-S6, YOLOv5-M6, YOLOv5-L6, YOLOX-L, YOLOR-P6 and HRNetV2P-W18
Training details
\nYOLOv5:
\nNetwork: YOLOv5-S6, YOLOv5-M6, YOLOv5-L6
\nTraining-size: 3600
\nInference-Size: 4800
\nOptimizer: SGD
\nScheduler: Warm Up + Linear LR + lr=0.01 + 15 epochs
\nAugmentation: hsv, translate, scale, flipud, fliplr, mosaic, mixup, water-augment, transpose
YOLOX:
\nNetwork: YOLOX-L
\nTraining-size: 1280
\nInference-Size: 1600
\nOptimizer: SGD
\nScheduler: yoloxwarmcos + 20 epochs
\nAugmentation: hsv, flip, degrees, translate, shear, mosaic, mixup, no_aug_epochs=5
YOLOR:
\nNetwork: YOLOR-P6
\nTraining-size: 2560
\nInference-Size: 2560
\nOptimizer: SGD
\nScheduler: Warm Up + Linear LR + lr=0.005 + 15 epochs
\nAugmentation: hsv, translate, scale, flipud, fliplr, mosaic, mixup, water-augment, transpose
HRNet:
\nNetwork: HRNetV2P-W18
\nTraining-size: 3600
\nInference-Size: 3600
\nOptimizer: SGD
\nScheduler: Warm Up + linear LR + lr=0.02 + 10 epochs
\nAugmentation: Resize, RandomFlip
If the detection model has found a box in previous frames and we can predict the box in the current frame by the following way:
\n1.Kalman Filter
\n2.Optic Flow
\n3.Finding homography matrix and then apply the matrix to get the box in the current frame.
I find the third method is the best for this dataset.
\nWe can get keypoint descriptors by using SuperPoint/SuperGlue and then find homography matrix.
The Tracking pipeline is:
\n1.If there is a box detected a frame t-10, then append a box using the homography matrix to frame t-9 unless there already a box is there(IOU greater than 0.4). Then repeat this procedure from t-9 until current frame.
\n2.Apply DeepSort to get a track.
\n3.Determining a box on a track is kept or not by the ratio of the model predicted boxes. If the ratio is too low, the track may be False Positive.
Calculating IOU between the boxes which predicted by different models in an image.
\nIf max IOU with other boxes of a box is less than 0.55,then drop this box.
\nThe remaining boxes are ensembled using WBF.
\n
Thanks for sharing your solution! 🙏
\n\n\nI did not stop my experiments, I tried to train GAN models to generate starfishes, although the generated samples were quite real, the score did not improve.
\n
May I ask what models were you using and what approaches were you taking? I was thinking about following this for the whale competition where many classes have just 1 image.
\nTIA!
", "rawMarkdown": "Thanks for sharing your solution! 🙏\n\n> I did not stop my experiments, I tried to train GAN models to generate starfishes, although the generated samples were quite real, the score did not improve. \n\nMay I ask what models were you using and what approaches were you taking? I was thinking about following this for the whale competition where many classes have just 1 image. \n\nTIA!", "votes": null }, { "id": "1693484", "postDate": "02/16/2022 17:40:09", "content": "GAN augmentation only seems to work in papers, I also never made it work for models, they are clever enough. Only solution I ever saw it work was in Bengali.
", "rawMarkdown": "GAN augmentation only seems to work in papers, I also never made it work for models, they are clever enough. Only solution I ever saw it work was in Bengali.", "votes": null }, { "id": "1693518", "postDate": "02/16/2022 17:57:02", "content": "Oh wow-I didn't know about the Bengali competition. I found the writeup by the winning team incase anyone reading here is interested.
\nThank you, Psi 🦁! 🍵
", "rawMarkdown": "Oh wow-I didn't know about the Bengali competition. I found the [writeup](https://www.kaggle.com/c/bengaliai-cv19/discussion/135984) by the winning team incase anyone reading here is interested. \n\nThank you, Psi 🦁! 🍵", "votes": null }, { "id": "1693529", "postDate": "02/16/2022 18:07:28", "content": "Hey @init27, StyleGAN2, I tried many models in https://www.kaggle.com/c/generative-dog-images competition and followed progresses after that. The main reason I entered this competition is to check whether GAN and image blending technologies can help solve this kind of problems.
", "rawMarkdown": "Hey @init27, StyleGAN2, I tried many models in https://www.kaggle.com/c/generative-dog-images competition and followed progresses after that. The main reason I entered this competition is to check whether GAN and image blending technologies can help solve this kind of problems.", "votes": null }, { "id": "1693556", "postDate": "02/16/2022 18:26:58", "content": "is there a paper or code reference for poisson blending? nice tabulation of results!
", "rawMarkdown": "is there a paper or code reference for poisson blending? nice tabulation of results!", "votes": null }, { "id": "1693838", "postDate": "02/17/2022 02:04:00", "content": "Great job, tkx
", "rawMarkdown": "Great job, tkx", "votes": null }, { "id": "1694195", "postDate": "02/17/2022 08:36:07", "content": "\n\nTraining a classification model to predict whether a box with COTS is real or fake.
\n
Can this part describe the process in detail?
\nIf you have relevant code for reference, it would be greatly appreciated.
", "rawMarkdown": "> Training a classification model to predict whether a box with COTS is real or fake.\n\nCan this part describe the process in detail?\n1. How to construct the target of training data, first pass copy-paste and then manually select whether it is real or fake?\n2. What model was chosen?\n3. Is it to classify the whole image, or only the bbox?\n4. proportion of generated data to original data, then fold all data?\n\nIf you have relevant code for reference, it would be greatly appreciated.", "votes": null }, { "id": "1694291", "postDate": "02/17/2022 10:22:33", "content": "https://www.cs.jhu.edu/~misha/Fall07/Papers/Perez03.pdf
\nhttps://github.com/PPPW/poisson-image-editing
The bbox in training set is real, so the target is 1, the poisson blending COTS boxes is fake, the target is 0.
Any classification model is OK for this task, such resnet34.
bbox, resize to 64x64.
1.4k fake images
OpenCV supports poisson blending as seamlessClone.
\nhttps://docs.opencv.org/4.5.5/df/da0/group__photo__clone.html#ga2bf426e4c93a6b1f21705513dfeca49d
\nhttps://learnopencv.com/seamless-cloning-using-opencv-python-cpp/
Congraatulations🤩🤩Thanks for sharing your code @bestfitting , feel free to check out my notebooks as well 😊
", "rawMarkdown": "Congraatulations🤩🤩Thanks for sharing your code @bestfitting , feel free to check out my [notebooks](https://www.kaggle.com/arunasivapragasam/notebooks) as well 😊", "votes": null }, { "id": "1696366", "postDate": "02/18/2022 19:22:55", "content": "So from what I understand @bestfitting, to increase the data on which model can train, you created a script that copies and paste the starfish on different video at a different location and then use poison blending so that it fits in that background. Is that right?
", "rawMarkdown": "So from what I understand @bestfitting, to increase the data on which model can train, you created a script that copies and paste the starfish on different video at a different location and then use poison blending so that it fits in that background. Is that right?", "votes": null }, { "id": "1696602", "postDate": "02/19/2022 00:41:04", "content": "@bestfitting from github link it seems we need mask to do image poissioning, how did you get the mask?
\n![\"\"](\"https://github.com/PPPW/poisson-image-editing/raw/master/figs/example1/all.png\")
I think we should fill the box containing COTS with 255 to generate the mask.
\nbtw, I did not use the lib indeed, I gave you as reference.
yes! Then I selected the generated COTS with different backgroud using the classification model.
", "rawMarkdown": "yes! Then I selected the generated COTS with different backgroud using the classification model.", "votes": null }, { "id": "1698109", "postDate": "02/20/2022 06:35:18", "content": "The poisson blending method is very interesting, and seems like a traditional CV-like GAN to me.
\nHow did you train the discriminator (classifier) for your pipeline?
", "rawMarkdown": "The poisson blending method is very interesting, and seems like a traditional CV-like GAN to me.\n\nHow did you train the discriminator (classifier) for your pipeline?", "votes": null }, { "id": "1698229", "postDate": "02/20/2022 08:14:55", "content": "The box containing real COTS as positive sample, the box containing fake/blending COTS as negative sample, training a binary classification model(Resnet34). Then use this model to predict a fake sample, if the score as positive sample is high, then use this fake sample.
", "rawMarkdown": "The box containing real COTS as positive sample, the box containing fake/blending COTS as negative sample, training a binary classification model(Resnet34). Then use this model to predict a fake sample, if the score as positive sample is high, then use this fake sample.", "votes": null }, { "id": "1698555", "postDate": "02/20/2022 13:34:33", "content": "@bestfitting I saw that how did you train the classification model to classify fake and real and at the end you got 1.4k fake images but how did you annotate 1.4 fake images for bbox? How did you resize bbox to 64x64?
", "rawMarkdown": "bestfitting I saw that how did you train the classification model to classify fake and real and at the end you got 1.4k fake images but how did you annotate 1.4 fake images for bbox? How did you resize bbox to 64x64?", "votes": null }, { "id": "1698698", "postDate": "02/20/2022 15:31:14", "content": "I generated many many fake images by using copy&paste then poisson blending, then selected 1.4k of them by the model.
\nSince the fake image is generated by pasting COTS to a background, the backgroud itself can be patch with size 64x64 or large, after we paste the COTS to it ,then resize to 64x64.
Thank you for sharing
", "rawMarkdown": "Thank you for sharing", "votes": null }, { "id": "1702354", "postDate": "02/23/2022 14:40:37", "content": "A novel idea. I want to be able to master it
", "rawMarkdown": "A novel idea. I want to be able to master it", "votes": null }, { "id": "1718534", "postDate": "03/10/2022 23:26:20", "content": "\n\n\"The main reason I entered this competition is to check whether GAN and image blending technologies can help solve this kind of problems.\"
\n
Hi, @bestfitting. congrats! We have some agreement on this.
\n1) GAN doesn't work for this. According to information theory, you train the GAN by these photos; then generate photos again? Actually, these two photo sets have the same \"information size\", nothing more.
\n2) image blending works. I agree.
hey,
\nIf we paste a generated COTS to other background, it's will help our models as an augmentation method.
\nI had planed to use other COTS from internet to generate samples, but was not sure whether an image or a video is permitted to use or not and I am very busy during this competition, so I did not use any of them.
\n", "rawMarkdown": "> \"I had planed to use other COTS from internet to generate samples\"\nYes, you could do it, I also tried it. But the problem is that you cannot find the right starfish picture with the same color style on the internet. Lots of high-quality photos of starfish are there, but you know you cannot add them into your training dataset as probably they would not be in the test datasets either. Finally, I believe your thought at that time is right.", "votes": null }, { "id": "1755191", "postDate": "04/14/2022 12:10:43", "content": "\"I had planed to use other COTS from internet to generate samples\"
\n
\n Yes, you could do it, I also tried it. But the problem is that you cannot find the right starfish picture with the same color style on the internet. Lots of high-quality photos of starfish are there, but you know you cannot add them into your training dataset as probably they would not be in the test datasets either. Finally, I believe your thought at that time is right.
Need a bigger bbox than the original one to include the feature edge, before resize, is it right?
", "rawMarkdown": "Need a bigger bbox than the original one to include the feature edge, before resize, is it right?", "votes": null }, { "id": "1763030", "postDate": "04/21/2022 08:12:36", "content": "@w3579628328
\nyes, bigger bbox to include some background…
Thanks for sharing your solution! 🙏
\n\n\nI did not stop my experiments, I tried to train GAN models to generate starfishes, although the generated samples were quite real, the score did not improve.
\n
May I ask what models were you using and what approaches were you taking? I was thinking about following this for the whale competition where many classes have just 1 image.
\nTIA!
", "votes": null, "replies": [ { "id": 1693484, "author_name": "philippsinger", "author_url": "", "post_date": "02/16/2022 17:40:09", "content": "GAN augmentation only seems to work in papers, I also never made it work for models, they are clever enough. Only solution I ever saw it work was in Bengali.
", "votes": null, "replies": [] }, { "id": 1693518, "author_name": "init27", "author_url": "", "post_date": "02/16/2022 17:57:02", "content": "Oh wow-I didn't know about the Bengali competition. I found the writeup by the winning team incase anyone reading here is interested.
\nThank you, Psi 🦁! 🍵
", "votes": null, "replies": [] }, { "id": 1693529, "author_name": "bestfitting", "author_url": "", "post_date": "02/16/2022 18:07:28", "content": "Hey @init27, StyleGAN2, I tried many models in https://www.kaggle.com/c/generative-dog-images competition and followed progresses after that. The main reason I entered this competition is to check whether GAN and image blending technologies can help solve this kind of problems.
", "votes": null, "replies": [] } ] }, { "id": 1693556, "author_name": "mpdroid", "author_url": "", "post_date": "02/16/2022 18:26:58", "content": "is there a paper or code reference for poisson blending? nice tabulation of results!
", "votes": null, "replies": [ { "id": 1694291, "author_name": "bestfitting", "author_url": "", "post_date": "02/17/2022 10:22:33", "content": "https://www.cs.jhu.edu/~misha/Fall07/Papers/Perez03.pdf
\nhttps://github.com/PPPW/poisson-image-editing
OpenCV supports poisson blending as seamlessClone.
\nhttps://docs.opencv.org/4.5.5/df/da0/group__photo__clone.html#ga2bf426e4c93a6b1f21705513dfeca49d
\nhttps://learnopencv.com/seamless-cloning-using-opencv-python-cpp/
@bestfitting from github link it seems we need mask to do image poissioning, how did you get the mask?
\n
I think we should fill the box containing COTS with 255 to generate the mask.
\nbtw, I did not use the lib indeed, I gave you as reference.
Great job, tkx
", "votes": null, "replies": [] }, { "id": 1694195, "author_name": "tikboa", "author_url": "", "post_date": "02/17/2022 08:36:07", "content": "\n\nTraining a classification model to predict whether a box with COTS is real or fake.
\n
Can this part describe the process in detail?
\nIf you have relevant code for reference, it would be greatly appreciated.
", "votes": null, "replies": [ { "id": 1694300, "author_name": "bestfitting", "author_url": "", "post_date": "02/17/2022 10:28:18", "content": "The bbox in training set is real, so the target is 1, the poisson blending COTS boxes is fake, the target is 0.
Any classification model is OK for this task, such resnet34.
bbox, resize to 64x64.
1.4k fake images
Need a bigger bbox than the original one to include the feature edge, before resize, is it right?
", "votes": null, "replies": [] }, { "id": 1763030, "author_name": "bestfitting", "author_url": "", "post_date": "04/21/2022 08:12:36", "content": "@w3579628328
\nyes, bigger bbox to include some background…
Congraatulations🤩🤩Thanks for sharing your code @bestfitting , feel free to check out my notebooks as well 😊
", "votes": null, "replies": [] }, { "id": 1696366, "author_name": "keagle", "author_url": "", "post_date": "02/18/2022 19:22:55", "content": "So from what I understand @bestfitting, to increase the data on which model can train, you created a script that copies and paste the starfish on different video at a different location and then use poison blending so that it fits in that background. Is that right?
", "votes": null, "replies": [ { "id": 1696700, "author_name": "bestfitting", "author_url": "", "post_date": "02/19/2022 03:33:52", "content": "yes! Then I selected the generated COTS with different backgroud using the classification model.
", "votes": null, "replies": [] } ] }, { "id": 1698109, "author_name": "kyoshioka47", "author_url": "", "post_date": "02/20/2022 06:35:18", "content": "The poisson blending method is very interesting, and seems like a traditional CV-like GAN to me.
\nHow did you train the discriminator (classifier) for your pipeline?
", "votes": null, "replies": [ { "id": 1698229, "author_name": "bestfitting", "author_url": "", "post_date": "02/20/2022 08:14:55", "content": "The box containing real COTS as positive sample, the box containing fake/blending COTS as negative sample, training a binary classification model(Resnet34). Then use this model to predict a fake sample, if the score as positive sample is high, then use this fake sample.
", "votes": null, "replies": [] } ] }, { "id": 1698555, "author_name": "nyanswanaung", "author_url": "", "post_date": "02/20/2022 13:34:33", "content": "@bestfitting I saw that how did you train the classification model to classify fake and real and at the end you got 1.4k fake images but how did you annotate 1.4 fake images for bbox? How did you resize bbox to 64x64?
", "votes": null, "replies": [ { "id": 1698698, "author_name": "bestfitting", "author_url": "", "post_date": "02/20/2022 15:31:14", "content": "I generated many many fake images by using copy&paste then poisson blending, then selected 1.4k of them by the model.
\nSince the fake image is generated by pasting COTS to a background, the backgroud itself can be patch with size 64x64 or large, after we paste the COTS to it ,then resize to 64x64.
Thank you for sharing
", "votes": null, "replies": [] }, { "id": 1702354, "author_name": "hiro310", "author_url": "", "post_date": "02/23/2022 14:40:37", "content": "A novel idea. I want to be able to master it
", "votes": null, "replies": [] }, { "id": 1718534, "author_name": "jeffreyhao", "author_url": "", "post_date": "03/10/2022 23:26:20", "content": "\n\n\"The main reason I entered this competition is to check whether GAN and image blending technologies can help solve this kind of problems.\"
\n
Hi, @bestfitting. congrats! We have some agreement on this.
\n1) GAN doesn't work for this. According to information theory, you train the GAN by these photos; then generate photos again? Actually, these two photo sets have the same \"information size\", nothing more.
\n2) image blending works. I agree.
hey,
\nIf we paste a generated COTS to other background, it's will help our models as an augmentation method.
\nI had planed to use other COTS from internet to generate samples, but was not sure whether an image or a video is permitted to use or not and I am very busy during this competition, so I did not use any of them.
\n", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1693417": "Congrats to all the winners, and thanks to organizers.\n\nThe following ideas and methods helped me survive from this challenging competition.\n1.Copy COTS box and paste into background image, apply poisson blending.\n2.Several detection models training and inference with different image sizes.\n3.Appending boxes to detection results by finding homography matrix.\n4.Trust Local CV.\n\n## Data\nAs the dataset is relative small, if we have more samples we can benefit from them.\nWe can generate more star fishes and put them in different under-sea images.\n\n\nThe simplest way is to crop the star fishes in train-set and paste it to other images, but we can find two obvious problems.\n1.The boundary and color is not real.\n2.The COTS is not on reasonable places.\n\nI solved these two problems by:\n\n1.Poisson blending\n2.Training a classification model to predict whether a box with COTS is real or fake.\n\nThis idea helped me to improve the score.\n\nI did not stop my experiments, I tried to train GAN models to generate starfishes, although the generated samples were quite real, the score did not improve. \nI think it’s limited by the count of unique star fishes are small, if we can use more COTS downloaded from internet, I guess we could improve the score. I am not sure whether an image is allowed to use or not, so I did not use any of them. \nOther methods such as image harmonization did not bring improvement. \nAnyway, despite quite a lot of time spent without too many gains in this competition, I quite enjoyed it.\n\n## Model training\n\n**Validation Strategy**\nSplit 5 folds by sequence\n\n**Models**\nYOLOv5-S6, YOLOv5-M6, YOLOv5-L6, YOLOX-L, YOLOR-P6 and HRNetV2P-W18\n\n**Training details**\nYOLOv5: \nNetwork: YOLOv5-S6, YOLOv5-M6, YOLOv5-L6\nTraining-size: 3600\nInference-Size: 4800\nOptimizer: SGD\nScheduler: Warm Up + Linear LR + lr=0.01 + 15 epochs\nAugmentation: hsv, translate, scale, flipud, fliplr, mosaic, mixup, water-augment, transpose\n\nYOLOX:\nNetwork: YOLOX-L\nTraining-size: 1280\nInference-Size: 1600\nOptimizer: SGD\nScheduler: yoloxwarmcos + 20 epochs\nAugmentation: hsv, flip, degrees, translate, shear, mosaic, mixup, no_aug_epochs=5\n\nYOLOR:\nNetwork: YOLOR-P6\nTraining-size: 2560\nInference-Size: 2560\nOptimizer: SGD\nScheduler: Warm Up + Linear LR + lr=0.005 + 15 epochs\nAugmentation: hsv, translate, scale, flipud, fliplr, mosaic, mixup, water-augment, transpose\n\nHRNet:\nNetwork: HRNetV2P-W18\nTraining-size: 3600\nInference-Size: 3600\nOptimizer: SGD\nScheduler: Warm Up + linear LR + lr=0.02 + 10 epochs\nAugmentation: Resize, RandomFlip\n\n## Tracking\nIf the detection model has found a box in previous frames and we can predict the box in the current frame by the following way:\n1.Kalman Filter\n2.Optic Flow\n3.Finding homography matrix and then apply the matrix to get the box in the current frame.\n\nI find the third method is the best for this dataset.\nWe can get keypoint descriptors by using SuperPoint/SuperGlue and then find homography matrix.\n\n**The Tracking pipeline is:**\n1.If there is a box detected a frame t-10, then append a box using the homography matrix to frame t-9 unless there already a box is there(IOU greater than 0.4). Then repeat this procedure from t-9 until current frame.\n2.Apply DeepSort to get a track.\n3.Determining a box on a track is kept or not by the ratio of the model predicted boxes. If the ratio is too low, the track may be False Positive.\n\n## Ensemble\nCalculating IOU between the boxes which predicted by different models in an image.\nIf max IOU with other boxes of a box is less than 0.55,then drop this box.\nThe remaining boxes are ensembled using WBF.\n\n\n## Results\n\n", "1693478": "Thanks for sharing your solution! 🙏\n\n> I did not stop my experiments, I tried to train GAN models to generate starfishes, although the generated samples were quite real, the score did not improve. \n\nMay I ask what models were you using and what approaches were you taking? I was thinking about following this for the whale competition where many classes have just 1 image. \n\nTIA!", "1693484": "GAN augmentation only seems to work in papers, I also never made it work for models, they are clever enough. Only solution I ever saw it work was in Bengali.", "1693518": "Oh wow-I didn't know about the Bengali competition. I found the [writeup](https://www.kaggle.com/c/bengaliai-cv19/discussion/135984) by the winning team incase anyone reading here is interested. \n\nThank you, Psi 🦁! 🍵", "1693529": "Hey @init27, StyleGAN2, I tried many models in https://www.kaggle.com/c/generative-dog-images competition and followed progresses after that. The main reason I entered this competition is to check whether GAN and image blending technologies can help solve this kind of problems.", "1693556": "is there a paper or code reference for poisson blending? nice tabulation of results!", "1693838": "Great job, tkx", "1694195": "> Training a classification model to predict whether a box with COTS is real or fake.\n\nCan this part describe the process in detail?\n1. How to construct the target of training data, first pass copy-paste and then manually select whether it is real or fake?\n2. What model was chosen?\n3. Is it to classify the whole image, or only the bbox?\n4. proportion of generated data to original data, then fold all data?\n\nIf you have relevant code for reference, it would be greatly appreciated.", "1694291": "https://www.cs.jhu.edu/~misha/Fall07/Papers/Perez03.pdf\nhttps://github.com/PPPW/poisson-image-editing", "1694300": "1. The bbox in training set is real, so the target is 1, the poisson blending COTS boxes is fake, the target is 0.\n\n2. Any classification model is OK for this task, such resnet34.\n\n3. bbox, resize to 64x64.\n4. 1.4k fake images", "1694530": "OpenCV supports poisson blending as `seamlessClone`.\nhttps://docs.opencv.org/4.5.5/df/da0/group__photo__clone.html#ga2bf426e4c93a6b1f21705513dfeca49d\nhttps://learnopencv.com/seamless-cloning-using-opencv-python-cpp/", "1695659": "Congraatulations🤩🤩Thanks for sharing your code @bestfitting , feel free to check out my [notebooks](https://www.kaggle.com/arunasivapragasam/notebooks) as well 😊", "1696366": "So from what I understand @bestfitting, to increase the data on which model can train, you created a script that copies and paste the starfish on different video at a different location and then use poison blending so that it fits in that background. Is that right?", "1696602": "bestfitting from github link it seems we need **mask** to do image poissioning, how did you get the mask?\n", "1696698": "I think we should fill the box containing COTS with 255 to generate the mask. \nbtw, I did not use the lib indeed, I gave you as reference.", "1696700": "yes! Then I selected the generated COTS with different backgroud using the classification model.", "1698109": "The poisson blending method is very interesting, and seems like a traditional CV-like GAN to me.\n\nHow did you train the discriminator (classifier) for your pipeline?", "1698229": "The box containing real COTS as positive sample, the box containing fake/blending COTS as negative sample, training a binary classification model(Resnet34). Then use this model to predict a fake sample, if the score as positive sample is high, then use this fake sample.", "1698555": "bestfitting I saw that how did you train the classification model to classify fake and real and at the end you got 1.4k fake images but how did you annotate 1.4 fake images for bbox? How did you resize bbox to 64x64?", "1698698": "I generated many many fake images by using copy&paste then poisson blending, then selected 1.4k of them by the model.\nSince the fake image is generated by pasting COTS to a background, the backgroud itself can be patch with size 64x64 or large, after we paste the COTS to it ,then resize to 64x64.", "1699134": "Thank you for sharing", "1702354": "A novel idea. I want to be able to master it", "1718534": "> \"The main reason I entered this competition is to check whether GAN and image blending technologies can help solve this kind of problems.\"\n\nHi, @bestfitting. congrats! We have some agreement on this.\n1) GAN doesn't work for this. According to information theory, you train the GAN by these photos; then generate photos again? Actually, these two photo sets have the same \"information size\", nothing more.\n2) image blending works. I agree.", "1719073": "hey,\nIf we paste a generated COTS to other background, it's will help our models as an augmentation method.\nI had planed to use other COTS from internet to generate samples, but was not sure whether an image or a video is permitted to use or not and I am very busy during this competition, so I did not use any of them.", "1719105": "> \"I had planed to use other COTS from internet to generate samples\"\nYes, you could do it, I also tried it. But the problem is that you cannot find the right starfish picture with the same color style on the internet. Lots of high-quality photos of starfish are there, but you know you cannot add them into your training dataset as probably they would not be in the test datasets either. Finally, I believe your thought at that time is right.", "1755191": "Need a bigger bbox than the original one to include the feature edge, before resize, is it right?", "1763030": "w3579628328 \nyes, bigger bbox to include some background..." }, "source": "meta" }\"I had planed to use other COTS from internet to generate samples\"
\n
\n Yes, you could do it, I also tried it. But the problem is that you cannot find the right starfish picture with the same color style on the internet. Lots of high-quality photos of starfish are there, but you know you cannot add them into your training dataset as probably they would not be in the test datasets either. Finally, I believe your thought at that time is right.