![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1182060%2Fdf20ee33a44d77843f547603e3b44e9b%2Fcbsd.png?generation=1606740430485871&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1182060%2F7dcaaab42543c774250f43ef930a3b22%2Fcbb.png?generation=1606740444469123&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1182060%2Fb3374380fb555d1b3ff36831f3572cb6%2Fcgm.png?generation=1606740455230574&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1182060%2F02aef0b81e4ad215fdf809f2c78c4cdb%2Fcmd.png?generation=1606740467241006&alt=media\")
I was thinking about doing some experimentations to use GANs and generate synthetic data and maybe use it for pre-training, I saw that some people already mentioned this idea, but I am curious as to what kind of GAN would work better here, I already have some experience with CycleGAN, but maybe there is a more recent architecture.
\nUpdate 1: Just found the GitHub of LeafGAN, unfortunately, they have not provided the code yet.
\nUpdate 2: I have just created a notebook Cassava Leaf Disease - CycleGAN data augmentation that implements CycleGAN and generates leaf images of the 4 diseases types from healthy images.
I was thinking about doing some experimentations to use GANs and generate synthetic data and maybe use it for pre-training, I saw that some people already mentioned this idea, but I am curious as to what kind of GAN would work better here, I already have some experience with CycleGAN, but maybe there is a more recent architecture.
\nUpdate 1: Just found the GitHub of LeafGAN, unfortunately, they have not provided the code yet.
\nUpdate 2: I have just created a notebook Cassava Leaf Disease - CycleGAN data augmentation that implements CycleGAN and generates leaf images of the 4 diseases types from healthy images.
According to Iyan Goodfellow (the creator of GANs for those who don't know) generating more training data using GANs will not make the model generalize better since the images generated by GANs will replicate already existing features and not bring new features to the table it could lead to overfitting. I tried using it in a previous competition plant pathology and it didn't work, nonetheless I hope you prove me wrong.
", "rawMarkdown": "According to Iyan Goodfellow (the creator of GANs for those who don't know) generating more training data using GANs will not make the model generalize better since the images generated by GANs will replicate already existing features and not bring new features to the table it could lead to overfitting. I tried using it in a previous competition plant pathology and it didn't work, nonetheless I hope you prove me wrong. ", "votes": 1, "replies": [ { "id": 1100603, "postDate": "2020-12-03T07:26:25.647Z", "content": "Actually the intuition behind using these generated images is that, in many cases they are better than hand-crafted synthetic examples, and can improve a downstream model’s generalization as well
", "rawMarkdown": "Actually the intuition behind using these generated images is that, in many cases they are better than hand-crafted synthetic examples, and can improve a downstream model’s generalization as well\n" }, { "id": 1100609, "postDate": "2020-12-03T07:30:11.410Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1101141, "postDate": "2020-12-03T16:54:45.173Z", "content": "Thanks for your input @aziz69 , I have seen people using GAN generated data with good results in some applications, and I also believe that in some cases it might be useful, here the idea is to translate data from one domain to another, it is different from simply generating data from random distribution to one domain.
\nAnyway, If I manage to get good results I can provide the data as TFRecords and it would be very easy to try and evaluate the results.
", "rawMarkdown": "Thanks for your input @aziz69 , I have seen people using GAN generated data with good results in some applications, and I also believe that in some cases it might be useful, here the idea is to translate data from one domain to another, it is different from simply generating data from random distribution to one domain.\n\nAnyway, If I manage to get good results I can provide the data as TFRecords and it would be very easy to try and evaluate the results.", "votes": 1 } ] }, { "id": 1098391, "postDate": "2020-12-01T16:05:53.070Z", "content": "Interesting idea, thanks for sharing with us
", "rawMarkdown": "Interesting idea, thanks for sharing with us", "votes": 1 }, { "id": 1096353, "postDate": "2020-11-30T12:48:08.950Z", "content": "A few generated samples from my notebook Cassava Leaf Disease - CycleGAN data augmentation, I have trained the models for only a few epochs because of the time, but it looks interesting.
\n
\n
\n
\n
Just saw LeafGAN, pointed by Heng here, seems like the ideal solution, not sure yet I can find the code implementation.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Ff715d0ba9e6f4d686698c988df642b0a%2FSelection_036.png?generation=1606365075088349&alt=media\")
Nice finding! If you try CycleGAN, try feeding the output to the input again to see if it works?
", "rawMarkdown": "Nice finding! If you try CycleGAN, try feeding the output to the input again to see if it works?", "votes": 1 }, { "id": 1094231, "postDate": "2020-11-28T12:33:34.477Z", "content": "Hey @aeryss , I think this has a chance of working if you are able to generate images with very high quality, lets see if I can get a good result, but thanks for the tip.
", "rawMarkdown": "Hey @aeryss , I think this has a chance of working if you are able to generate images with very high quality, lets see if I can get a good result, but thanks for the tip." }, { "id": 1096912, "postDate": "2020-11-30T21:45:35.053Z", "content": "\" I think this has a chance of working if you are able to generate images with very high quality,\"
\nactually it would help even if generate images with low quality. you just need a classifier to classify the quality of the images. for image with low quality, the cross entropy loss is weighted less.
\none can generally treat GAN images as images with noisy labels or unlabelled (but with a new class known as \"none of the target class\")
", "rawMarkdown": "\" I think this has a chance of working if you are able to generate images with very high quality,\"\n\nactually it would help even if generate images with low quality. you just need a classifier to classify the quality of the images. for image with low quality, the cross entropy loss is weighted less. \n\none can generally treat GAN images as images with noisy labels or unlabelled (but with a new class known as \"none of the target class\")", "votes": 1 }, { "id": 1097158, "postDate": "2020-12-01T01:00:26.387Z", "content": "That is a great point @hengck23 , I am a big fan of multi-task, using image quality as an auxiliary target sounds good.
\nI am thinking about doing some experiments on pre-train as unsupervised learning, maybe using GAN data can help.
", "rawMarkdown": "That is a great point @hengck23 , I am a big fan of multi-task, using image quality as an auxiliary target sounds good.\n\nI am thinking about doing some experiments on pre-train as unsupervised learning, maybe using GAN data can help." }, { "id": 1097259, "postDate": "2020-12-01T01:54:55.230Z", "content": "check this work as well
\nhttps://hirokatsukataoka16.github.io/Pretraining-without-Natural-Images/
\nPre-training without Natural Images
Asian Conference on Computer Vision (ACCV) 2020
\nBest Paper Honorable Mention Award
\nOral Presentation
Image space is Fractal?
", "rawMarkdown": "check this work as well\n\nhttps://hirokatsukataoka16.github.io/Pretraining-without-Natural-Images/\nPre-training without Natural Images\n\nAsian Conference on Computer Vision (ACCV) 2020\nBest Paper Honorable Mention Award\nOral Presentation \n\nImage space is Fractal?", "votes": 2 }, { "id": 1098731, "postDate": "2020-12-01T19:53:10.713Z", "content": "Thanks @hengck23 , this paper seems amazing, I am not sure but feels like something like pre-pre-training with fractal patterns, is a good idea, those patterns may be a good starting point to \"initialize\" the network with patterns that are very basic yet generic.
\nI loved the idea, very creative.
Hi @epocxy , I am not very familiar with many GANs types, but by what I saw it may be a nice option, the issue with CycleGAN is that you have to train many on it.
", "rawMarkdown": "Hi @epocxy , I am not very familiar with many GANs types, but by what I saw it may be a nice option, the issue with CycleGAN is that you have to train many on it." } ] } ], "comments": [ { "id": 1100578, "author_name": "Aziz_Belaweid", "author_url": "", "post_date": "2020-12-03T06:56:15.580000", "content": "According to Iyan Goodfellow (the creator of GANs for those who don't know) generating more training data using GANs will not make the model generalize better since the images generated by GANs will replicate already existing features and not bring new features to the table it could lead to overfitting. I tried using it in a previous competition plant pathology and it didn't work, nonetheless I hope you prove me wrong.
", "votes": 1, "replies": [ { "id": 1100603, "author_name": "Sepideh Nahali", "author_url": "", "post_date": "2020-12-03T07:26:25.647000", "content": "Actually the intuition behind using these generated images is that, in many cases they are better than hand-crafted synthetic examples, and can improve a downstream model’s generalization as well
", "votes": 0, "replies": [] }, { "id": 1100609, "author_name": "", "author_url": "", "post_date": "2020-12-03T07:30:11.410000", "content": "", "votes": 0, "replies": [] }, { "id": 1101141, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-12-03T16:54:45.173000", "content": "Thanks for your input @aziz69 , I have seen people using GAN generated data with good results in some applications, and I also believe that in some cases it might be useful, here the idea is to translate data from one domain to another, it is different from simply generating data from random distribution to one domain.
\nAnyway, If I manage to get good results I can provide the data as TFRecords and it would be very easy to try and evaluate the results.
", "votes": 1, "replies": [] } ] }, { "id": 1098391, "author_name": "GoogleCloudAutoML", "author_url": "", "post_date": "2020-12-01T16:05:53.070000", "content": "Interesting idea, thanks for sharing with us
", "votes": 1, "replies": [] }, { "id": 1096353, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-11-30T12:48:08.950000", "content": "A few generated samples from my notebook Cassava Leaf Disease - CycleGAN data augmentation, I have trained the models for only a few epochs because of the time, but it looks interesting.
\n
\n
\n
\n
Just saw LeafGAN, pointed by Heng here, seems like the ideal solution, not sure yet I can find the code implementation.
\n
Nice finding! If you try CycleGAN, try feeding the output to the input again to see if it works?
", "votes": 1, "replies": [] }, { "id": 1094231, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-11-28T12:33:34.477000", "content": "Hey @aeryss , I think this has a chance of working if you are able to generate images with very high quality, lets see if I can get a good result, but thanks for the tip.
", "votes": 0, "replies": [] }, { "id": 1096912, "author_name": "hengck23", "author_url": "", "post_date": "2020-11-30T21:45:35.053000", "content": "\" I think this has a chance of working if you are able to generate images with very high quality,\"
\nactually it would help even if generate images with low quality. you just need a classifier to classify the quality of the images. for image with low quality, the cross entropy loss is weighted less.
\none can generally treat GAN images as images with noisy labels or unlabelled (but with a new class known as \"none of the target class\")
", "votes": 1, "replies": [] }, { "id": 1097158, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-12-01T01:00:26.387000", "content": "That is a great point @hengck23 , I am a big fan of multi-task, using image quality as an auxiliary target sounds good.
\nI am thinking about doing some experiments on pre-train as unsupervised learning, maybe using GAN data can help.
", "votes": 0, "replies": [] }, { "id": 1097259, "author_name": "hengck23", "author_url": "", "post_date": "2020-12-01T01:54:55.230000", "content": "check this work as well
\nhttps://hirokatsukataoka16.github.io/Pretraining-without-Natural-Images/
\nPre-training without Natural Images
Asian Conference on Computer Vision (ACCV) 2020
\nBest Paper Honorable Mention Award
\nOral Presentation
Image space is Fractal?
", "votes": 2, "replies": [] }, { "id": 1098731, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-12-01T19:53:10.713000", "content": "Thanks @hengck23 , this paper seems amazing, I am not sure but feels like something like pre-pre-training with fractal patterns, is a good idea, those patterns may be a good starting point to \"initialize\" the network with patterns that are very basic yet generic.
\nI loved the idea, very creative.
Hi @epocxy , I am not very familiar with many GANs types, but by what I saw it may be a nice option, the issue with CycleGAN is that you have to train many on it.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1093652": "I was thinking about doing some experimentations to use GANs and generate synthetic data and maybe use it for pre-training, I saw that some people already [mentioned](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/199276#1090329) this [idea](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/199131), but I am curious as to what kind of GAN would work better here, I already have [some experience](https://www.kaggle.com/dimitreoliveira/introduction-to-cyclegan-monet-paintings) with [CycleGAN](https://www.kaggle.com/dimitreoliveira/improving-cyclegan-monet-paintings), but maybe there is a more recent architecture.\n\nUpdate 1: Just found the [GitHub of LeafGAN](https://github.com/IyatomiLab/LeafGAN), unfortunately, they have not provided the code yet.\nUpdate 2: I have just created a notebook [Cassava Leaf Disease - CycleGAN data augmentation](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-cyclegan-data-augmentation/notebook) that implements CycleGAN and generates leaf images of the 4 diseases types from healthy images.", "1100578": "According to Iyan Goodfellow (the creator of GANs for those who don't know) generating more training data using GANs will not make the model generalize better since the images generated by GANs will replicate already existing features and not bring new features to the table it could lead to overfitting. I tried using it in a previous competition plant pathology and it didn't work, nonetheless I hope you prove me wrong. ", "1098391": "Interesting idea, thanks for sharing with us", "1096353": "A few generated samples from my notebook [Cassava Leaf Disease - CycleGAN data augmentation](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-cyclegan-data-augmentation/notebook#Evaluating-generator-models), I have trained the models for only a few epochs because of the time, but it looks interesting.\n\n\n\n\n", "1093657": "Just saw LeafGAN, pointed by Heng [here](https://www.kaggle.com/kmat2019/cycle-gan-to-enlarge-training-data/comments), seems like the ideal solution, not sure yet I can find the code implementation.\n\n", "1096869": "" } }