Could someone give me some advice?\nI tried some augment about mixup, cutmix, cutout and tuned the parameter.\nMy model is se_resnext50 with three fc layers. \nMy best score is CV:0.980 and LB:0.9718.\nI do not know how to do next. \nShould I continue to try different augment or combine with them?\nOr should I tune my model, such as add some Conv layers at the head? \nI don't have a GPU. So I will spend about two days to try an idea.\nIf you give me some advice, I will be very grateful.
", "messages": [ { "id": "765372", "postDate": "03/06/2020 14:41:20", "content": "Could someone give me some advice?\nI tried some augment about mixup, cutmix, cutout and tuned the parameter.\nMy model is se_resnext50 with three fc layers. \nMy best score is CV:0.980 and LB:0.9718.\nI do not know how to do next. \nShould I continue to try different augment or combine with them?\nOr should I tune my model, such as add some Conv layers at the head? \nI don't have a GPU. So I will spend about two days to try an idea.\nIf you give me some advice, I will be very grateful.
", "rawMarkdown": "Could someone give me some advice?\nI tried some augment about mixup, cutmix, cutout and tuned the parameter.\nMy model is se_resnext50 with three fc layers. \nMy best score is CV:0.980 and LB:0.9718.\nI do not know how to do next. \nShould I continue to try different augment or combine with them?\nOr should I tune my model, such as add some Conv layers at the head? \nI don't have a GPU. So I will spend about two days to try an idea.\nIf you give me some advice, I will be very grateful.", "votes": null }, { "id": "765545", "postDate": "03/06/2020 19:27:52", "content": "What is the image input resolution that you use?
\n\nHave you tried putting GeM at the head? See, discussion here.
\n\nI submitted a single model that only uses cutmix with constant intensity as augmentation gets LB 0.9841\nConstant intensity = cutmix every picture. However, since the box is random, often, it actually does not cut out anything.
\n\nWhen, I train an efficient net B3 by a 2080 Ti, it takes about 36 hours/140 epochs to see the validation score stop increasing. I am pretty new to image recognition though and I didn't use many tricks to accelerate this process. I don't think I can get your score using CPU :)
\n\nGood luck!
", "rawMarkdown": "What is the image input resolution that you use?\n\nHave you tried putting GeM at the head? See, [discussion here](https://www.kaggle.com/c/bengaliai-cv19/discussion/123432#708872). \n\nI submitted a single model that only uses cutmix with constant intensity as augmentation gets LB 0.9841\nConstant intensity = cutmix every picture. However, since the box is random, often, it actually does not cut out anything.\n\nWhen, I train an efficient net B3 by a 2080 Ti, it takes about 36 hours/140 epochs to see the validation score stop increasing. I am pretty new to image recognition though and I didn't use many tricks to accelerate this process. I don't think I can get your score using CPU :)\n\nGood luck!", "votes": null }, { "id": "765677", "postDate": "03/07/2020 00:26:16", "content": "Thanks for your reply.\nThe image size is 128 by 128\n I will make a try. And I use colab, not CPU. it takes 30 minutes for one epoch.
", "rawMarkdown": "Thanks for your reply.\nThe image size is 128 by 128\n I will make a try. And I use colab, not CPU. it takes 30 minutes for one epoch.", "votes": null }, { "id": "766424", "postDate": "03/08/2020 05:42:21", "content": "Can you share your cutmix version please? What's mix loss did you use and how you cut images (between one batches, between 2 samples, etc...)? @barnwellguy
", "rawMarkdown": "Can you share your cutmix version please? What's mix loss did you use and how you cut images (between one batches, between 2 samples, etc...)? @barnwellguy", "votes": null }, { "id": "766640", "postDate": "03/08/2020 14:02:42", "content": "NIce
", "rawMarkdown": "NIce", "votes": null }, { "id": "766678", "postDate": "03/08/2020 14:50:48", "content": "Hi, @mnikita , I wrote a simple cutmix by myself.\n- For example, if my batch size is 5, then, I shuffle 1,2,3,4,5 so that every picture has a cutmix target within the batch, e.g, 1 <- 3, 2 <- 1, 3 <- 2, 4 <- 4, 5 <- 5. Next, for every picture, I pick cutmix box height and width fractions from two [0,1] uniform distributions after which I pick their left upper corner from another two uniform distribution so that the whole box is within the limit of the picture. Pixels in the box are replaced with that from the target picture.\n- the resulting label is the average label with the weight to be proportional to the area.
", "rawMarkdown": "Hi, @mnikita , I wrote a simple cutmix by myself.\n- For example, if my batch size is 5, then, I shuffle 1,2,3,4,5 so that every picture has a cutmix target within the batch, e.g, 1 <- 3, 2 <- 1, 3 <- 2, 4 <- 4, 5 <- 5. Next, for every picture, I pick cutmix box height and width fractions from two [0,1] uniform distributions after which I pick their left upper corner from another two uniform distribution so that the whole box is within the limit of the picture. Pixels in the box are replaced with that from the target picture.\n- the resulting label is the average label with the weight to be proportional to the area.", "votes": null }, { "id": "766772", "postDate": "03/08/2020 17:08:20", "content": "For GPU I highly recommend using Colab Pro .. its only 10$ a month, they say only available in US but works in many places regardless. If you don't want to commit to Colab Pro just try out Colab first its free.
\n\nSee the very helpful discussion here https://www.kaggle.com/c/bengaliai-cv19/discussion/133857
", "rawMarkdown": "For GPU I highly recommend using Colab Pro .. its only 10$ a month, they say only available in US but works in many places regardless. If you don't want to commit to Colab Pro just try out Colab first its free. \n\nSee the very helpful discussion here https://www.kaggle.com/c/bengaliai-cv19/discussion/133857", "votes": null }, { "id": "766797", "postDate": "03/08/2020 18:25:29", "content": "@dipamc77 Is it available in India? if yes, how to get it
", "rawMarkdown": "dipamc77 Is it available in India? if yes, how to get it", "votes": null }, { "id": "766830", "postDate": "03/08/2020 19:49:27", "content": "As far as I know, it is available everywhere if you provide an US address. Whether that is legal or not, I can't really say.
", "rawMarkdown": "As far as I know, it is available everywhere if you provide an US address. Whether that is legal or not, I can't really say.", "votes": null }, { "id": "766936", "postDate": "03/09/2020 00:33:40", "content": "Yes, I'm using it from India, just enter some US pincode when asked. Search for Colab Pro on google and use the option to pay. One other thing, would suggest using an account where you have more space available on google drive. (Create a new one if needed)
", "rawMarkdown": "Yes, I'm using it from India, just enter some US pincode when asked. Search for Colab Pro on google and use the option to pay. One other thing, would suggest using an account where you have more space available on google drive. (Create a new one if needed)", "votes": null }, { "id": "766958", "postDate": "03/09/2020 01:29:02", "content": "Hi @barnwellguy , \nYou did not enforce your cutmix box cutting part of character?
", "rawMarkdown": "Hi @barnwellguy , \nYou did not enforce your cutmix box cutting part of character?", "votes": null }, { "id": "767286", "postDate": "03/09/2020 12:42:58", "content": "Hi, @kasim0226 , thanks for asking. But I didn't quite get it. 😳 \nI just skimmed through the cutmix paper again and might have missed something.\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1557202%2F25219301d2ebc8653879155feb57c199%2FScreenshot%20from%202020-03-09%2007-37-30.png?generation=1583757595193009&alt=media\")
\nMy difference from this original paper is, the cutting box width-height ratio is not proportional to the original picture, but two random number is drawn to determined its shape. So, it has a larger variety of shapes. It is not intentional, but I feel perhaps it is not a bad idea too :) Is this what you are referring to?
Hi @barnwellguy ,\nFrom your reply, I think I get what I want to know.\nThank you
", "rawMarkdown": "Hi @barnwellguy ,\nFrom your reply, I think I get what I want to know.\nThank you", "votes": null }, { "id": "767752", "postDate": "03/10/2020 03:59:06", "content": "I suggest using google colab if you don't have your gpus. Also, try small image size such as 64x64 for experiments to validate your ideas. Then using larger image size for lb score improvement.
", "rawMarkdown": "I suggest using google colab if you don't have your gpus. Also, try small image size such as 64x64 for experiments to validate your ideas. Then using larger image size for lb score improvement.", "votes": null } ], "comments": [ { "id": 765545, "author_name": "barnwellguy", "author_url": "", "post_date": "03/06/2020 19:27:52", "content": "What is the image input resolution that you use?
\n\nHave you tried putting GeM at the head? See, discussion here.
\n\nI submitted a single model that only uses cutmix with constant intensity as augmentation gets LB 0.9841\nConstant intensity = cutmix every picture. However, since the box is random, often, it actually does not cut out anything.
\n\nWhen, I train an efficient net B3 by a 2080 Ti, it takes about 36 hours/140 epochs to see the validation score stop increasing. I am pretty new to image recognition though and I didn't use many tricks to accelerate this process. I don't think I can get your score using CPU :)
\n\nGood luck!
", "votes": null, "replies": [ { "id": 765677, "author_name": "luzerochen", "author_url": "", "post_date": "03/07/2020 00:26:16", "content": "Thanks for your reply.\nThe image size is 128 by 128\n I will make a try. And I use colab, not CPU. it takes 30 minutes for one epoch.
", "votes": null, "replies": [] }, { "id": 766424, "author_name": "mnikita", "author_url": "", "post_date": "03/08/2020 05:42:21", "content": "Can you share your cutmix version please? What's mix loss did you use and how you cut images (between one batches, between 2 samples, etc...)? @barnwellguy
", "votes": null, "replies": [] }, { "id": 766678, "author_name": "barnwellguy", "author_url": "", "post_date": "03/08/2020 14:50:48", "content": "Hi, @mnikita , I wrote a simple cutmix by myself.\n- For example, if my batch size is 5, then, I shuffle 1,2,3,4,5 so that every picture has a cutmix target within the batch, e.g, 1 <- 3, 2 <- 1, 3 <- 2, 4 <- 4, 5 <- 5. Next, for every picture, I pick cutmix box height and width fractions from two [0,1] uniform distributions after which I pick their left upper corner from another two uniform distribution so that the whole box is within the limit of the picture. Pixels in the box are replaced with that from the target picture.\n- the resulting label is the average label with the weight to be proportional to the area.
", "votes": null, "replies": [] }, { "id": 766958, "author_name": "kasim0226", "author_url": "", "post_date": "03/09/2020 01:29:02", "content": "Hi @barnwellguy , \nYou did not enforce your cutmix box cutting part of character?
", "votes": null, "replies": [] }, { "id": 767286, "author_name": "barnwellguy", "author_url": "", "post_date": "03/09/2020 12:42:58", "content": "Hi, @kasim0226 , thanks for asking. But I didn't quite get it. 😳 \nI just skimmed through the cutmix paper again and might have missed something.\n\nMy difference from this original paper is, the cutting box width-height ratio is not proportional to the original picture, but two random number is drawn to determined its shape. So, it has a larger variety of shapes. It is not intentional, but I feel perhaps it is not a bad idea too :) Is this what you are referring to?
Hi @barnwellguy ,\nFrom your reply, I think I get what I want to know.\nThank you
", "votes": null, "replies": [] } ] }, { "id": 766640, "author_name": "dhanrajkumar", "author_url": "", "post_date": "03/08/2020 14:02:42", "content": "NIce
", "votes": null, "replies": [] }, { "id": 766772, "author_name": "dipamc77", "author_url": "", "post_date": "03/08/2020 17:08:20", "content": "For GPU I highly recommend using Colab Pro .. its only 10$ a month, they say only available in US but works in many places regardless. If you don't want to commit to Colab Pro just try out Colab first its free.
\n\nSee the very helpful discussion here https://www.kaggle.com/c/bengaliai-cv19/discussion/133857
", "votes": null, "replies": [ { "id": 766797, "author_name": "hermione36", "author_url": "", "post_date": "03/08/2020 18:25:29", "content": "@dipamc77 Is it available in India? if yes, how to get it
", "votes": null, "replies": [] }, { "id": 766830, "author_name": "utsavnandi", "author_url": "", "post_date": "03/08/2020 19:49:27", "content": "As far as I know, it is available everywhere if you provide an US address. Whether that is legal or not, I can't really say.
", "votes": null, "replies": [] }, { "id": 766936, "author_name": "dipamc77", "author_url": "", "post_date": "03/09/2020 00:33:40", "content": "Yes, I'm using it from India, just enter some US pincode when asked. Search for Colab Pro on google and use the option to pay. One other thing, would suggest using an account where you have more space available on google drive. (Create a new one if needed)
", "votes": null, "replies": [] } ] }, { "id": 767752, "author_name": "syoya1997", "author_url": "", "post_date": "03/10/2020 03:59:06", "content": "I suggest using google colab if you don't have your gpus. Also, try small image size such as 64x64 for experiments to validate your ideas. Then using larger image size for lb score improvement.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "765372": "Could someone give me some advice?\nI tried some augment about mixup, cutmix, cutout and tuned the parameter.\nMy model is se_resnext50 with three fc layers. \nMy best score is CV:0.980 and LB:0.9718.\nI do not know how to do next. \nShould I continue to try different augment or combine with them?\nOr should I tune my model, such as add some Conv layers at the head? \nI don't have a GPU. So I will spend about two days to try an idea.\nIf you give me some advice, I will be very grateful.", "765545": "What is the image input resolution that you use?\n\nHave you tried putting GeM at the head? See, [discussion here](https://www.kaggle.com/c/bengaliai-cv19/discussion/123432#708872). \n\nI submitted a single model that only uses cutmix with constant intensity as augmentation gets LB 0.9841\nConstant intensity = cutmix every picture. However, since the box is random, often, it actually does not cut out anything.\n\nWhen, I train an efficient net B3 by a 2080 Ti, it takes about 36 hours/140 epochs to see the validation score stop increasing. I am pretty new to image recognition though and I didn't use many tricks to accelerate this process. I don't think I can get your score using CPU :)\n\nGood luck!", "765677": "Thanks for your reply.\nThe image size is 128 by 128\n I will make a try. And I use colab, not CPU. it takes 30 minutes for one epoch.", "766424": "Can you share your cutmix version please? What's mix loss did you use and how you cut images (between one batches, between 2 samples, etc...)? @barnwellguy", "766640": "NIce", "766678": "Hi, @mnikita , I wrote a simple cutmix by myself.\n- For example, if my batch size is 5, then, I shuffle 1,2,3,4,5 so that every picture has a cutmix target within the batch, e.g, 1 <- 3, 2 <- 1, 3 <- 2, 4 <- 4, 5 <- 5. Next, for every picture, I pick cutmix box height and width fractions from two [0,1] uniform distributions after which I pick their left upper corner from another two uniform distribution so that the whole box is within the limit of the picture. Pixels in the box are replaced with that from the target picture.\n- the resulting label is the average label with the weight to be proportional to the area.", "766772": "For GPU I highly recommend using Colab Pro .. its only 10$ a month, they say only available in US but works in many places regardless. If you don't want to commit to Colab Pro just try out Colab first its free. \n\nSee the very helpful discussion here https://www.kaggle.com/c/bengaliai-cv19/discussion/133857", "766797": "dipamc77 Is it available in India? if yes, how to get it", "766830": "As far as I know, it is available everywhere if you provide an US address. Whether that is legal or not, I can't really say.", "766936": "Yes, I'm using it from India, just enter some US pincode when asked. Search for Colab Pro on google and use the option to pay. One other thing, would suggest using an account where you have more space available on google drive. (Create a new one if needed)", "766958": "Hi @barnwellguy , \nYou did not enforce your cutmix box cutting part of character?", "767286": "Hi, @kasim0226 , thanks for asking. But I didn't quite get it. 😳 \nI just skimmed through the cutmix paper again and might have missed something.\n\nMy difference from this original paper is, the cutting box width-height ratio is not proportional to the original picture, but two random number is drawn to determined its shape. So, it has a larger variety of shapes. It is not intentional, but I feel perhaps it is not a bad idea too :) Is this what you are referring to?", "767438": "Hi @barnwellguy ,\nFrom your reply, I think I get what I want to know.\nThank you", "767752": "I suggest using google colab if you don't have your gpus. Also, try small image size such as 64x64 for experiments to validate your ideas. Then using larger image size for lb score improvement." }, "source": "meta" }