Takes less time to train. So I am able to get more time to experiment
\n\nDensenet 121
\n\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9745\nLB: 0.9672\n5 Fold {New Split}\n30 epochs
\n\nDensenet with Mixup\nSize :137 x 236\nCV: 0.9750\nLB: 0.9665\n5 Fold {New Split}\n30 epochs
\n\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9721\nLB: 0.9655\n5 Fold { Old split}\n30 epochs
\n\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9764\nLB: 0.9683\n5 Fold { New split}\n30 epochs\nReduced the amount of dropout. Too much dropout and too high augmentation, prevents the model from learning.
\n\nDensenet with Cutmix + Differential Learning {New technique I came up with }\nSize :137 x 236\nCV: 0.9804 #CV:0.9783 for 32 epochs\nLB: 0.9725\n5 Fold { New split}\n100 epochs
\n\nAt last, I got a good baseline to work with. Now, I need to march fast towards the top as much as possible.
\n\nLooking for team-mates who are using Densenet
\n\nI am searching for a good baseline. As of now, I have done mostly architectural changes only. I haven't seen anyone discuss anything similar to the differential learning which I have formulated and it is boosting the score a lot.
\n\nI am looking for team-mates who are interested in long term friendship and to learn together.
\n\nKindly share your Densenet ideas here, so everyone making use of Densenet can get help out of this.\nAll the best
", "messages": [ { "id": "758729", "postDate": "02/28/2020 04:56:18", "content": "Takes less time to train. So I am able to get more time to experiment
\n\nDensenet 121
\n\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9745\nLB: 0.9672\n5 Fold {New Split}\n30 epochs
\n\nDensenet with Mixup\nSize :137 x 236\nCV: 0.9750\nLB: 0.9665\n5 Fold {New Split}\n30 epochs
\n\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9721\nLB: 0.9655\n5 Fold { Old split}\n30 epochs
\n\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9764\nLB: 0.9683\n5 Fold { New split}\n30 epochs\nReduced the amount of dropout. Too much dropout and too high augmentation, prevents the model from learning.
\n\nDensenet with Cutmix + Differential Learning {New technique I came up with }\nSize :137 x 236\nCV: 0.9804 #CV:0.9783 for 32 epochs\nLB: 0.9725\n5 Fold { New split}\n100 epochs
\n\nAt last, I got a good baseline to work with. Now, I need to march fast towards the top as much as possible.
\n\nLooking for team-mates who are using Densenet
\n\nI am searching for a good baseline. As of now, I have done mostly architectural changes only. I haven't seen anyone discuss anything similar to the differential learning which I have formulated and it is boosting the score a lot.
\n\nI am looking for team-mates who are interested in long term friendship and to learn together.
\n\nKindly share your Densenet ideas here, so everyone making use of Densenet can get help out of this.\nAll the best
", "rawMarkdown": "Takes less time to train. So I am able to get more time to experiment\n\nDensenet 121\n\nTrial - 1:\n-------------------------------------------------------------------------------\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9745\nLB: 0.9672\n5 Fold {New Split}\n30 epochs\n\nTrial - 2:\n-------------------------------------------------------------------------------\nDensenet with Mixup\nSize :137 x 236\nCV: 0.9750\nLB: 0.9665\n5 Fold {New Split}\n30 epochs\n\nTrial - 3:\n-------------------------------------------------------------------------------\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9721\nLB: 0.9655\n5 Fold { Old split}\n30 epochs\n\nTrial - 4:\n-------------------------------------------------------------------------------\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9764\nLB: 0.9683\n5 Fold { New split}\n30 epochs\nReduced the amount of dropout. Too much dropout and too high augmentation, prevents the model from learning.\n\nTrial - 5:\n-------------------------------------------------------------------------------\nDensenet with Cutmix + Differential Learning {New technique I came up with }\nSize :137 x 236\nCV: 0.9804 #CV:0.9783 for 32 epochs\nLB: 0.9725\n5 Fold { New split}\n100 epochs\n\n\nKey Points:\n-------------------------------------------------------------------------------\n1. Fold matters a lot. 0.9655 to 0.9672. Nearly boost of 0.0017\n2. Reduced the amount of dropout. Too much dropout and too high augmentation prevent the model from learning.\n3. Differential Learning approach boosted performance a lot. CV boosted from 0.9764 to 0.9783. Nearly boost of 0.0019 in 32 epochs.\n4. Any comparison between Densenet 121 and other models for this challenge ??\n\nAt last, I got a good baseline to work with. Now, I need to march fast towards the top as much as possible.\n\n************************************************\nLooking for team-mates who are using Densenet\n\nI am searching for a good baseline. As of now, I have done mostly architectural changes only. I haven't seen anyone discuss anything similar to the differential learning which I have formulated and it is boosting the score a lot.\n\nI am looking for team-mates who are interested in long term friendship and to learn together.\n\n************************************************\n\nKindly share your Densenet ideas here, so everyone making use of Densenet can get help out of this.\nAll the best", "votes": null }, { "id": "758940", "postDate": "02/28/2020 11:20:01", "content": "agree dropout didn't work for me.
", "rawMarkdown": "agree dropout didn't work for me.", "votes": null }, { "id": "758943", "postDate": "02/28/2020 11:25:52", "content": "Great. I tried DenseNet169 with the following set up and got a poor LB score: 0.9584 😣
\n\n\nsplit: randomly 8:2\nimg: 128 x 128 (iafaos preposess image)\nopt: Adam\naug: gridmas + augmix\nepoch: ~80\n\nThe things that bug me is the preprocess image files of @iafoss, is it one of the reasons for the low score according to some other's experiment. Next, the splitting strategy! Next to the augmentation themself.
I can`t beat even 0.965 LB with densenet(
", "rawMarkdown": "I can`t beat even 0.965 LB with densenet(", "votes": null }, { "id": "761033", "postDate": "03/02/2020 04:01:59", "content": "For me densenet with cutmix isnt really working compared to a resnet with cutmix
", "rawMarkdown": "For me densenet with cutmix isnt really working compared to a resnet with cutmix", "votes": null }, { "id": "761151", "postDate": "03/02/2020 07:36:27", "content": "@kupchanski Better make use of @iafoss pipeline for densenet. He has given methods to get around 0.9670 i guess.. Check out all his kernels.
\n\nOnce done. Add my suggestions and see, whether it boosts the score. All the best
", "rawMarkdown": "kupchanski Better make use of @iafoss pipeline for densenet. He has given methods to get around 0.9670 i guess.. Check out all his kernels.\n\nOnce done. Add my suggestions and see, whether it boosts the score. All the best", "votes": null }, { "id": "761152", "postDate": "03/02/2020 07:37:12", "content": "Yup. Having too much augmentation and too much dropout may stop your model from learning. Its a big stopping point.
", "rawMarkdown": "Yup. Having too much augmentation and too much dropout may stop your model from learning. Its a big stopping point.", "votes": null }, { "id": "761409", "postDate": "03/02/2020 13:34:59", "content": "I previously used @iafoss re-process method. Original images gave a better score than his pre-processing image for my models.
", "rawMarkdown": "I previously used @iafoss re-process method. Original images gave a better score than his pre-processing image for my models.", "votes": null }, { "id": "761436", "postDate": "03/02/2020 14:16:34", "content": "@yannmajewski \nCould you share your technique that boosted your score for the current position ?
", "rawMarkdown": "yannmajewski \nCould you share your technique that boosted your score for the current position ?", "votes": null }, { "id": "761540", "postDate": "03/02/2020 16:33:19", "content": "Right now im just testing basic augmentations like cutout, shifscalerotate, distortion, and perspective. Also, having a tail like, BN/Dropout/Linear/ReLu/BN/Dropout/Linear helped to get a better score!\nThese two things got me 97.6 lb. I think i might have to remove dropout layers if i want to use cutmix, will update you on that once I test it :)
", "rawMarkdown": "Right now im just testing basic augmentations like cutout, shifscalerotate, distortion, and perspective. Also, having a tail like, BN/Dropout/Linear/ReLu/BN/Dropout/Linear helped to get a better score!\nThese two things got me 97.6 lb. I think i might have to remove dropout layers if i want to use cutmix, will update you on that once I test it :)", "votes": null }, { "id": "761982", "postDate": "03/03/2020 05:16:29", "content": "If you are interested we can form a team. As the time is less, we can experiment more as a team.
", "rawMarkdown": "If you are interested we can form a team. As the time is less, we can experiment more as a team.", "votes": null }, { "id": "765105", "postDate": "03/06/2020 09:08:07", "content": "Hi @dhakshiin1601,
\n\nI've tried with DenseNet169:\n\nimg: 128 x 128\nAug: Cutout + RandomRotate\nEpochs: 80\nCV: 0.9751\nLB: 0.9702\n\nAnd, I agree with you on too much dropout.
Your Differential Learning is pretty amazing. I think I should study on that asap 🙌
I will share the details of it in this thread, at the end of the competition. Its working very good with Densenet. Was very hard to setup with seresnext-50, as the training time was quite high. All the best @moximo13
", "rawMarkdown": "I will share the details of it in this thread, at the end of the competition. Its working very good with Densenet. Was very hard to setup with seresnext-50, as the training time was quite high. All the best @moximo13", "votes": null }, { "id": "765469", "postDate": "03/06/2020 16:54:37", "content": "\n\n\nI haven't seen anyone discuss anything similar to the differential learning which I have formulated and it is boosting the score a lot.
\n
Probably because no one has a clue what you're talking about ... 😁
", "rawMarkdown": "> I haven't seen anyone discuss anything similar to the differential learning which I have formulated and it is boosting the score a lot.\n\nProbably because no one has a clue what you're talking about ... **😁**", "votes": null }, { "id": "765696", "postDate": "03/07/2020 01:14:23", "content": "My current LB of 0.9819 is a Densenet121 using only Cutout and Dropblock! I suggest you try it
", "rawMarkdown": "My current LB of 0.9819 is a Densenet121 using only Cutout and Dropblock! I suggest you try it", "votes": null }, { "id": "765699", "postDate": "03/07/2020 01:27:33", "content": "wow, great. Would you share your Cutout setup? Here is what I have tried though.
\n\n```\ndef cutout(org_img, magnitude=None):\n magnitudes = np.linspace(0, 60/331, 11)
\n\nimg = np.copy(org_img)\nmask_val = img.mean()\n\nif magnitude is None:\n mask_size = 16\nelse:\n mask_size = int(round(img.shape[0]*random.uniform(magnitudes[magnitude], magnitudes[magnitude+1])))\ntop = np.random.randint(0 - mask_size//2, img.shape[0] - mask_size)\nleft = np.random.randint(0 - mask_size//2, img.shape[1] - mask_size)\nbottom = top + mask_size\nright = left + mask_size\nif top < 0:\n top = 0\nif left < 0:\n left = 0\n\nimg[top:bottom, left:right, :].fill(mask_val)\nreturn img\n```
", "rawMarkdown": "wow, great. Would you share your Cutout setup? Here is what I have tried though. \n\n```\ndef cutout(org_img, magnitude=None):\n magnitudes = np.linspace(0, 60/331, 11)\n\n img = np.copy(org_img)\n mask_val = img.mean()\n\n if magnitude is None:\n mask_size = 16\n else:\n mask_size = int(round(img.shape[0]*random.uniform(magnitudes[magnitude], magnitudes[magnitude+1])))\n top = np.random.randint(0 - mask_size//2, img.shape[0] - mask_size)\n left = np.random.randint(0 - mask_size//2, img.shape[1] - mask_size)\n bottom = top + mask_size\n right = left + mask_size\n if top < 0:\n top = 0\n if left < 0:\n left = 0\n\n img[top:bottom, left:right, :].fill(mask_val)\n return img\n```", "votes": null }, { "id": "765721", "postDate": "03/07/2020 02:00:57", "content": "I cutout 2 holes of about 1/3 of my image size as max hole size!
", "rawMarkdown": "I cutout 2 holes of about 1/3 of my image size as max hole size!", "votes": null }, { "id": "765723", "postDate": "03/07/2020 02:02:14", "content": "ok, that would help. Thank you.
", "rawMarkdown": "ok, that would help. Thank you.", "votes": null }, { "id": "765729", "postDate": "03/07/2020 02:09:23", "content": "Very instresting, i wonder how much of a boost did Differential Learning give you?
", "rawMarkdown": "Very instresting, i wonder how much of a boost did Differential Learning give you?", "votes": null }, { "id": "765740", "postDate": "03/07/2020 02:29:40", "content": "Interesting for me a simple head worked best but maybe I should go back and experiment with it...
", "rawMarkdown": "Interesting for me a simple head worked best but maybe I should go back and experiment with it...", "votes": null }, { "id": "765746", "postDate": "03/07/2020 02:51:29", "content": "I actually changed a lot of my architecture so last comment doesnt represent my current score, I now have a simple tail! So dont waste more time on those experiments :p
", "rawMarkdown": "I actually changed a lot of my architecture so last comment doesnt represent my current score, I now have a simple tail! So dont waste more time on those experiments :p", "votes": null }, { "id": "765748", "postDate": "03/07/2020 02:53:13", "content": "Oups nevermind, just saw that you said that in your post haha!
", "rawMarkdown": "Oups nevermind, just saw that you said that in your post haha!", "votes": null } ], "comments": [ { "id": 758940, "author_name": "yl1202", "author_url": "", "post_date": "02/28/2020 11:20:01", "content": "agree dropout didn't work for me.
", "votes": null, "replies": [ { "id": 761152, "author_name": "dhakshiin1601", "author_url": "", "post_date": "03/02/2020 07:37:12", "content": "Yup. Having too much augmentation and too much dropout may stop your model from learning. Its a big stopping point.
", "votes": null, "replies": [] } ] }, { "id": 758943, "author_name": "ipythonx", "author_url": "", "post_date": "02/28/2020 11:25:52", "content": "Great. I tried DenseNet169 with the following set up and got a poor LB score: 0.9584 😣
\n\n\nsplit: randomly 8:2\nimg: 128 x 128 (iafaos preposess image)\nopt: Adam\naug: gridmas + augmix\nepoch: ~80\n\nThe things that bug me is the preprocess image files of @iafoss, is it one of the reasons for the low score according to some other's experiment. Next, the splitting strategy! Next to the augmentation themself.
I previously used @iafoss re-process method. Original images gave a better score than his pre-processing image for my models.
", "votes": null, "replies": [] } ] }, { "id": 759148, "author_name": "kupchanski", "author_url": "", "post_date": "02/28/2020 16:37:39", "content": "I can`t beat even 0.965 LB with densenet(
", "votes": null, "replies": [ { "id": 761151, "author_name": "dhakshiin1601", "author_url": "", "post_date": "03/02/2020 07:36:27", "content": "@kupchanski Better make use of @iafoss pipeline for densenet. He has given methods to get around 0.9670 i guess.. Check out all his kernels.
\n\nOnce done. Add my suggestions and see, whether it boosts the score. All the best
", "votes": null, "replies": [] } ] }, { "id": 761033, "author_name": "yannmajewski", "author_url": "", "post_date": "03/02/2020 04:01:59", "content": "For me densenet with cutmix isnt really working compared to a resnet with cutmix
", "votes": null, "replies": [ { "id": 761436, "author_name": "dhakshiin1601", "author_url": "", "post_date": "03/02/2020 14:16:34", "content": "@yannmajewski \nCould you share your technique that boosted your score for the current position ?
", "votes": null, "replies": [] }, { "id": 761540, "author_name": "yannmajewski", "author_url": "", "post_date": "03/02/2020 16:33:19", "content": "Right now im just testing basic augmentations like cutout, shifscalerotate, distortion, and perspective. Also, having a tail like, BN/Dropout/Linear/ReLu/BN/Dropout/Linear helped to get a better score!\nThese two things got me 97.6 lb. I think i might have to remove dropout layers if i want to use cutmix, will update you on that once I test it :)
", "votes": null, "replies": [] }, { "id": 761982, "author_name": "dhakshiin1601", "author_url": "", "post_date": "03/03/2020 05:16:29", "content": "If you are interested we can form a team. As the time is less, we can experiment more as a team.
", "votes": null, "replies": [] }, { "id": 765740, "author_name": "greatgamedota", "author_url": "", "post_date": "03/07/2020 02:29:40", "content": "Interesting for me a simple head worked best but maybe I should go back and experiment with it...
", "votes": null, "replies": [] }, { "id": 765746, "author_name": "yannmajewski", "author_url": "", "post_date": "03/07/2020 02:51:29", "content": "I actually changed a lot of my architecture so last comment doesnt represent my current score, I now have a simple tail! So dont waste more time on those experiments :p
", "votes": null, "replies": [] } ] }, { "id": 765105, "author_name": "moximo13", "author_url": "", "post_date": "03/06/2020 09:08:07", "content": "Hi @dhakshiin1601,
\n\nI've tried with DenseNet169:\n\nimg: 128 x 128\nAug: Cutout + RandomRotate\nEpochs: 80\nCV: 0.9751\nLB: 0.9702\n\nAnd, I agree with you on too much dropout.
Your Differential Learning is pretty amazing. I think I should study on that asap 🙌
I will share the details of it in this thread, at the end of the competition. Its working very good with Densenet. Was very hard to setup with seresnext-50, as the training time was quite high. All the best @moximo13
", "votes": null, "replies": [] }, { "id": 765729, "author_name": "yannmajewski", "author_url": "", "post_date": "03/07/2020 02:09:23", "content": "Very instresting, i wonder how much of a boost did Differential Learning give you?
", "votes": null, "replies": [] }, { "id": 765748, "author_name": "yannmajewski", "author_url": "", "post_date": "03/07/2020 02:53:13", "content": "Oups nevermind, just saw that you said that in your post haha!
", "votes": null, "replies": [] } ] }, { "id": 765469, "author_name": "ljschuster", "author_url": "", "post_date": "03/06/2020 16:54:37", "content": "\n\n\nI haven't seen anyone discuss anything similar to the differential learning which I have formulated and it is boosting the score a lot.
\n
Probably because no one has a clue what you're talking about ... 😁
", "votes": null, "replies": [] }, { "id": 765696, "author_name": "yannmajewski", "author_url": "", "post_date": "03/07/2020 01:14:23", "content": "My current LB of 0.9819 is a Densenet121 using only Cutout and Dropblock! I suggest you try it
", "votes": null, "replies": [ { "id": 765699, "author_name": "ipythonx", "author_url": "", "post_date": "03/07/2020 01:27:33", "content": "wow, great. Would you share your Cutout setup? Here is what I have tried though.
\n\n```\ndef cutout(org_img, magnitude=None):\n magnitudes = np.linspace(0, 60/331, 11)
\n\nimg = np.copy(org_img)\nmask_val = img.mean()\n\nif magnitude is None:\n mask_size = 16\nelse:\n mask_size = int(round(img.shape[0]*random.uniform(magnitudes[magnitude], magnitudes[magnitude+1])))\ntop = np.random.randint(0 - mask_size//2, img.shape[0] - mask_size)\nleft = np.random.randint(0 - mask_size//2, img.shape[1] - mask_size)\nbottom = top + mask_size\nright = left + mask_size\nif top < 0:\n top = 0\nif left < 0:\n left = 0\n\nimg[top:bottom, left:right, :].fill(mask_val)\nreturn img\n```
", "votes": null, "replies": [] }, { "id": 765721, "author_name": "yannmajewski", "author_url": "", "post_date": "03/07/2020 02:00:57", "content": "I cutout 2 holes of about 1/3 of my image size as max hole size!
", "votes": null, "replies": [] }, { "id": 765723, "author_name": "ipythonx", "author_url": "", "post_date": "03/07/2020 02:02:14", "content": "ok, that would help. Thank you.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "758729": "Takes less time to train. So I am able to get more time to experiment\n\nDensenet 121\n\nTrial - 1:\n-------------------------------------------------------------------------------\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9745\nLB: 0.9672\n5 Fold {New Split}\n30 epochs\n\nTrial - 2:\n-------------------------------------------------------------------------------\nDensenet with Mixup\nSize :137 x 236\nCV: 0.9750\nLB: 0.9665\n5 Fold {New Split}\n30 epochs\n\nTrial - 3:\n-------------------------------------------------------------------------------\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9721\nLB: 0.9655\n5 Fold { Old split}\n30 epochs\n\nTrial - 4:\n-------------------------------------------------------------------------------\nDensenet with Cutmix\nSize :137 x 236\nCV: 0.9764\nLB: 0.9683\n5 Fold { New split}\n30 epochs\nReduced the amount of dropout. Too much dropout and too high augmentation, prevents the model from learning.\n\nTrial - 5:\n-------------------------------------------------------------------------------\nDensenet with Cutmix + Differential Learning {New technique I came up with }\nSize :137 x 236\nCV: 0.9804 #CV:0.9783 for 32 epochs\nLB: 0.9725\n5 Fold { New split}\n100 epochs\n\n\nKey Points:\n-------------------------------------------------------------------------------\n1. Fold matters a lot. 0.9655 to 0.9672. Nearly boost of 0.0017\n2. Reduced the amount of dropout. Too much dropout and too high augmentation prevent the model from learning.\n3. Differential Learning approach boosted performance a lot. CV boosted from 0.9764 to 0.9783. Nearly boost of 0.0019 in 32 epochs.\n4. Any comparison between Densenet 121 and other models for this challenge ??\n\nAt last, I got a good baseline to work with. Now, I need to march fast towards the top as much as possible.\n\n************************************************\nLooking for team-mates who are using Densenet\n\nI am searching for a good baseline. As of now, I have done mostly architectural changes only. I haven't seen anyone discuss anything similar to the differential learning which I have formulated and it is boosting the score a lot.\n\nI am looking for team-mates who are interested in long term friendship and to learn together.\n\n************************************************\n\nKindly share your Densenet ideas here, so everyone making use of Densenet can get help out of this.\nAll the best", "758940": "agree dropout didn't work for me.", "758943": "Great. I tried DenseNet169 with the following set up and got a poor LB score: 0.9584 😣 \n\n```\nsplit: randomly 8:2\nimg: 128 x 128 (iafaos preposess image)\nopt: Adam\naug: gridmas + augmix\nepoch: ~80\n```\nThe things that bug me is the preprocess image files of @iafoss, is it one of the reasons for the low score according to some other's experiment. Next, the splitting strategy! Next to the augmentation themself.", "759148": "I can`t beat even 0.965 LB with densenet(", "761033": "For me densenet with cutmix isnt really working compared to a resnet with cutmix", "761151": "kupchanski Better make use of @iafoss pipeline for densenet. He has given methods to get around 0.9670 i guess.. Check out all his kernels.\n\nOnce done. Add my suggestions and see, whether it boosts the score. All the best", "761152": "Yup. Having too much augmentation and too much dropout may stop your model from learning. Its a big stopping point.", "761409": "I previously used @iafoss re-process method. Original images gave a better score than his pre-processing image for my models.", "761436": "yannmajewski \nCould you share your technique that boosted your score for the current position ?", "761540": "Right now im just testing basic augmentations like cutout, shifscalerotate, distortion, and perspective. Also, having a tail like, BN/Dropout/Linear/ReLu/BN/Dropout/Linear helped to get a better score!\nThese two things got me 97.6 lb. I think i might have to remove dropout layers if i want to use cutmix, will update you on that once I test it :)", "761982": "If you are interested we can form a team. As the time is less, we can experiment more as a team.", "765105": "Hi @dhakshiin1601,\n\nI've tried with DenseNet169:\n```\nimg: 128 x 128\nAug: Cutout + RandomRotate\nEpochs: 80\nCV: 0.9751\nLB: 0.9702\n```\nAnd, I agree with you on too much dropout.\n\nYour `Differential Learning` is pretty amazing. I think I should study on that asap 🙌", "765176": "I will share the details of it in this thread, at the end of the competition. Its working very good with Densenet. Was very hard to setup with seresnext-50, as the training time was quite high. All the best @moximo13", "765469": "> I haven't seen anyone discuss anything similar to the differential learning which I have formulated and it is boosting the score a lot.\n\nProbably because no one has a clue what you're talking about ... **😁**", "765696": "My current LB of 0.9819 is a Densenet121 using only Cutout and Dropblock! I suggest you try it", "765699": "wow, great. Would you share your Cutout setup? Here is what I have tried though. \n\n```\ndef cutout(org_img, magnitude=None):\n magnitudes = np.linspace(0, 60/331, 11)\n\n img = np.copy(org_img)\n mask_val = img.mean()\n\n if magnitude is None:\n mask_size = 16\n else:\n mask_size = int(round(img.shape[0]*random.uniform(magnitudes[magnitude], magnitudes[magnitude+1])))\n top = np.random.randint(0 - mask_size//2, img.shape[0] - mask_size)\n left = np.random.randint(0 - mask_size//2, img.shape[1] - mask_size)\n bottom = top + mask_size\n right = left + mask_size\n if top < 0:\n top = 0\n if left < 0:\n left = 0\n\n img[top:bottom, left:right, :].fill(mask_val)\n return img\n```", "765721": "I cutout 2 holes of about 1/3 of my image size as max hole size!", "765723": "ok, that would help. Thank you.", "765729": "Very instresting, i wonder how much of a boost did Differential Learning give you?", "765740": "Interesting for me a simple head worked best but maybe I should go back and experiment with it...", "765746": "I actually changed a lot of my architecture so last comment doesnt represent my current score, I now have a simple tail! So dont waste more time on those experiments :p", "765748": "Oups nevermind, just saw that you said that in your post haha!" }, "source": "meta" }