My model scored 0.962 from the beginning. Many kernels performs beyond 0.95~0.96 were available already, they helped me a lot.
\n\nThe 0.962 model from a single cross-validation set is improved by\n0.962 : DenseNet 121 Backbone + CutMix\n0.967 : Use Fast AutoAugment's Found Policy(ImageNet)\n0.968+alpha : Larger Input Size, Parameter Tuning
\n\nAfter that I ensembled three models from different cross-validation sets by averaging softmax output. This gives me stable 0.97. Other small tweaks were applied such as mish activation and etcs.
\n\nLB seems to be very fragment. I expect participants from 0.96 to 0.97xx have similar approach with me. But for now, I'm not sure how to achieve more than 0.98. Simple network architectural changes(eg. other architecture, activation, input size, normalization, ...) or optimization methods(hyperparameter tuning, different optimizer, ...) addition would not be enough.
\n\nWhat's next?
", "messages": [ { "id": 718058, "postDate": "2020-01-13T23:56:12.367Z", "content": "My model scored 0.962 from the beginning. Many kernels performs beyond 0.95~0.96 were available already, they helped me a lot.
\n\nThe 0.962 model from a single cross-validation set is improved by\n0.962 : DenseNet 121 Backbone + CutMix\n0.967 : Use Fast AutoAugment's Found Policy(ImageNet)\n0.968+alpha : Larger Input Size, Parameter Tuning
\n\nAfter that I ensembled three models from different cross-validation sets by averaging softmax output. This gives me stable 0.97. Other small tweaks were applied such as mish activation and etcs.
\n\nLB seems to be very fragment. I expect participants from 0.96 to 0.97xx have similar approach with me. But for now, I'm not sure how to achieve more than 0.98. Simple network architectural changes(eg. other architecture, activation, input size, normalization, ...) or optimization methods(hyperparameter tuning, different optimizer, ...) addition would not be enough.
\n\nWhat's next?
", "rawMarkdown": "My model scored 0.962 from the beginning. Many kernels performs beyond 0.95~0.96 were available already, they helped me a lot. \n\nThe 0.962 model from a single cross-validation set is improved by\n0.962 : DenseNet 121 Backbone + CutMix\n0.967 : Use Fast AutoAugment's Found Policy(ImageNet)\n0.968+alpha : Larger Input Size, Parameter Tuning\n\nAfter that I ensembled three models from different cross-validation sets by averaging softmax output. This gives me stable 0.97. Other small tweaks were applied such as mish activation and etcs.\n\nLB seems to be very fragment. I expect participants from 0.96 to 0.97xx have similar approach with me. But for now, I'm not sure how to achieve more than 0.98. Simple network architectural changes(eg. other architecture, activation, input size, normalization, ...) or optimization methods(hyperparameter tuning, different optimizer, ...) addition would not be enough.\n\nWhat's next?\n\n", "votes": 45 }, { "id": 718545, "postDate": "2020-01-14T14:14:04.830Z", "content": "I am in similar position like you... my model score between 0.965-0.9697. I have tried bunch of things...
\n\nI am not sure if success to getting 0.98+ relies on big models or some kind of data property which has to be explored....
", "rawMarkdown": "I am in similar position like you... my model score between 0.965-0.9697. I have tried bunch of things... \n\nI am not sure if success to getting 0.98+ relies on big models or some kind of data property which has to be explored....\n", "votes": 2, "replies": [ { "id": 720162, "postDate": "2020-01-16T08:02:39.563Z", "content": "In my case, Model capacity doesn't impact much. But intensive regularization might be helpful. I will examine on this after this weekdays.
", "rawMarkdown": "In my case, Model capacity doesn't impact much. But intensive regularization might be helpful. I will examine on this after this weekdays.", "votes": 2 }, { "id": 721606, "postDate": "2020-01-17T14:25:43.617Z", "content": "well it seems like we both found something =)
", "rawMarkdown": "well it seems like we both found something =) ", "votes": 3 } ] }, { "id": 718076, "postDate": "2020-01-14T00:42:32.220Z", "content": "Maybe mixup could be the one you need. btw, how much did Fast AutoAugment's Found Policy(ImageNet) helped on local CV/LB? I've read the paper and looks very interesting
\n", "rawMarkdown": "Maybe mixup could be the one you need. btw, how much did ` Fast AutoAugment's Found Policy(ImageNet)` helped on local CV/LB? I've read the paper and looks very interesting \n> 0.962 -> 0.967 ??\n", "votes": 1, "replies": [ { "id": 718156, "postDate": "2020-01-14T05:01:01.863Z", "content": "0.962 -> 0.967 ??
\n
Yes from 0.962 to 0.967 (LB). CV have a similar tendency. We can optimize policy by searching optimal policy using Fast AutoAugment, but this simple approach also gives moderate improvement.
", "rawMarkdown": "Yes from 0.962 to 0.967 (LB). CV have a similar tendency. We can optimize policy by searching optimal policy using Fast AutoAugment, but this simple approach also gives moderate improvement.", "votes": 2 }, { "id": 718194, "postDate": "2020-01-14T06:55:34.517Z", "content": "Wow!! that is a huge boost. It would be nice if you could make a kernel(?) about how to use Fast Auto Augment, since this is a relatively new concept and haven't seen much in Kaggle. I think you are the perfect person to explain Fast Auto Augment😄 😄
", "rawMarkdown": "Wow!! that is a huge boost. It would be nice if you could make a kernel(?) about how to use Fast Auto Augment, since this is a relatively new concept and haven't seen much in Kaggle. I think you are the perfect person to explain Fast Auto Augment😄 😄 ", "votes": 3 }, { "id": 718259, "postDate": "2020-01-14T08:18:30.083Z", "content": "he already used cutmix so mixup is not really needed
", "rawMarkdown": "he already used cutmix so mixup is not really needed", "votes": 7 }, { "id": 720161, "postDate": "2020-01-16T08:01:45.640Z", "content": "Right, but as @bibek777 's post, it might be beneficial using cutmix and mixup randomly.
", "rawMarkdown": "Right, but as @bibek777 's post, it might be beneficial using cutmix and mixup randomly." } ] }, { "id": 734167, "postDate": "2020-02-01T03:38:59.137Z", "content": "How many epochs did you train your model?
", "rawMarkdown": "How many epochs did you train your model?" }, { "id": 725499, "postDate": "2020-01-22T07:09:44.820Z", "content": "Did you train all these models on Kaggle? @ildoonet
", "rawMarkdown": "Did you train all these models on Kaggle? @ildoonet ", "replies": [ { "id": 727739, "postDate": "2020-01-24T02:08:33.693Z", "content": "Yes bit I also use my own local machines.
", "rawMarkdown": "Yes bit I also use my own local machines. " }, { "id": 727754, "postDate": "2020-01-24T02:22:52.503Z", "rawMarkdown": "" } ] }, { "id": 718454, "postDate": "2020-01-14T12:32:55.830Z", "rawMarkdown": "", "isDeleted": true }, { "id": 733588, "postDate": "2020-01-31T10:24:54.850Z", "content": "Thanks for sharing
", "rawMarkdown": " Thanks for sharing", "votes": 1 } ], "comments": [ { "id": 718545, "author_name": "DrHB", "author_url": "", "post_date": "2020-01-14T14:14:04.830000", "content": "I am in similar position like you... my model score between 0.965-0.9697. I have tried bunch of things...
\n\nI am not sure if success to getting 0.98+ relies on big models or some kind of data property which has to be explored....
", "votes": 2, "replies": [ { "id": 720162, "author_name": "Ildoo Kim", "author_url": "", "post_date": "2020-01-16T08:02:39.563000", "content": "In my case, Model capacity doesn't impact much. But intensive regularization might be helpful. I will examine on this after this weekdays.
", "votes": 2, "replies": [] }, { "id": 721606, "author_name": "DrHB", "author_url": "", "post_date": "2020-01-17T14:25:43.617000", "content": "well it seems like we both found something =)
", "votes": 3, "replies": [] } ] }, { "id": 718076, "author_name": "Bibek", "author_url": "", "post_date": "2020-01-14T00:42:32.220000", "content": "Maybe mixup could be the one you need. btw, how much did Fast AutoAugment's Found Policy(ImageNet) helped on local CV/LB? I've read the paper and looks very interesting
\n", "votes": 1, "replies": [ { "id": 718156, "author_name": "Ildoo Kim", "author_url": "", "post_date": "2020-01-14T05:01:01.863000", "content": "0.962 -> 0.967 ??
\n
Yes from 0.962 to 0.967 (LB). CV have a similar tendency. We can optimize policy by searching optimal policy using Fast AutoAugment, but this simple approach also gives moderate improvement.
", "votes": 2, "replies": [] }, { "id": 718194, "author_name": "Bibek", "author_url": "", "post_date": "2020-01-14T06:55:34.517000", "content": "Wow!! that is a huge boost. It would be nice if you could make a kernel(?) about how to use Fast Auto Augment, since this is a relatively new concept and haven't seen much in Kaggle. I think you are the perfect person to explain Fast Auto Augment😄 😄
", "votes": 3, "replies": [] }, { "id": 718259, "author_name": "DatNT", "author_url": "", "post_date": "2020-01-14T08:18:30.083000", "content": "he already used cutmix so mixup is not really needed
", "votes": 7, "replies": [] }, { "id": 720161, "author_name": "Ildoo Kim", "author_url": "", "post_date": "2020-01-16T08:01:45.640000", "content": "Right, but as @bibek777 's post, it might be beneficial using cutmix and mixup randomly.
", "votes": 0, "replies": [] } ] }, { "id": 734167, "author_name": "Wang Xinliang", "author_url": "", "post_date": "2020-02-01T03:38:59.137000", "content": "How many epochs did you train your model?
", "votes": 0, "replies": [] }, { "id": 725499, "author_name": "sl02", "author_url": "", "post_date": "2020-01-22T07:09:44.820000", "content": "Did you train all these models on Kaggle? @ildoonet
", "votes": 0, "replies": [ { "id": 727739, "author_name": "Ildoo Kim", "author_url": "", "post_date": "2020-01-24T02:08:33.693000", "content": "Yes bit I also use my own local machines.
", "votes": 0, "replies": [] }, { "id": 727754, "author_name": "Innat", "author_url": "", "post_date": "2020-01-24T02:22:52.503000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 718454, "author_name": "", "author_url": "", "post_date": "2020-01-14T12:32:55.830000", "content": "", "votes": 0, "replies": [] }, { "id": 733588, "author_name": "manoj kumar", "author_url": "", "post_date": "2020-01-31T10:24:54.850000", "content": "Thanks for sharing
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "718058": "My model scored 0.962 from the beginning. Many kernels performs beyond 0.95~0.96 were available already, they helped me a lot. \n\nThe 0.962 model from a single cross-validation set is improved by\n0.962 : DenseNet 121 Backbone + CutMix\n0.967 : Use Fast AutoAugment's Found Policy(ImageNet)\n0.968+alpha : Larger Input Size, Parameter Tuning\n\nAfter that I ensembled three models from different cross-validation sets by averaging softmax output. This gives me stable 0.97. Other small tweaks were applied such as mish activation and etcs.\n\nLB seems to be very fragment. I expect participants from 0.96 to 0.97xx have similar approach with me. But for now, I'm not sure how to achieve more than 0.98. Simple network architectural changes(eg. other architecture, activation, input size, normalization, ...) or optimization methods(hyperparameter tuning, different optimizer, ...) addition would not be enough.\n\nWhat's next?\n\n", "718545": "I am in similar position like you... my model score between 0.965-0.9697. I have tried bunch of things... \n\nI am not sure if success to getting 0.98+ relies on big models or some kind of data property which has to be explored....\n", "718076": "Maybe mixup could be the one you need. btw, how much did ` Fast AutoAugment's Found Policy(ImageNet)` helped on local CV/LB? I've read the paper and looks very interesting \n> 0.962 -> 0.967 ??\n", "734167": "How many epochs did you train your model?", "725499": "Did you train all these models on Kaggle? @ildoonet ", "718454": "", "733588": " Thanks for sharing" } }