Just a quick question - have you succeeded with any other model except se_resnext50? My se101 shows worse result. Don't know yet whether it's related to model or as already mentioned to augmentations.
", "messages": [ { "id": "763096", "postDate": "03/04/2020 06:09:18", "content": "Just a quick question - have you succeeded with any other model except se_resnext50? My se101 shows worse result. Don't know yet whether it's related to model or as already mentioned to augmentations.
", "rawMarkdown": "Just a quick question - have you succeeded with any other model except se_resnext50? My se101 shows worse result. Don't know yet whether it's related to model or as already mentioned to augmentations.", "votes": null }, { "id": "763123", "postDate": "03/04/2020 06:51:04", "content": "I have worse results with the se-series in general. I think it also has something to do with the config (lr, schedule, augmentation etc.) My current score is not from se-resnexts
", "rawMarkdown": "I have worse results with the se-series in general. I think it also has something to do with the config (lr, schedule, augmentation etc.) My current score is not from se-resnexts", "votes": null }, { "id": "763125", "postDate": "03/04/2020 06:53:28", "content": "That's interesting! ty
", "rawMarkdown": "That's interesting! ty", "votes": null }, { "id": "763127", "postDate": "03/04/2020 06:55:15", "content": "", "rawMarkdown": "", "votes": null }, { "id": "763141", "postDate": "03/04/2020 07:13:43", "content": "What series is better for you?
", "rawMarkdown": "What series is better for you?", "votes": null }, { "id": "763151", "postDate": "03/04/2020 07:28:33", "content": "Efficientnet, but I guess it really depends on the setup. Have not been able to get seresnext working in past competitions, neither
", "rawMarkdown": "Efficientnet, but I guess it really depends on the setup. Have not been able to get seresnext working in past competitions, neither", "votes": null }, { "id": "763160", "postDate": "03/04/2020 07:44:22", "content": "Different architectures need different lr tuning, that's why it is so hard to switch unfortunately.
", "rawMarkdown": "Different architectures need different lr tuning, that's why it is so hard to switch unfortunately.", "votes": null }, { "id": "763226", "postDate": "03/04/2020 09:11:55", "content": "For me, ghostnet < efficientnet = seresnext
", "rawMarkdown": "For me, ghostnet < efficientnet = seresnext", "votes": null }, { "id": "763240", "postDate": "03/04/2020 09:26:57", "content": "I haven't tuned yet, maybe I'll try efficientnet now.
", "rawMarkdown": "I haven't tuned yet, maybe I'll try efficientnet now.", "votes": null }, { "id": "763546", "postDate": "03/04/2020 15:33:29", "content": "@roguekk007 @jadechoi \nI found effnets quite bulky here, high on mem and slower on epochs than seresnexts. I couldn't run it for longer epochs with the limited resources that I have. Could you share your experience?
", "rawMarkdown": "roguekk007 @jadechoi \nI found effnets quite bulky here, high on mem and slower on epochs than seresnexts. I couldn't run it for longer epochs with the limited resources that I have. Could you share your experience?", "votes": null }, { "id": "763555", "postDate": "03/04/2020 15:42:58", "content": "I remember Dr. HB pointing out something suboptimal about swish implementation in efficientnet, you can check that. I substituted all swish in efficientnet to mish_cuda (you can find it on github) which is more optimized. Efficientnets just might not be as well-optimized as the se-resnexts, i don't know. Taking a random guess here.
", "rawMarkdown": "I remember Dr. HB pointing out something suboptimal about swish implementation in efficientnet, you can check that. I substituted all swish in efficientnet to mish_cuda (you can find it on github) which is more optimized. Efficientnets just might not be as well-optimized as the se-resnexts, i don't know. Taking a random guess here.", "votes": null }, { "id": "763563", "postDate": "03/04/2020 15:51:05", "content": "yes,\u001b effnet needs more time than seresnext. so I use seresnext now😂 (because I got the same LB score)
", "rawMarkdown": "yes,\u001b effnet needs more time than seresnext. so I use seresnext now😂 (because I got the same LB score)", "votes": null }, { "id": "763571", "postDate": "03/04/2020 16:03:03", "content": "@roguekk007 possibly you're referring to this?\nI did try out Mish too, but not mish_cuda. I've a B1 in training, let's see how it goes.\nBtw, what was the highest LB you got using effnets?
", "rawMarkdown": "roguekk007 possibly you're referring to [this](https://www.kaggle.com/c/bengaliai-cv19/discussion/128911)?\nI did try out Mish too, but not mish_cuda. I've a B1 in training, let's see how it goes.\nBtw, what was the highest LB you got using effnets?", "votes": null }, { "id": "763654", "postDate": "03/04/2020 17:51:09", "content": "\n\n\"I have worse results with the se-series in general.\" ... \"Efficientnet, but I guess it really depends on the setup. Have not been able to get seresnext\"
\n\ni believe it is because there is drop connect in training model of efficient-net. If you setup correct regularisation in seresnext, i think you will get similar performance
", "rawMarkdown": "roguekk007 \n\n\"I have worse results with the se-series in general.\" ... \"Efficientnet, but I guess it really depends on the setup. Have not been able to get seresnext\"\n\ni believe it is because there is drop connect in training model of efficient-net. If you setup correct regularisation in seresnext, i think you will get similar performance", "votes": null }, { "id": "763923", "postDate": "03/05/2020 01:27:42", "content": "In my experience, efficientNet≒se-resnext, but DenseNet is not so good.\nIs it just me?
", "rawMarkdown": "In my experience, efficientNet≒se-resnext, but DenseNet is not so good.\nIs it just me?", "votes": null }, { "id": "764265", "postDate": "03/05/2020 09:29:13", "content": "My B3 got me a CV ~0.974
", "rawMarkdown": "My B3 got me a CV ~0.974", "votes": null } ], "comments": [ { "id": 763123, "author_name": "roguekk007", "author_url": "", "post_date": "03/04/2020 06:51:04", "content": "I have worse results with the se-series in general. I think it also has something to do with the config (lr, schedule, augmentation etc.) My current score is not from se-resnexts
", "votes": null, "replies": [ { "id": 763125, "author_name": "yaroshevskiy", "author_url": "", "post_date": "03/04/2020 06:53:28", "content": "That's interesting! ty
", "votes": null, "replies": [] }, { "id": 763127, "author_name": "roguekk007", "author_url": "", "post_date": "03/04/2020 06:55:15", "content": "", "votes": null, "replies": [] }, { "id": 763141, "author_name": "cpmpml", "author_url": "", "post_date": "03/04/2020 07:13:43", "content": "What series is better for you?
", "votes": null, "replies": [] }, { "id": 763151, "author_name": "roguekk007", "author_url": "", "post_date": "03/04/2020 07:28:33", "content": "Efficientnet, but I guess it really depends on the setup. Have not been able to get seresnext working in past competitions, neither
", "votes": null, "replies": [] }, { "id": 763160, "author_name": "philippsinger", "author_url": "", "post_date": "03/04/2020 07:44:22", "content": "Different architectures need different lr tuning, that's why it is so hard to switch unfortunately.
", "votes": null, "replies": [] }, { "id": 763240, "author_name": "cpmpml", "author_url": "", "post_date": "03/04/2020 09:26:57", "content": "I haven't tuned yet, maybe I'll try efficientnet now.
", "votes": null, "replies": [] }, { "id": 763654, "author_name": "hengck23", "author_url": "", "post_date": "03/04/2020 17:51:09", "content": "\n\n\"I have worse results with the se-series in general.\" ... \"Efficientnet, but I guess it really depends on the setup. Have not been able to get seresnext\"
\n\ni believe it is because there is drop connect in training model of efficient-net. If you setup correct regularisation in seresnext, i think you will get similar performance
", "votes": null, "replies": [] } ] }, { "id": 763226, "author_name": "jadechoi", "author_url": "", "post_date": "03/04/2020 09:11:55", "content": "For me, ghostnet < efficientnet = seresnext
", "votes": null, "replies": [] }, { "id": 763546, "author_name": "mightyrains", "author_url": "", "post_date": "03/04/2020 15:33:29", "content": "@roguekk007 @jadechoi \nI found effnets quite bulky here, high on mem and slower on epochs than seresnexts. I couldn't run it for longer epochs with the limited resources that I have. Could you share your experience?
", "votes": null, "replies": [ { "id": 763555, "author_name": "roguekk007", "author_url": "", "post_date": "03/04/2020 15:42:58", "content": "I remember Dr. HB pointing out something suboptimal about swish implementation in efficientnet, you can check that. I substituted all swish in efficientnet to mish_cuda (you can find it on github) which is more optimized. Efficientnets just might not be as well-optimized as the se-resnexts, i don't know. Taking a random guess here.
", "votes": null, "replies": [] }, { "id": 763563, "author_name": "jadechoi", "author_url": "", "post_date": "03/04/2020 15:51:05", "content": "yes,\u001b effnet needs more time than seresnext. so I use seresnext now😂 (because I got the same LB score)
", "votes": null, "replies": [] }, { "id": 763571, "author_name": "mightyrains", "author_url": "", "post_date": "03/04/2020 16:03:03", "content": "@roguekk007 possibly you're referring to this?\nI did try out Mish too, but not mish_cuda. I've a B1 in training, let's see how it goes.\nBtw, what was the highest LB you got using effnets?
", "votes": null, "replies": [] }, { "id": 764265, "author_name": "mightyrains", "author_url": "", "post_date": "03/05/2020 09:29:13", "content": "My B3 got me a CV ~0.974
", "votes": null, "replies": [] } ] }, { "id": 763923, "author_name": "kyosato", "author_url": "", "post_date": "03/05/2020 01:27:42", "content": "In my experience, efficientNet≒se-resnext, but DenseNet is not so good.\nIs it just me?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "763096": "Just a quick question - have you succeeded with any other model except se_resnext50? My se101 shows worse result. Don't know yet whether it's related to model or as already mentioned to augmentations.", "763123": "I have worse results with the se-series in general. I think it also has something to do with the config (lr, schedule, augmentation etc.) My current score is not from se-resnexts", "763125": "That's interesting! ty", "763127": "", "763141": "What series is better for you?", "763151": "Efficientnet, but I guess it really depends on the setup. Have not been able to get seresnext working in past competitions, neither", "763160": "Different architectures need different lr tuning, that's why it is so hard to switch unfortunately.", "763226": "For me, ghostnet < efficientnet = seresnext", "763240": "I haven't tuned yet, maybe I'll try efficientnet now.", "763546": "roguekk007 @jadechoi \nI found effnets quite bulky here, high on mem and slower on epochs than seresnexts. I couldn't run it for longer epochs with the limited resources that I have. Could you share your experience?", "763555": "I remember Dr. HB pointing out something suboptimal about swish implementation in efficientnet, you can check that. I substituted all swish in efficientnet to mish_cuda (you can find it on github) which is more optimized. Efficientnets just might not be as well-optimized as the se-resnexts, i don't know. Taking a random guess here.", "763563": "yes,\u001b effnet needs more time than seresnext. so I use seresnext now😂 (because I got the same LB score)", "763571": "roguekk007 possibly you're referring to [this](https://www.kaggle.com/c/bengaliai-cv19/discussion/128911)?\nI did try out Mish too, but not mish_cuda. I've a B1 in training, let's see how it goes.\nBtw, what was the highest LB you got using effnets?", "763654": "roguekk007 \n\n\"I have worse results with the se-series in general.\" ... \"Efficientnet, but I guess it really depends on the setup. Have not been able to get seresnext\"\n\ni believe it is because there is drop connect in training model of efficient-net. If you setup correct regularisation in seresnext, i think you will get similar performance", "763923": "In my experience, efficientNet≒se-resnext, but DenseNet is not so good.\nIs it just me?", "764265": "My B3 got me a CV ~0.974" }, "source": "meta" }