{ "id": 209041, "title": "Is a larger model unnecessary?", "url": "/competitions/rfcx-species-audio-detection/discussion/209041", "author_name": "shinmura0", "post_date": "2021-01-06T03:07:49.849000", "votes": 40, "comment_count": 16, "views": 0, "content": "

In this competition, a smaller model(EfficientNet and MobileNetV2) has good performance.
\nBut, is a larger model(like ResNest50) unnecessary? The answer is \"NO\".

\n

In ensemble phase, a larger model is necessary.

\n

\"\"

\n\n

In this competition, the ensemble is storng. And mixing a larger model is effective.
\nThe ensemble code is below.

\n
def ensemble(DataFrame1, DataFrame2, ..., DataFrame5):\n    for i in range(24):\n        name = \"s\" + str(i)\n        DataFrame1[name] = 0.2*(DataFrame1[name]+ ... + DataFrame5[name])\n    return DataFrame1\n
\n

This is simple average.

", "messages": [ { "id": 1140473, "postDate": "2021-01-06T03:07:49.850Z", "content": "

In this competition, a smaller model(EfficientNet and MobileNetV2) has good performance.
\nBut, is a larger model(like ResNest50) unnecessary? The answer is \"NO\".

\n

In ensemble phase, a larger model is necessary.

\n

\"\"

\n\n

In this competition, the ensemble is storng. And mixing a larger model is effective.
\nThe ensemble code is below.

\n
def ensemble(DataFrame1, DataFrame2, ..., DataFrame5):\n    for i in range(24):\n        name = \"s\" + str(i)\n        DataFrame1[name] = 0.2*(DataFrame1[name]+ ... + DataFrame5[name])\n    return DataFrame1\n
\n

This is simple average.

", "rawMarkdown": "In this competition, a smaller model(EfficientNet and MobileNetV2) has good performance.\nBut, is a larger model(like ResNest50) unnecessary? The answer is \"NO\".\n\nIn ensemble phase, a larger model is necessary.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4704212%2F1720b48aade45ca0e78208fdbfac0383%2Ffig1.png?generation=1609899400061046&alt=media)\n\n+ ResNest50 is not good performance(LB:0.867)\n+ But \"EfficientNetB0-2(**LB:0.901**) + ResNest50(**LB:0.867**) = **LB:0.903**\"\n+ ResNest50 is important role\n\nIn this competition, the ensemble is storng. And **mixing a larger model is effective**.\nThe ensemble code is below.\n\n```\ndef ensemble(DataFrame1, DataFrame2, ..., DataFrame5):\n for i in range(24):\n name = \"s\" + str(i)\n DataFrame1[name] = 0.2*(DataFrame1[name]+ ... + DataFrame5[name])\n return DataFrame1\n```\n\nThis is simple average.", "votes": 40 }, { "id": 1140942, "postDate": "2021-01-06T11:45:23.557Z", "content": "

I think your ensemble benefits more from diversity (ResNests and EfficientNets are quite different) than model size.

\n

If you want to compare model size, I would advise checking the performance of EfficientNet-b3 or even b4.

", "rawMarkdown": "I think your ensemble benefits more from diversity (ResNests and EfficientNets are quite different) than model size.\n\nIf you want to compare model size, I would advise checking the performance of EfficientNet-b3 or even b4.", "votes": 7, "replies": [ { "id": 1142241, "postDate": "2021-01-07T08:48:15.307Z", "content": "

It seems that the effect of diversity is greater than the size of the model.

", "rawMarkdown": "It seems that the effect of diversity is greater than the size of the model.", "votes": 2 }, { "id": 1175133, "postDate": "2021-01-29T00:05:03.457Z", "content": "

Maybe I should start looking beyond effnets…

", "rawMarkdown": "Maybe I should start looking beyond effnets..." } ] }, { "id": 1174090, "postDate": "2021-01-28T09:02:16.883Z", "content": "

Thanks for sharing @shinmurashinmura , may I ask how did you train your 5 different models? I guess you use StratifiedKFold as 5 folds, and each of your model are trained with all 5 folds, then ensemble them (total models are 5*5=25), or you only train each model on only 1 fold(total models are 5)?

", "rawMarkdown": "Thanks for sharing @shinmurashinmura , may I ask how did you train your 5 different models? I guess you use StratifiedKFold as 5 folds, and each of your model are trained with all 5 folds, then ensemble them (total models are 5*5=25), or you only train each model on only 1 fold(total models are 5)?", "replies": [ { "id": 1180362, "postDate": "2021-02-01T08:17:59.517Z", "content": "
\n

then ensemble them (total models are 5*5=25), or you only train each model on only 1 fold(total models are 5)?

\n
\n

In this case, I trained 25(=5*5) models. And I average each prediction.

", "rawMarkdown": "> then ensemble them (total models are 5*5=25), or you only train each model on only 1 fold(total models are 5)?\n\nIn this case, I trained 25(=5*5) models. And I average each prediction." } ] }, { "id": 1143918, "postDate": "2021-01-08T06:12:10.113Z", "content": "

How did you train EfficinetNet-B0 it takes too much RAM

", "rawMarkdown": "How did you train EfficinetNet-B0 it takes too much RAM", "replies": [ { "id": 1144084, "postDate": "2021-01-08T08:17:43.197Z", "content": "

I used batch_size=16. It is a small size.

", "rawMarkdown": "I used batch_size=16. It is a small size.", "votes": 1 } ] }, { "id": 1143668, "postDate": "2021-01-08T01:56:48.417Z", "content": "

Hi!
\nMay I know how you generated the graph?

", "rawMarkdown": "Hi!\nMay I know how you generated the graph?", "replies": [ { "id": 1143722, "postDate": "2021-01-08T03:04:01.187Z", "content": "

I used \"Google slide\".

", "rawMarkdown": "I used \"Google slide\".", "votes": 1 } ] }, { "id": 1141277, "postDate": "2021-01-06T15:47:17.507Z", "content": "

Can you briefly explain the SED part architecture wise? Does it have any specific properties? I still don't understand it properly :( I currently use ResNests backbone with the linear layers and a sigmoid at the end and then use a BCELoss. Comparing the differences or pointing out to any resource would be too much appreciated. Thanks.

", "rawMarkdown": "Can you briefly explain the SED part architecture wise? Does it have any specific properties? I still don't understand it properly :( I currently use ResNests backbone with the linear layers and a sigmoid at the end and then use a BCELoss. Comparing the differences or pointing out to any resource would be too much appreciated. Thanks.", "replies": [ { "id": 1142227, "postDate": "2021-01-07T08:38:14.967Z", "content": "

Now, I'm writing new topic \"How to use SED\". Please wait.

", "rawMarkdown": "Now, I'm writing new topic \"How to use SED\". Please wait.", "votes": 5 }, { "id": 1142810, "postDate": "2021-01-07T15:47:18.590Z", "content": "

Great, I'm waiting for it 🤘

", "rawMarkdown": "Great, I'm waiting for it 🤘" } ] }, { "id": 1140653, "postDate": "2021-01-06T06:59:16.013Z", "content": "

Thank you i was struggling with this alot

", "rawMarkdown": "Thank you i was struggling with this alot" }, { "id": 1144627, "postDate": "2021-01-08T15:16:08.727Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1140647, "postDate": "2021-01-06T06:54:13.577Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1140646, "postDate": "2021-01-06T06:51:29.123Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1140942, "author_name": "Theo Viel", "author_url": "", "post_date": "2021-01-06T11:45:23.557000", "content": "

I think your ensemble benefits more from diversity (ResNests and EfficientNets are quite different) than model size.

\n

If you want to compare model size, I would advise checking the performance of EfficientNet-b3 or even b4.

", "votes": 7, "replies": [ { "id": 1142241, "author_name": "shinmura0", "author_url": "", "post_date": "2021-01-07T08:48:15.307000", "content": "

It seems that the effect of diversity is greater than the size of the model.

", "votes": 2, "replies": [] }, { "id": 1175133, "author_name": "CPMP", "author_url": "", "post_date": "2021-01-29T00:05:03.457000", "content": "

Maybe I should start looking beyond effnets…

", "votes": 0, "replies": [] } ] }, { "id": 1174090, "author_name": "Hao", "author_url": "", "post_date": "2021-01-28T09:02:16.883000", "content": "

Thanks for sharing @shinmurashinmura , may I ask how did you train your 5 different models? I guess you use StratifiedKFold as 5 folds, and each of your model are trained with all 5 folds, then ensemble them (total models are 5*5=25), or you only train each model on only 1 fold(total models are 5)?

", "votes": 0, "replies": [ { "id": 1180362, "author_name": "shinmura0", "author_url": "", "post_date": "2021-02-01T08:17:59.517000", "content": "
\n

then ensemble them (total models are 5*5=25), or you only train each model on only 1 fold(total models are 5)?

\n
\n

In this case, I trained 25(=5*5) models. And I average each prediction.

", "votes": 0, "replies": [] } ] }, { "id": 1143918, "author_name": "Sayantan Mazumdar", "author_url": "", "post_date": "2021-01-08T06:12:10.113000", "content": "

How did you train EfficinetNet-B0 it takes too much RAM

", "votes": 0, "replies": [ { "id": 1144084, "author_name": "shinmura0", "author_url": "", "post_date": "2021-01-08T08:17:43.197000", "content": "

I used batch_size=16. It is a small size.

", "votes": 1, "replies": [] } ] }, { "id": 1143668, "author_name": "GVKannan", "author_url": "", "post_date": "2021-01-08T01:56:48.417000", "content": "

Hi!
\nMay I know how you generated the graph?

", "votes": 0, "replies": [ { "id": 1143722, "author_name": "shinmura0", "author_url": "", "post_date": "2021-01-08T03:04:01.187000", "content": "

I used \"Google slide\".

", "votes": 1, "replies": [] } ] }, { "id": 1141277, "author_name": "Sinan Calisir", "author_url": "", "post_date": "2021-01-06T15:47:17.507000", "content": "

Can you briefly explain the SED part architecture wise? Does it have any specific properties? I still don't understand it properly :( I currently use ResNests backbone with the linear layers and a sigmoid at the end and then use a BCELoss. Comparing the differences or pointing out to any resource would be too much appreciated. Thanks.

", "votes": 0, "replies": [ { "id": 1142227, "author_name": "shinmura0", "author_url": "", "post_date": "2021-01-07T08:38:14.967000", "content": "

Now, I'm writing new topic \"How to use SED\". Please wait.

", "votes": 5, "replies": [] }, { "id": 1142810, "author_name": "Sinan Calisir", "author_url": "", "post_date": "2021-01-07T15:47:18.590000", "content": "

Great, I'm waiting for it 🤘

", "votes": 0, "replies": [] } ] }, { "id": 1140653, "author_name": "Sayantan Mazumdar", "author_url": "", "post_date": "2021-01-06T06:59:16.013000", "content": "

Thank you i was struggling with this alot

", "votes": 0, "replies": [] }, { "id": 1144627, "author_name": "", "author_url": "", "post_date": "2021-01-08T15:16:08.727000", "content": "", "votes": 0, "replies": [] }, { "id": 1140647, "author_name": "", "author_url": "", "post_date": "2021-01-06T06:54:13.577000", "content": "", "votes": 0, "replies": [] }, { "id": 1140646, "author_name": "", "author_url": "", "post_date": "2021-01-06T06:51:29.123000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1140473": "In this competition, a smaller model(EfficientNet and MobileNetV2) has good performance.\nBut, is a larger model(like ResNest50) unnecessary? The answer is \"NO\".\n\nIn ensemble phase, a larger model is necessary.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4704212%2F1720b48aade45ca0e78208fdbfac0383%2Ffig1.png?generation=1609899400061046&alt=media)\n\n+ ResNest50 is not good performance(LB:0.867)\n+ But \"EfficientNetB0-2(**LB:0.901**) + ResNest50(**LB:0.867**) = **LB:0.903**\"\n+ ResNest50 is important role\n\nIn this competition, the ensemble is storng. And **mixing a larger model is effective**.\nThe ensemble code is below.\n\n```\ndef ensemble(DataFrame1, DataFrame2, ..., DataFrame5):\n for i in range(24):\n name = \"s\" + str(i)\n DataFrame1[name] = 0.2*(DataFrame1[name]+ ... + DataFrame5[name])\n return DataFrame1\n```\n\nThis is simple average.", "1140942": "I think your ensemble benefits more from diversity (ResNests and EfficientNets are quite different) than model size.\n\nIf you want to compare model size, I would advise checking the performance of EfficientNet-b3 or even b4.", "1174090": "Thanks for sharing @shinmurashinmura , may I ask how did you train your 5 different models? I guess you use StratifiedKFold as 5 folds, and each of your model are trained with all 5 folds, then ensemble them (total models are 5*5=25), or you only train each model on only 1 fold(total models are 5)?", "1143918": "How did you train EfficinetNet-B0 it takes too much RAM", "1143668": "Hi!\nMay I know how you generated the graph?", "1141277": "Can you briefly explain the SED part architecture wise? Does it have any specific properties? I still don't understand it properly :( I currently use ResNests backbone with the linear layers and a sigmoid at the end and then use a BCELoss. Comparing the differences or pointing out to any resource would be too much appreciated. Thanks.", "1140653": "Thank you i was struggling with this alot", "1144627": "", "1140647": "", "1140646": "" } }