{ "id": 205161, "title": "Ensemble voting or cat models?", "url": "/competitions/cassava-leaf-disease-classification/discussion/205161", "author_name": "", "post_date": "2020-12-18T18:47:59.792595500Z", "votes": null, "comment_count": 2, "views": 0, "content": "

Hello everyone.
\nI just investigate possibility to use ensemble of EfficientNet and ResNet50
\nThe idea is simple. Load pre trained weights, cut off last layers, start train in parallel, cat features and make prediction.
\nFor example

\n

Model

\n
       ## EfficientNet\n       self.effnet = EfficientNet.from_pretrained(\"efficientnet-b3\")\n       ## Resnet \n       self.resnet = torchvision.models.resnet50(pretrained=True)\n       for param in self.resnet.parameters():\n            param.requires_grad_(False)\n       self.resnet.fc = nn.Identity()\n       self.out = nn.Linear(2048+1536, num_classes)\n
\n

Train

\n
        x1 = self.effnet.extract_features(image.clone())\n        x1 = F.adaptive_avg_pool2d(x1, 1).reshape(batch_size, -1)\n        x2 = self.resnet(image)\n        x2 = x2.view(x2.size(0), -1)\n        x = torch.cat((x1, x2), dim=1)\n\n        outputs = self.out(x)\n
\n

But the scores are not good.
\nMaybe it's not good practices to cat features? Or better use voting?
\nwhich approaches are better?

\n

Thanks

", "messages": [ { "id": "1118121", "postDate": "12/18/2020 18:47:59", "content": "

Hello everyone.
\nI just investigate possibility to use ensemble of EfficientNet and ResNet50
\nThe idea is simple. Load pre trained weights, cut off last layers, start train in parallel, cat features and make prediction.
\nFor example

\n

Model

\n
       ## EfficientNet\n       self.effnet = EfficientNet.from_pretrained(\"efficientnet-b3\")\n       ## Resnet \n       self.resnet = torchvision.models.resnet50(pretrained=True)\n       for param in self.resnet.parameters():\n            param.requires_grad_(False)\n       self.resnet.fc = nn.Identity()\n       self.out = nn.Linear(2048+1536, num_classes)\n
\n

Train

\n
        x1 = self.effnet.extract_features(image.clone())\n        x1 = F.adaptive_avg_pool2d(x1, 1).reshape(batch_size, -1)\n        x2 = self.resnet(image)\n        x2 = x2.view(x2.size(0), -1)\n        x = torch.cat((x1, x2), dim=1)\n\n        outputs = self.out(x)\n
\n

But the scores are not good.
\nMaybe it's not good practices to cat features? Or better use voting?
\nwhich approaches are better?

\n

Thanks

", "rawMarkdown": "Hello everyone. \nI just investigate possibility to use ensemble of EfficientNet and ResNet50\nThe idea is simple. Load pre trained weights, cut off last layers, start train in parallel, cat features and make prediction. \nFor example \n\n\n**Model**\n```\n ## EfficientNet\n self.effnet = EfficientNet.from_pretrained(\"efficientnet-b3\")\n ## Resnet \n self.resnet = torchvision.models.resnet50(pretrained=True)\n for param in self.resnet.parameters():\n param.requires_grad_(False)\n self.resnet.fc = nn.Identity()\n self.out = nn.Linear(2048+1536, num_classes)\n```\n**Train**\n```\n x1 = self.effnet.extract_features(image.clone())\n x1 = F.adaptive_avg_pool2d(x1, 1).reshape(batch_size, -1)\n x2 = self.resnet(image)\n x2 = x2.view(x2.size(0), -1)\n x = torch.cat((x1, x2), dim=1)\n \n outputs = self.out(x)\n```\n\nBut the scores are not good. \nMaybe it's not good practices to cat features? Or better use voting? \nwhich approaches are better? \n\nThanks", "votes": null }, { "id": "1118560", "postDate": "12/19/2020 07:28:47", "content": "

the two models are doing mostly the same thing, redundant features rather than really important feeatures for classification obviously doesn't make much sense.

", "rawMarkdown": "the two models are doing mostly the same thing, redundant features rather than really important feeatures for classification obviously doesn't make much sense.", "votes": null }, { "id": "1118620", "postDate": "12/19/2020 08:48:44", "content": "

@plugin1689 Thank you for response, looks like you are right. Eventually no improvements in redundant features.

", "rawMarkdown": "plugin1689 Thank you for response, looks like you are right. Eventually no improvements in redundant features.", "votes": null } ], "comments": [ { "id": 1118560, "author_name": "plugin1689", "author_url": "", "post_date": "12/19/2020 07:28:47", "content": "

the two models are doing mostly the same thing, redundant features rather than really important feeatures for classification obviously doesn't make much sense.

", "votes": null, "replies": [ { "id": 1118620, "author_name": "maverix", "author_url": "", "post_date": "12/19/2020 08:48:44", "content": "

@plugin1689 Thank you for response, looks like you are right. Eventually no improvements in redundant features.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1118121": "Hello everyone. \nI just investigate possibility to use ensemble of EfficientNet and ResNet50\nThe idea is simple. Load pre trained weights, cut off last layers, start train in parallel, cat features and make prediction. \nFor example \n\n\n**Model**\n```\n ## EfficientNet\n self.effnet = EfficientNet.from_pretrained(\"efficientnet-b3\")\n ## Resnet \n self.resnet = torchvision.models.resnet50(pretrained=True)\n for param in self.resnet.parameters():\n param.requires_grad_(False)\n self.resnet.fc = nn.Identity()\n self.out = nn.Linear(2048+1536, num_classes)\n```\n**Train**\n```\n x1 = self.effnet.extract_features(image.clone())\n x1 = F.adaptive_avg_pool2d(x1, 1).reshape(batch_size, -1)\n x2 = self.resnet(image)\n x2 = x2.view(x2.size(0), -1)\n x = torch.cat((x1, x2), dim=1)\n \n outputs = self.out(x)\n```\n\nBut the scores are not good. \nMaybe it's not good practices to cat features? Or better use voting? \nwhich approaches are better? \n\nThanks", "1118560": "the two models are doing mostly the same thing, redundant features rather than really important feeatures for classification obviously doesn't make much sense.", "1118620": "plugin1689 Thank you for response, looks like you are right. Eventually no improvements in redundant features." }, "source": "meta" }