In the torchvision version, all the FasterRcnn is done on the resnet and mobilenet variants. And have internally declared them. (https://github.com/pytorch/vision/blob/e65a857b5487a8493bc8a80a95d64d9f049de347/torchvision/models/detection/backbone_utils.py#L59)
\nI wanted to experiment with other different backbone networks from timm, I am not getting how to do it.
\nAnyone doing it or can help?
PS. I am trying to hack the BackboneWithFPN() function (https://github.com/pytorch/vision/blob/e65a857b5487a8493bc8a80a95d64d9f049de347/torchvision/models/detection/backbone_utils.py#L13)
\nBut there are weird errors when I use timm, they are so big I don't even want to show them here.
I think if we can bypass this we can add any backbone from timm to our fasterrcnn variants and can try more diverse models.Thanks
", "messages": [ { "id": "1636714", "postDate": "01/03/2022 07:29:25", "content": "In the torchvision version, all the FasterRcnn is done on the resnet and mobilenet variants. And have internally declared them. (https://github.com/pytorch/vision/blob/e65a857b5487a8493bc8a80a95d64d9f049de347/torchvision/models/detection/backbone_utils.py#L59)
\nI wanted to experiment with other different backbone networks from timm, I am not getting how to do it.
\nAnyone doing it or can help?
PS. I am trying to hack the BackboneWithFPN() function (https://github.com/pytorch/vision/blob/e65a857b5487a8493bc8a80a95d64d9f049de347/torchvision/models/detection/backbone_utils.py#L13)
\nBut there are weird errors when I use timm, they are so big I don't even want to show them here.
I think if we can bypass this we can add any backbone from timm to our fasterrcnn variants and can try more diverse models.Thanks
", "rawMarkdown": "In the torchvision version, all the FasterRcnn is done on the resnet and mobilenet variants. And have internally declared them. (https://github.com/pytorch/vision/blob/e65a857b5487a8493bc8a80a95d64d9f049de347/torchvision/models/detection/backbone_utils.py#L59) \nI wanted to experiment with other different backbone networks from timm, I am not getting how to do it.\nAnyone doing it or can help?\n\nPS. I am trying to hack the BackboneWithFPN() function (https://github.com/pytorch/vision/blob/e65a857b5487a8493bc8a80a95d64d9f049de347/torchvision/models/detection/backbone_utils.py#L13)\nBut there are weird errors when I use timm, they are so big I don't even want to show them here.\n\nI think if we can bypass this we can add any backbone from timm to our fasterrcnn variants and can try more diverse models.Thanks", "votes": null }, { "id": "1636738", "postDate": "01/03/2022 08:21:47", "content": "This is my FasterRCNN model you can switch resnets inside
\nbackbone = torchvision.models.detection.backbone_utils.resnet_fpn_backbone('resnet152',pretrained=True)\n\n\nanchor_sizes = ( (24,32),(48,64), (96,128), (192,256), (384,512)) \naspect_ratios = ((0.5,0.75,1.0,1.5,2.0,)) * len(anchor_sizes)\nanchor_generator = AnchorGenerator(anchor_sizes , aspect_ratios)\n\nroi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0','1','2','3','pool'],\n output_size=7,\n sampling_ratio=2)\nmodel = FasterRCNN(backbone,\n num_classes=2,\n rpn_anchor_generator=anchor_generator,\n box_roi_pool=roi_pooler)\nThanks, Lukasz! Have you tried some other backbones other than resnets?
", "rawMarkdown": "Thanks, Lukasz! Have you tried some other backbones other than resnets?", "votes": null }, { "id": "1636842", "postDate": "01/03/2022 10:14:22", "content": "Yeah i tried different backbones with and without FPN tried retina etc. :)
", "rawMarkdown": "Yeah i tried different backbones with and without FPN tried retina etc. :)", "votes": null }, { "id": "1639892", "postDate": "01/06/2022 03:03:49", "content": "Hi mrinath!
\nMaybe you can try the official implementation, https://rwightman.github.io/pytorch-image-models/feature_extraction/ with the \"features_only\" param in timm.create_model, we can get all the features from each stage of the backbone, and then, we can use the torchvision.ops.FPN to make the FPN layer.
\nWish it helps.
Hi @jarviskevin, thank you for replying, I tried to do that but like this,
\nbackbone = timm.create_model('resnet152', features_only = True, pretrained = False)\nm = torchvision.ops.FeaturePyramidNetwork([10, 20, 30], 5)\nbut I don't understand where should I pass this FPN and backbone, yes I know that we can pass backbone in the FasterRCNN but where should I pass the FPN?
\nI also tried this,
\nbackbone = timm.create_model('resnet152',features_only = True, pretrained = False)\nbackbone.out_channels = 1280\n\nanchor_generator = AnchorGenerator(sizes=((128, 256, 512),),\n aspect_ratios=((0.5, 1.0, 2.0),))\n\n\nroi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0','1','2','3','pool'],\n output_size=7,\n sampling_ratio=2)\nmodel = FasterRCNN(backbone,\n num_classes=2,\n rpn_anchor_generator=anchor_generator,\n box_roi_pool=roi_pooler)\n\nmodel = model.to(DEVICE)\nThis code was able to load the model But in the training loop when calculating loss loss_dict = model(images, targets), it is giving me this error 'List' object has no attribute 'Values' error. I don't really understand where I'm going wrong.
Hi @mrinath thank you for opening the discussion thread. I was wondering if you have solved the issue of adding backbone using timm? It would be really helpful if you can share some insights. Thank you.
", "rawMarkdown": "Hi @mrinath thank you for opening the discussion thread. I was wondering if you have solved the issue of adding backbone using timm? It would be really helpful if you can share some insights. Thank you.", "votes": null }, { "id": "1649295", "postDate": "01/14/2022 06:59:26", "content": "Hey guys, I think I am successful to join timm and FPN!
\nI will release the code shortly,
\nthere was a lot of manual things to do though, like picking the layers we want to get the features from, and the name of the features, since each series for example effnets or densenet, have different names and different sizes of feature maps, a bit of manual work is needed.
\nBut none the less I was able to join the effnet family of timm with the FPN.
Thank you for the quick reply.
\nIt will be interesting to see the code.
\nBTW, any luck with transformer-based model, like swin? The only way I'm seeing the use of swin as backbone for fasterRCNN is via MMdetection.
I have to check with transformer based models, but with CNNs my code should work well
\nI published here https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/300768
This is my FasterRCNN model you can switch resnets inside
\nbackbone = torchvision.models.detection.backbone_utils.resnet_fpn_backbone('resnet152',pretrained=True)\n\n\nanchor_sizes = ( (24,32),(48,64), (96,128), (192,256), (384,512)) \naspect_ratios = ((0.5,0.75,1.0,1.5,2.0,)) * len(anchor_sizes)\nanchor_generator = AnchorGenerator(anchor_sizes , aspect_ratios)\n\nroi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0','1','2','3','pool'],\n output_size=7,\n sampling_ratio=2)\nmodel = FasterRCNN(backbone,\n num_classes=2,\n rpn_anchor_generator=anchor_generator,\n box_roi_pool=roi_pooler)\nThanks, Lukasz! Have you tried some other backbones other than resnets?
", "votes": null, "replies": [] }, { "id": 1636842, "author_name": "lukaszborecki", "author_url": "", "post_date": "01/03/2022 10:14:22", "content": "Yeah i tried different backbones with and without FPN tried retina etc. :)
", "votes": null, "replies": [] } ] }, { "id": 1639892, "author_name": "jarviskevin", "author_url": "", "post_date": "01/06/2022 03:03:49", "content": "Hi mrinath!
\nMaybe you can try the official implementation, https://rwightman.github.io/pytorch-image-models/feature_extraction/ with the \"features_only\" param in timm.create_model, we can get all the features from each stage of the backbone, and then, we can use the torchvision.ops.FPN to make the FPN layer.
\nWish it helps.
Hi @jarviskevin, thank you for replying, I tried to do that but like this,
\nbackbone = timm.create_model('resnet152', features_only = True, pretrained = False)\nm = torchvision.ops.FeaturePyramidNetwork([10, 20, 30], 5)\nbut I don't understand where should I pass this FPN and backbone, yes I know that we can pass backbone in the FasterRCNN but where should I pass the FPN?
\nI also tried this,
\nbackbone = timm.create_model('resnet152',features_only = True, pretrained = False)\nbackbone.out_channels = 1280\n\nanchor_generator = AnchorGenerator(sizes=((128, 256, 512),),\n aspect_ratios=((0.5, 1.0, 2.0),))\n\n\nroi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0','1','2','3','pool'],\n output_size=7,\n sampling_ratio=2)\nmodel = FasterRCNN(backbone,\n num_classes=2,\n rpn_anchor_generator=anchor_generator,\n box_roi_pool=roi_pooler)\n\nmodel = model.to(DEVICE)\nThis code was able to load the model But in the training loop when calculating loss loss_dict = model(images, targets), it is giving me this error 'List' object has no attribute 'Values' error. I don't really understand where I'm going wrong.
Hey guys, I think I am successful to join timm and FPN!
\nI will release the code shortly,
\nthere was a lot of manual things to do though, like picking the layers we want to get the features from, and the name of the features, since each series for example effnets or densenet, have different names and different sizes of feature maps, a bit of manual work is needed.
\nBut none the less I was able to join the effnet family of timm with the FPN.
Thank you for the quick reply.
\nIt will be interesting to see the code.
\nBTW, any luck with transformer-based model, like swin? The only way I'm seeing the use of swin as backbone for fasterRCNN is via MMdetection.
I have to check with transformer based models, but with CNNs my code should work well
\nI published here https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/300768
Hi @mrinath thank you for opening the discussion thread. I was wondering if you have solved the issue of adding backbone using timm? It would be really helpful if you can share some insights. Thank you.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1636714": "In the torchvision version, all the FasterRcnn is done on the resnet and mobilenet variants. And have internally declared them. (https://github.com/pytorch/vision/blob/e65a857b5487a8493bc8a80a95d64d9f049de347/torchvision/models/detection/backbone_utils.py#L59) \nI wanted to experiment with other different backbone networks from timm, I am not getting how to do it.\nAnyone doing it or can help?\n\nPS. I am trying to hack the BackboneWithFPN() function (https://github.com/pytorch/vision/blob/e65a857b5487a8493bc8a80a95d64d9f049de347/torchvision/models/detection/backbone_utils.py#L13)\nBut there are weird errors when I use timm, they are so big I don't even want to show them here.\n\nI think if we can bypass this we can add any backbone from timm to our fasterrcnn variants and can try more diverse models.Thanks", "1636738": "This is my FasterRCNN model you can switch resnets inside\n\n```\nbackbone = torchvision.models.detection.backbone_utils.resnet_fpn_backbone('resnet152',pretrained=True)\n\n\nanchor_sizes = ( (24,32),(48,64), (96,128), (192,256), (384,512)) \naspect_ratios = ((0.5,0.75,1.0,1.5,2.0,)) * len(anchor_sizes)\nanchor_generator = AnchorGenerator(anchor_sizes , aspect_ratios)\n\nroi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0','1','2','3','pool'],\n output_size=7,\n sampling_ratio=2)\nmodel = FasterRCNN(backbone,\n num_classes=2,\n rpn_anchor_generator=anchor_generator,\n box_roi_pool=roi_pooler)\n```", "1636822": "Thanks, Lukasz! Have you tried some other backbones other than resnets?", "1636842": "Yeah i tried different backbones with and without FPN tried retina etc. :)", "1639892": "Hi mrinath!\nMaybe you can try the official implementation, [https://rwightman.github.io/pytorch-image-models/feature_extraction/](url) with the \"features_only\" param in timm.create_model, we can get all the features from each stage of the backbone, and then, we can use the torchvision.ops.FPN to make the FPN layer. \nWish it helps.", "1649283": "Hi @jarviskevin, thank you for replying, I tried to do that but like this,\n```\nbackbone = timm.create_model('resnet152', features_only = True, pretrained = False)\nm = torchvision.ops.FeaturePyramidNetwork([10, 20, 30], 5)\n```\nbut I don't understand where should I pass this FPN and backbone, yes I know that we can pass backbone in the FasterRCNN but where should I pass the FPN? \n\n----\n\nI also tried this,\n```\nbackbone = timm.create_model('resnet152',features_only = True, pretrained = False)\nbackbone.out_channels = 1280\n\nanchor_generator = AnchorGenerator(sizes=((128, 256, 512),),\n aspect_ratios=((0.5, 1.0, 2.0),))\n\n\nroi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0','1','2','3','pool'],\n output_size=7,\n sampling_ratio=2)\nmodel = FasterRCNN(backbone,\n num_classes=2,\n rpn_anchor_generator=anchor_generator,\n box_roi_pool=roi_pooler)\n\nmodel = model.to(DEVICE)\n```\nThis code was able to load the model But in the training loop when calculating loss `loss_dict = model(images, targets)`, it is giving me this error `'List' object has no attribute 'Values' error`. I don't really understand where I'm going wrong.\n\n----\n\n### It would be really very helpful if you can give some more hints about using timm for creating backbone.", "1649291": "Hi @mrinath thank you for opening the discussion thread. I was wondering if you have solved the issue of adding backbone using timm? It would be really helpful if you can share some insights. Thank you.", "1649295": "Hey guys, I think I am successful to join timm and FPN!\nI will release the code shortly, \nthere was a lot of manual things to do though, like picking the layers we want to get the features from, and the name of the features, since each series for example effnets or densenet, have different names and different sizes of feature maps, a bit of manual work is needed.\nBut none the less I was able to join the effnet family of timm with the FPN.", "1649302": "Thank you for the quick reply. \nIt will be interesting to see the code. \nBTW, any luck with transformer-based model, like swin? The only way I'm seeing the use of `swin` as backbone for fasterRCNN is via MMdetection.", "1649308": "I have to check with transformer based models, but with CNNs my code should work well\nI published here https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/300768" }, "source": "meta" }