{
  "id": 170730,
  "title": "Starter with TResNet: A GPU-Dedicated Arch",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/170730",
  "author_name": "",
  "post_date": "2020-07-28T19:25:06.325995400Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Paper: <a href=\"https://arxiv.org/pdf/2003.13630.pdf\">TResNet: High-Performance GPU-Dedicated Architecture</a></p>\n\n<p>I've found some of the ideas of this paper really interesting.  And it was easy to try out with <a href=\"https://github.com/rwightman/pytorch-image-models\"><code>timm</code></a> packages, so I did on <a href=\"https://www.kaggle.com/shonenkov/training-cv-melanoma-starter\">Alex</a>. </p>\n\n<p>Training kernel: <a href=\"https://www.kaggle.com/ipythonx/melanoma-starter-tresnet-hp-gpu-dedicated-arch\">https://www.kaggle.com/ipythonx/melanoma-starter-tresnet-hp-gpu-dedicated-arch</a></p>\n\n<p>Eye-Catching:\n<code>\n- SpaceToDepth Stem\n- Anti-Alias Downsampling\n- In-Place Activated BatchNorm\n- New Block-type Selection\n- Optimized SE Layers\n</code></p>",
  "messages": [
    {
      "id": "949657",
      "postDate": "07/28/2020 19:25:06",
      "content": "<p>Paper: <a href=\"https://arxiv.org/pdf/2003.13630.pdf\">TResNet: High-Performance GPU-Dedicated Architecture</a></p>\n\n<p>I've found some of the ideas of this paper really interesting.  And it was easy to try out with <a href=\"https://github.com/rwightman/pytorch-image-models\"><code>timm</code></a> packages, so I did on <a href=\"https://www.kaggle.com/shonenkov/training-cv-melanoma-starter\">Alex</a>. </p>\n\n<p>Training kernel: <a href=\"https://www.kaggle.com/ipythonx/melanoma-starter-tresnet-hp-gpu-dedicated-arch\">https://www.kaggle.com/ipythonx/melanoma-starter-tresnet-hp-gpu-dedicated-arch</a></p>\n\n<p>Eye-Catching:\n<code>\n- SpaceToDepth Stem\n- Anti-Alias Downsampling\n- In-Place Activated BatchNorm\n- New Block-type Selection\n- Optimized SE Layers\n</code></p>",
      "rawMarkdown": "Paper: [TResNet: High-Performance GPU-Dedicated Architecture](https://arxiv.org/pdf/2003.13630.pdf)\n\nI've found some of the ideas of this paper really interesting.  And it was easy to try out with [`timm`](https://github.com/rwightman/pytorch-image-models) packages, so I did on [Alex](https://www.kaggle.com/shonenkov/training-cv-melanoma-starter). \n\nTraining kernel: https://www.kaggle.com/ipythonx/melanoma-starter-tresnet-hp-gpu-dedicated-arch\n\nEye-Catching:\n```\n- SpaceToDepth Stem\n- Anti-Alias Downsampling\n- In-Place Activated BatchNorm\n- New Block-type Selection\n- Optimized SE Layers\n```",
      "votes": null
    },
    {
      "id": "949676",
      "postDate": "07/28/2020 19:44:52",
      "content": "<p>Interesting that while their <a href=\"https://sotabench.com/benchmarks/image-classification-on-imagenet\">ImageNet accuracy</a> (52nd rank) is less than EfficientNets and FixEfficientNets,\nbut their inference speed is an order of magnitude faster!</p>\n\n<p>By the way, does this model work with TPUs or only the GPU Accelerator?</p>",
      "rawMarkdown": "Interesting that while their [ImageNet accuracy](https://sotabench.com/benchmarks/image-classification-on-imagenet) (52nd rank) is less than EfficientNets and FixEfficientNets,\nbut their inference speed is an order of magnitude faster!\n\nBy the way, does this model work with TPUs or only the GPU Accelerator?",
      "votes": null
    },
    {
      "id": "949688",
      "postDate": "07/28/2020 20:00:19",
      "content": "<p>hmm, if you go through the paperwork, section 3.4, they literally showed the comparison with EfficientNet models in the context of modeling-speed-accuracy. </p>",
      "rawMarkdown": "hmm, if you go through the paperwork, section 3.4, they literally showed the comparison with EfficientNet models in the context of modeling-speed-accuracy.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 949676,
      "author_name": "sirishks",
      "author_url": "",
      "post_date": "07/28/2020 19:44:52",
      "content": "<p>Interesting that while their <a href=\"https://sotabench.com/benchmarks/image-classification-on-imagenet\">ImageNet accuracy</a> (52nd rank) is less than EfficientNets and FixEfficientNets,\nbut their inference speed is an order of magnitude faster!</p>\n\n<p>By the way, does this model work with TPUs or only the GPU Accelerator?</p>",
      "votes": null,
      "replies": [
        {
          "id": 949688,
          "author_name": "ipythonx",
          "author_url": "",
          "post_date": "07/28/2020 20:00:19",
          "content": "<p>hmm, if you go through the paperwork, section 3.4, they literally showed the comparison with EfficientNet models in the context of modeling-speed-accuracy. </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "949657": "Paper: [TResNet: High-Performance GPU-Dedicated Architecture](https://arxiv.org/pdf/2003.13630.pdf)\n\nI've found some of the ideas of this paper really interesting.  And it was easy to try out with [`timm`](https://github.com/rwightman/pytorch-image-models) packages, so I did on [Alex](https://www.kaggle.com/shonenkov/training-cv-melanoma-starter). \n\nTraining kernel: https://www.kaggle.com/ipythonx/melanoma-starter-tresnet-hp-gpu-dedicated-arch\n\nEye-Catching:\n```\n- SpaceToDepth Stem\n- Anti-Alias Downsampling\n- In-Place Activated BatchNorm\n- New Block-type Selection\n- Optimized SE Layers\n```",
    "949676": "Interesting that while their [ImageNet accuracy](https://sotabench.com/benchmarks/image-classification-on-imagenet) (52nd rank) is less than EfficientNets and FixEfficientNets,\nbut their inference speed is an order of magnitude faster!\n\nBy the way, does this model work with TPUs or only the GPU Accelerator?",
    "949688": "hmm, if you go through the paperwork, section 3.4, they literally showed the comparison with EfficientNet models in the context of modeling-speed-accuracy."
  },
  "source": "meta"
}