{
  "id": 207615,
  "title": "A quick way to do TTA with pytorch and how I do it!",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/207615",
  "author_name": "",
  "post_date": "2020-12-30T15:03:32.113073800Z",
  "votes": 13,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I have seen a few people talking about how to do TTA so i wanted to share how I do it with pytorch. This might not be the best way but it works..</p>\n<p>First set up your augs (here im using albumentations). Im using HFlip, VFlip and Transpose as additionnal augmentations of the image:</p>\n<pre><code>transforms_test = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA1 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.HorizontalFlip(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA2 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.VerticalFlip(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA3 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.Transpose(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n</code></pre>\n<p>Then we need to change the dataset to return a stack of frames:</p>\n<pre><code>class CASSAVADataset(Dataset):\n    def __init__(self, df,image_path, transform):\n        self.transform = transform\n        self.df=df\n        self.image_path=image_path\n\n    def __len__(self) -&gt; int:\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        img_id, label, fold  = self.df.loc[idx,['image_id','label','fold']].values\n        img_path=self.image_path+img_id\n        image = cv2.imread(img_path)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n        imglist=[]\n        imglist.append(self.transform(image=image)['image'])\n        imglist.append(TTA1(image=image)['image'])\n        imglist.append(TTA2(image=image)['image'])\n        imglist.append(TTA3(image=image)['image'])\n        image=torch.stack(imglist)\n\n        return image, img_id\n</code></pre>\n<p>Then we change the shape of the input so it can be foward passed in the model and do the mean of all the softmax values:</p>\n<pre><code>for images,image_names in tqdm(loader):\n        images = images.cuda()\n        with torch.no_grad():\n            batch_size, n_crops, c, h, w = images.size()\n            images = images.view(-1, c, h, w)\n            output= F.softmax(model(images),dim=1)\n            output = output.view(batch_size, n_crops,-1).mean(1)\n            pred = output.argmax(1).cpu().numpy()\n</code></pre>\n<p>Hope this helped. Im not sure its the best way but it works :)</p>",
  "messages": [
    {
      "id": "1132643",
      "postDate": "12/30/2020 15:03:32",
      "content": "<p>I have seen a few people talking about how to do TTA so i wanted to share how I do it with pytorch. This might not be the best way but it works..</p>\n<p>First set up your augs (here im using albumentations). Im using HFlip, VFlip and Transpose as additionnal augmentations of the image:</p>\n<pre><code>transforms_test = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA1 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.HorizontalFlip(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA2 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.VerticalFlip(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA3 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.Transpose(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n</code></pre>\n<p>Then we need to change the dataset to return a stack of frames:</p>\n<pre><code>class CASSAVADataset(Dataset):\n    def __init__(self, df,image_path, transform):\n        self.transform = transform\n        self.df=df\n        self.image_path=image_path\n\n    def __len__(self) -&gt; int:\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        img_id, label, fold  = self.df.loc[idx,['image_id','label','fold']].values\n        img_path=self.image_path+img_id\n        image = cv2.imread(img_path)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n        imglist=[]\n        imglist.append(self.transform(image=image)['image'])\n        imglist.append(TTA1(image=image)['image'])\n        imglist.append(TTA2(image=image)['image'])\n        imglist.append(TTA3(image=image)['image'])\n        image=torch.stack(imglist)\n\n        return image, img_id\n</code></pre>\n<p>Then we change the shape of the input so it can be foward passed in the model and do the mean of all the softmax values:</p>\n<pre><code>for images,image_names in tqdm(loader):\n        images = images.cuda()\n        with torch.no_grad():\n            batch_size, n_crops, c, h, w = images.size()\n            images = images.view(-1, c, h, w)\n            output= F.softmax(model(images),dim=1)\n            output = output.view(batch_size, n_crops,-1).mean(1)\n            pred = output.argmax(1).cpu().numpy()\n</code></pre>\n<p>Hope this helped. Im not sure its the best way but it works :)</p>",
      "rawMarkdown": "I have seen a few people talking about how to do TTA so i wanted to share how I do it with pytorch. This might not be the best way but it works..\n\nFirst set up your augs (here im using albumentations). Im using HFlip, VFlip and Transpose as additionnal augmentations of the image:\n\n```\ntransforms_test = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA1 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.HorizontalFlip(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA2 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.VerticalFlip(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA3 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.Transpose(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n```\n\nThen we need to change the dataset to return a stack of frames:\n\n```\nclass CASSAVADataset(Dataset):\n    def __init__(self, df,image_path, transform):\n        self.transform = transform\n        self.df=df\n        self.image_path=image_path\n        \n    def __len__(self) -> int:\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        img_id, label, fold  = self.df.loc[idx,['image_id','label','fold']].values\n        img_path=self.image_path+img_id\n        image = cv2.imread(img_path)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n        \n        imglist=[]\n        imglist.append(self.transform(image=image)['image'])\n        imglist.append(TTA1(image=image)['image'])\n        imglist.append(TTA2(image=image)['image'])\n        imglist.append(TTA3(image=image)['image'])\n        image=torch.stack(imglist)\n        \n        return image, img_id\n```\n\nThen we change the shape of the input so it can be foward passed in the model and do the mean of all the softmax values:\n\n```\nfor images,image_names in tqdm(loader):\n        images = images.cuda()\n        with torch.no_grad():\n            batch_size, n_crops, c, h, w = images.size()\n            images = images.view(-1, c, h, w)\n            output= F.softmax(model(images),dim=1)\n            output = output.view(batch_size, n_crops,-1).mean(1)\n            pred = output.argmax(1).cpu().numpy()\n```\n\nHope this helped. Im not sure its the best way but it works :)",
      "votes": null
    },
    {
      "id": "1132682",
      "postDate": "12/30/2020 15:56:42",
      "content": "<p>You can speed up as follows</p>\n<pre><code>class CASSAVADataset(Dataset):\n    def __init__(self, df,image_path, transform):\n        self.transform = transform\n        self.df=data.values  # update\n        self.image_path=image_path\n        self.ttas = [self.transform, TTA1, TTA2, TTA3]\n\n    def __len__(self) -&gt; int:\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        img_id, label = self.data[idx, :2].values  # update\n        img_path=self.image_path+img_id\n        image = cv2.imread(img_path)\n        image = image[:, :, ::-1]  # update\n\n        imglist=[tta(image=image)['image'] for tta in self.ttas]  # update\n\n        image=torch.stack(imglist)\n\n        return image, img_id\n</code></pre>\n<p>Tips</p>\n<ul>\n<li><code>df.loc</code> is lower than <code>numpy</code>. I changed <code>df</code> as <code>data</code> because df is no longer <code>DataFrame</code>.</li>\n<li>when converting bgr to rgb, using slice may speed up a little.</li>\n<li>list comprehension is faster than <code>append</code>.<br>\nI hope this helps you :)</li>\n</ul>",
      "rawMarkdown": "You can speed up as follows\n```\nclass CASSAVADataset(Dataset):\n    def __init__(self, df,image_path, transform):\n        self.transform = transform\n        self.df=data.values  # update\n        self.image_path=image_path\n        self.ttas = [self.transform, TTA1, TTA2, TTA3]\n\n    def __len__(self) -> int:\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        img_id, label = self.data[idx, :2].values  # update\n        img_path=self.image_path+img_id\n        image = cv2.imread(img_path)\n        image = image[:, :, ::-1]  # update\n\n        imglist=[tta(image=image)['image'] for tta in self.ttas]  # update\n\n        image=torch.stack(imglist)\n\n        return image, img_id\n```\nTips\n* `df.loc` is lower than `numpy`. I changed `df` as `data` because df is no longer `DataFrame`.\n* when converting bgr to rgb, using slice may speed up a little.\n* list comprehension is faster than `append`.\nI hope this helps you :)",
      "votes": null
    },
    {
      "id": "1132686",
      "postDate": "12/30/2020 15:58:54",
      "content": "<p>Thanks for your help, always nice to see what others do! :)</p>",
      "rawMarkdown": "Thanks for your help, always nice to see what others do! :)",
      "votes": null
    },
    {
      "id": "1185156",
      "postDate": "02/04/2021 02:00:02",
      "content": "<p>Thank you for sharing nice code! Let me make sure one thing.<br>\nWhy you use (not \"Resize\" but) \"CenterCrop\" in every TTA?<br>\nI'm not sure, but I think if you use CenterCrop, models can't see the edge of the image.<br>\nThanks!</p>",
      "rawMarkdown": "Thank you for sharing nice code! Let me make sure one thing.\nWhy you use (not \"Resize\" but) \"CenterCrop\" in every TTA?\nI'm not sure, but I think if you use CenterCrop, models can't see the edge of the image.\nThanks!",
      "votes": null
    },
    {
      "id": "1185206",
      "postDate": "02/04/2021 03:12:09",
      "content": "<p>I used to use resize but im getting a better score with centercrop!</p>",
      "rawMarkdown": "I used to use resize but im getting a better score with centercrop!",
      "votes": null
    },
    {
      "id": "1185216",
      "postDate": "02/04/2021 03:32:59",
      "content": "<p>I got it! Thank you very much :)</p>",
      "rawMarkdown": "I got it! Thank you very much :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1132682,
      "author_name": "tmhrkt",
      "author_url": "",
      "post_date": "12/30/2020 15:56:42",
      "content": "<p>You can speed up as follows</p>\n<pre><code>class CASSAVADataset(Dataset):\n    def __init__(self, df,image_path, transform):\n        self.transform = transform\n        self.df=data.values  # update\n        self.image_path=image_path\n        self.ttas = [self.transform, TTA1, TTA2, TTA3]\n\n    def __len__(self) -&gt; int:\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        img_id, label = self.data[idx, :2].values  # update\n        img_path=self.image_path+img_id\n        image = cv2.imread(img_path)\n        image = image[:, :, ::-1]  # update\n\n        imglist=[tta(image=image)['image'] for tta in self.ttas]  # update\n\n        image=torch.stack(imglist)\n\n        return image, img_id\n</code></pre>\n<p>Tips</p>\n<ul>\n<li><code>df.loc</code> is lower than <code>numpy</code>. I changed <code>df</code> as <code>data</code> because df is no longer <code>DataFrame</code>.</li>\n<li>when converting bgr to rgb, using slice may speed up a little.</li>\n<li>list comprehension is faster than <code>append</code>.<br>\nI hope this helps you :)</li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 1132686,
          "author_name": "yannmajewski",
          "author_url": "",
          "post_date": "12/30/2020 15:58:54",
          "content": "<p>Thanks for your help, always nice to see what others do! :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1185156,
      "author_name": "tt0721",
      "author_url": "",
      "post_date": "02/04/2021 02:00:02",
      "content": "<p>Thank you for sharing nice code! Let me make sure one thing.<br>\nWhy you use (not \"Resize\" but) \"CenterCrop\" in every TTA?<br>\nI'm not sure, but I think if you use CenterCrop, models can't see the edge of the image.<br>\nThanks!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1185206,
          "author_name": "yannmajewski",
          "author_url": "",
          "post_date": "02/04/2021 03:12:09",
          "content": "<p>I used to use resize but im getting a better score with centercrop!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1185216,
          "author_name": "tt0721",
          "author_url": "",
          "post_date": "02/04/2021 03:32:59",
          "content": "<p>I got it! Thank you very much :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1132643": "I have seen a few people talking about how to do TTA so i wanted to share how I do it with pytorch. This might not be the best way but it works..\n\nFirst set up your augs (here im using albumentations). Im using HFlip, VFlip and Transpose as additionnal augmentations of the image:\n\n```\ntransforms_test = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA1 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.HorizontalFlip(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA2 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.VerticalFlip(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n\nTTA3 = A.Compose([\n    A.CenterCrop(img_size,img_size,p=1),\n    A.Transpose(p=1),\n    A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),\n    ToTensorV2(),\n])\n```\n\nThen we need to change the dataset to return a stack of frames:\n\n```\nclass CASSAVADataset(Dataset):\n    def __init__(self, df,image_path, transform):\n        self.transform = transform\n        self.df=df\n        self.image_path=image_path\n        \n    def __len__(self) -> int:\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        img_id, label, fold  = self.df.loc[idx,['image_id','label','fold']].values\n        img_path=self.image_path+img_id\n        image = cv2.imread(img_path)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n        \n        imglist=[]\n        imglist.append(self.transform(image=image)['image'])\n        imglist.append(TTA1(image=image)['image'])\n        imglist.append(TTA2(image=image)['image'])\n        imglist.append(TTA3(image=image)['image'])\n        image=torch.stack(imglist)\n        \n        return image, img_id\n```\n\nThen we change the shape of the input so it can be foward passed in the model and do the mean of all the softmax values:\n\n```\nfor images,image_names in tqdm(loader):\n        images = images.cuda()\n        with torch.no_grad():\n            batch_size, n_crops, c, h, w = images.size()\n            images = images.view(-1, c, h, w)\n            output= F.softmax(model(images),dim=1)\n            output = output.view(batch_size, n_crops,-1).mean(1)\n            pred = output.argmax(1).cpu().numpy()\n```\n\nHope this helped. Im not sure its the best way but it works :)",
    "1132682": "You can speed up as follows\n```\nclass CASSAVADataset(Dataset):\n    def __init__(self, df,image_path, transform):\n        self.transform = transform\n        self.df=data.values  # update\n        self.image_path=image_path\n        self.ttas = [self.transform, TTA1, TTA2, TTA3]\n\n    def __len__(self) -> int:\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        img_id, label = self.data[idx, :2].values  # update\n        img_path=self.image_path+img_id\n        image = cv2.imread(img_path)\n        image = image[:, :, ::-1]  # update\n\n        imglist=[tta(image=image)['image'] for tta in self.ttas]  # update\n\n        image=torch.stack(imglist)\n\n        return image, img_id\n```\nTips\n* `df.loc` is lower than `numpy`. I changed `df` as `data` because df is no longer `DataFrame`.\n* when converting bgr to rgb, using slice may speed up a little.\n* list comprehension is faster than `append`.\nI hope this helps you :)",
    "1132686": "Thanks for your help, always nice to see what others do! :)",
    "1185156": "Thank you for sharing nice code! Let me make sure one thing.\nWhy you use (not \"Resize\" but) \"CenterCrop\" in every TTA?\nI'm not sure, but I think if you use CenterCrop, models can't see the edge of the image.\nThanks!",
    "1185206": "I used to use resize but im getting a better score with centercrop!",
    "1185216": "I got it! Thank you very much :)"
  },
  "source": "meta"
}