{
  "id": 111777,
  "title": "Adding TTA to the model before optimisation could help",
  "url": "/competitions/understanding_cloud_organization/discussion/111777",
  "author_name": "",
  "post_date": "2019-10-08T23:15:25.320083700Z",
  "votes": 17,
  "comment_count": 20,
  "views": 0,
  "content": "<p>With TTA and optimisation, I have increased my 0.598 resnet18 prediction to 0.653, refering to V13 of my public kernel.</p>",
  "messages": [
    {
      "id": "644514",
      "postDate": "10/08/2019 23:15:25",
      "content": "<p>With TTA and optimisation, I have increased my 0.598 resnet18 prediction to 0.653, refering to V13 of my public kernel.</p>",
      "rawMarkdown": "With TTA and optimisation, I have increased my 0.598 resnet18 prediction to 0.653, refering to V13 of my public kernel.",
      "votes": null
    },
    {
      "id": "644599",
      "postDate": "10/09/2019 03:50:09",
      "content": "<p>sounds interesting. Can you elaborate more on what is TTA in this case and how exactly can help?\nregards,\nA</p>",
      "rawMarkdown": "sounds interesting. Can you elaborate more on what is TTA in this case and how exactly can help?\nregards,\nA",
      "votes": null
    },
    {
      "id": "644626",
      "postDate": "10/09/2019 05:22:31",
      "content": "<p>Hi, thanks! Which types of TTAs did you use?</p>",
      "rawMarkdown": "Hi, thanks! Which types of TTAs did you use?",
      "votes": null
    },
    {
      "id": "644656",
      "postDate": "10/09/2019 06:26:11",
      "content": "<p>TTA - Test Time Augmentation\nBasically, you apply some transformations to your test images before making a prediction. Maybe, make predictions for multiple copies of your test image (after augmentations) and take max/average of all.</p>",
      "rawMarkdown": "TTA - Test Time Augmentation\nBasically, you apply some transformations to your test images before making a prediction. Maybe, make predictions for multiple copies of your test image (after augmentations) and take max/average of all.",
      "votes": null
    },
    {
      "id": "644762",
      "postDate": "10/09/2019 09:50:56",
      "content": "<p><a href=\"/gogo827jz\">@gogo827jz</a> Great idea, thanks for sharing!</p>\n\n<p>Code is in the public kernel <a href=\"https://www.kaggle.com/gogo827jz/resunet-keras-with-some-new-ideas\">ResUNet Keras with some new ideas</a>. </p>\n\n<p>The qubvel's <a href=\"https://github.com/qubvel/tta_wrapper\">TTA wrapper</a> is used:\n<code>model = tta_segmentation(model, h_flip=True, h_shift=(-10, 10), merge='mean')</code></p>",
      "rawMarkdown": "gogo827jz Great idea, thanks for sharing!\n\nCode is in the public kernel [ResUNet Keras with some new ideas](https://www.kaggle.com/gogo827jz/resunet-keras-with-some-new-ideas). \n\nThe qubvel's [TTA wrapper](https://github.com/qubvel/tta_wrapper) is used:\n```model = tta_segmentation(model, h_flip=True, h_shift=(-10, 10), merge='mean')```",
      "votes": null
    },
    {
      "id": "644821",
      "postDate": "10/09/2019 11:52:22",
      "content": "<p>Without TTA and with the same threshold, the score is 0.650.</p>",
      "rawMarkdown": "Without TTA and with the same threshold, the score is 0.650.",
      "votes": null
    },
    {
      "id": "644822",
      "postDate": "10/09/2019 11:53:39",
      "content": "<p>You can also try different types of parameters. Mine may not be the optimal.</p>",
      "rawMarkdown": "You can also try different types of parameters. Mine may not be the optimal.",
      "votes": null
    },
    {
      "id": "644832",
      "postDate": "10/09/2019 12:10:18",
      "content": "<p>Since our models have learnt augmentatioms of images during training, we can also apply augmentations when predicting on test data and takes the average. This helps on the bias-variance tradeoff.</p>",
      "rawMarkdown": "Since our models have learnt augmentatioms of images during training, we can also apply augmentations when predicting on test data and takes the average. This helps on the bias-variance tradeoff.",
      "votes": null
    },
    {
      "id": "644863",
      "postDate": "10/09/2019 12:47:14",
      "content": "<p>Nice <a href=\"/gogo827jz\">@gogo827jz</a> , I was holding a bit to try TTA, but I guess I also should be already doing.</p>",
      "rawMarkdown": "Nice @gogo827jz , I was holding a bit to try TTA, but I guess I also should be already doing.",
      "votes": null
    },
    {
      "id": "644878",
      "postDate": "10/09/2019 13:09:41",
      "content": "<p>thanks a lot. Knew the terminology in general, it was interesting the application in your case. \nDoes it mean that if the time is the bigger - than the results should be better?\nthanks\nA</p>",
      "rawMarkdown": "thanks a lot. Knew the terminology in general, it was interesting the application in your case. \nDoes it mean that if the time is the bigger - than the results should be better?\nthanks\nA",
      "votes": null
    },
    {
      "id": "644879",
      "postDate": "10/09/2019 13:10:51",
      "content": "<p>Normally, it can't help increase your score that much. I was also surprised that it can get 0.653 without too much tuning.</p>",
      "rawMarkdown": "Normally, it can't help increase your score that much. I was also surprised that it can get 0.653 without too much tuning.",
      "votes": null
    },
    {
      "id": "644888",
      "postDate": "10/09/2019 13:34:02",
      "content": "<p>What do you mean by the time is bigger?</p>",
      "rawMarkdown": "What do you mean by the time is bigger?",
      "votes": null
    },
    {
      "id": "644890",
      "postDate": "10/09/2019 13:35:31",
      "content": "<p>processed in more time </p>",
      "rawMarkdown": "processed in more time",
      "votes": null
    },
    {
      "id": "644891",
      "postDate": "10/09/2019 13:37:30",
      "content": "<p>Agreed, but definitely it's worth the try 😄 .</p>",
      "rawMarkdown": "Agreed, but definitely it's worth the try 😄 .",
      "votes": null
    },
    {
      "id": "644892",
      "postDate": "10/09/2019 13:40:05",
      "content": "<p>If you mean adding more TTA options, I would say it depends. I have also tried more options such as <code>v_flip</code> and <code>v_shift</code>, however, they are not that helpful.</p>",
      "rawMarkdown": "If you mean adding more TTA options, I would say it depends. I have also tried more options such as `v_flip` and `v_shift`, however, they are not that helpful.",
      "votes": null
    },
    {
      "id": "647555",
      "postDate": "10/12/2019 20:50:11",
      "content": "<p><a href=\"/gogo827jz\">@gogo827jz</a> One question about how you use TTA, do you do threshold and mask size search with the regular model or do you search with the model wrapped on TTA, I feel this is important because the TTA wrapped model masks seem to be more reliable.</p>",
      "rawMarkdown": "gogo827jz One question about how you use TTA, do you do threshold and mask size search with the regular model or do you search with the model wrapped on TTA, I feel this is important because the TTA wrapped model masks seem to be more reliable.",
      "votes": null
    },
    {
      "id": "648728",
      "postDate": "10/14/2019 14:54:58",
      "content": "<p>I searched the threshold with TTA-wrapped model in my public kernel. But now, I just get lazy and start to use 0.5 because I don't want to search the threshold using OOF for CV models and it usually takes really long time...</p>",
      "rawMarkdown": "I searched the threshold with TTA-wrapped model in my public kernel. But now, I just get lazy and start to use 0.5 because I don't want to search the threshold using OOF for CV models and it usually takes really long time...",
      "votes": null
    },
    {
      "id": "649287",
      "postDate": "10/15/2019 07:18:13",
      "content": "<p><a href=\"/gogo827jz\">@gogo827jz</a> thanks for sharing your approach and tta code. I tried the same and it improved my score from lower 0.658 to upper 0.658. Can you share some more tips and tricks, I'm just stuck at 0.658 score.\nThanks and Happy Kaggling! :)</p>",
      "rawMarkdown": "gogo827jz thanks for sharing your approach and tta code. I tried the same and it improved my score from lower 0.658 to upper 0.658. Can you share some more tips and tricks, I'm just stuck at 0.658 score.\nThanks and Happy Kaggling! :)",
      "votes": null
    },
    {
      "id": "649435",
      "postDate": "10/15/2019 10:46:53",
      "content": "<p>Thanks. Classification is important.</p>",
      "rawMarkdown": "Thanks. Classification is important.",
      "votes": null
    },
    {
      "id": "658501",
      "postDate": "10/26/2019 04:24:49",
      "content": "<p>Hi, zhang, which version of Resnet18 get 0.653,  segment or classify?</p>",
      "rawMarkdown": "Hi, zhang, which version of Resnet18 get 0.653,  segment or classify?",
      "votes": null
    },
    {
      "id": "658691",
      "postDate": "10/26/2019 11:31:07",
      "content": "<p>only segmentation.</p>",
      "rawMarkdown": "only segmentation.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 644599,
      "author_name": "zinovadr",
      "author_url": "",
      "post_date": "10/09/2019 03:50:09",
      "content": "<p>sounds interesting. Can you elaborate more on what is TTA in this case and how exactly can help?\nregards,\nA</p>",
      "votes": null,
      "replies": [
        {
          "id": 644656,
          "author_name": "timetraveller98",
          "author_url": "",
          "post_date": "10/09/2019 06:26:11",
          "content": "<p>TTA - Test Time Augmentation\nBasically, you apply some transformations to your test images before making a prediction. Maybe, make predictions for multiple copies of your test image (after augmentations) and take max/average of all.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 644832,
          "author_name": "gogo827jz",
          "author_url": "",
          "post_date": "10/09/2019 12:10:18",
          "content": "<p>Since our models have learnt augmentatioms of images during training, we can also apply augmentations when predicting on test data and takes the average. This helps on the bias-variance tradeoff.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 644878,
          "author_name": "zinovadr",
          "author_url": "",
          "post_date": "10/09/2019 13:09:41",
          "content": "<p>thanks a lot. Knew the terminology in general, it was interesting the application in your case. \nDoes it mean that if the time is the bigger - than the results should be better?\nthanks\nA</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 644888,
          "author_name": "gogo827jz",
          "author_url": "",
          "post_date": "10/09/2019 13:34:02",
          "content": "<p>What do you mean by the time is bigger?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 644890,
          "author_name": "zinovadr",
          "author_url": "",
          "post_date": "10/09/2019 13:35:31",
          "content": "<p>processed in more time </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 644892,
          "author_name": "gogo827jz",
          "author_url": "",
          "post_date": "10/09/2019 13:40:05",
          "content": "<p>If you mean adding more TTA options, I would say it depends. I have also tried more options such as <code>v_flip</code> and <code>v_shift</code>, however, they are not that helpful.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 644626,
      "author_name": "siarheifedartsou",
      "author_url": "",
      "post_date": "10/09/2019 05:22:31",
      "content": "<p>Hi, thanks! Which types of TTAs did you use?</p>",
      "votes": null,
      "replies": [
        {
          "id": 644762,
          "author_name": "alt250",
          "author_url": "",
          "post_date": "10/09/2019 09:50:56",
          "content": "<p><a href=\"/gogo827jz\">@gogo827jz</a> Great idea, thanks for sharing!</p>\n\n<p>Code is in the public kernel <a href=\"https://www.kaggle.com/gogo827jz/resunet-keras-with-some-new-ideas\">ResUNet Keras with some new ideas</a>. </p>\n\n<p>The qubvel's <a href=\"https://github.com/qubvel/tta_wrapper\">TTA wrapper</a> is used:\n<code>model = tta_segmentation(model, h_flip=True, h_shift=(-10, 10), merge='mean')</code></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 644822,
          "author_name": "gogo827jz",
          "author_url": "",
          "post_date": "10/09/2019 11:53:39",
          "content": "<p>You can also try different types of parameters. Mine may not be the optimal.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 644821,
      "author_name": "gogo827jz",
      "author_url": "",
      "post_date": "10/09/2019 11:52:22",
      "content": "<p>Without TTA and with the same threshold, the score is 0.650.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 644863,
      "author_name": "dimitreoliveira",
      "author_url": "",
      "post_date": "10/09/2019 12:47:14",
      "content": "<p>Nice <a href=\"/gogo827jz\">@gogo827jz</a> , I was holding a bit to try TTA, but I guess I also should be already doing.</p>",
      "votes": null,
      "replies": [
        {
          "id": 644879,
          "author_name": "gogo827jz",
          "author_url": "",
          "post_date": "10/09/2019 13:10:51",
          "content": "<p>Normally, it can't help increase your score that much. I was also surprised that it can get 0.653 without too much tuning.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 644891,
          "author_name": "dimitreoliveira",
          "author_url": "",
          "post_date": "10/09/2019 13:37:30",
          "content": "<p>Agreed, but definitely it's worth the try 😄 .</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 647555,
      "author_name": "dimitreoliveira",
      "author_url": "",
      "post_date": "10/12/2019 20:50:11",
      "content": "<p><a href=\"/gogo827jz\">@gogo827jz</a> One question about how you use TTA, do you do threshold and mask size search with the regular model or do you search with the model wrapped on TTA, I feel this is important because the TTA wrapped model masks seem to be more reliable.</p>",
      "votes": null,
      "replies": [
        {
          "id": 648728,
          "author_name": "gogo827jz",
          "author_url": "",
          "post_date": "10/14/2019 14:54:58",
          "content": "<p>I searched the threshold with TTA-wrapped model in my public kernel. But now, I just get lazy and start to use 0.5 because I don't want to search the threshold using OOF for CV models and it usually takes really long time...</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 649287,
      "author_name": "adish333",
      "author_url": "",
      "post_date": "10/15/2019 07:18:13",
      "content": "<p><a href=\"/gogo827jz\">@gogo827jz</a> thanks for sharing your approach and tta code. I tried the same and it improved my score from lower 0.658 to upper 0.658. Can you share some more tips and tricks, I'm just stuck at 0.658 score.\nThanks and Happy Kaggling! :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 649435,
          "author_name": "gogo827jz",
          "author_url": "",
          "post_date": "10/15/2019 10:46:53",
          "content": "<p>Thanks. Classification is important.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 658501,
      "author_name": "cswwp347724",
      "author_url": "",
      "post_date": "10/26/2019 04:24:49",
      "content": "<p>Hi, zhang, which version of Resnet18 get 0.653,  segment or classify?</p>",
      "votes": null,
      "replies": [
        {
          "id": 658691,
          "author_name": "gogo827jz",
          "author_url": "",
          "post_date": "10/26/2019 11:31:07",
          "content": "<p>only segmentation.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "644514": "With TTA and optimisation, I have increased my 0.598 resnet18 prediction to 0.653, refering to V13 of my public kernel.",
    "644599": "sounds interesting. Can you elaborate more on what is TTA in this case and how exactly can help?\nregards,\nA",
    "644626": "Hi, thanks! Which types of TTAs did you use?",
    "644656": "TTA - Test Time Augmentation\nBasically, you apply some transformations to your test images before making a prediction. Maybe, make predictions for multiple copies of your test image (after augmentations) and take max/average of all.",
    "644762": "gogo827jz Great idea, thanks for sharing!\n\nCode is in the public kernel [ResUNet Keras with some new ideas](https://www.kaggle.com/gogo827jz/resunet-keras-with-some-new-ideas). \n\nThe qubvel's [TTA wrapper](https://github.com/qubvel/tta_wrapper) is used:\n```model = tta_segmentation(model, h_flip=True, h_shift=(-10, 10), merge='mean')```",
    "644821": "Without TTA and with the same threshold, the score is 0.650.",
    "644822": "You can also try different types of parameters. Mine may not be the optimal.",
    "644832": "Since our models have learnt augmentatioms of images during training, we can also apply augmentations when predicting on test data and takes the average. This helps on the bias-variance tradeoff.",
    "644863": "Nice @gogo827jz , I was holding a bit to try TTA, but I guess I also should be already doing.",
    "644878": "thanks a lot. Knew the terminology in general, it was interesting the application in your case. \nDoes it mean that if the time is the bigger - than the results should be better?\nthanks\nA",
    "644879": "Normally, it can't help increase your score that much. I was also surprised that it can get 0.653 without too much tuning.",
    "644888": "What do you mean by the time is bigger?",
    "644890": "processed in more time",
    "644891": "Agreed, but definitely it's worth the try 😄 .",
    "644892": "If you mean adding more TTA options, I would say it depends. I have also tried more options such as `v_flip` and `v_shift`, however, they are not that helpful.",
    "647555": "gogo827jz One question about how you use TTA, do you do threshold and mask size search with the regular model or do you search with the model wrapped on TTA, I feel this is important because the TTA wrapped model masks seem to be more reliable.",
    "648728": "I searched the threshold with TTA-wrapped model in my public kernel. But now, I just get lazy and start to use 0.5 because I don't want to search the threshold using OOF for CV models and it usually takes really long time...",
    "649287": "gogo827jz thanks for sharing your approach and tta code. I tried the same and it improved my score from lower 0.658 to upper 0.658. Can you share some more tips and tricks, I'm just stuck at 0.658 score.\nThanks and Happy Kaggling! :)",
    "649435": "Thanks. Classification is important.",
    "658501": "Hi, zhang, which version of Resnet18 get 0.653,  segment or classify?",
    "658691": "only segmentation."
  },
  "source": "meta"
}