{
  "id": 249826,
  "title": "What is the LB limit without TTA, Ensembles and External Data!?",
  "url": "/competitions/siim-covid19-detection/discussion/249826",
  "author_name": "",
  "post_date": "2021-06-30T01:30:51.044638100Z",
  "votes": 18,
  "comment_count": 10,
  "views": 0,
  "content": "<p>In case some of you are wondering just how much you can improve your score without tta, ensembles and external data, then the answer is pretty much .629…</p>\n<p>Thoughts and Questions are welcome.</p>",
  "messages": [
    {
      "id": "1370179",
      "postDate": "06/30/2021 01:30:51",
      "content": "<p>In case some of you are wondering just how much you can improve your score without tta, ensembles and external data, then the answer is pretty much .629…</p>\n<p>Thoughts and Questions are welcome.</p>",
      "rawMarkdown": "In case some of you are wondering just how much you can improve your score without tta, ensembles and external data, then the answer is pretty much .629...\n\nThoughts and Questions are welcome.",
      "votes": null
    },
    {
      "id": "1370193",
      "postDate": "06/30/2021 02:05:11",
      "content": "<blockquote>\n  <blockquote>\n    <p>answer is pretty much .629…</p>\n  </blockquote>\n</blockquote>\n<p>I agree on that. you probably can squeeze a bit more like +0.005.<br>\nbut maybe it is more \"efficient\" to try other methods</p>\n<p>time to move on to semi-supervised learning on external dataset!</p>\n<hr>\n<p>peformance is limited by data.<br>\nsometimes it is meaningless to compare with and without TTA/ensemble.</p>\n<p>if a model with TTA/ensemble can achieve x, it just means that another single model without TTA/ensemble can also achieve x.</p>\n<p>if x is the data limit, then that is the score all models will reached with or without TTA/ensemble</p>",
      "rawMarkdown": ">> answer is pretty much .629…\n\nI agree on that. you probably can squeeze a bit more like +0.005.\nbut maybe it is more \"efficient\" to try other methods\n\ntime to move on to semi-supervised learning on external dataset!\n\n---\n\npeformance is limited by data.\nsometimes it is meaningless to compare with and without TTA/ensemble.\n\nif a model with TTA/ensemble can achieve x, it just means that another single model without TTA/ensemble can also achieve x.\n\nif x is the data limit, then that is the score all models will reached with or without TTA/ensemble",
      "votes": null
    },
    {
      "id": "1370201",
      "postDate": "06/30/2021 02:32:45",
      "content": "<p>I agree Squeezing out +0.005 is theoretically possible which was what I was going after for last 3 weeks, but its nearly not practically possible, atleast at my current level… and it would be way more efficient to try things, in my opinion in a week or two pretty much everyone at the top lb is gonna start using external datasets and ensembles and we will be able to see the leaderboard shaking up places way more than we are seeing this month.</p>",
      "rawMarkdown": "I agree Squeezing out +0.005 is theoretically possible which was what I was going after for last 3 weeks, but its nearly not practically possible, atleast at my current level... and it would be way more efficient to try things, in my opinion in a week or two pretty much everyone at the top lb is gonna start using external datasets and ensembles and we will be able to see the leaderboard shaking up places way more than we are seeing this month.",
      "votes": null
    },
    {
      "id": "1370618",
      "postDate": "06/30/2021 09:19:38",
      "content": "<p>Good job on achieving this LB upper bound then ! I wonder if you attained this number using an aux loss for 4 class classification ? As in my case, it improved the CV but didn't improve LB at all.  </p>",
      "rawMarkdown": "Good job on achieving this LB upper bound then ! I wonder if you attained this number using an aux loss for 4 class classification ? As in my case, it improved the CV but didn't improve LB at all.",
      "votes": null
    },
    {
      "id": "1370753",
      "postDate": "06/30/2021 11:41:37",
      "content": "<p>I did use aux loss, there are many ways you can make a mask, suspecting that you are using heng's method which did not work for me either. I did use the same for 2 cls model too.</p>",
      "rawMarkdown": "I did use aux loss, there are many ways you can make a mask, suspecting that you are using heng's method which did not work for me either. I did use the same for 2 cls model too.",
      "votes": null
    },
    {
      "id": "1371074",
      "postDate": "06/30/2021 16:19:52",
      "content": "<p>My score without hflip tta is about .630, very close to your score.</p>",
      "rawMarkdown": "My score without hflip tta is about .630, very close to your score.",
      "votes": null
    },
    {
      "id": "1371232",
      "postDate": "06/30/2021 19:17:21",
      "content": "<p>Thanks for the info!</p>",
      "rawMarkdown": "Thanks for the info!",
      "votes": null
    },
    {
      "id": "1373143",
      "postDate": "07/02/2021 08:53:29",
      "content": "<p>what's your original score without these actions ?</p>",
      "rawMarkdown": "what's your original score without these actions ?",
      "votes": null
    },
    {
      "id": "1373464",
      "postDate": "07/02/2021 14:02:10",
      "content": "<p>Did not the get the question, please rephrase with clarity.</p>",
      "rawMarkdown": "Did not the get the question, please rephrase with clarity.",
      "votes": null
    },
    {
      "id": "1374102",
      "postDate": "07/03/2021 03:09:19",
      "content": "<p>I think he means he reach .629 just without these actions, showing how high score you can get without them. And you suppose he means reaching .629 with them.</p>",
      "rawMarkdown": "I think he means he reach .629 just without these actions, showing how high score you can get without them. And you suppose he means reaching .629 with them.",
      "votes": null
    },
    {
      "id": "1379967",
      "postDate": "07/07/2021 17:38:34",
      "content": "<p><a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a>  if it is alright with you i am curious what is the split b/w study level and image level scores in your final score of .63?</p>",
      "rawMarkdown": "steamedsheep  if it is alright with you i am curious what is the split b/w study level and image level scores in your final score of .63?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1370193,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "06/30/2021 02:05:11",
      "content": "<blockquote>\n  <blockquote>\n    <p>answer is pretty much .629…</p>\n  </blockquote>\n</blockquote>\n<p>I agree on that. you probably can squeeze a bit more like +0.005.<br>\nbut maybe it is more \"efficient\" to try other methods</p>\n<p>time to move on to semi-supervised learning on external dataset!</p>\n<hr>\n<p>peformance is limited by data.<br>\nsometimes it is meaningless to compare with and without TTA/ensemble.</p>\n<p>if a model with TTA/ensemble can achieve x, it just means that another single model without TTA/ensemble can also achieve x.</p>\n<p>if x is the data limit, then that is the score all models will reached with or without TTA/ensemble</p>",
      "votes": null,
      "replies": [
        {
          "id": 1370201,
          "author_name": "harshitsheoran",
          "author_url": "",
          "post_date": "06/30/2021 02:32:45",
          "content": "<p>I agree Squeezing out +0.005 is theoretically possible which was what I was going after for last 3 weeks, but its nearly not practically possible, atleast at my current level… and it would be way more efficient to try things, in my opinion in a week or two pretty much everyone at the top lb is gonna start using external datasets and ensembles and we will be able to see the leaderboard shaking up places way more than we are seeing this month.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1370618,
      "author_name": "tomdarmon",
      "author_url": "",
      "post_date": "06/30/2021 09:19:38",
      "content": "<p>Good job on achieving this LB upper bound then ! I wonder if you attained this number using an aux loss for 4 class classification ? As in my case, it improved the CV but didn't improve LB at all.  </p>",
      "votes": null,
      "replies": [
        {
          "id": 1370753,
          "author_name": "harshitsheoran",
          "author_url": "",
          "post_date": "06/30/2021 11:41:37",
          "content": "<p>I did use aux loss, there are many ways you can make a mask, suspecting that you are using heng's method which did not work for me either. I did use the same for 2 cls model too.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1371074,
      "author_name": "steamedsheep",
      "author_url": "",
      "post_date": "06/30/2021 16:19:52",
      "content": "<p>My score without hflip tta is about .630, very close to your score.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1371232,
          "author_name": "harshitsheoran",
          "author_url": "",
          "post_date": "06/30/2021 19:17:21",
          "content": "<p>Thanks for the info!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1379967,
          "author_name": "varundutt9213",
          "author_url": "",
          "post_date": "07/07/2021 17:38:34",
          "content": "<p><a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a>  if it is alright with you i am curious what is the split b/w study level and image level scores in your final score of .63?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1373143,
      "author_name": "southsakura",
      "author_url": "",
      "post_date": "07/02/2021 08:53:29",
      "content": "<p>what's your original score without these actions ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1373464,
          "author_name": "harshitsheoran",
          "author_url": "",
          "post_date": "07/02/2021 14:02:10",
          "content": "<p>Did not the get the question, please rephrase with clarity.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1374102,
          "author_name": "louieshao",
          "author_url": "",
          "post_date": "07/03/2021 03:09:19",
          "content": "<p>I think he means he reach .629 just without these actions, showing how high score you can get without them. And you suppose he means reaching .629 with them.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1370179": "In case some of you are wondering just how much you can improve your score without tta, ensembles and external data, then the answer is pretty much .629...\n\nThoughts and Questions are welcome.",
    "1370193": ">> answer is pretty much .629…\n\nI agree on that. you probably can squeeze a bit more like +0.005.\nbut maybe it is more \"efficient\" to try other methods\n\ntime to move on to semi-supervised learning on external dataset!\n\n---\n\npeformance is limited by data.\nsometimes it is meaningless to compare with and without TTA/ensemble.\n\nif a model with TTA/ensemble can achieve x, it just means that another single model without TTA/ensemble can also achieve x.\n\nif x is the data limit, then that is the score all models will reached with or without TTA/ensemble",
    "1370201": "I agree Squeezing out +0.005 is theoretically possible which was what I was going after for last 3 weeks, but its nearly not practically possible, atleast at my current level... and it would be way more efficient to try things, in my opinion in a week or two pretty much everyone at the top lb is gonna start using external datasets and ensembles and we will be able to see the leaderboard shaking up places way more than we are seeing this month.",
    "1370618": "Good job on achieving this LB upper bound then ! I wonder if you attained this number using an aux loss for 4 class classification ? As in my case, it improved the CV but didn't improve LB at all.",
    "1370753": "I did use aux loss, there are many ways you can make a mask, suspecting that you are using heng's method which did not work for me either. I did use the same for 2 cls model too.",
    "1371074": "My score without hflip tta is about .630, very close to your score.",
    "1371232": "Thanks for the info!",
    "1373143": "what's your original score without these actions ?",
    "1373464": "Did not the get the question, please rephrase with clarity.",
    "1374102": "I think he means he reach .629 just without these actions, showing how high score you can get without them. And you suppose he means reaching .629 with them.",
    "1379967": "steamedsheep  if it is alright with you i am curious what is the split b/w study level and image level scores in your final score of .63?"
  },
  "source": "meta"
}