{
  "id": 65137,
  "title": "Experiment: If your confidence is low, does shrinking bounding box improve score?",
  "url": "/competitions/rsna-pneumonia-detection-challenge/discussion/65137",
  "author_name": "",
  "post_date": "2018-09-06T15:03:05.466867800Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I had this question, so I tested it.</p>\n\n<p>The result is: no, it does not affect the public LB score. I got a 0.001 improvement from the public kernel, which does not mean very much.</p>\n\n<p>You can view the kernel here: <a href=\"https://www.kaggle.com/returnofsputnik/smaller-bounding-boxes-better-score\">https://www.kaggle.com/returnofsputnik/smaller-bounding-boxes-better-score</a></p>\n\n<p>Please let me know if you think I made some mistakes. Also let me know any opinions or questions you have about the experiment</p>",
  "messages": [
    {
      "id": "382541",
      "postDate": "09/06/2018 15:03:05",
      "content": "<p>I had this question, so I tested it.</p>\n\n<p>The result is: no, it does not affect the public LB score. I got a 0.001 improvement from the public kernel, which does not mean very much.</p>\n\n<p>You can view the kernel here: <a href=\"https://www.kaggle.com/returnofsputnik/smaller-bounding-boxes-better-score\">https://www.kaggle.com/returnofsputnik/smaller-bounding-boxes-better-score</a></p>\n\n<p>Please let me know if you think I made some mistakes. Also let me know any opinions or questions you have about the experiment</p>",
      "rawMarkdown": "I had this question, so I tested it.\n\nThe result is: no, it does not affect the public LB score. I got a 0.001 improvement from the public kernel, which does not mean very much.\n\nYou can view the kernel here: https://www.kaggle.com/returnofsputnik/smaller-bounding-boxes-better-score\n\nPlease let me know if you think I made some mistakes. Also let me know any opinions or questions you have about the experiment",
      "votes": null
    },
    {
      "id": "382571",
      "postDate": "09/06/2018 16:53:09",
      "content": "<p>Hello, Carey. That's good thinking -  maybe, we can improve our prediction, after algorithm worked. But, if we gonna create some prediction transformation (decrease bounded boxes in your case) , we need to test our aproach. I mean that you don't have to test your transformation on LB -  just separate part of your training set and test your aproach on this part of data. Or you can test transformation on crossvalidation as well as we do with getting algorithm score. But if we consider you particular case -  boxes decrease  - I think, that improvement on LB reason is lucky. I would take some transformation depending on some parameter and choose this parameter by crossvalidation. Good luck! (but not on LB:)) </p>",
      "rawMarkdown": "Hello, Carey. That's good thinking -  maybe, we can improve our prediction, after algorithm worked. But, if we gonna create some prediction transformation (decrease bounded boxes in your case) , we need to test our aproach. I mean that you don't have to test your transformation on LB -  just separate part of your training set and test your aproach on this part of data. Or you can test transformation on crossvalidation as well as we do with getting algorithm score. But if we consider you particular case -  boxes decrease  - I think, that improvement on LB reason is lucky. I would take some transformation depending on some parameter and choose this parameter by crossvalidation. Good luck! (but not on LB:))",
      "votes": null
    },
    {
      "id": "382578",
      "postDate": "09/06/2018 17:07:28",
      "content": "<p>@Dmitrij That is a good idea, but I am unsure how to confidently perform cross-validation in this competition, because the grading metric is complicated. That is why I chose just to submit to LB =)</p>",
      "rawMarkdown": "Dmitrij That is a good idea, but I am unsure how to confidently perform cross-validation in this competition, because the grading metric is complicated. That is why I chose just to submit to LB =)",
      "votes": null
    },
    {
      "id": "382646",
      "postDate": "09/06/2018 19:29:19",
      "content": "<p>I have <a href=\"https://www.kaggle.com/aharless/jonne-cnn-validation-optimize-mask-threshold\">a kernel</a> that attempts to do validation (just hold-out validation, but the same approach could be applied to k-fold). Whether we can be confident in the result is unclear, but then again, I'm not confident in LB scores either.</p>",
      "rawMarkdown": "I have [a kernel][1] that attempts to do validation (just hold-out validation, but the same approach could be applied to k-fold). Whether we can be confident in the result is unclear, but then again, I'm not confident in LB scores either.\n\n [1]: https://www.kaggle.com/aharless/jonne-cnn-validation-optimize-mask-threshold",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 382571,
      "author_name": "koza4ukdmitrij",
      "author_url": "",
      "post_date": "09/06/2018 16:53:09",
      "content": "<p>Hello, Carey. That's good thinking -  maybe, we can improve our prediction, after algorithm worked. But, if we gonna create some prediction transformation (decrease bounded boxes in your case) , we need to test our aproach. I mean that you don't have to test your transformation on LB -  just separate part of your training set and test your aproach on this part of data. Or you can test transformation on crossvalidation as well as we do with getting algorithm score. But if we consider you particular case -  boxes decrease  - I think, that improvement on LB reason is lucky. I would take some transformation depending on some parameter and choose this parameter by crossvalidation. Good luck! (but not on LB:)) </p>",
      "votes": null,
      "replies": [
        {
          "id": 382578,
          "author_name": "returnofsputnik",
          "author_url": "",
          "post_date": "09/06/2018 17:07:28",
          "content": "<p>@Dmitrij That is a good idea, but I am unsure how to confidently perform cross-validation in this competition, because the grading metric is complicated. That is why I chose just to submit to LB =)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 382646,
          "author_name": "aharless",
          "author_url": "",
          "post_date": "09/06/2018 19:29:19",
          "content": "<p>I have <a href=\"https://www.kaggle.com/aharless/jonne-cnn-validation-optimize-mask-threshold\">a kernel</a> that attempts to do validation (just hold-out validation, but the same approach could be applied to k-fold). Whether we can be confident in the result is unclear, but then again, I'm not confident in LB scores either.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "382541": "I had this question, so I tested it.\n\nThe result is: no, it does not affect the public LB score. I got a 0.001 improvement from the public kernel, which does not mean very much.\n\nYou can view the kernel here: https://www.kaggle.com/returnofsputnik/smaller-bounding-boxes-better-score\n\nPlease let me know if you think I made some mistakes. Also let me know any opinions or questions you have about the experiment",
    "382571": "Hello, Carey. That's good thinking -  maybe, we can improve our prediction, after algorithm worked. But, if we gonna create some prediction transformation (decrease bounded boxes in your case) , we need to test our aproach. I mean that you don't have to test your transformation on LB -  just separate part of your training set and test your aproach on this part of data. Or you can test transformation on crossvalidation as well as we do with getting algorithm score. But if we consider you particular case -  boxes decrease  - I think, that improvement on LB reason is lucky. I would take some transformation depending on some parameter and choose this parameter by crossvalidation. Good luck! (but not on LB:))",
    "382578": "Dmitrij That is a good idea, but I am unsure how to confidently perform cross-validation in this competition, because the grading metric is complicated. That is why I chose just to submit to LB =)",
    "382646": "I have [a kernel][1] that attempts to do validation (just hold-out validation, but the same approach could be applied to k-fold). Whether we can be confident in the result is unclear, but then again, I'm not confident in LB scores either.\n\n [1]: https://www.kaggle.com/aharless/jonne-cnn-validation-optimize-mask-threshold"
  },
  "source": "meta"
}