![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F70497%2F7c638addd8366aa19a5a64fe7e9d90a2%2FScreenshot%20from%202020-07-25%2013-43-03.png?generation=1595681023658105&alt=media\")
I pasted into a notebook and read through it. The metric looks completely wrong and dangerous.
\n\nCan somebody explain why you would want to floor the sigma? It should be maxed. Over-confidence MUST be punished severely.
\n\nAnd extremely wrong values of the mean FVC MUST be punished. This metric is completely wrong. Dangerous way of selecting a good algo.
\n\nIs this actually what is intended or are their typos?
\n\n
I pasted into a notebook and read through it. The metric looks completely wrong and dangerous.
\n\nCan somebody explain why you would want to floor the sigma? It should be maxed. Over-confidence MUST be punished severely.
\n\nAnd extremely wrong values of the mean FVC MUST be punished. This metric is completely wrong. Dangerous way of selecting a good algo.
\n\nIs this actually what is intended or are their typos?
\n\n
It's not as simple as just maximizing sigma. Taking the derivative of the metric, you find that it's maximized when sigma = sqrt(2)*delta. This means the optimal value of sigma is lower if the difference is low and higher if the difference is high, which is intuitive given that you want to predict higher values of sigma if you're less confident in your answer.
", "rawMarkdown": "It's not as simple as just maximizing sigma. Taking the derivative of the metric, you find that it's maximized when sigma = sqrt(2)*delta. This means the optimal value of sigma is lower if the difference is low and higher if the difference is high, which is intuitive given that you want to predict higher values of sigma if you're less confident in your answer.", "votes": null }, { "id": "947043", "postDate": "07/27/2020 03:29:32", "content": "So higher sigma value, meaning you are less confident in your predictions?
", "rawMarkdown": "So higher sigma value, meaning you are less confident in your predictions?", "votes": null }, { "id": "947340", "postDate": "07/27/2020 07:35:55", "content": "> So higher sigma value, meaning you are less confident in your predictions?
\n\nYes. That's right. The term confidence is very confusing and unintuitive (I guess because higher value of confidence implies you are actually less confident!), I had recommended to think about it as uncertainty instead.\nYou can check the competition host's discussion on this here.
", "rawMarkdown": "> So higher sigma value, meaning you are less confident in your predictions?\n\nYes. That's right. The term *confidence* is very confusing and unintuitive (I guess because higher value of confidence implies you are actually less confident!), I had recommended to think about it as *uncertainty* instead.\nYou can check the competition host's discussion on this [here](https://www.kaggle.com/rohanrao/osic-understanding-laplace-log-likelihood/comments#921776).", "votes": null } ], "comments": [ { "id": 945019, "author_name": "vaillant", "author_url": "", "post_date": "07/25/2020 14:07:35", "content": "It's not as simple as just maximizing sigma. Taking the derivative of the metric, you find that it's maximized when sigma = sqrt(2)*delta. This means the optimal value of sigma is lower if the difference is low and higher if the difference is high, which is intuitive given that you want to predict higher values of sigma if you're less confident in your answer.
", "votes": null, "replies": [ { "id": 947043, "author_name": "kurianbenoy", "author_url": "", "post_date": "07/27/2020 03:29:32", "content": "So higher sigma value, meaning you are less confident in your predictions?
", "votes": null, "replies": [] }, { "id": 947340, "author_name": "rohanrao", "author_url": "", "post_date": "07/27/2020 07:35:55", "content": "> So higher sigma value, meaning you are less confident in your predictions?
\n\nYes. That's right. The term confidence is very confusing and unintuitive (I guess because higher value of confidence implies you are actually less confident!), I had recommended to think about it as uncertainty instead.\nYou can check the competition host's discussion on this here.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "944905": "I pasted into a notebook and read through it. The metric looks completely wrong and dangerous. \n\nCan somebody explain why you would want to floor the sigma? It should be maxed. Over-confidence MUST be punished severely. \n\nAnd extremely wrong values of the mean FVC MUST be punished. This metric is completely wrong. Dangerous way of selecting a good algo.\n\nIs this actually what is intended or are their typos? \n\n", "945019": "It's not as simple as just maximizing sigma. Taking the derivative of the metric, you find that it's maximized when sigma = sqrt(2)*delta. This means the optimal value of sigma is lower if the difference is low and higher if the difference is high, which is intuitive given that you want to predict higher values of sigma if you're less confident in your answer.", "947043": "So higher sigma value, meaning you are less confident in your predictions?", "947340": "> So higher sigma value, meaning you are less confident in your predictions?\n\nYes. That's right. The term *confidence* is very confusing and unintuitive (I guess because higher value of confidence implies you are actually less confident!), I had recommended to think about it as *uncertainty* instead.\nYou can check the competition host's discussion on this [here](https://www.kaggle.com/rohanrao/osic-understanding-laplace-log-likelihood/comments#921776)." }, "source": "meta" }