{
  "id": 73853,
  "title": "A single line of code which effects your score most.",
  "url": "/competitions/PLAsTiCC-2018/discussion/73853",
  "author_name": "",
  "post_date": "2018-12-06T06:04:36.592350100Z",
  "votes": 2,
  "comment_count": 7,
  "views": 0,
  "content": "<p>In a categorical entropy, difference between prob 0 and 0.01 is well below all variations like label noises and validation errors, but in the cross entropy, which is used in the competition,  they will differ tremendously,  log(10^-15)=-34.5 and log(0.01)=-4.6.  The organizer would not want a parameter of nearly no practical value but guessed (or elaborated) luckily will affect evaluation inadvertently, and may need to modify lower clipping value. Until then, try:</p>\n\n<p>preds = np.clip(preds, 0.001 [place for your hit], None)</p>\n\n<p>and share what happens. </p>",
  "messages": [
    {
      "id": "434268",
      "postDate": "12/06/2018 06:04:36",
      "content": "<p>In a categorical entropy, difference between prob 0 and 0.01 is well below all variations like label noises and validation errors, but in the cross entropy, which is used in the competition,  they will differ tremendously,  log(10^-15)=-34.5 and log(0.01)=-4.6.  The organizer would not want a parameter of nearly no practical value but guessed (or elaborated) luckily will affect evaluation inadvertently, and may need to modify lower clipping value. Until then, try:</p>\n\n<p>preds = np.clip(preds, 0.001 [place for your hit], None)</p>\n\n<p>and share what happens. </p>",
      "rawMarkdown": "In a categorical entropy, difference between prob 0 and 0.01 is well below all variations like label noises and validation errors, but in the cross entropy, which is used in the competition,  they will differ tremendously,  log(10^-15)=-34.5 and log(0.01)=-4.6.  The organizer would not want a parameter of nearly no practical value but guessed (or elaborated) luckily will affect evaluation inadvertently, and may need to modify lower clipping value. Until then, try:\n\npreds = np.clip(preds, 0.001 [place for your hit], None)\n\nand share what happens.",
      "votes": null
    },
    {
      "id": "434276",
      "postDate": "12/06/2018 06:19:50",
      "content": "<p>Clipping degrades our LB score ;)  Only a tiny bit, but it does.</p>\n\n<p>If you are using softmax or sigmoid output, odds of having a prediction below 1e-6 or above 1 - 1e-6 are extremely low.  </p>",
      "rawMarkdown": "Clipping degrades our LB score ;)  Only a tiny bit, but it does.\n\nIf you are using softmax or sigmoid output, odds of having a prediction below 1e-6 or above 1 - 1e-6 are extremely low.",
      "votes": null
    },
    {
      "id": "434430",
      "postDate": "12/06/2018 11:34:05",
      "content": "<p>\"In a categorical entropy, difference between prob 0 and 0.01 is well below all variations like label noises and validation errors...\"</p>\n\n<p>The difference between linear predictors leading to 0 or 0.01 is infinitely large if using a softmax or sigmoid transformation.</p>",
      "rawMarkdown": "\"In a categorical entropy, difference between prob 0 and 0.01 is well below all variations like label noises and validation errors...\"\n\nThe difference between linear predictors leading to 0 or 0.01 is infinitely large if using a softmax or sigmoid transformation.",
      "votes": null
    },
    {
      "id": "434482",
      "postDate": "12/06/2018 13:24:47",
      "content": "<p>The value of the trick might depend on ones score. Your score is already extremely good, so the trick might be less important to your team. I will try it out in one of the coming days.</p>",
      "rawMarkdown": "The value of the trick might depend on ones score. Your score is already extremely good, so the trick might be less important to your team. I will try it out in one of the coming days.",
      "votes": null
    },
    {
      "id": "434643",
      "postDate": "12/06/2018 18:08:20",
      "content": "<p>I bet clipping will degrade your performance as well.  Absolute score has nothing to do with it, it is just that if the model is confident enough to predict a low value, then it must be pretty confident about it.</p>",
      "rawMarkdown": "I bet clipping will degrade your performance as well.  Absolute score has nothing to do with it, it is just that if the model is confident enough to predict a low value, then it must be pretty confident about it.",
      "votes": null
    },
    {
      "id": "434799",
      "postDate": "12/07/2018 00:49:52",
      "content": "<p>I clip my scores in 1e-5 and 1-1e-5. It don't affect LB much in my case.</p>",
      "rawMarkdown": "I clip my scores in 1e-5 and 1-1e-5. It don't affect LB much in my case.",
      "votes": null
    },
    {
      "id": "435052",
      "postDate": "12/07/2018 12:05:27",
      "content": "<p>As the difference galactic/extra galactic is very safe, one could perhaps use Gibas 1e-5 value for jumps between galaxies and a higher value \"inside\" each of the two categories. Compared to using the high value everywhere that would give a somewhat less degrading influence.  </p>",
      "rawMarkdown": "As the difference galactic/extra galactic is very safe, one could perhaps use Gibas 1e-5 value for jumps between galaxies and a higher value \"inside\" each of the two categories. Compared to using the high value everywhere that would give a somewhat less degrading influence.",
      "votes": null
    },
    {
      "id": "440244",
      "postDate": "12/17/2018 09:30:45",
      "content": "<p>why there is None. should not it be  (1 - .001) ?</p>",
      "rawMarkdown": "why there is None. should not it be  (1 - .001) ?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 434276,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "12/06/2018 06:19:50",
      "content": "<p>Clipping degrades our LB score ;)  Only a tiny bit, but it does.</p>\n\n<p>If you are using softmax or sigmoid output, odds of having a prediction below 1e-6 or above 1 - 1e-6 are extremely low.  </p>",
      "votes": null,
      "replies": [
        {
          "id": 434482,
          "author_name": "petersorensen360",
          "author_url": "",
          "post_date": "12/06/2018 13:24:47",
          "content": "<p>The value of the trick might depend on ones score. Your score is already extremely good, so the trick might be less important to your team. I will try it out in one of the coming days.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 434643,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "12/06/2018 18:08:20",
          "content": "<p>I bet clipping will degrade your performance as well.  Absolute score has nothing to do with it, it is just that if the model is confident enough to predict a low value, then it must be pretty confident about it.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 434799,
          "author_name": "titericz",
          "author_url": "",
          "post_date": "12/07/2018 00:49:52",
          "content": "<p>I clip my scores in 1e-5 and 1-1e-5. It don't affect LB much in my case.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435052,
          "author_name": "petersorensen360",
          "author_url": "",
          "post_date": "12/07/2018 12:05:27",
          "content": "<p>As the difference galactic/extra galactic is very safe, one could perhaps use Gibas 1e-5 value for jumps between galaxies and a higher value \"inside\" each of the two categories. Compared to using the high value everywhere that would give a somewhat less degrading influence.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 434430,
      "author_name": "jlomond",
      "author_url": "",
      "post_date": "12/06/2018 11:34:05",
      "content": "<p>\"In a categorical entropy, difference between prob 0 and 0.01 is well below all variations like label noises and validation errors...\"</p>\n\n<p>The difference between linear predictors leading to 0 or 0.01 is infinitely large if using a softmax or sigmoid transformation.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 440244,
      "author_name": "mks2192",
      "author_url": "",
      "post_date": "12/17/2018 09:30:45",
      "content": "<p>why there is None. should not it be  (1 - .001) ?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "434268": "In a categorical entropy, difference between prob 0 and 0.01 is well below all variations like label noises and validation errors, but in the cross entropy, which is used in the competition,  they will differ tremendously,  log(10^-15)=-34.5 and log(0.01)=-4.6.  The organizer would not want a parameter of nearly no practical value but guessed (or elaborated) luckily will affect evaluation inadvertently, and may need to modify lower clipping value. Until then, try:\n\npreds = np.clip(preds, 0.001 [place for your hit], None)\n\nand share what happens.",
    "434276": "Clipping degrades our LB score ;)  Only a tiny bit, but it does.\n\nIf you are using softmax or sigmoid output, odds of having a prediction below 1e-6 or above 1 - 1e-6 are extremely low.",
    "434430": "\"In a categorical entropy, difference between prob 0 and 0.01 is well below all variations like label noises and validation errors...\"\n\nThe difference between linear predictors leading to 0 or 0.01 is infinitely large if using a softmax or sigmoid transformation.",
    "434482": "The value of the trick might depend on ones score. Your score is already extremely good, so the trick might be less important to your team. I will try it out in one of the coming days.",
    "434643": "I bet clipping will degrade your performance as well.  Absolute score has nothing to do with it, it is just that if the model is confident enough to predict a low value, then it must be pretty confident about it.",
    "434799": "I clip my scores in 1e-5 and 1-1e-5. It don't affect LB much in my case.",
    "435052": "As the difference galactic/extra galactic is very safe, one could perhaps use Gibas 1e-5 value for jumps between galaxies and a higher value \"inside\" each of the two categories. Compared to using the high value everywhere that would give a somewhat less degrading influence.",
    "440244": "why there is None. should not it be  (1 - .001) ?"
  },
  "source": "meta"
}