{
  "id": 210183,
  "title": "Help understanding cap_and_update_priors in the benchmark code",
  "url": "/competitions/predict-closed-questions-on-stack-overflow/discussion/210183",
  "author_name": "dotslan",
  "post_date": "2021-01-10T00:00:43.994000",
  "votes": 1,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I know this competition is 8 years old but thought it couldn't hurt to ask.</p>\n<p>I'm trying to understand the <a href=\"https://github.com/benhamner/Stack-Overflow-Competition\" target=\"_blank\">benchmark code</a> and have gotten to the cap_and_update_priors function.</p>\n<p>I understand that the cap part is scaling the probabilities between epsilon (0.001) and 1-epsilon (0.999), but I don't understand where the general reasoning comes from for this function and what the <code>evidence_ratio</code> is for. Would appreciate any links to literature for review.</p>\n<pre><code>def cap_and_update_priors(old_priors, old_posteriors, new_priors, epsilon):\n    old_posteriors = cap_predictions(old_posteriors, epsilon)\n    old_priors = np.kron(np.ones((np.size(old_posteriors, 0), 1)), old_priors)\n    new_priors = np.kron(np.ones((np.size(old_posteriors, 0), 1)), new_priors)\n    evidence_ratio = (old_priors*(1-old_posteriors)) / (old_posteriors*(1-old_priors))\n    new_posteriors = new_priors / (new_priors + (1-new_priors)*evidence_ratio)\n    new_posteriors = cap_predictions(new_posteriors, epsilon)\n    return new_posteriors\n</code></pre>\n<p>edit: for other noobs like me, some further reading about the evaluation method (multi class log loss) explains the capping.</p>",
  "messages": [
    {
      "id": 1146682,
      "postDate": "2021-01-10T00:00:43.993Z",
      "content": "<p>I know this competition is 8 years old but thought it couldn't hurt to ask.</p>\n<p>I'm trying to understand the <a href=\"https://github.com/benhamner/Stack-Overflow-Competition\" target=\"_blank\">benchmark code</a> and have gotten to the cap_and_update_priors function.</p>\n<p>I understand that the cap part is scaling the probabilities between epsilon (0.001) and 1-epsilon (0.999), but I don't understand where the general reasoning comes from for this function and what the <code>evidence_ratio</code> is for. Would appreciate any links to literature for review.</p>\n<pre><code>def cap_and_update_priors(old_priors, old_posteriors, new_priors, epsilon):\n    old_posteriors = cap_predictions(old_posteriors, epsilon)\n    old_priors = np.kron(np.ones((np.size(old_posteriors, 0), 1)), old_priors)\n    new_priors = np.kron(np.ones((np.size(old_posteriors, 0), 1)), new_priors)\n    evidence_ratio = (old_priors*(1-old_posteriors)) / (old_posteriors*(1-old_priors))\n    new_posteriors = new_priors / (new_priors + (1-new_priors)*evidence_ratio)\n    new_posteriors = cap_predictions(new_posteriors, epsilon)\n    return new_posteriors\n</code></pre>\n<p>edit: for other noobs like me, some further reading about the evaluation method (multi class log loss) explains the capping.</p>",
      "rawMarkdown": "I know this competition is 8 years old but thought it couldn't hurt to ask.\n\nI'm trying to understand the [benchmark code](https://github.com/benhamner/Stack-Overflow-Competition) and have gotten to the cap_and_update_priors function.\n\nI understand that the cap part is scaling the probabilities between epsilon (0.001) and 1-epsilon (0.999), but I don't understand where the general reasoning comes from for this function and what the `evidence_ratio` is for. Would appreciate any links to literature for review.\n\n```\ndef cap_and_update_priors(old_priors, old_posteriors, new_priors, epsilon):\n    old_posteriors = cap_predictions(old_posteriors, epsilon)\n    old_priors = np.kron(np.ones((np.size(old_posteriors, 0), 1)), old_priors)\n    new_priors = np.kron(np.ones((np.size(old_posteriors, 0), 1)), new_priors)\n    evidence_ratio = (old_priors*(1-old_posteriors)) / (old_posteriors*(1-old_priors))\n    new_posteriors = new_priors / (new_priors + (1-new_priors)*evidence_ratio)\n    new_posteriors = cap_predictions(new_posteriors, epsilon)\n    return new_posteriors\n```\n\nedit: for other noobs like me, some further reading about the evaluation method (multi class log loss) explains the capping."
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1146682": "I know this competition is 8 years old but thought it couldn't hurt to ask.\n\nI'm trying to understand the [benchmark code](https://github.com/benhamner/Stack-Overflow-Competition) and have gotten to the cap_and_update_priors function.\n\nI understand that the cap part is scaling the probabilities between epsilon (0.001) and 1-epsilon (0.999), but I don't understand where the general reasoning comes from for this function and what the `evidence_ratio` is for. Would appreciate any links to literature for review.\n\n```\ndef cap_and_update_priors(old_priors, old_posteriors, new_priors, epsilon):\n    old_posteriors = cap_predictions(old_posteriors, epsilon)\n    old_priors = np.kron(np.ones((np.size(old_posteriors, 0), 1)), old_priors)\n    new_priors = np.kron(np.ones((np.size(old_posteriors, 0), 1)), new_priors)\n    evidence_ratio = (old_priors*(1-old_posteriors)) / (old_posteriors*(1-old_priors))\n    new_posteriors = new_priors / (new_priors + (1-new_priors)*evidence_ratio)\n    new_posteriors = cap_predictions(new_posteriors, epsilon)\n    return new_posteriors\n```\n\nedit: for other noobs like me, some further reading about the evaluation method (multi class log loss) explains the capping."
  }
}