{
  "id": 86657,
  "title": "why upload non-binary answers?",
  "url": "/competitions/histopathologic-cancer-detection/discussion/86657",
  "author_name": "",
  "post_date": "2019-03-25T17:46:05.473423900Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hello everyone. \nI am a novice, and I learn from the kernels, which were published in this competition by DS masters. I noticed, that most top-LB kernels suggest calculation of  continuous probabilities for cancer/non-cancer cells. Many have  model.add(Dense(1, ...)) at the last layer.</p>\n\n<p>Moreover, when I tried to reproduce several kernels (for example, an excellent kernel of Henrique Mello at <a href=\"https://www.kaggle.com/hrmello/base-cnn-classification-from-scratch\">https://www.kaggle.com/hrmello/base-cnn-classification-from-scratch</a>), I got much worse result for binary data. I tried to add something simple like this:</p>\n\n<p>porog = 0.5\nfor i in range(len(predictions)):\n    if predictions[i] &gt; porog:\n        predictions[i] = 1\n    else:\n       predictions[i] = 0 <br>\n...</p>\n\n<p>At the same time in the \"Data Description\"  we read:\n<strong>\"A positive label indicates that the center 32x32px region of a patch contains at least one pixel of tumor tissue. \"</strong> That means \"classification\" approach, not \"regression\" (continuous).</p>\n\n<p>In the evaluation overview of the competition we read:\n\"<em>you must predict a probability that center 32x32px region of a patch contains at least one pixel of tumor tissue</em>\"</p>\n\n<p>So why \"binarization\" provide worse result, then \"regression - style\" approach?</p>",
  "messages": [
    {
      "id": "500188",
      "postDate": "03/25/2019 17:46:05",
      "content": "<p>Hello everyone. \nI am a novice, and I learn from the kernels, which were published in this competition by DS masters. I noticed, that most top-LB kernels suggest calculation of  continuous probabilities for cancer/non-cancer cells. Many have  model.add(Dense(1, ...)) at the last layer.</p>\n\n<p>Moreover, when I tried to reproduce several kernels (for example, an excellent kernel of Henrique Mello at <a href=\"https://www.kaggle.com/hrmello/base-cnn-classification-from-scratch\">https://www.kaggle.com/hrmello/base-cnn-classification-from-scratch</a>), I got much worse result for binary data. I tried to add something simple like this:</p>\n\n<p>porog = 0.5\nfor i in range(len(predictions)):\n    if predictions[i] &gt; porog:\n        predictions[i] = 1\n    else:\n       predictions[i] = 0 <br>\n...</p>\n\n<p>At the same time in the \"Data Description\"  we read:\n<strong>\"A positive label indicates that the center 32x32px region of a patch contains at least one pixel of tumor tissue. \"</strong> That means \"classification\" approach, not \"regression\" (continuous).</p>\n\n<p>In the evaluation overview of the competition we read:\n\"<em>you must predict a probability that center 32x32px region of a patch contains at least one pixel of tumor tissue</em>\"</p>\n\n<p>So why \"binarization\" provide worse result, then \"regression - style\" approach?</p>",
      "rawMarkdown": "Hello everyone. \nI am a novice, and I learn from the kernels, which were published in this competition by DS masters. I noticed, that most top-LB kernels suggest calculation of  continuous probabilities for cancer/non-cancer cells. Many have  model.add(Dense(1, ...)) at the last layer.\n\nMoreover, when I tried to reproduce several kernels (for example, an excellent kernel of Henrique Mello at https://www.kaggle.com/hrmello/base-cnn-classification-from-scratch), I got much worse result for binary data. I tried to add something simple like this:\n\nporog = 0.5\nfor i in range(len(predictions)):\n    if predictions[i] &gt; porog:\n        predictions[i] = 1\n    else:\n       predictions[i] = 0   \n...\n\nAt the same time in the \"Data Description\"  we read:\n**\"A positive label indicates that the center 32x32px region of a patch contains at least one pixel of tumor tissue. \"** That means \"classification\" approach, not \"regression\" (continuous).\n\nIn the evaluation overview of the competition we read:\n\"*you must predict a probability that center 32x32px region of a patch contains at least one pixel of tumor tissue*\"\n\nSo why \"binarization\" provide worse result, then \"regression - style\" approach?",
      "votes": null
    },
    {
      "id": "500209",
      "postDate": "03/25/2019 18:02:44",
      "content": "<p>The evaluation metric is AUC, which requires class probabilities to correctly calculate. </p>\n\n<p>By predicting the class label you're actually predicting class probability first, then applying threshold of 0.5, which throws away a lot of information.</p>\n\n<p><a href=\"https://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5\">https://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5</a> </p>",
      "rawMarkdown": "The evaluation metric is AUC, which requires class probabilities to correctly calculate. \n\nBy predicting the class label you're actually predicting class probability first, then applying threshold of 0.5, which throws away a lot of information.\n\nhttps://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5",
      "votes": null
    },
    {
      "id": "500343",
      "postDate": "03/25/2019 21:59:57",
      "content": "<p>auroc also gives information about how a binary classifier fails, that is, how likely are false positives or false negatives given a true positive or negative. Submitting only binary values means you only get credit for correctly classified samples. If you set the threshold at 0.5 say, only true positives your model scores above this will count, if your model gives some true positive a value of say 0.49 but gives some other true negative a value of 0.48, this is desirable and will be reflected in the score.</p>",
      "rawMarkdown": "auroc also gives information about how a binary classifier fails, that is, how likely are false positives or false negatives given a true positive or negative. Submitting only binary values means you only get credit for correctly classified samples. If you set the threshold at 0.5 say, only true positives your model scores above this will count, if your model gives some true positive a value of say 0.49 but gives some other true negative a value of 0.48, this is desirable and will be reflected in the score.",
      "votes": null
    },
    {
      "id": "501045",
      "postDate": "03/26/2019 20:22:12",
      "content": "<p>GarethJones, interneuron,\nthank you for your explanations! It's more clear now, the difference among accuracy and ROC.</p>",
      "rawMarkdown": "GarethJones, interneuron,\nthank you for your explanations! It's more clear now, the difference among accuracy and ROC.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 500209,
      "author_name": "garethjns",
      "author_url": "",
      "post_date": "03/25/2019 18:02:44",
      "content": "<p>The evaluation metric is AUC, which requires class probabilities to correctly calculate. </p>\n\n<p>By predicting the class label you're actually predicting class probability first, then applying threshold of 0.5, which throws away a lot of information.</p>\n\n<p><a href=\"https://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5\">https://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5</a> </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 500343,
      "author_name": "interneuron",
      "author_url": "",
      "post_date": "03/25/2019 21:59:57",
      "content": "<p>auroc also gives information about how a binary classifier fails, that is, how likely are false positives or false negatives given a true positive or negative. Submitting only binary values means you only get credit for correctly classified samples. If you set the threshold at 0.5 say, only true positives your model scores above this will count, if your model gives some true positive a value of say 0.49 but gives some other true negative a value of 0.48, this is desirable and will be reflected in the score.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 501045,
      "author_name": "vbelobragin",
      "author_url": "",
      "post_date": "03/26/2019 20:22:12",
      "content": "<p>GarethJones, interneuron,\nthank you for your explanations! It's more clear now, the difference among accuracy and ROC.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "500188": "Hello everyone. \nI am a novice, and I learn from the kernels, which were published in this competition by DS masters. I noticed, that most top-LB kernels suggest calculation of  continuous probabilities for cancer/non-cancer cells. Many have  model.add(Dense(1, ...)) at the last layer.\n\nMoreover, when I tried to reproduce several kernels (for example, an excellent kernel of Henrique Mello at https://www.kaggle.com/hrmello/base-cnn-classification-from-scratch), I got much worse result for binary data. I tried to add something simple like this:\n\nporog = 0.5\nfor i in range(len(predictions)):\n    if predictions[i] &gt; porog:\n        predictions[i] = 1\n    else:\n       predictions[i] = 0   \n...\n\nAt the same time in the \"Data Description\"  we read:\n**\"A positive label indicates that the center 32x32px region of a patch contains at least one pixel of tumor tissue. \"** That means \"classification\" approach, not \"regression\" (continuous).\n\nIn the evaluation overview of the competition we read:\n\"*you must predict a probability that center 32x32px region of a patch contains at least one pixel of tumor tissue*\"\n\nSo why \"binarization\" provide worse result, then \"regression - style\" approach?",
    "500209": "The evaluation metric is AUC, which requires class probabilities to correctly calculate. \n\nBy predicting the class label you're actually predicting class probability first, then applying threshold of 0.5, which throws away a lot of information.\n\nhttps://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5",
    "500343": "auroc also gives information about how a binary classifier fails, that is, how likely are false positives or false negatives given a true positive or negative. Submitting only binary values means you only get credit for correctly classified samples. If you set the threshold at 0.5 say, only true positives your model scores above this will count, if your model gives some true positive a value of say 0.49 but gives some other true negative a value of 0.48, this is desirable and will be reflected in the score.",
    "501045": "GarethJones, interneuron,\nthank you for your explanations! It's more clear now, the difference among accuracy and ROC."
  },
  "source": "meta"
}