{
  "id": 12586,
  "title": "Non-integer forecasts?",
  "url": "/competitions/diabetic-retinopathy-detection/discussion/12586",
  "author_name": "",
  "post_date": "2015-02-22T04:28:29.883Z",
  "votes": 1,
  "comment_count": 3,
  "views": 1355,
  "content": "<p>Must all predictions be one of the integers 0, 1, 2, 3, or 4?&nbsp; This was not clear to me from the supplied documentation.&nbsp; Since the ratings are on a numeric scale, it could be plausible for predictions to take on non-integer values, e.g. 1.7, just as in some other contests with numeric outcomes.</p>\n<p>Thanks for clarifying this.</p>",
  "messages": [
    {
      "id": "64727",
      "postDate": "02/22/2015 04:28:29",
      "content": "<p>Must all predictions be one of the integers 0, 1, 2, 3, or 4?&nbsp; This was not clear to me from the supplied documentation.&nbsp; Since the ratings are on a numeric scale, it could be plausible for predictions to take on non-integer values, e.g. 1.7, just as in some other contests with numeric outcomes.</p>\n<p>Thanks for clarifying this.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64736",
      "postDate": "02/22/2015 15:45:17",
      "content": "<p>[quote=David J. Slate;64727]</p>\n<p>Must all predictions be one of the integers 0, 1, 2, 3, or 4?</p>\n<p>[/quote]</p>\n<p>The evaluation page says</p>\n<p>&quot;Images have five possible ratings, 0,1,2,3,4. Each image is characterized by a tuple (ea,eb), which corresponds to its scores by Rater A (human) and Rater B (predicted). &quot;</p>\n<p>So I think the predicted levels must be integers from 0 to 4.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64738",
      "postDate": "02/22/2015 16:33:28",
      "content": "<p>Correct, ratings must be integers. This is done because grading scales like this aren't necessarily &quot;linear&quot; in response (e.g. what would it mean to average &quot;No DR&quot; with &quot;Mild&quot;, and should that be the same as averaging other ratings?)</p>\n<p>Of course, nothing stops you from treating this as regression and rounding your outputs. My team did this&nbsp;in the ASAP Essay Grading competition years ago. Interestingly, we found that the ideal rounding thresholds were not the same for different scores!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64749",
      "postDate": "02/22/2015 23:14:28",
      "content": "<p>Thanks William; that's what I thought, but I wasn't sure.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 64736,
      "author_name": "jfkingiii",
      "author_url": "",
      "post_date": "02/22/2015 15:45:17",
      "content": "<p>[quote=David J. Slate;64727]</p>\n<p>Must all predictions be one of the integers 0, 1, 2, 3, or 4?</p>\n<p>[/quote]</p>\n<p>The evaluation page says</p>\n<p>&quot;Images have five possible ratings, 0,1,2,3,4. Each image is characterized by a tuple (ea,eb), which corresponds to its scores by Rater A (human) and Rater B (predicted). &quot;</p>\n<p>So I think the predicted levels must be integers from 0 to 4.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64738,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "02/22/2015 16:33:28",
      "content": "<p>Correct, ratings must be integers. This is done because grading scales like this aren't necessarily &quot;linear&quot; in response (e.g. what would it mean to average &quot;No DR&quot; with &quot;Mild&quot;, and should that be the same as averaging other ratings?)</p>\n<p>Of course, nothing stops you from treating this as regression and rounding your outputs. My team did this&nbsp;in the ASAP Essay Grading competition years ago. Interestingly, we found that the ideal rounding thresholds were not the same for different scores!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64749,
      "author_name": "dslate",
      "author_url": "",
      "post_date": "02/22/2015 23:14:28",
      "content": "<p>Thanks William; that's what I thought, but I wasn't sure.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "64727": "",
    "64736": "",
    "64738": "",
    "64749": ""
  },
  "source": "meta"
}