{
  "id": 29017,
  "title": "Private LB and evaluation exception",
  "url": "/competitions/dstl-satellite-imagery-feature-detection/discussion/29017",
  "author_name": "",
  "post_date": "2017-02-19T21:11:12.677760900Z",
  "votes": 7,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hello,</p>\n\n<p>Thank you, this competition is good fun.\nI have a concern about the evaluation execption that we sometimes face.\nAs far as I understand, the exceptions might be raised by the polygons resulting from the intersection between the prediction and the expected polygons. It may ocur even when we provide valid polygons (checked with shapely and the tpex tool).\nIs my understanding alright ?\nIf it is, this the responsability of the validation tool to produce valid polygons, or at least to fix invalid polygons if it produces some when doing the intersection.\nI have another question : is it possible that we get exceptions on private LB with predictions files  that were \"accepted\" for public LB ? If yes, this is a big problem.</p>",
  "messages": [
    {
      "id": "162541",
      "postDate": "02/19/2017 21:11:12",
      "content": "<p>Hello,</p>\n\n<p>Thank you, this competition is good fun.\nI have a concern about the evaluation execption that we sometimes face.\nAs far as I understand, the exceptions might be raised by the polygons resulting from the intersection between the prediction and the expected polygons. It may ocur even when we provide valid polygons (checked with shapely and the tpex tool).\nIs my understanding alright ?\nIf it is, this the responsability of the validation tool to produce valid polygons, or at least to fix invalid polygons if it produces some when doing the intersection.\nI have another question : is it possible that we get exceptions on private LB with predictions files  that were \"accepted\" for public LB ? If yes, this is a big problem.</p>",
      "rawMarkdown": "Hello,\n\nThank you, this competition is good fun.\nI have a concern about the evaluation execption that we sometimes face.\nAs far as I understand, the exceptions might be raised by the polygons resulting from the intersection between the prediction and the expected polygons. It may ocur even when we provide valid polygons (checked with shapely and the tpex tool).\nIs my understanding alright ?\nIf it is, this the responsability of the validation tool to produce valid polygons, or at least to fix invalid polygons if it produces some when doing the intersection.\nI have another question : is it possible that we get exceptions on private LB with predictions files  that were \"accepted\" for public LB ? If yes, this is a big problem.",
      "votes": null
    },
    {
      "id": "162601",
      "postDate": "02/20/2017 08:16:25",
      "content": "<p>Hi Thomas,</p>\n\n<p>I expect that our private score has been already calculated potentially.<br>\nIf there is possibility we get <em>invalid polygons error</em> in accepted files, nobody could pass the final validation.</p>",
      "rawMarkdown": "Hi Thomas,\n\nI expect that our private score has been already calculated potentially.<br>\nIf there is possibility we get *invalid polygons error* in accepted files, nobody could pass the final validation.",
      "votes": null
    },
    {
      "id": "162676",
      "postDate": "02/20/2017 16:13:08",
      "content": "<p>Or maybe a lucky one passes it, and wins with a lower score...</p>",
      "rawMarkdown": "Or maybe a lucky one passes it, and wins with a lower score...",
      "votes": null
    },
    {
      "id": "162804",
      "postDate": "02/21/2017 06:43:00",
      "content": "<p>I would like that but having a doubt, I ask.\nI have this doubt as I saw that at least one of my \"successful\" submission had one topology error reported by tpex tool.\nCould a Kaggle admin check and answer the question ?</p>",
      "rawMarkdown": "I would like that but having a doubt, I ask.\nI have this doubt as I saw that at least one of my \"successful\" submission had one topology error reported by tpex tool.\nCould a Kaggle admin check and answer the question ?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 162601,
      "author_name": "toshik",
      "author_url": "",
      "post_date": "02/20/2017 08:16:25",
      "content": "<p>Hi Thomas,</p>\n\n<p>I expect that our private score has been already calculated potentially.<br>\nIf there is possibility we get <em>invalid polygons error</em> in accepted files, nobody could pass the final validation.</p>",
      "votes": null,
      "replies": [
        {
          "id": 162676,
          "author_name": "ironbar",
          "author_url": "",
          "post_date": "02/20/2017 16:13:08",
          "content": "<p>Or maybe a lucky one passes it, and wins with a lower score...</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 162804,
          "author_name": "cogitae",
          "author_url": "",
          "post_date": "02/21/2017 06:43:00",
          "content": "<p>I would like that but having a doubt, I ask.\nI have this doubt as I saw that at least one of my \"successful\" submission had one topology error reported by tpex tool.\nCould a Kaggle admin check and answer the question ?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "162541": "Hello,\n\nThank you, this competition is good fun.\nI have a concern about the evaluation execption that we sometimes face.\nAs far as I understand, the exceptions might be raised by the polygons resulting from the intersection between the prediction and the expected polygons. It may ocur even when we provide valid polygons (checked with shapely and the tpex tool).\nIs my understanding alright ?\nIf it is, this the responsability of the validation tool to produce valid polygons, or at least to fix invalid polygons if it produces some when doing the intersection.\nI have another question : is it possible that we get exceptions on private LB with predictions files  that were \"accepted\" for public LB ? If yes, this is a big problem.",
    "162601": "Hi Thomas,\n\nI expect that our private score has been already calculated potentially.<br>\nIf there is possibility we get *invalid polygons error* in accepted files, nobody could pass the final validation.",
    "162676": "Or maybe a lucky one passes it, and wins with a lower score...",
    "162804": "I would like that but having a doubt, I ask.\nI have this doubt as I saw that at least one of my \"successful\" submission had one topology error reported by tpex tool.\nCould a Kaggle admin check and answer the question ?"
  },
  "source": "meta"
}