{
  "id": 69832,
  "title": "Is the data set a heavily skewed?",
  "url": "/competitions/airbus-ship-detection/discussion/69832",
  "author_name": "",
  "post_date": "2018-10-27T20:10:08.558455700Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>As far as I can assess, 22% of the masks in the train data are non-empty.  On average, a non-empty mask is about 0.3% covered with mask pixels.  So, optimization would seem mostly to favor predicting empty masks.  Can someone suggest a good way to deal with this heavy skew toward predicting empty masks?</p>",
  "messages": [
    {
      "id": "411303",
      "postDate": "10/27/2018 20:10:08",
      "content": "<p>As far as I can assess, 22% of the masks in the train data are non-empty.  On average, a non-empty mask is about 0.3% covered with mask pixels.  So, optimization would seem mostly to favor predicting empty masks.  Can someone suggest a good way to deal with this heavy skew toward predicting empty masks?</p>",
      "rawMarkdown": "As far as I can assess, 22% of the masks in the train data are non-empty.  On average, a non-empty mask is about 0.3% covered with mask pixels.  So, optimization would seem mostly to favor predicting empty masks.  Can someone suggest a good way to deal with this heavy skew toward predicting empty masks?",
      "votes": null
    },
    {
      "id": "411309",
      "postDate": "10/27/2018 20:24:31",
      "content": "<p>You can check <a href=\"https://www.kaggle.com/iafoss/unet34-dice-0-87\">https://www.kaggle.com/iafoss/unet34-dice-0-87</a> kernel, where I explain how to deal with this issue.</p>",
      "rawMarkdown": "You can check https://www.kaggle.com/iafoss/unet34-dice-0-87 kernel, where I explain how to deal with this issue.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 411309,
      "author_name": "iafoss",
      "author_url": "",
      "post_date": "10/27/2018 20:24:31",
      "content": "<p>You can check <a href=\"https://www.kaggle.com/iafoss/unet34-dice-0-87\">https://www.kaggle.com/iafoss/unet34-dice-0-87</a> kernel, where I explain how to deal with this issue.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "411303": "As far as I can assess, 22% of the masks in the train data are non-empty.  On average, a non-empty mask is about 0.3% covered with mask pixels.  So, optimization would seem mostly to favor predicting empty masks.  Can someone suggest a good way to deal with this heavy skew toward predicting empty masks?",
    "411309": "You can check https://www.kaggle.com/iafoss/unet34-dice-0-87 kernel, where I explain how to deal with this issue."
  },
  "source": "meta"
}