{
  "id": 245793,
  "title": "Robustness of the postprocessing",
  "url": "/competitions/google-smartphone-decimeter-challenge/discussion/245793",
  "author_name": "",
  "post_date": "2021-06-12T12:38:23.709332400Z",
  "votes": 7,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Can we really say that current postprocessing methods can be considered to be robust?</p>\n<p>The very fact that final LB result depends on the order of the pipeline (and in some combinations we receive the CV increase but LB becomes worse) makes me a bit worried about the whole thing. Removing outliers shouldnt make a difference, but the rest of the methods sometimes seem to merge in a weird way producing the misleading results.</p>\n<p>Did anyone have the same experience and what are your thoughts on possibility of overfitting the baseline at all?</p>",
  "messages": [
    {
      "id": "1346478",
      "postDate": "06/12/2021 12:38:23",
      "content": "<p>Can we really say that current postprocessing methods can be considered to be robust?</p>\n<p>The very fact that final LB result depends on the order of the pipeline (and in some combinations we receive the CV increase but LB becomes worse) makes me a bit worried about the whole thing. Removing outliers shouldnt make a difference, but the rest of the methods sometimes seem to merge in a weird way producing the misleading results.</p>\n<p>Did anyone have the same experience and what are your thoughts on possibility of overfitting the baseline at all?</p>",
      "rawMarkdown": "Can we really say that current postprocessing methods can be considered to be robust?\n\nThe very fact that final LB result depends on the order of the pipeline (and in some combinations we receive the CV increase but LB becomes worse) makes me a bit worried about the whole thing. Removing outliers shouldnt make a difference, but the rest of the methods sometimes seem to merge in a weird way producing the misleading results.\n\nDid anyone have the same experience and what are your thoughts on possibility of overfitting the baseline at all?",
      "votes": null
    },
    {
      "id": "1347021",
      "postDate": "06/12/2021 20:49:36",
      "content": "<p>In my case, \"Removing device\" and \"Position shift\" have no effect due to other post-processing.<br>\nHowever, I don't suspect overlearning because both LB and CV have the same effect.</p>",
      "rawMarkdown": "In my case, \"Removing device\" and \"Position shift\" have no effect due to other post-processing.\nHowever, I don't suspect overlearning because both LB and CV have the same effect.",
      "votes": null
    },
    {
      "id": "1359101",
      "postDate": "06/21/2021 04:46:34",
      "content": "<p>It sounds possible to me that the post-processing will cause overfitting, I chose to submit one solution without major post-processing.</p>",
      "rawMarkdown": "It sounds possible to me that the post-processing will cause overfitting, I chose to submit one solution without major post-processing.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1347021,
      "author_name": "dehokanta",
      "author_url": "",
      "post_date": "06/12/2021 20:49:36",
      "content": "<p>In my case, \"Removing device\" and \"Position shift\" have no effect due to other post-processing.<br>\nHowever, I don't suspect overlearning because both LB and CV have the same effect.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1359101,
      "author_name": "avivlevi815",
      "author_url": "",
      "post_date": "06/21/2021 04:46:34",
      "content": "<p>It sounds possible to me that the post-processing will cause overfitting, I chose to submit one solution without major post-processing.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1346478": "Can we really say that current postprocessing methods can be considered to be robust?\n\nThe very fact that final LB result depends on the order of the pipeline (and in some combinations we receive the CV increase but LB becomes worse) makes me a bit worried about the whole thing. Removing outliers shouldnt make a difference, but the rest of the methods sometimes seem to merge in a weird way producing the misleading results.\n\nDid anyone have the same experience and what are your thoughts on possibility of overfitting the baseline at all?",
    "1347021": "In my case, \"Removing device\" and \"Position shift\" have no effect due to other post-processing.\nHowever, I don't suspect overlearning because both LB and CV have the same effect.",
    "1359101": "It sounds possible to me that the post-processing will cause overfitting, I chose to submit one solution without major post-processing."
  },
  "source": "meta"
}