{
  "id": 59894,
  "title": "How did you fill missing values ? ",
  "url": "/competitions/avito-demand-prediction/discussion/59894",
  "author_name": "",
  "post_date": "2018-06-28T03:56:42.419110800Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I found if i fill missing value(image_top_1, param_1,param_2, param_3) at first time , then make some groupby feature engineer, it may cause overfit, I think it is because filling these with static values bring big noise or wrong information in groupby stage.</p>\n\n<p>So I just first do groupby feature engineer, and then filling missing value with a reasonable static value in each feature columns , seems like got better gerneralization,  So how did you fill missing values? could you share something? </p>\n\n<p>In some threads, I see someone using predict method to predict missing value without static filling. But I can not figure out how to do ? </p>",
  "messages": [
    {
      "id": "349385",
      "postDate": "06/28/2018 03:56:42",
      "content": "<p>I found if i fill missing value(image_top_1, param_1,param_2, param_3) at first time , then make some groupby feature engineer, it may cause overfit, I think it is because filling these with static values bring big noise or wrong information in groupby stage.</p>\n\n<p>So I just first do groupby feature engineer, and then filling missing value with a reasonable static value in each feature columns , seems like got better gerneralization,  So how did you fill missing values? could you share something? </p>\n\n<p>In some threads, I see someone using predict method to predict missing value without static filling. But I can not figure out how to do ? </p>",
      "rawMarkdown": "I found if i fill missing value(image_top_1, param_1,param_2, param_3) at first time , then make some groupby feature engineer, it may cause overfit, I think it is because filling these with static values bring big noise or wrong information in groupby stage.\n\nSo I just first do groupby feature engineer, and then filling missing value with a reasonable static value in each feature columns , seems like got better gerneralization,  So how did you fill missing values? could you share something? \n\nIn some threads, I see someone using predict method to predict missing value without static filling. But I can not figure out how to do ?",
      "votes": null
    },
    {
      "id": "349389",
      "postDate": "06/28/2018 04:05:04",
      "content": "<p>My assumption was that params image_top and price should be strongly correlated with title and description so I simply built a neural network on these to predict image_top etc. and then used it to fill those missings.</p>",
      "rawMarkdown": "My assumption was that params image_top and price should be strongly correlated with title and description so I simply built a neural network on these to predict image_top etc. and then used it to fill those missings.",
      "votes": null
    },
    {
      "id": "349417",
      "postDate": "06/28/2018 04:54:29",
      "content": "<p>Oh，i got it, thanks very much !</p>",
      "rawMarkdown": "Oh，i got it, thanks very much !",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 349389,
      "author_name": "arroqc",
      "author_url": "",
      "post_date": "06/28/2018 04:05:04",
      "content": "<p>My assumption was that params image_top and price should be strongly correlated with title and description so I simply built a neural network on these to predict image_top etc. and then used it to fill those missings.</p>",
      "votes": null,
      "replies": [
        {
          "id": 349417,
          "author_name": "qfzgs1994",
          "author_url": "",
          "post_date": "06/28/2018 04:54:29",
          "content": "<p>Oh，i got it, thanks very much !</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "349385": "I found if i fill missing value(image_top_1, param_1,param_2, param_3) at first time , then make some groupby feature engineer, it may cause overfit, I think it is because filling these with static values bring big noise or wrong information in groupby stage.\n\nSo I just first do groupby feature engineer, and then filling missing value with a reasonable static value in each feature columns , seems like got better gerneralization,  So how did you fill missing values? could you share something? \n\nIn some threads, I see someone using predict method to predict missing value without static filling. But I can not figure out how to do ?",
    "349389": "My assumption was that params image_top and price should be strongly correlated with title and description so I simply built a neural network on these to predict image_top etc. and then used it to fill those missings.",
    "349417": "Oh，i got it, thanks very much !"
  },
  "source": "meta"
}