{
  "id": 232995,
  "title": "Best Single Model (without post processing)",
  "url": "/competitions/indoor-location-navigation/discussion/232995",
  "author_name": "Eric Freeman",
  "post_date": "2021-04-16T13:51:25.009000",
  "votes": 24,
  "comment_count": 15,
  "views": 0,
  "content": "<p>The postprocessing scripts are doing an amazing job lowering the scores.  I'm curious where people are before postprocessing.  I have a LSTM at 6.351 and a GBM at 6.670</p>",
  "messages": [
    {
      "id": 1275610,
      "postDate": "2021-04-16T13:51:25.010Z",
      "content": "<p>The postprocessing scripts are doing an amazing job lowering the scores.  I'm curious where people are before postprocessing.  I have a LSTM at 6.351 and a GBM at 6.670</p>",
      "rawMarkdown": "The postprocessing scripts are doing an amazing job lowering the scores.  I'm curious where people are before postprocessing.  I have a LSTM at 6.351 and a GBM at 6.670",
      "votes": 24
    },
    {
      "id": 1281411,
      "postDate": "2021-04-22T23:12:08.450Z",
      "content": "<p>I think a key question is if the \"delta\" features as defined in the <a href=\"https://www.kaggle.com/saitodevel01/indoor-post-processing-by-cost-minimization\" target=\"_blank\">cost minimization notebook</a> are used in training the single model. For example, my single models are something as follows:</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>Trained using Delta?</th>\n<th>Applied PP?</th>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Model A</td>\n<td>Yes</td>\n<td>No</td>\n<td>5.5</td>\n<td>4.2</td>\n</tr>\n<tr>\n<td>Model A</td>\n<td>Yes</td>\n<td>Yes</td>\n<td>2.6</td>\n<td>2.8</td>\n</tr>\n<tr>\n<td>Model B</td>\n<td>No</td>\n<td>No</td>\n<td>6.5</td>\n<td></td>\n</tr>\n<tr>\n<td>Model B</td>\n<td>No</td>\n<td>Yes</td>\n<td>2.8</td>\n<td>2.9</td>\n</tr>\n</tbody>\n</table>\n<p>where \"PP\" stands for post-processing.<br>\n<a href=\"https://www.kaggle.com/chenxin1991\" target=\"_blank\">@chenxin1991</a> Please see above.</p>",
      "rawMarkdown": "I think a key question is if the \"delta\" features as defined in the [cost minimization notebook](https://www.kaggle.com/saitodevel01/indoor-post-processing-by-cost-minimization) are used in training the single model. For example, my single models are something as follows:\n\n|  | Trained using Delta? | Applied PP? | CV | LB |\n| --- | --- | --- | --- |\n| Model A | Yes | No | 5.5 | 4.2 |\n| Model A | Yes | Yes | 2.6 | 2.8 |\n| Model B | No | No | 6.5 |  |\n| Model B | No | Yes | 2.8 | 2.9  |\n\nwhere \"PP\" stands for post-processing.\n@chenxin1991 Please see above.",
      "votes": 9,
      "replies": [
        {
          "id": 1281451,
          "postDate": "2021-04-23T00:50:30.637Z",
          "content": "<p>Thanks! I notice the \"delta\" features too.It's worth trying.</p>",
          "rawMarkdown": "Thanks! I notice the \"delta\" features too.It's worth trying."
        },
        {
          "id": 1281862,
          "postDate": "2021-04-23T11:20:48.770Z",
          "content": "<p>Thank you for sharing, <a href=\"https://www.kaggle.com/jiweiliu\" target=\"_blank\">@jiweiliu</a>. I think with your table, the delta should not be a key, or I am missing something? Because by comparing two models with and without delta, the CV and LB of these two more are not so different. You have 6.5 CV with a model without delta and pp, we somehow have a similar CV, but our pp is not as good as yours. So, I think pp is still a key here, delta waypoints may help, but not so much. However, please correct me if I am wrong.</p>",
          "rawMarkdown": "Thank you for sharing, @jiweiliu. I think with your table, the delta should not be a key, or I am missing something? Because by comparing two models with and without delta, the CV and LB of these two more are not so different. You have 6.5 CV with a model without delta and pp, we somehow have a similar CV, but our pp is not as good as yours. So, I think pp is still a key here, delta waypoints may help, but not so much. However, please correct me if I am wrong.",
          "votes": 3
        },
        {
          "id": 1281893,
          "postDate": "2021-04-23T12:09:10.033Z",
          "content": "<p>Yeah, absolutely right. I just mean when talking about single models in this thread, it is better to clarify if delta is used in training.</p>",
          "rawMarkdown": "Yeah, absolutely right. I just mean when talking about single models in this thread, it is better to clarify if delta is used in training.",
          "votes": 4
        }
      ]
    },
    {
      "id": 1275625,
      "postDate": "2021-04-16T14:01:47.873Z",
      "content": "<p>That's about what I was getting with a single model too… I keep thinking there has to be a way to get a single giant model to work better, but I keep having a really hard time getting one to train :)</p>",
      "rawMarkdown": "That's about what I was getting with a single model too... I keep thinking there has to be a way to get a single giant model to work better, but I keep having a really hard time getting one to train :)",
      "votes": 5,
      "replies": [
        {
          "id": 1275960,
          "postDate": "2021-04-16T22:43:32.147Z",
          "content": "<p>Interesting I have a single model with LB at 5.728. So apparently there's room for post-processing for my case.</p>",
          "rawMarkdown": "Interesting I have a single model with LB at 5.728. So apparently there's room for post-processing for my case.",
          "votes": 1
        },
        {
          "id": 1277048,
          "postDate": "2021-04-18T10:40:18.507Z",
          "content": "<p><a href=\"https://www.kaggle.com/higepon\" target=\"_blank\">@higepon</a> My best single rnn model is 5.7 too, so the key of this competions is the method of prost-processing? </p>",
          "rawMarkdown": "@higepon My best single rnn model is 5.7 too, so the key of this competions is the method of prost-processing? "
        },
        {
          "id": 1277060,
          "postDate": "2021-04-18T10:55:32.323Z",
          "content": "<p>That is true so far.<br>\nYou may want to check my notebook <a href=\"https://www.kaggle.com/higepon/visualize-submissions-with-post-processing\" target=\"_blank\">here</a> to see how other post processing may work.</p>",
          "rawMarkdown": "That is true so far.\nYou may want to check my notebook [here](https://www.kaggle.com/higepon/visualize-submissions-with-post-processing) to see how other post processing may work."
        },
        {
          "id": 1277892,
          "postDate": "2021-04-19T10:48:30.803Z",
          "content": "<p>Those are nice results from just the base model, congratz! May I ask if you are using just BSSID-RSSID Wifi features or you incorporated more features?</p>",
          "rawMarkdown": "Those are nice results from just the base model, congratz! May I ask if you are using just BSSID-RSSID Wifi features or you incorporated more features?"
        },
        {
          "id": 1278475,
          "postDate": "2021-04-19T23:55:01.990Z",
          "content": "<p>I'm using slightly different feature set for my model.</p>",
          "rawMarkdown": "I'm using slightly different feature set for my model."
        }
      ]
    },
    {
      "id": 1280421,
      "postDate": "2021-04-21T23:51:45.247Z",
      "content": "<p>My best single model is a custom NN which score 5.4X (is a single model, no seed averaging, neither training in different versions of dataset). I haven't worked yet on post processing.</p>",
      "rawMarkdown": "My best single model is a custom NN which score 5.4X (is a single model, no seed averaging, neither training in different versions of dataset). I haven't worked yet on post processing.",
      "votes": 1
    },
    {
      "id": 1276987,
      "postDate": "2021-04-18T08:23:18.247Z",
      "content": "<p>I have a MLP at 5.972. My post-processing seems very poor.</p>",
      "rawMarkdown": "I have a MLP at 5.972. My post-processing seems very poor."
    },
    {
      "id": 1276171,
      "postDate": "2021-04-17T07:36:33.617Z",
      "content": "<p>Lol i had exactly the same socres! </p>",
      "rawMarkdown": "Lol i had exactly the same socres! "
    },
    {
      "id": 1276981,
      "postDate": "2021-04-18T08:04:30.293Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 1276374,
      "postDate": "2021-04-17T13:32:02.007Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1281411,
      "author_name": "Jiwei Liu",
      "author_url": "",
      "post_date": "2021-04-22T23:12:08.450000",
      "content": "<p>I think a key question is if the \"delta\" features as defined in the <a href=\"https://www.kaggle.com/saitodevel01/indoor-post-processing-by-cost-minimization\" target=\"_blank\">cost minimization notebook</a> are used in training the single model. For example, my single models are something as follows:</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>Trained using Delta?</th>\n<th>Applied PP?</th>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Model A</td>\n<td>Yes</td>\n<td>No</td>\n<td>5.5</td>\n<td>4.2</td>\n</tr>\n<tr>\n<td>Model A</td>\n<td>Yes</td>\n<td>Yes</td>\n<td>2.6</td>\n<td>2.8</td>\n</tr>\n<tr>\n<td>Model B</td>\n<td>No</td>\n<td>No</td>\n<td>6.5</td>\n<td></td>\n</tr>\n<tr>\n<td>Model B</td>\n<td>No</td>\n<td>Yes</td>\n<td>2.8</td>\n<td>2.9</td>\n</tr>\n</tbody>\n</table>\n<p>where \"PP\" stands for post-processing.<br>\n<a href=\"https://www.kaggle.com/chenxin1991\" target=\"_blank\">@chenxin1991</a> Please see above.</p>",
      "votes": 9,
      "replies": [
        {
          "id": 1281451,
          "author_name": "Ethan",
          "author_url": "",
          "post_date": "2021-04-23T00:50:30.637000",
          "content": "<p>Thanks! I notice the \"delta\" features too.It's worth trying.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1281862,
          "author_name": "Minh Tri Phan",
          "author_url": "",
          "post_date": "2021-04-23T11:20:48.770000",
          "content": "<p>Thank you for sharing, <a href=\"https://www.kaggle.com/jiweiliu\" target=\"_blank\">@jiweiliu</a>. I think with your table, the delta should not be a key, or I am missing something? Because by comparing two models with and without delta, the CV and LB of these two more are not so different. You have 6.5 CV with a model without delta and pp, we somehow have a similar CV, but our pp is not as good as yours. So, I think pp is still a key here, delta waypoints may help, but not so much. However, please correct me if I am wrong.</p>",
          "votes": 3,
          "replies": []
        },
        {
          "id": 1281893,
          "author_name": "Jiwei Liu",
          "author_url": "",
          "post_date": "2021-04-23T12:09:10.033000",
          "content": "<p>Yeah, absolutely right. I just mean when talking about single models in this thread, it is better to clarify if delta is used in training.</p>",
          "votes": 4,
          "replies": []
        }
      ]
    },
    {
      "id": 1275625,
      "author_name": "chris",
      "author_url": "",
      "post_date": "2021-04-16T14:01:47.873000",
      "content": "<p>That's about what I was getting with a single model too… I keep thinking there has to be a way to get a single giant model to work better, but I keep having a really hard time getting one to train :)</p>",
      "votes": 5,
      "replies": [
        {
          "id": 1275960,
          "author_name": "higepon",
          "author_url": "",
          "post_date": "2021-04-16T22:43:32.147000",
          "content": "<p>Interesting I have a single model with LB at 5.728. So apparently there's room for post-processing for my case.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1277048,
          "author_name": "Ethan",
          "author_url": "",
          "post_date": "2021-04-18T10:40:18.507000",
          "content": "<p><a href=\"https://www.kaggle.com/higepon\" target=\"_blank\">@higepon</a> My best single rnn model is 5.7 too, so the key of this competions is the method of prost-processing? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1277060,
          "author_name": "higepon",
          "author_url": "",
          "post_date": "2021-04-18T10:55:32.323000",
          "content": "<p>That is true so far.<br>\nYou may want to check my notebook <a href=\"https://www.kaggle.com/higepon/visualize-submissions-with-post-processing\" target=\"_blank\">here</a> to see how other post processing may work.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1277892,
          "author_name": "JQ",
          "author_url": "",
          "post_date": "2021-04-19T10:48:30.803000",
          "content": "<p>Those are nice results from just the base model, congratz! May I ask if you are using just BSSID-RSSID Wifi features or you incorporated more features?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1278475,
          "author_name": "higepon",
          "author_url": "",
          "post_date": "2021-04-19T23:55:01.990000",
          "content": "<p>I'm using slightly different feature set for my model.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1280421,
      "author_name": "mavillan",
      "author_url": "",
      "post_date": "2021-04-21T23:51:45.247000",
      "content": "<p>My best single model is a custom NN which score 5.4X (is a single model, no seed averaging, neither training in different versions of dataset). I haven't worked yet on post processing.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1276987,
      "author_name": "Ebi",
      "author_url": "",
      "post_date": "2021-04-18T08:23:18.247000",
      "content": "<p>I have a MLP at 5.972. My post-processing seems very poor.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1276171,
      "author_name": "Abhishek252",
      "author_url": "",
      "post_date": "2021-04-17T07:36:33.617000",
      "content": "<p>Lol i had exactly the same socres! </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1276981,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-04-18T08:04:30.293000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1276374,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-04-17T13:32:02.007000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1275610": "The postprocessing scripts are doing an amazing job lowering the scores.  I'm curious where people are before postprocessing.  I have a LSTM at 6.351 and a GBM at 6.670",
    "1281411": "I think a key question is if the \"delta\" features as defined in the [cost minimization notebook](https://www.kaggle.com/saitodevel01/indoor-post-processing-by-cost-minimization) are used in training the single model. For example, my single models are something as follows:\n\n|  | Trained using Delta? | Applied PP? | CV | LB |\n| --- | --- | --- | --- |\n| Model A | Yes | No | 5.5 | 4.2 |\n| Model A | Yes | Yes | 2.6 | 2.8 |\n| Model B | No | No | 6.5 |  |\n| Model B | No | Yes | 2.8 | 2.9  |\n\nwhere \"PP\" stands for post-processing.\n@chenxin1991 Please see above.",
    "1275625": "That's about what I was getting with a single model too... I keep thinking there has to be a way to get a single giant model to work better, but I keep having a really hard time getting one to train :)",
    "1280421": "My best single model is a custom NN which score 5.4X (is a single model, no seed averaging, neither training in different versions of dataset). I haven't worked yet on post processing.",
    "1276987": "I have a MLP at 5.972. My post-processing seems very poor.",
    "1276171": "Lol i had exactly the same socres! ",
    "1276981": "",
    "1276374": ""
  }
}