{
  "id": 540668,
  "title": "unclear points about the data ",
  "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/540668",
  "author_name": "",
  "post_date": "2024-10-15T16:17:57.398447600Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>first point: shall we use other responders as features?<br>\nWill the other responders, besides responder_6, be available at inference time? Can we use them as training features or not?</p>\n<p>second point: how shall we deal with the rows that include nans. They are many that you can't just drop them all or impute them all, and since the features names are anonymous we can't really make a good reasoning for why are they missing and so we can't handle them.</p>",
  "messages": [
    {
      "id": "3018238",
      "postDate": "10/15/2024 16:17:57",
      "content": "<p>first point: shall we use other responders as features?<br>\nWill the other responders, besides responder_6, be available at inference time? Can we use them as training features or not?</p>\n<p>second point: how shall we deal with the rows that include nans. They are many that you can't just drop them all or impute them all, and since the features names are anonymous we can't really make a good reasoning for why are they missing and so we can't handle them.</p>",
      "rawMarkdown": "first point: shall we use other responders as features?\nWill the other responders, besides responder_6, be available at inference time? Can we use them as training features or not?\n\nsecond point: how shall we deal with the rows that include nans. They are many that you can't just drop them all or impute them all, and since the features names are anonymous we can't really make a good reasoning for why are they missing and so we can't handle them.",
      "votes": null
    },
    {
      "id": "3018627",
      "postDate": "10/15/2024 23:27:24",
      "content": "<p>first: other responders are not available during inference. <br>\nsecond: it is your choice how to deal with NaNs (yes we don't know why these data are missing, but still :(</p>",
      "rawMarkdown": "first: other responders are not available during inference. \nsecond: it is your choice how to deal with NaNs (yes we don't know why these data are missing, but still :(",
      "votes": null
    },
    {
      "id": "3019072",
      "postDate": "10/16/2024 08:28:33",
      "content": "<p>I mean you can't deal with missing data without understanding why they are missing and without understanding what the feature is about in the first place </p>",
      "rawMarkdown": "I mean you can't deal with missing data without understanding why they are missing and without understanding what the feature is about in the first place",
      "votes": null
    },
    {
      "id": "3019126",
      "postDate": "10/16/2024 09:27:52",
      "content": "<p>Yes. But I dont think the host will share much info about the features. EDA is the only weapon (or maybe some reverse enginnering) to understand the feature. As for why they are missing, I think it is simple to answer: because it is how the market works :(</p>",
      "rawMarkdown": "Yes. But I dont think the host will share much info about the features. EDA is the only weapon (or maybe some reverse enginnering) to understand the feature. As for why they are missing, I think it is simple to answer: because it is how the market works :(",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3018627,
      "author_name": "shiyili",
      "author_url": "",
      "post_date": "10/15/2024 23:27:24",
      "content": "<p>first: other responders are not available during inference. <br>\nsecond: it is your choice how to deal with NaNs (yes we don't know why these data are missing, but still :(</p>",
      "votes": null,
      "replies": [
        {
          "id": 3019072,
          "author_name": "aymanallawi",
          "author_url": "",
          "post_date": "10/16/2024 08:28:33",
          "content": "<p>I mean you can't deal with missing data without understanding why they are missing and without understanding what the feature is about in the first place </p>",
          "votes": null,
          "replies": [
            {
              "id": 3019126,
              "author_name": "shiyili",
              "author_url": "",
              "post_date": "10/16/2024 09:27:52",
              "content": "<p>Yes. But I dont think the host will share much info about the features. EDA is the only weapon (or maybe some reverse enginnering) to understand the feature. As for why they are missing, I think it is simple to answer: because it is how the market works :(</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3018238": "first point: shall we use other responders as features?\nWill the other responders, besides responder_6, be available at inference time? Can we use them as training features or not?\n\nsecond point: how shall we deal with the rows that include nans. They are many that you can't just drop them all or impute them all, and since the features names are anonymous we can't really make a good reasoning for why are they missing and so we can't handle them.",
    "3018627": "first: other responders are not available during inference. \nsecond: it is your choice how to deal with NaNs (yes we don't know why these data are missing, but still :(",
    "3019072": "I mean you can't deal with missing data without understanding why they are missing and without understanding what the feature is about in the first place",
    "3019126": "Yes. But I dont think the host will share much info about the features. EDA is the only weapon (or maybe some reverse enginnering) to understand the feature. As for why they are missing, I think it is simple to answer: because it is how the market works :("
  },
  "source": "meta"
}