{
  "id": 545117,
  "title": "Feature types",
  "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/545117",
  "author_name": "",
  "post_date": "2024-11-08T12:45:20.184395600Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>What is a type of feature_09?<br>\nIn train.parquet it has type i8<br>\nIn test.parquet it has type f64</p>\n<p>Why types are different?</p>",
  "messages": [
    {
      "id": "3039850",
      "postDate": "11/08/2024 12:45:20",
      "content": "<p>What is a type of feature_09?<br>\nIn train.parquet it has type i8<br>\nIn test.parquet it has type f64</p>\n<p>Why types are different?</p>",
      "rawMarkdown": "What is a type of feature_09?\nIn train.parquet it has type i8\nIn test.parquet it has type f64\n\nWhy types are different?",
      "votes": null
    },
    {
      "id": "3041499",
      "postDate": "11/10/2024 12:56:16",
      "content": "<p>I am also having a horrendous time here. I am recasting all features to Float32 just to be safe</p>\n<pre><code> () -&gt; pl.DataFrame:\n     df.with_columns(\n        pl.col().cast(pl.Int8),\n        pl.col().cast(pl.Int64),\n        pl.col().cast(pl.Int16),\n        pl.col().cast(pl.Int16),\n        *[pl.col().cast(pl.Float32)  i  ()],\n        *[pl.col().cast(pl.Float32)  i  ()],\n    )\n</code></pre>\n<p>The row_id also seems to be changing between UInt32 and Int64 between days in the private test set</p>",
      "rawMarkdown": "I am also having a horrendous time here. I am recasting all features to Float32 just to be safe\n\n```python\ndef recast_data(df: pl.DataFrame) -> pl.DataFrame:\n    return df.with_columns(\n        pl.col(\"symbol_id\").cast(pl.Int8),\n        pl.col(\"row_id\").cast(pl.Int64),\n        pl.col(\"date_id\").cast(pl.Int16),\n        pl.col(\"time_id\").cast(pl.Int16),\n        *[pl.col(f\"feature_{i:02}\").cast(pl.Float32) for i in range(79)],\n        *[pl.col(f\"responder_{i}\").cast(pl.Float32) for i in range(9)],\n    )\n```\n\nThe row_id also seems to be changing between UInt32 and Int64 between days in the private test set",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3041499,
      "author_name": "michaeltimbs",
      "author_url": "",
      "post_date": "11/10/2024 12:56:16",
      "content": "<p>I am also having a horrendous time here. I am recasting all features to Float32 just to be safe</p>\n<pre><code> () -&gt; pl.DataFrame:\n     df.with_columns(\n        pl.col().cast(pl.Int8),\n        pl.col().cast(pl.Int64),\n        pl.col().cast(pl.Int16),\n        pl.col().cast(pl.Int16),\n        *[pl.col().cast(pl.Float32)  i  ()],\n        *[pl.col().cast(pl.Float32)  i  ()],\n    )\n</code></pre>\n<p>The row_id also seems to be changing between UInt32 and Int64 between days in the private test set</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3039850": "What is a type of feature_09?\nIn train.parquet it has type i8\nIn test.parquet it has type f64\n\nWhy types are different?",
    "3041499": "I am also having a horrendous time here. I am recasting all features to Float32 just to be safe\n\n```python\ndef recast_data(df: pl.DataFrame) -> pl.DataFrame:\n    return df.with_columns(\n        pl.col(\"symbol_id\").cast(pl.Int8),\n        pl.col(\"row_id\").cast(pl.Int64),\n        pl.col(\"date_id\").cast(pl.Int16),\n        pl.col(\"time_id\").cast(pl.Int16),\n        *[pl.col(f\"feature_{i:02}\").cast(pl.Float32) for i in range(79)],\n        *[pl.col(f\"responder_{i}\").cast(pl.Float32) for i in range(9)],\n    )\n```\n\nThe row_id also seems to be changing between UInt32 and Int64 between days in the private test set"
  },
  "source": "meta"
}