{
  "id": 552217,
  "title": "Notebook timeout, what is the time limit?",
  "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/552217",
  "author_name": "Yicong Huang",
  "post_date": "2024-12-18T09:04:23.319000",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi, </p>\n<p>I met \"notebook timeout\" problems for recent submissions.<br>\nThe submission run for 8 hours and time out.<br>\nI wrote a simple mock predict test, that submit test data and increase 'date_id' one by one.</p>\n<pre><code> ():\n    test_data = pl.read_parquet(test_path + )\n    lag_data = pl.read_parquet(lag_path + )\n    batch_len = (test_data)\n    t1 = time.time()\n     i  (count):\n        \n        test_data = test_data.with_columns(\n            pl.lit(i).cast(pl.Int16).alias()\n        )\n        lag_data = lag_data.with_columns(\n            pl.lit(i).cast(pl.Int16).alias()\n        )\n        predict(test_data, lag_data)\n        \n    t2 = time.time()  \n    (%(count, t2-t1))\n    \n</code></pre>\n<p>The experiment result shows my code is able to do 1000 predict in about 68 sec.<br>\nThe speed looks not bad, and I've no idea why \"notebook timeout\" in the submission.</p>",
  "messages": [
    {
      "id": 3078874,
      "postDate": "2024-12-23T00:35:57.250Z",
      "content": "<p>Problem was solved.<br>\nBy optimizing the code, I'm able to reduce the time cost of 1000 predict from 68 sec to 46 sec.<br>\nAnd then the submission succeed without timeout.</p>",
      "rawMarkdown": "Problem was solved.\nBy optimizing the code, I'm able to reduce the time cost of 1000 predict from 68 sec to 46 sec.\nAnd then the submission succeed without timeout.",
      "votes": 2
    },
    {
      "id": 3074957,
      "postDate": "2024-12-18T09:04:23.320Z",
      "content": "<p>Hi, </p>\n<p>I met \"notebook timeout\" problems for recent submissions.<br>\nThe submission run for 8 hours and time out.<br>\nI wrote a simple mock predict test, that submit test data and increase 'date_id' one by one.</p>\n<pre><code> ():\n    test_data = pl.read_parquet(test_path + )\n    lag_data = pl.read_parquet(lag_path + )\n    batch_len = (test_data)\n    t1 = time.time()\n     i  (count):\n        \n        test_data = test_data.with_columns(\n            pl.lit(i).cast(pl.Int16).alias()\n        )\n        lag_data = lag_data.with_columns(\n            pl.lit(i).cast(pl.Int16).alias()\n        )\n        predict(test_data, lag_data)\n        \n    t2 = time.time()  \n    (%(count, t2-t1))\n    \n</code></pre>\n<p>The experiment result shows my code is able to do 1000 predict in about 68 sec.<br>\nThe speed looks not bad, and I've no idea why \"notebook timeout\" in the submission.</p>",
      "rawMarkdown": "Hi, \n\nI met \"notebook timeout\" problems for recent submissions.\nThe submission run for 8 hours and time out.\nI wrote a simple mock predict test, that submit test data and increase 'date_id' one by one.\n```\ndef mock_predict_test(test_path, lag_path, count, start_id):\n    test_data = pl.read_parquet(test_path + '/date_id=0/part-0.parquet')\n    lag_data = pl.read_parquet(lag_path + '/date_id=0/part-0.parquet')\n    batch_len = len(test_data)\n    t1 = time.time()\n    for i in range(count):\n        # test data_id start from 0\n        test_data = test_data.with_columns(\n            pl.lit(i).cast(pl.Int16).alias('date_id')\n        )\n        lag_data = lag_data.with_columns(\n            pl.lit(i).cast(pl.Int16).alias('date_id')\n        )\n        predict(test_data, lag_data)\n        '''\n        test_data = test_data.with_columns(\n            row_id = pl.col('row_id') + batch_len,\n        )\n        '''\n    t2 = time.time()  \n    print('Mock predict %d samples in %.2f seconds.'%(count, t2-t1))\n    return\n```\nThe experiment result shows my code is able to do 1000 predict in about 68 sec.\nThe speed looks not bad, and I've no idea why \"notebook timeout\" in the submission.",
      "votes": 2
    },
    {
      "id": 3074970,
      "postDate": "2024-12-18T09:28:23.760Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 3078874,
      "author_name": "Yicong Huang",
      "author_url": "",
      "post_date": "2024-12-23T00:35:57.250000",
      "content": "<p>Problem was solved.<br>\nBy optimizing the code, I'm able to reduce the time cost of 1000 predict from 68 sec to 46 sec.<br>\nAnd then the submission succeed without timeout.</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 3074970,
      "author_name": "",
      "author_url": "",
      "post_date": "2024-12-18T09:28:23.760000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3078874": "Problem was solved.\nBy optimizing the code, I'm able to reduce the time cost of 1000 predict from 68 sec to 46 sec.\nAnd then the submission succeed without timeout.",
    "3074957": "Hi, \n\nI met \"notebook timeout\" problems for recent submissions.\nThe submission run for 8 hours and time out.\nI wrote a simple mock predict test, that submit test data and increase 'date_id' one by one.\n```\ndef mock_predict_test(test_path, lag_path, count, start_id):\n    test_data = pl.read_parquet(test_path + '/date_id=0/part-0.parquet')\n    lag_data = pl.read_parquet(lag_path + '/date_id=0/part-0.parquet')\n    batch_len = len(test_data)\n    t1 = time.time()\n    for i in range(count):\n        # test data_id start from 0\n        test_data = test_data.with_columns(\n            pl.lit(i).cast(pl.Int16).alias('date_id')\n        )\n        lag_data = lag_data.with_columns(\n            pl.lit(i).cast(pl.Int16).alias('date_id')\n        )\n        predict(test_data, lag_data)\n        '''\n        test_data = test_data.with_columns(\n            row_id = pl.col('row_id') + batch_len,\n        )\n        '''\n    t2 = time.time()  \n    print('Mock predict %d samples in %.2f seconds.'%(count, t2-t1))\n    return\n```\nThe experiment result shows my code is able to do 1000 predict in about 68 sec.\nThe speed looks not bad, and I've no idea why \"notebook timeout\" in the submission.",
    "3074970": ""
  }
}