{
  "id": 554278,
  "title": "How long does a test submission take to process?",
  "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/554278",
  "author_name": "",
  "post_date": "2024-12-31T15:57:44.643489500Z",
  "votes": 2,
  "comment_count": 16,
  "views": 0,
  "content": "<p>First time participating in a competition…</p>\n<p>I just copied one of the public notebooks (<a href=\"https://www.kaggle.com/code/taikimori/lb-0-0077-micro-boost-tiny-change-for-a-0-001\" target=\"_blank\">https://www.kaggle.com/code/taikimori/lb-0-0077-micro-boost-tiny-change-for-a-0-001</a>) and submitted it.</p>\n<p>This notebook takes ~2 minutes to run on Kaggle.</p>\n<p>It has now been running for over an hour after submitting.</p>\n<p>Is this normal? If not, what might I be missing?</p>",
  "messages": [
    {
      "id": "3085039",
      "postDate": "12/31/2024 15:57:44",
      "content": "<p>First time participating in a competition…</p>\n<p>I just copied one of the public notebooks (<a href=\"https://www.kaggle.com/code/taikimori/lb-0-0077-micro-boost-tiny-change-for-a-0-001\" target=\"_blank\">https://www.kaggle.com/code/taikimori/lb-0-0077-micro-boost-tiny-change-for-a-0-001</a>) and submitted it.</p>\n<p>This notebook takes ~2 minutes to run on Kaggle.</p>\n<p>It has now been running for over an hour after submitting.</p>\n<p>Is this normal? If not, what might I be missing?</p>",
      "rawMarkdown": "First time participating in a competition...\n\nI just copied one of the public notebooks (https://www.kaggle.com/code/taikimori/lb-0-0077-micro-boost-tiny-change-for-a-0-001) and submitted it.\n\nThis notebook takes ~2 minutes to run on Kaggle.\n\nIt has now been running for over an hour after submitting.\n\nIs this normal? If not, what might I be missing?",
      "votes": null
    },
    {
      "id": "3085048",
      "postDate": "12/31/2024 16:14:32",
      "content": "<p>My notebook usually take about 80min, and used to be very stable. But today they are running much longger and still run not sure if by newly added mistakes or due to platform issues.</p>",
      "rawMarkdown": "My notebook usually take about 80min, and used to be very stable. But today they are running much longger and still run not sure if by newly added mistakes or due to platform issues.",
      "votes": null
    },
    {
      "id": "3085055",
      "postDate": "12/31/2024 16:21:47",
      "content": "<p>Thanks. This is extremely helpful. </p>",
      "rawMarkdown": "Thanks. This is extremely helpful.",
      "votes": null
    },
    {
      "id": "3085059",
      "postDate": "12/31/2024 16:24:17",
      "content": "<p>Normaly I suppose this public notebook will take more then 2 hours as it has nn and tree model and nn is 5 folds.</p>",
      "rawMarkdown": "Normaly I suppose this public notebook will take more then 2 hours as it has nn and tree model and nn is 5 folds.",
      "votes": null
    },
    {
      "id": "3085129",
      "postDate": "12/31/2024 17:52:39",
      "content": "<p>3 hours and counting…</p>",
      "rawMarkdown": "3 hours and counting...",
      "votes": null
    },
    {
      "id": "3085154",
      "postDate": "12/31/2024 18:52:38",
      "content": "<p>Now four hours…</p>",
      "rawMarkdown": "Now four hours...",
      "votes": null
    },
    {
      "id": "3085173",
      "postDate": "12/31/2024 19:34:57",
      "content": "<p>My inference is over 6 hours, and I don't know if players on top LB have such a similar high duration.</p>",
      "rawMarkdown": "My inference is over 6 hours, and I don't know if players on top LB have such a similar high duration.",
      "votes": null
    },
    {
      "id": "3085182",
      "postDate": "12/31/2024 19:43:19",
      "content": "<p>Interesting information.</p>\n<p>But I think I'm saying something different. I'm saying that on Kaggle, my notebook runs in two minutes. But when I submit the notebook, it takes over 4 hours to run.</p>\n<p>Is there a difference between running your notebook on Kaggle (or locally) and running it as a submission? If so, why do you think this is happening?</p>",
      "rawMarkdown": "Interesting information.\n\nBut I think I'm saying something different. I'm saying that on Kaggle, my notebook runs in two minutes. But when I submit the notebook, it takes over 4 hours to run.\n\nIs there a difference between running your notebook on Kaggle (or locally) and running it as a submission? If so, why do you think this is happening?",
      "votes": null
    },
    {
      "id": "3085189",
      "postDate": "12/31/2024 19:58:05",
      "content": "<p>I suspect there might be an issue with the infrastructure. My notebook, which usually takes around 2 hours to complete scoring, is now taking over 4 hours despite no changes to the code. <br>\nIt might be worth reaching out to support to investigate further.</p>",
      "rawMarkdown": "I suspect there might be an issue with the infrastructure. My notebook, which usually takes around 2 hours to complete scoring, is now taking over 4 hours despite no changes to the code. \nIt might be worth reaching out to support to investigate further.",
      "votes": null
    },
    {
      "id": "3085190",
      "postDate": "12/31/2024 20:00:15",
      "content": "<p>Thanks, very helpful information.</p>",
      "rawMarkdown": "Thanks, very helpful information.",
      "votes": null
    },
    {
      "id": "3085191",
      "postDate": "12/31/2024 20:00:24",
      "content": "<p>Five hours…</p>",
      "rawMarkdown": "Five hours...",
      "votes": null
    },
    {
      "id": "3085196",
      "postDate": "12/31/2024 20:23:04",
      "content": "<p>Regarding to your question, it's because when you run your pipeline before submission in kaggle, it will run with a test sample, which is visible to you and quite small. But when you submit, the test data will be replaced with much larger dataset (currently it's about 200 date_ids) that is invisible to you offiline. This is quite normal in most competitions in kaggle nowadays to avoid overfitting to test data. So normally it will take several hours in current stage depending on how complicating your models are, which is expected. </p>",
      "rawMarkdown": "Regarding to your question, it's because when you run your pipeline before submission in kaggle, it will run with a test sample, which is visible to you and quite small. But when you submit, the test data will be replaced with much larger dataset (currently it's about 200 date_ids) that is invisible to you offiline. This is quite normal in most competitions in kaggle nowadays to avoid overfitting to test data. So normally it will take several hours in current stage depending on how complicating your models are, which is expected.",
      "votes": null
    },
    {
      "id": "3085425",
      "postDate": "01/01/2025 06:29:58",
      "content": "<p>There's new data released periodically, could be one reason</p>",
      "rawMarkdown": "There's new data released periodically, could be one reason",
      "votes": null
    },
    {
      "id": "3085574",
      "postDate": "01/01/2025 10:36:16",
      "content": "<p>6hrs would be too long for the forecast period. Maximum running time is 9hrs, and there are more data during the forecasting period.</p>",
      "rawMarkdown": "6hrs would be too long for the forecast period. Maximum running time is 9hrs, and there are more data during the forecasting period.",
      "votes": null
    },
    {
      "id": "3085591",
      "postDate": "01/01/2025 10:54:50",
      "content": "<p>Hey, one remainder in case you missed this important info </p>\n<blockquote>\n  <p>is_scored - Indicates whether this row is included in the evaluation metric calculation.</p>\n</blockquote>\n<p>So technically, you only need to forecast for the final about 120 date_ids data and for the rest with is_scored == False, since it doesn't really matter, so you can just predict with 0, for example, which is most people did in previous Optiver competition. </p>",
      "rawMarkdown": "Hey, one remainder in case you missed this important info \n>is_scored - Indicates whether this row is included in the evaluation metric calculation.\n\nSo technically, you only need to forecast for the final about 120 date_ids data and for the rest with is_scored == False, since it doesn't really matter, so you can just predict with 0, for example, which is most people did in previous Optiver competition.",
      "votes": null
    },
    {
      "id": "3085691",
      "postDate": "01/01/2025 13:03:10",
      "content": "<p>Thank you. This helps a lot!</p>",
      "rawMarkdown": "Thank you. This helps a lot!",
      "votes": null
    },
    {
      "id": "3085692",
      "postDate": "01/01/2025 13:08:22",
      "content": "<p>Thanks! will surely do that :)</p>",
      "rawMarkdown": "Thanks! will surely do that :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3085048,
      "author_name": "goldenlock",
      "author_url": "",
      "post_date": "12/31/2024 16:14:32",
      "content": "<p>My notebook usually take about 80min, and used to be very stable. But today they are running much longger and still run not sure if by newly added mistakes or due to platform issues.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3085055,
          "author_name": "michaeldelamaza",
          "author_url": "",
          "post_date": "12/31/2024 16:21:47",
          "content": "<p>Thanks. This is extremely helpful. </p>",
          "votes": null,
          "replies": [
            {
              "id": 3085059,
              "author_name": "goldenlock",
              "author_url": "",
              "post_date": "12/31/2024 16:24:17",
              "content": "<p>Normaly I suppose this public notebook will take more then 2 hours as it has nn and tree model and nn is 5 folds.</p>",
              "votes": null,
              "replies": [
                {
                  "id": 3085129,
                  "author_name": "michaeldelamaza",
                  "author_url": "",
                  "post_date": "12/31/2024 17:52:39",
                  "content": "<p>3 hours and counting…</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 3085154,
                      "author_name": "michaeldelamaza",
                      "author_url": "",
                      "post_date": "12/31/2024 18:52:38",
                      "content": "<p>Now four hours…</p>",
                      "votes": null,
                      "replies": [
                        {
                          "id": 3085191,
                          "author_name": "michaeldelamaza",
                          "author_url": "",
                          "post_date": "12/31/2024 20:00:24",
                          "content": "<p>Five hours…</p>",
                          "votes": null,
                          "replies": []
                        }
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 3085173,
      "author_name": "xstargate",
      "author_url": "",
      "post_date": "12/31/2024 19:34:57",
      "content": "<p>My inference is over 6 hours, and I don't know if players on top LB have such a similar high duration.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3085182,
          "author_name": "michaeldelamaza",
          "author_url": "",
          "post_date": "12/31/2024 19:43:19",
          "content": "<p>Interesting information.</p>\n<p>But I think I'm saying something different. I'm saying that on Kaggle, my notebook runs in two minutes. But when I submit the notebook, it takes over 4 hours to run.</p>\n<p>Is there a difference between running your notebook on Kaggle (or locally) and running it as a submission? If so, why do you think this is happening?</p>",
          "votes": null,
          "replies": [
            {
              "id": 3085196,
              "author_name": "lihaorocky",
              "author_url": "",
              "post_date": "12/31/2024 20:23:04",
              "content": "<p>Regarding to your question, it's because when you run your pipeline before submission in kaggle, it will run with a test sample, which is visible to you and quite small. But when you submit, the test data will be replaced with much larger dataset (currently it's about 200 date_ids) that is invisible to you offiline. This is quite normal in most competitions in kaggle nowadays to avoid overfitting to test data. So normally it will take several hours in current stage depending on how complicating your models are, which is expected. </p>",
              "votes": null,
              "replies": [
                {
                  "id": 3085691,
                  "author_name": "michaeldelamaza",
                  "author_url": "",
                  "post_date": "01/01/2025 13:03:10",
                  "content": "<p>Thank you. This helps a lot!</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        },
        {
          "id": 3085574,
          "author_name": "shiyili",
          "author_url": "",
          "post_date": "01/01/2025 10:36:16",
          "content": "<p>6hrs would be too long for the forecast period. Maximum running time is 9hrs, and there are more data during the forecasting period.</p>",
          "votes": null,
          "replies": [
            {
              "id": 3085591,
              "author_name": "lihaorocky",
              "author_url": "",
              "post_date": "01/01/2025 10:54:50",
              "content": "<p>Hey, one remainder in case you missed this important info </p>\n<blockquote>\n  <p>is_scored - Indicates whether this row is included in the evaluation metric calculation.</p>\n</blockquote>\n<p>So technically, you only need to forecast for the final about 120 date_ids data and for the rest with is_scored == False, since it doesn't really matter, so you can just predict with 0, for example, which is most people did in previous Optiver competition. </p>",
              "votes": null,
              "replies": [
                {
                  "id": 3085692,
                  "author_name": "shiyili",
                  "author_url": "",
                  "post_date": "01/01/2025 13:08:22",
                  "content": "<p>Thanks! will surely do that :)</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 3085189,
      "author_name": "daebak2021",
      "author_url": "",
      "post_date": "12/31/2024 19:58:05",
      "content": "<p>I suspect there might be an issue with the infrastructure. My notebook, which usually takes around 2 hours to complete scoring, is now taking over 4 hours despite no changes to the code. <br>\nIt might be worth reaching out to support to investigate further.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3085190,
          "author_name": "michaeldelamaza",
          "author_url": "",
          "post_date": "12/31/2024 20:00:15",
          "content": "<p>Thanks, very helpful information.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3085425,
      "author_name": "zhangyue199",
      "author_url": "",
      "post_date": "01/01/2025 06:29:58",
      "content": "<p>There's new data released periodically, could be one reason</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3085039": "First time participating in a competition...\n\nI just copied one of the public notebooks (https://www.kaggle.com/code/taikimori/lb-0-0077-micro-boost-tiny-change-for-a-0-001) and submitted it.\n\nThis notebook takes ~2 minutes to run on Kaggle.\n\nIt has now been running for over an hour after submitting.\n\nIs this normal? If not, what might I be missing?",
    "3085048": "My notebook usually take about 80min, and used to be very stable. But today they are running much longger and still run not sure if by newly added mistakes or due to platform issues.",
    "3085055": "Thanks. This is extremely helpful.",
    "3085059": "Normaly I suppose this public notebook will take more then 2 hours as it has nn and tree model and nn is 5 folds.",
    "3085129": "3 hours and counting...",
    "3085154": "Now four hours...",
    "3085173": "My inference is over 6 hours, and I don't know if players on top LB have such a similar high duration.",
    "3085182": "Interesting information.\n\nBut I think I'm saying something different. I'm saying that on Kaggle, my notebook runs in two minutes. But when I submit the notebook, it takes over 4 hours to run.\n\nIs there a difference between running your notebook on Kaggle (or locally) and running it as a submission? If so, why do you think this is happening?",
    "3085189": "I suspect there might be an issue with the infrastructure. My notebook, which usually takes around 2 hours to complete scoring, is now taking over 4 hours despite no changes to the code. \nIt might be worth reaching out to support to investigate further.",
    "3085190": "Thanks, very helpful information.",
    "3085191": "Five hours...",
    "3085196": "Regarding to your question, it's because when you run your pipeline before submission in kaggle, it will run with a test sample, which is visible to you and quite small. But when you submit, the test data will be replaced with much larger dataset (currently it's about 200 date_ids) that is invisible to you offiline. This is quite normal in most competitions in kaggle nowadays to avoid overfitting to test data. So normally it will take several hours in current stage depending on how complicating your models are, which is expected.",
    "3085425": "There's new data released periodically, could be one reason",
    "3085574": "6hrs would be too long for the forecast period. Maximum running time is 9hrs, and there are more data during the forecasting period.",
    "3085591": "Hey, one remainder in case you missed this important info \n>is_scored - Indicates whether this row is included in the evaluation metric calculation.\n\nSo technically, you only need to forecast for the final about 120 date_ids data and for the rest with is_scored == False, since it doesn't really matter, so you can just predict with 0, for example, which is most people did in previous Optiver competition.",
    "3085691": "Thank you. This helps a lot!",
    "3085692": "Thanks! will surely do that :)"
  },
  "source": "meta"
}