{
  "id": 579166,
  "title": "Regarding future notebook execution time limits",
  "url": "/competitions/stanford-rna-3d-folding/discussion/579166",
  "author_name": "",
  "post_date": "2025-05-15T16:14:20.135034700Z",
  "votes": 1,
  "comment_count": 7,
  "views": 0,
  "content": "<p>To the competition hosts</p>\n<p>First of all, thank you for hosting such a worthwhile competition</p>\n<p>I have one question.<br>\nIf the execution time limit is met with the current test data, can we assume that it will also be met in future data evaluation phases?</p>\n<p><a href=\"https://www.kaggle.com/rhijudas\" target=\"_blank\">@rhijudas</a></p>",
  "messages": [
    {
      "id": "3202601",
      "postDate": "05/15/2025 16:14:20",
      "content": "<p>To the competition hosts</p>\n<p>First of all, thank you for hosting such a worthwhile competition</p>\n<p>I have one question.<br>\nIf the execution time limit is met with the current test data, can we assume that it will also be met in future data evaluation phases?</p>\n<p><a href=\"https://www.kaggle.com/rhijudas\" target=\"_blank\">@rhijudas</a></p>",
      "rawMarkdown": "To the competition hosts\n\nFirst of all, thank you for hosting such a worthwhile competition\n\nI have one question.\nIf the execution time limit is met with the current test data, can we assume that it will also be met in future data evaluation phases?\n\n@rhijudas",
      "votes": null
    },
    {
      "id": "3202756",
      "postDate": "05/15/2025 21:01:58",
      "content": "<p>That is one of my main concerns. Execution time depends drastically on the length of each sequence unless this is previously accounted for. If the number of sequences and their length distribution remains the same I think it would not be a big deal. </p>",
      "rawMarkdown": "That is one of my main concerns. Execution time depends drastically on the length of each sequence unless this is previously accounted for. If the number of sequences and their length distribution remains the same I think it would not be a big deal.",
      "votes": null
    },
    {
      "id": "3203427",
      "postDate": "05/16/2025 18:05:03",
      "content": "<p>The number of sequences and length distribution in the future data set are expected to be similar to the current hidden test set.</p>",
      "rawMarkdown": "The number of sequences and length distribution in the future data set are expected to be similar to the current hidden test set.",
      "votes": null
    },
    {
      "id": "3203491",
      "postDate": "05/16/2025 19:49:48",
      "content": "<p>Thanks for the clarification <a href=\"https://www.kaggle.com/rhijudas\" target=\"_blank\">@rhijudas</a> !</p>",
      "rawMarkdown": "Thanks for the clarification @rhijudas !",
      "votes": null
    },
    {
      "id": "3203982",
      "postDate": "05/17/2025 15:57:20",
      "content": "<p>Thanks for both of your replies!</p>\n<p>So, if the time limit is met with the current notebook, does that mean it will be met with future test data as well?</p>",
      "rawMarkdown": "Thanks for both of your replies!\n\nSo, if the time limit is met with the current notebook, does that mean it will be met with future test data as well?",
      "votes": null
    },
    {
      "id": "3205360",
      "postDate": "05/19/2025 18:27:00",
      "content": "<p>Hello <a href=\"https://www.kaggle.com/rhijudas\" target=\"_blank\">@rhijudas</a> , do I understand it well that the current hidden test set including: public test set for LB + private test set?<br>\nI'm a little bit confused when I read the description <a href=\"https://www.kaggle.com/competitions/stanford-rna-3d-folding/data\" target=\"_blank\">here</a>:<br>\n<strong>Model training phase 2</strong>. On April 23, 2025, we updated the hidden test set and reset the leaderboard. Sequences in the current public test set were added to the train data, sequences currently in the private set were rolled into the new public set, and new sequences were added to <strong>the public test set</strong>. Here, did you mean \"and new sequences were added to <strong>the private test set</strong>\"?</p>",
      "rawMarkdown": "Hello @rhijudas , do I understand it well that the current hidden test set including: public test set for LB + private test set?\nI'm a little bit confused when I read the description [here](https://www.kaggle.com/competitions/stanford-rna-3d-folding/data):\n**Model training phase 2**. On April 23, 2025, we updated the hidden test set and reset the leaderboard. Sequences in the current public test set were added to the train data, sequences currently in the private set were rolled into the new public set, and new sequences were added to **the public test set**. Here, did you mean \"and new sequences were added to **the private test set**\"?",
      "votes": null
    },
    {
      "id": "3205363",
      "postDate": "05/19/2025 18:33:31",
      "content": "<p>Actually, new sequences were added to <strong>the public test set.</strong> This was to help competitors get some feedback from new targets based on public LB scores during this training phase. </p>\n<p>Note that the final test set for evaluation (normally called the private test set) will be based on future targets that become available after competition entry deadline. Even we hosts don't know what they will be.</p>",
      "rawMarkdown": "Actually, new sequences were added to **the public test set.** This was to help competitors get some feedback from new targets based on public LB scores during this training phase. \n\nNote that the final test set for evaluation (normally called the private test set) will be based on future targets that become available after competition entry deadline. Even we hosts don't know what they will be.",
      "votes": null
    },
    {
      "id": "3212130",
      "postDate": "05/29/2025 10:52:39",
      "content": "<p>So that means there’s no OOM now, and there won’t be later, right?</p>",
      "rawMarkdown": "So that means there’s no OOM now, and there won’t be later, right?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3202756,
      "author_name": "alejopaullier",
      "author_url": "",
      "post_date": "05/15/2025 21:01:58",
      "content": "<p>That is one of my main concerns. Execution time depends drastically on the length of each sequence unless this is previously accounted for. If the number of sequences and their length distribution remains the same I think it would not be a big deal. </p>",
      "votes": null,
      "replies": [
        {
          "id": 3203427,
          "author_name": "rhijudas",
          "author_url": "",
          "post_date": "05/16/2025 18:05:03",
          "content": "<p>The number of sequences and length distribution in the future data set are expected to be similar to the current hidden test set.</p>",
          "votes": null,
          "replies": [
            {
              "id": 3203491,
              "author_name": "alejopaullier",
              "author_url": "",
              "post_date": "05/16/2025 19:49:48",
              "content": "<p>Thanks for the clarification <a href=\"https://www.kaggle.com/rhijudas\" target=\"_blank\">@rhijudas</a> !</p>",
              "votes": null,
              "replies": []
            },
            {
              "id": 3203982,
              "author_name": "koooeo",
              "author_url": "",
              "post_date": "05/17/2025 15:57:20",
              "content": "<p>Thanks for both of your replies!</p>\n<p>So, if the time limit is met with the current notebook, does that mean it will be met with future test data as well?</p>",
              "votes": null,
              "replies": []
            },
            {
              "id": 3205360,
              "author_name": "nguyenhoa",
              "author_url": "",
              "post_date": "05/19/2025 18:27:00",
              "content": "<p>Hello <a href=\"https://www.kaggle.com/rhijudas\" target=\"_blank\">@rhijudas</a> , do I understand it well that the current hidden test set including: public test set for LB + private test set?<br>\nI'm a little bit confused when I read the description <a href=\"https://www.kaggle.com/competitions/stanford-rna-3d-folding/data\" target=\"_blank\">here</a>:<br>\n<strong>Model training phase 2</strong>. On April 23, 2025, we updated the hidden test set and reset the leaderboard. Sequences in the current public test set were added to the train data, sequences currently in the private set were rolled into the new public set, and new sequences were added to <strong>the public test set</strong>. Here, did you mean \"and new sequences were added to <strong>the private test set</strong>\"?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 3205363,
                  "author_name": "rhijudas",
                  "author_url": "",
                  "post_date": "05/19/2025 18:33:31",
                  "content": "<p>Actually, new sequences were added to <strong>the public test set.</strong> This was to help competitors get some feedback from new targets based on public LB scores during this training phase. </p>\n<p>Note that the final test set for evaluation (normally called the private test set) will be based on future targets that become available after competition entry deadline. Even we hosts don't know what they will be.</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            },
            {
              "id": 3212130,
              "author_name": "daoheliu",
              "author_url": "",
              "post_date": "05/29/2025 10:52:39",
              "content": "<p>So that means there’s no OOM now, and there won’t be later, right?</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3202601": "To the competition hosts\n\nFirst of all, thank you for hosting such a worthwhile competition\n\nI have one question.\nIf the execution time limit is met with the current test data, can we assume that it will also be met in future data evaluation phases?\n\n@rhijudas",
    "3202756": "That is one of my main concerns. Execution time depends drastically on the length of each sequence unless this is previously accounted for. If the number of sequences and their length distribution remains the same I think it would not be a big deal.",
    "3203427": "The number of sequences and length distribution in the future data set are expected to be similar to the current hidden test set.",
    "3203491": "Thanks for the clarification @rhijudas !",
    "3203982": "Thanks for both of your replies!\n\nSo, if the time limit is met with the current notebook, does that mean it will be met with future test data as well?",
    "3205360": "Hello @rhijudas , do I understand it well that the current hidden test set including: public test set for LB + private test set?\nI'm a little bit confused when I read the description [here](https://www.kaggle.com/competitions/stanford-rna-3d-folding/data):\n**Model training phase 2**. On April 23, 2025, we updated the hidden test set and reset the leaderboard. Sequences in the current public test set were added to the train data, sequences currently in the private set were rolled into the new public set, and new sequences were added to **the public test set**. Here, did you mean \"and new sequences were added to **the private test set**\"?",
    "3205363": "Actually, new sequences were added to **the public test set.** This was to help competitors get some feedback from new targets based on public LB scores during this training phase. \n\nNote that the final test set for evaluation (normally called the private test set) will be based on future targets that become available after competition entry deadline. Even we hosts don't know what they will be.",
    "3212130": "So that means there’s no OOM now, and there won’t be later, right?"
  },
  "source": "meta"
}