{
  "id": 578784,
  "title": "What's the best single-model performance we've seen on the leaderboard so far?",
  "url": "/competitions/stanford-rna-3d-folding/discussion/578784",
  "author_name": "Shuxian Zou",
  "post_date": "2025-05-13T08:24:46.827000",
  "votes": 2,
  "comment_count": 12,
  "views": 0,
  "content": "<p>I'm curious—what's the best single-model performance we've seen on the leaderboard so far?</p>\n<p>How about we each share our current best score along with the method category (e.g., Deep Learning, Physics-based, Hybrid, etc.)—without revealing the specific details of the method itself?</p>\n<p>I'd love to get a sense of how far single-model approaches (without human input) can really push the performance. </p>\n<p>I will share mine first: 0.387 LB, Deep Learning Based</p>",
  "messages": [
    {
      "id": 3201131,
      "postDate": "2025-05-13T12:59:35.277Z",
      "content": "<p>My best lb scored  0.412, but it only got around 0.387 on the validation dataset. However, another model scored 0.391 but achieved 0.403 on the validation dataset. I'm a bit confused.🥲</p>",
      "rawMarkdown": "My best lb scored  0.412, but it only got around 0.387 on the validation dataset. However, another model scored 0.391 but achieved 0.403 on the validation dataset. I'm a bit confused.🥲",
      "votes": 3,
      "replies": [
        {
          "id": 3201186,
          "postDate": "2025-05-13T14:00:30.747Z",
          "content": "<p>Impressive! I met a similar issue to yours. The provided validation set (and the previous LB) is not informative for model selection. I think a major problem for this competition is using what dataset to select models that can guarantee generalization to the hidden test set.  </p>",
          "rawMarkdown": "Impressive! I met a similar issue to yours. The provided validation set (and the previous LB) is not informative for model selection. I think a major problem for this competition is using what dataset to select models that can guarantee generalization to the hidden test set.  ",
          "votes": 1,
          "replies": [
            {
              "id": 3204774,
              "postDate": "2025-05-18T19:20:59.343Z",
              "content": "<p>I don't know if you know this, so just adding…<br>\nThe general idea of splitting data into train and validation does not work on this problem statements. We need to do the splitting by the cutoff date.<br>\nIf you took the data from year 2015 to 2018 in training set you cannot use this time period for validation. Take 2018 to 2020 for validation. Take above 2020 for test set. <br>\nNow, how to use this to properly validate the model that's hard, as adding data for a large time period will anyways improve the model. So difficult to fit the whole Cross-validation thing.</p>",
              "rawMarkdown": "I don't know if you know this, so just adding...\nThe general idea of splitting data into train and validation does not work on this problem statements. We need to do the splitting by the cutoff date.\nIf you took the data from year 2015 to 2018 in training set you cannot use this time period for validation. Take 2018 to 2020 for validation. Take above 2020 for test set. \nNow, how to use this to properly validate the model that's hard, as adding data for a large time period will anyways improve the model. So difficult to fit the whole Cross-validation thing."
            }
          ]
        }
      ]
    },
    {
      "id": 3200967,
      "postDate": "2025-05-13T09:20:51.963Z",
      "content": "<p>LB 0.388, Deep Learning Based</p>",
      "rawMarkdown": "LB 0.388, Deep Learning Based",
      "votes": 3,
      "replies": [
        {
          "id": 3206226,
          "postDate": "2025-05-21T04:02:53.957Z",
          "content": "<p>how many epochs did you train your model for?</p>",
          "rawMarkdown": "how many epochs did you train your model for?",
          "replies": [
            {
              "id": 3207675,
              "postDate": "2025-05-23T05:48:39.443Z",
              "rawMarkdown": "",
              "isDeleted": true
            }
          ]
        },
        {
          "id": 3207682,
          "postDate": "2025-05-23T05:57:01.540Z",
          "content": "<p>Thanks! I will try but is hard to achieve it in the remaining time…</p>",
          "rawMarkdown": "Thanks! I will try but is hard to achieve it in the remaining time...",
          "votes": 1
        }
      ]
    },
    {
      "id": 3202801,
      "postDate": "2025-05-15T23:43:12.387Z",
      "content": "<p>0.428 LB Deep Learning</p>",
      "rawMarkdown": "0.428 LB Deep Learning",
      "votes": 2,
      "replies": [
        {
          "id": 3202862,
          "postDate": "2025-05-16T03:23:20.280Z",
          "content": "<p>is it single model? Wow,, mine haven't got over 0.400, with only one deep learning model</p>",
          "rawMarkdown": "is it single model? Wow,, mine haven't got over 0.400, with only one deep learning model"
        },
        {
          "id": 3202863,
          "postDate": "2025-05-16T03:26:58.630Z",
          "content": "<p>Amazing, bro! </p>",
          "rawMarkdown": "Amazing, bro! "
        },
        {
          "id": 3207678,
          "postDate": "2025-05-23T05:52:44.220Z",
          "content": "<p>Amazing, that's great!</p>",
          "rawMarkdown": "Amazing, that's great!",
          "votes": 1
        }
      ]
    },
    {
      "id": 3200926,
      "postDate": "2025-05-13T08:24:46.827Z",
      "content": "<p>I'm curious—what's the best single-model performance we've seen on the leaderboard so far?</p>\n<p>How about we each share our current best score along with the method category (e.g., Deep Learning, Physics-based, Hybrid, etc.)—without revealing the specific details of the method itself?</p>\n<p>I'd love to get a sense of how far single-model approaches (without human input) can really push the performance. </p>\n<p>I will share mine first: 0.387 LB, Deep Learning Based</p>",
      "rawMarkdown": "I'm curious—what's the best single-model performance we've seen on the leaderboard so far?\n\nHow about we each share our current best score along with the method category (e.g., Deep Learning, Physics-based, Hybrid, etc.)—without revealing the specific details of the method itself?\n\nI'd love to get a sense of how far single-model approaches (without human input) can really push the performance. \n\nI will share mine first: 0.387 LB, Deep Learning Based",
      "votes": 2
    },
    {
      "id": 3277693,
      "postDate": "2025-08-28T14:05:13.007Z",
      "content": "<p>LB0.411, protenix</p>",
      "rawMarkdown": "LB0.411, protenix"
    }
  ],
  "comments": [
    {
      "id": 3201131,
      "author_name": "Timmy Juicehouse",
      "author_url": "",
      "post_date": "2025-05-13T12:59:35.277000",
      "content": "<p>My best lb scored  0.412, but it only got around 0.387 on the validation dataset. However, another model scored 0.391 but achieved 0.403 on the validation dataset. I'm a bit confused.🥲</p>",
      "votes": 3,
      "replies": [
        {
          "id": 3201186,
          "author_name": "Shuxian Zou",
          "author_url": "",
          "post_date": "2025-05-13T14:00:30.747000",
          "content": "<p>Impressive! I met a similar issue to yours. The provided validation set (and the previous LB) is not informative for model selection. I think a major problem for this competition is using what dataset to select models that can guarantee generalization to the hidden test set.  </p>",
          "votes": 1,
          "replies": [
            {
              "id": 3204774,
              "author_name": "Siddhantoon",
              "author_url": "",
              "post_date": "2025-05-18T19:20:59.343000",
              "content": "<p>I don't know if you know this, so just adding…<br>\nThe general idea of splitting data into train and validation does not work on this problem statements. We need to do the splitting by the cutoff date.<br>\nIf you took the data from year 2015 to 2018 in training set you cannot use this time period for validation. Take 2018 to 2020 for validation. Take above 2020 for test set. <br>\nNow, how to use this to properly validate the model that's hard, as adding data for a large time period will anyways improve the model. So difficult to fit the whole Cross-validation thing.</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3200967,
      "author_name": "CodeHacker",
      "author_url": "",
      "post_date": "2025-05-13T09:20:51.963000",
      "content": "<p>LB 0.388, Deep Learning Based</p>",
      "votes": 3,
      "replies": [
        {
          "id": 3206226,
          "author_name": "MengYe",
          "author_url": "",
          "post_date": "2025-05-21T04:02:53.957000",
          "content": "<p>how many epochs did you train your model for?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3207675,
              "author_name": "",
              "author_url": "",
              "post_date": "2025-05-23T05:48:39.443000",
              "content": "",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 3207682,
          "author_name": "",
          "author_url": "",
          "post_date": "2025-05-23T05:57:01.540000",
          "content": "<p>Thanks! I will try but is hard to achieve it in the remaining time…</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 3202801,
      "author_name": "Shun Kuraishi",
      "author_url": "",
      "post_date": "2025-05-15T23:43:12.387000",
      "content": "<p>0.428 LB Deep Learning</p>",
      "votes": 2,
      "replies": [
        {
          "id": 3202862,
          "author_name": "doheon114",
          "author_url": "",
          "post_date": "2025-05-16T03:23:20.280000",
          "content": "<p>is it single model? Wow,, mine haven't got over 0.400, with only one deep learning model</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3202863,
          "author_name": "Timmy Juicehouse",
          "author_url": "",
          "post_date": "2025-05-16T03:26:58.630000",
          "content": "<p>Amazing, bro! </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3207678,
          "author_name": "",
          "author_url": "",
          "post_date": "2025-05-23T05:52:44.220000",
          "content": "<p>Amazing, that's great!</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 3277693,
      "author_name": "DaoHe Liu",
      "author_url": "",
      "post_date": "2025-08-28T14:05:13.007000",
      "content": "<p>LB0.411, protenix</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3201131": "My best lb scored  0.412, but it only got around 0.387 on the validation dataset. However, another model scored 0.391 but achieved 0.403 on the validation dataset. I'm a bit confused.🥲",
    "3200967": "LB 0.388, Deep Learning Based",
    "3202801": "0.428 LB Deep Learning",
    "3200926": "I'm curious—what's the best single-model performance we've seen on the leaderboard so far?\n\nHow about we each share our current best score along with the method category (e.g., Deep Learning, Physics-based, Hybrid, etc.)—without revealing the specific details of the method itself?\n\nI'd love to get a sense of how far single-model approaches (without human input) can really push the performance. \n\nI will share mine first: 0.387 LB, Deep Learning Based",
    "3277693": "LB0.411, protenix"
  }
}