{
  "id": 547752,
  "title": "Train Data and LB ",
  "url": "/competitions/czii-cryo-et-object-identification/discussion/547752",
  "author_name": "",
  "post_date": "2024-11-23T11:27:21.422489200Z",
  "votes": 3,
  "comment_count": 4,
  "views": 0,
  "content": "<p>There is a very strange problem, I use the exact same model and training code, 7 training data according to 6:1 training and verification division, get 7 groups of training data, so I get 7 models. They all scored between 0.5 and 0.6 for local DiceMetric scores, but when I submitted them, I got lb between 0.47 and 0.63. This is strange, maybe there is a strong correlation between certain data and lb?🤔</p>",
  "messages": [
    {
      "id": "3053276",
      "postDate": "11/23/2024 11:27:21",
      "content": "<p>There is a very strange problem, I use the exact same model and training code, 7 training data according to 6:1 training and verification division, get 7 groups of training data, so I get 7 models. They all scored between 0.5 and 0.6 for local DiceMetric scores, but when I submitted them, I got lb between 0.47 and 0.63. This is strange, maybe there is a strong correlation between certain data and lb?🤔</p>",
      "rawMarkdown": "There is a very strange problem, I use the exact same model and training code, 7 training data according to 6:1 training and verification division, get 7 groups of training data, so I get 7 models. They all scored between 0.5 and 0.6 for local DiceMetric scores, but when I submitted them, I got lb between 0.47 and 0.63. This is strange, maybe there is a strong correlation between certain data and lb?🤔",
      "votes": null
    },
    {
      "id": "3053298",
      "postDate": "11/23/2024 11:48:04",
      "content": "<p>dice and fbeta score are not the same thing.</p>\n<p>as an extreme example, you make only make one pixe in error in dice, but this causes coord to be wrongly estimated and falls out of 0.5 radius threshold, affecting fbeta score</p>",
      "rawMarkdown": "dice and fbeta score are not the same thing.\n\nas an extreme example, you make only make one pixe in error in dice, but this causes coord to be wrongly estimated and falls out of 0.5 radius threshold, affecting fbeta score",
      "votes": null
    },
    {
      "id": "3053320",
      "postDate": "11/23/2024 11:54:07",
      "content": "<p>Thank you for your reply. I will modify my index and conduct a new experiment in the future</p>",
      "rawMarkdown": "Thank you for your reply. I will modify my index and conduct a new experiment in the future",
      "votes": null
    },
    {
      "id": "3053328",
      "postDate": "11/23/2024 11:56:01",
      "content": "<p>dice 0.6 is not strong enough. if you have dice 0.8 or 0.9, maybe you see corelation between different mtrics</p>",
      "rawMarkdown": "dice 0.6 is not strong enough. if you have dice 0.8 or 0.9, maybe you see corelation between different mtrics",
      "votes": null
    },
    {
      "id": "3064562",
      "postDate": "12/05/2024 18:08:57",
      "content": "<p>I apologize for the interruption. After several days of continuous attempts, I have now modified dice into the officially provided score function. I submitted two results yesterday, and their local scores were 0.67 and 0.73, respectively. I obtained lb values of 0.501 and 0.599. I think that my cv and lb may have begun to correlate with each other, but today I used the previously mentioned data partitioning method that achieved low local dice scores but high lb values to retrain. I obtained a local score of 0.70, and I think that if lb is at 0.5*, it can overturn my previous thinking that some data and test sets have a strong correlation. However, I obtained lb 0.621.🤔</p>",
      "rawMarkdown": "I apologize for the interruption. After several days of continuous attempts, I have now modified dice into the officially provided score function. I submitted two results yesterday, and their local scores were 0.67 and 0.73, respectively. I obtained lb values of 0.501 and 0.599. I think that my cv and lb may have begun to correlate with each other, but today I used the previously mentioned data partitioning method that achieved low local dice scores but high lb values to retrain. I obtained a local score of 0.70, and I think that if lb is at 0.5*, it can overturn my previous thinking that some data and test sets have a strong correlation. However, I obtained lb 0.621.🤔",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3053298,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "11/23/2024 11:48:04",
      "content": "<p>dice and fbeta score are not the same thing.</p>\n<p>as an extreme example, you make only make one pixe in error in dice, but this causes coord to be wrongly estimated and falls out of 0.5 radius threshold, affecting fbeta score</p>",
      "votes": null,
      "replies": [
        {
          "id": 3053320,
          "author_name": "peilwang",
          "author_url": "",
          "post_date": "11/23/2024 11:54:07",
          "content": "<p>Thank you for your reply. I will modify my index and conduct a new experiment in the future</p>",
          "votes": null,
          "replies": [
            {
              "id": 3053328,
              "author_name": "hengck23",
              "author_url": "",
              "post_date": "11/23/2024 11:56:01",
              "content": "<p>dice 0.6 is not strong enough. if you have dice 0.8 or 0.9, maybe you see corelation between different mtrics</p>",
              "votes": null,
              "replies": [
                {
                  "id": 3064562,
                  "author_name": "peilwang",
                  "author_url": "",
                  "post_date": "12/05/2024 18:08:57",
                  "content": "<p>I apologize for the interruption. After several days of continuous attempts, I have now modified dice into the officially provided score function. I submitted two results yesterday, and their local scores were 0.67 and 0.73, respectively. I obtained lb values of 0.501 and 0.599. I think that my cv and lb may have begun to correlate with each other, but today I used the previously mentioned data partitioning method that achieved low local dice scores but high lb values to retrain. I obtained a local score of 0.70, and I think that if lb is at 0.5*, it can overturn my previous thinking that some data and test sets have a strong correlation. However, I obtained lb 0.621.🤔</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3053276": "There is a very strange problem, I use the exact same model and training code, 7 training data according to 6:1 training and verification division, get 7 groups of training data, so I get 7 models. They all scored between 0.5 and 0.6 for local DiceMetric scores, but when I submitted them, I got lb between 0.47 and 0.63. This is strange, maybe there is a strong correlation between certain data and lb?🤔",
    "3053298": "dice and fbeta score are not the same thing.\n\nas an extreme example, you make only make one pixe in error in dice, but this causes coord to be wrongly estimated and falls out of 0.5 radius threshold, affecting fbeta score",
    "3053320": "Thank you for your reply. I will modify my index and conduct a new experiment in the future",
    "3053328": "dice 0.6 is not strong enough. if you have dice 0.8 or 0.9, maybe you see corelation between different mtrics",
    "3064562": "I apologize for the interruption. After several days of continuous attempts, I have now modified dice into the officially provided score function. I submitted two results yesterday, and their local scores were 0.67 and 0.73, respectively. I obtained lb values of 0.501 and 0.599. I think that my cv and lb may have begun to correlate with each other, but today I used the previously mentioned data partitioning method that achieved low local dice scores but high lb values to retrain. I obtained a local score of 0.70, and I think that if lb is at 0.5*, it can overturn my previous thinking that some data and test sets have a strong correlation. However, I obtained lb 0.621.🤔"
  },
  "source": "meta"
}