{
  "id": 455894,
  "title": "Latest on Submission Issue",
  "url": "/competitions/blood-vessel-segmentation/discussion/455894",
  "author_name": "",
  "post_date": "2023-11-17T00:53:39.351233400Z",
  "votes": 6,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I've undertaken several measures in an attempt to resolve the ongoing submission issues on for <strong>\"Submission scoring error\"</strong>:</p>\n<ul>\n<li><p><strong>Submissions</strong>: I have made nearly 30 submission attempts, with only one initially being successful, scoring 0.003. However, this success was not replicable in subsequent submissions.</p></li>\n<li><p><strong>Current Testing:</strong>. (The notebook doesn´t work anymore)</p></li>\n<li><p><strong>Troubleshooting Steps:</strong><br>\n    1.  RLE Format: I've ensured that my RLEs are in the correct format, specifically in pairs.<br>\n    2. Tensor Inspection: All tensors have been thoroughly checked for NaN and infinite values.<br>\n    3. ID Verification: I developed custom code to identify any missing IDs in the submissions, appending them with \"1 1\" as the RLE.<br>\n     4. Competition metric usage: I tried to use the competition to investigate any issues, \"1 0\" will give a nan but beside that I successfully run it locally on my validation set. </p></li>\n\n<li><p><strong>Observations:</strong> The situation remains perplexing, especially considering that some participants have managed to achieve scores as high as 0.7, while others are struggling.</p></li>\n</ul>",
  "messages": [
    {
      "id": "2527929",
      "postDate": "11/17/2023 00:53:39",
      "content": "<p>I've undertaken several measures in an attempt to resolve the ongoing submission issues on for <strong>\"Submission scoring error\"</strong>:</p>\n<ul>\n<li><p><strong>Submissions</strong>: I have made nearly 30 submission attempts, with only one initially being successful, scoring 0.003. However, this success was not replicable in subsequent submissions.</p></li>\n<li><p><strong>Current Testing:</strong>. (The notebook doesn´t work anymore)</p></li>\n<li><p><strong>Troubleshooting Steps:</strong><br>\n    1.  RLE Format: I've ensured that my RLEs are in the correct format, specifically in pairs.<br>\n    2. Tensor Inspection: All tensors have been thoroughly checked for NaN and infinite values.<br>\n    3. ID Verification: I developed custom code to identify any missing IDs in the submissions, appending them with \"1 1\" as the RLE.<br>\n     4. Competition metric usage: I tried to use the competition to investigate any issues, \"1 0\" will give a nan but beside that I successfully run it locally on my validation set. </p></li>\n\n<li><p><strong>Observations:</strong> The situation remains perplexing, especially considering that some participants have managed to achieve scores as high as 0.7, while others are struggling.</p></li>\n</ul>",
      "rawMarkdown": "I've undertaken several measures in an attempt to resolve the ongoing submission issues on for **\"Submission scoring error\"**:\n\n- **Submissions**: I have made nearly 30 submission attempts, with only one initially being successful, scoring 0.003. However, this success was not replicable in subsequent submissions.\n\n- **Current Testing:**~~ I am evaluating the effectiveness of @kashiwaba's notebook to determine if it offers a different outcome~~. (The notebook doesn´t work anymore)\n\n- **Troubleshooting Steps:**\n        1.  RLE Format: I've ensured that my RLEs are in the correct format, specifically in pairs.\n        2. Tensor Inspection: All tensors have been thoroughly checked for NaN and infinite values.\n        3. ID Verification: I developed custom code to identify any missing IDs in the submissions, appending them with \"1 1\" as the RLE.\n         4. Competition metric usage: I tried to use the competition to investigate any issues, \"1 0\" will give a nan but beside that I successfully run it locally on my validation set. \n\n\n\n- **Observations:** The situation remains perplexing, especially considering that some participants have managed to achieve scores as high as 0.7, while others are struggling.",
      "votes": null
    },
    {
      "id": "2527989",
      "postDate": "11/17/2023 03:06:57",
      "content": "<p>Thanks for posting this and you're troubleshooting efforts. I haven't been able to submit at all and still get the dreaded 'Submission scoring error'. It would be good to get some more descriptive error messages as I have no idea what I'm doing wrong. Cheers! <a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a> </p>",
      "rawMarkdown": "Thanks for posting this and you're troubleshooting efforts. I haven't been able to submit at all and still get the dreaded 'Submission scoring error'. It would be good to get some more descriptive error messages as I have no idea what I'm doing wrong. Cheers! @ryanholbrook",
      "votes": null
    },
    {
      "id": "2528851",
      "postDate": "11/17/2023 16:56:35",
      "content": "<p>You're welcome, <a href=\"https://www.kaggle.com/joebeachcapital\" target=\"_blank\">@joebeachcapital</a>! I completely understand the frustration that comes with the vague 'Submission scoring error'. It's indeed challenging to pinpoint the issue without more descriptive error messages. I hope my troubleshooting insights can help in some way.</p>",
      "rawMarkdown": "You're welcome, @joebeachcapital! I completely understand the frustration that comes with the vague 'Submission scoring error'. It's indeed challenging to pinpoint the issue without more descriptive error messages. I hope my troubleshooting insights can help in some way.",
      "votes": null
    },
    {
      "id": "2528861",
      "postDate": "11/17/2023 17:06:46",
      "content": "<p>Same issue. I think the scoring function encounters a memory out. Because when I use <a href=\"https://www.kaggle.com/code/metric/surface-dice-metric\" target=\"_blank\">https://www.kaggle.com/code/metric/surface-dice-metric</a> to check the score on a large set, memory out occurs.</p>",
      "rawMarkdown": "Same issue. I think the scoring function encounters a memory out. Because when I use https://www.kaggle.com/code/metric/surface-dice-metric to check the score on a large set, memory out occurs.",
      "votes": null
    },
    {
      "id": "2529099",
      "postDate": "11/18/2023 00:04:12",
      "content": "<p>Yes, it could be… If they just raise a generic error from the scoring metric function, everything would be a scoring error lol </p>",
      "rawMarkdown": "Yes, it could be... If they just raise a generic error from the scoring metric function, everything would be a scoring error lol",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2527989,
      "author_name": "joebeachcapital",
      "author_url": "",
      "post_date": "11/17/2023 03:06:57",
      "content": "<p>Thanks for posting this and you're troubleshooting efforts. I haven't been able to submit at all and still get the dreaded 'Submission scoring error'. It would be good to get some more descriptive error messages as I have no idea what I'm doing wrong. Cheers! <a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a> </p>",
      "votes": null,
      "replies": [
        {
          "id": 2528851,
          "author_name": "abdrah",
          "author_url": "",
          "post_date": "11/17/2023 16:56:35",
          "content": "<p>You're welcome, <a href=\"https://www.kaggle.com/joebeachcapital\" target=\"_blank\">@joebeachcapital</a>! I completely understand the frustration that comes with the vague 'Submission scoring error'. It's indeed challenging to pinpoint the issue without more descriptive error messages. I hope my troubleshooting insights can help in some way.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2528861,
      "author_name": "snorfyang",
      "author_url": "",
      "post_date": "11/17/2023 17:06:46",
      "content": "<p>Same issue. I think the scoring function encounters a memory out. Because when I use <a href=\"https://www.kaggle.com/code/metric/surface-dice-metric\" target=\"_blank\">https://www.kaggle.com/code/metric/surface-dice-metric</a> to check the score on a large set, memory out occurs.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2529099,
          "author_name": "abdrah",
          "author_url": "",
          "post_date": "11/18/2023 00:04:12",
          "content": "<p>Yes, it could be… If they just raise a generic error from the scoring metric function, everything would be a scoring error lol </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2527929": "I've undertaken several measures in an attempt to resolve the ongoing submission issues on for **\"Submission scoring error\"**:\n\n- **Submissions**: I have made nearly 30 submission attempts, with only one initially being successful, scoring 0.003. However, this success was not replicable in subsequent submissions.\n\n- **Current Testing:**~~ I am evaluating the effectiveness of @kashiwaba's notebook to determine if it offers a different outcome~~. (The notebook doesn´t work anymore)\n\n- **Troubleshooting Steps:**\n        1.  RLE Format: I've ensured that my RLEs are in the correct format, specifically in pairs.\n        2. Tensor Inspection: All tensors have been thoroughly checked for NaN and infinite values.\n        3. ID Verification: I developed custom code to identify any missing IDs in the submissions, appending them with \"1 1\" as the RLE.\n         4. Competition metric usage: I tried to use the competition to investigate any issues, \"1 0\" will give a nan but beside that I successfully run it locally on my validation set. \n\n\n\n- **Observations:** The situation remains perplexing, especially considering that some participants have managed to achieve scores as high as 0.7, while others are struggling.",
    "2527989": "Thanks for posting this and you're troubleshooting efforts. I haven't been able to submit at all and still get the dreaded 'Submission scoring error'. It would be good to get some more descriptive error messages as I have no idea what I'm doing wrong. Cheers! @ryanholbrook",
    "2528851": "You're welcome, @joebeachcapital! I completely understand the frustration that comes with the vague 'Submission scoring error'. It's indeed challenging to pinpoint the issue without more descriptive error messages. I hope my troubleshooting insights can help in some way.",
    "2528861": "Same issue. I think the scoring function encounters a memory out. Because when I use https://www.kaggle.com/code/metric/surface-dice-metric to check the score on a large set, memory out occurs.",
    "2529099": "Yes, it could be... If they just raise a generic error from the scoring metric function, everything would be a scoring error lol"
  },
  "source": "meta"
}