{
  "id": 215833,
  "title": "A possible reason for the \"Submission Scoring Error\"",
  "url": "/competitions/hpa-single-cell-image-classification/discussion/215833",
  "author_name": "Raman",
  "post_date": "2021-01-31T12:35:37.949000",
  "votes": 9,
  "comment_count": 8,
  "views": 0,
  "content": "<p>Hello everyone,</p>\n<p>it took me a few attempts to overcome the \"Submission Scoring Error\". In my case, it was because I blindly used the <code>encode_binary_mask</code> function from the <a href=\"https://www.kaggle.com/c/hpa-single-cell-image-classification/overview/evaluation\" target=\"_blank\">evaluation page</a>. <br>\nThe function returns a bytes representation of the string. So, what I needed was to add the<br>\n<code>return base64_str</code><strong><code>.decode()</code></strong>.</p>\n<p>Thank you, <a href=\"https://www.kaggle.com/its7171\" target=\"_blank\">@its7171</a>, you saved me with your reference <a href=\"https://www.kaggle.com/its7171/mmdetection-for-segmentation-inference\" target=\"_blank\">notebook</a>!</p>\n<p>P.S. Please, I don't want the organizers of the awesome competition to get me wrong: I'm super grateful for providing all the helpful routines, and it was my inattentive fault!</p>\n<p>I decided I'd share it, just in case it might save someone a few minutes and a couple of nerve cells 😊</p>",
  "messages": [
    {
      "id": 1179227,
      "postDate": "2021-01-31T12:35:37.950Z",
      "content": "<p>Hello everyone,</p>\n<p>it took me a few attempts to overcome the \"Submission Scoring Error\". In my case, it was because I blindly used the <code>encode_binary_mask</code> function from the <a href=\"https://www.kaggle.com/c/hpa-single-cell-image-classification/overview/evaluation\" target=\"_blank\">evaluation page</a>. <br>\nThe function returns a bytes representation of the string. So, what I needed was to add the<br>\n<code>return base64_str</code><strong><code>.decode()</code></strong>.</p>\n<p>Thank you, <a href=\"https://www.kaggle.com/its7171\" target=\"_blank\">@its7171</a>, you saved me with your reference <a href=\"https://www.kaggle.com/its7171/mmdetection-for-segmentation-inference\" target=\"_blank\">notebook</a>!</p>\n<p>P.S. Please, I don't want the organizers of the awesome competition to get me wrong: I'm super grateful for providing all the helpful routines, and it was my inattentive fault!</p>\n<p>I decided I'd share it, just in case it might save someone a few minutes and a couple of nerve cells 😊</p>",
      "rawMarkdown": "Hello everyone,\n\nit took me a few attempts to overcome the \"Submission Scoring Error\". In my case, it was because I blindly used the `encode_binary_mask` function from the [evaluation page](https://www.kaggle.com/c/hpa-single-cell-image-classification/overview/evaluation). \nThe function returns a bytes representation of the string. So, what I needed was to add the\n`return base64_str`**`.decode()`**.\n\nThank you, @its7171, you saved me with your reference [notebook](https://www.kaggle.com/its7171/mmdetection-for-segmentation-inference)!\n\n\n\nP.S. Please, I don't want the organizers of the awesome competition to get me wrong: I'm super grateful for providing all the helpful routines, and it was my inattentive fault!\n\nI decided I'd share it, just in case it might save someone a few minutes and a couple of nerve cells 😊",
      "votes": 9
    },
    {
      "id": 1199311,
      "postDate": "2021-02-13T17:50:09.960Z",
      "content": "<p>Hello all,<br>\nI am also getting a submission error. When submitting I noticed a comment \"In this competition, we will privately re-run your selected Notebook Version with a hidden test set substituted into the competition dataset. We then extract your chosen Output File from the re-run and use that to determine your score.\"<br>\nDoes it mean that the test set is not the same as the one that was published? If so then what is the purpose of the provided test set?.</p>",
      "rawMarkdown": "Hello all,\nI am also getting a submission error. When submitting I noticed a comment \"In this competition, we will privately re-run your selected Notebook Version with a hidden test set substituted into the competition dataset. We then extract your chosen Output File from the re-run and use that to determine your score.\"\nDoes it mean that the test set is not the same as the one that was published? If so then what is the purpose of the provided test set?.",
      "replies": [
        {
          "id": 1199330,
          "postDate": "2021-02-13T18:20:55.683Z",
          "content": "<p>Hi,</p>\n<p>the public test set lets us debug the whole prediction pipeline. <br>\nAre you getting the \"Submission Scoring Error\"?</p>",
          "rawMarkdown": "Hi,\n\nthe public test set lets us debug the whole prediction pipeline. \nAre you getting the \"Submission Scoring Error\"?"
        },
        {
          "id": 1199331,
          "postDate": "2021-02-13T18:22:50.357Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 1199333,
          "postDate": "2021-02-13T18:30:21.913Z",
          "content": "<p>Yes even when using sample_submission.csv.</p>",
          "rawMarkdown": "Yes even when using sample_submission.csv."
        },
        {
          "id": 1199337,
          "postDate": "2021-02-13T18:42:34.433Z",
          "content": "<p>Hm… that's weird… I would expect the sample_submission to be scored correctly. Have you by any chance modified it anyhow? Did you set index=None when outputting the submission.csv?</p>\n<p>\"Submission Scoring Error\" basically means that the automatic grading script fails when processing your submission.csv</p>",
          "rawMarkdown": "Hm... that's weird... I would expect the sample_submission to be scored correctly. Have you by any chance modified it anyhow? Did you set index=None when outputting the submission.csv?\n\n\"Submission Scoring Error\" basically means that the automatic grading script fails when processing your submission.csv"
        },
        {
          "id": 1199346,
          "postDate": "2021-02-13T18:51:47.140Z",
          "content": "<p>Yes I used no index when exporting. Are there any logs explaining the failure?. Also any idea why submission errors are counted towards our five daily submits?.</p>",
          "rawMarkdown": "Yes I used no index when exporting. Are there any logs explaining the failure?. Also any idea why submission errors are counted towards our five daily submits?."
        },
        {
          "id": 1199356,
          "postDate": "2021-02-13T19:02:19.100Z",
          "content": "<p>Unfortunately, I'm not aware of any… </p>\n<p>Hopefully, you'll manage to get it up and running… I know the frustration you're likely experiencing at the moment. I'll keep my fingers crossed!</p>",
          "rawMarkdown": "Unfortunately, I'm not aware of any... \n\nHopefully, you'll manage to get it up and running... I know the frustration you're likely experiencing at the moment. I'll keep my fingers crossed!"
        },
        {
          "id": 1284684,
          "postDate": "2021-04-26T07:42:47.083Z",
          "content": "<p><a href=\"https://www.kaggle.com/zurkin11\" target=\"_blank\">@zurkin11</a> I faced the same issue as you did. I output submission.csv with index=False and I get images by reading sample_submission.csv.</p>\n<p>Did you tackle the problem? Please help me. Thank you.</p>",
          "rawMarkdown": "@zurkin11 I faced the same issue as you did. I output submission.csv with index=False and I get images by reading sample_submission.csv.\n\nDid you tackle the problem? Please help me. Thank you."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1199311,
      "author_name": "Zurkin11",
      "author_url": "",
      "post_date": "2021-02-13T17:50:09.960000",
      "content": "<p>Hello all,<br>\nI am also getting a submission error. When submitting I noticed a comment \"In this competition, we will privately re-run your selected Notebook Version with a hidden test set substituted into the competition dataset. We then extract your chosen Output File from the re-run and use that to determine your score.\"<br>\nDoes it mean that the test set is not the same as the one that was published? If so then what is the purpose of the provided test set?.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1199330,
          "author_name": "Raman",
          "author_url": "",
          "post_date": "2021-02-13T18:20:55.683000",
          "content": "<p>Hi,</p>\n<p>the public test set lets us debug the whole prediction pipeline. <br>\nAre you getting the \"Submission Scoring Error\"?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1199331,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-02-13T18:22:50.357000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1199333,
          "author_name": "Zurkin11",
          "author_url": "",
          "post_date": "2021-02-13T18:30:21.913000",
          "content": "<p>Yes even when using sample_submission.csv.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1199337,
          "author_name": "Raman",
          "author_url": "",
          "post_date": "2021-02-13T18:42:34.433000",
          "content": "<p>Hm… that's weird… I would expect the sample_submission to be scored correctly. Have you by any chance modified it anyhow? Did you set index=None when outputting the submission.csv?</p>\n<p>\"Submission Scoring Error\" basically means that the automatic grading script fails when processing your submission.csv</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1199346,
          "author_name": "Zurkin11",
          "author_url": "",
          "post_date": "2021-02-13T18:51:47.140000",
          "content": "<p>Yes I used no index when exporting. Are there any logs explaining the failure?. Also any idea why submission errors are counted towards our five daily submits?.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1199356,
          "author_name": "Raman",
          "author_url": "",
          "post_date": "2021-02-13T19:02:19.100000",
          "content": "<p>Unfortunately, I'm not aware of any… </p>\n<p>Hopefully, you'll manage to get it up and running… I know the frustration you're likely experiencing at the moment. I'll keep my fingers crossed!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1284684,
          "author_name": "Anh-Vu Mai-Nguyen",
          "author_url": "",
          "post_date": "2021-04-26T07:42:47.083000",
          "content": "<p><a href=\"https://www.kaggle.com/zurkin11\" target=\"_blank\">@zurkin11</a> I faced the same issue as you did. I output submission.csv with index=False and I get images by reading sample_submission.csv.</p>\n<p>Did you tackle the problem? Please help me. Thank you.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1179227": "Hello everyone,\n\nit took me a few attempts to overcome the \"Submission Scoring Error\". In my case, it was because I blindly used the `encode_binary_mask` function from the [evaluation page](https://www.kaggle.com/c/hpa-single-cell-image-classification/overview/evaluation). \nThe function returns a bytes representation of the string. So, what I needed was to add the\n`return base64_str`**`.decode()`**.\n\nThank you, @its7171, you saved me with your reference [notebook](https://www.kaggle.com/its7171/mmdetection-for-segmentation-inference)!\n\n\n\nP.S. Please, I don't want the organizers of the awesome competition to get me wrong: I'm super grateful for providing all the helpful routines, and it was my inattentive fault!\n\nI decided I'd share it, just in case it might save someone a few minutes and a couple of nerve cells 😊",
    "1199311": "Hello all,\nI am also getting a submission error. When submitting I noticed a comment \"In this competition, we will privately re-run your selected Notebook Version with a hidden test set substituted into the competition dataset. We then extract your chosen Output File from the re-run and use that to determine your score.\"\nDoes it mean that the test set is not the same as the one that was published? If so then what is the purpose of the provided test set?."
  }
}