{
  "id": 240332,
  "title": "[Error] Submission Scoring Error & Question for submission format",
  "url": "/competitions/siim-covid19-detection/discussion/240332",
  "author_name": "",
  "post_date": "2021-05-19T10:41:19.445150200Z",
  "votes": 4,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Hi, I have some issues for submission.</p>\n<p>For testing, I predict only <code>image_level</code> PredictionString like this:</p>\n<table>\n<thead>\n<tr>\n<th>id</th>\n<th>PredictionString</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>00188a671292_study</td>\n<td>negative 1 0 0 1 1 (default string)</td>\n</tr>\n<tr>\n<td>…</td>\n<td>…</td>\n</tr>\n<tr>\n<td>xxx_image</td>\n<td>opacity 0.3535 439 426 1149 1533</td>\n</tr>\n<tr>\n<td>yyy_image</td>\n<td>none 1.0 0 0 1 1</td>\n</tr>\n<tr>\n<td>zzz_image</td>\n<td>opacity 0.3987 368 695 1092 2139 opacity 0.351…</td>\n</tr>\n<tr>\n<td>…</td>\n<td>…</td>\n</tr>\n</tbody>\n</table>\n<p>If I don't predict study_level, does the problem arise?<br>\nHas anyone else encountered <code>submission scoring error</code>?</p>\n<p>p.s. There are few submissions for testing, so I post this question. 😹</p>",
  "messages": [
    {
      "id": "1314720",
      "postDate": "05/19/2021 10:41:19",
      "content": "<p>Hi, I have some issues for submission.</p>\n<p>For testing, I predict only <code>image_level</code> PredictionString like this:</p>\n<table>\n<thead>\n<tr>\n<th>id</th>\n<th>PredictionString</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>00188a671292_study</td>\n<td>negative 1 0 0 1 1 (default string)</td>\n</tr>\n<tr>\n<td>…</td>\n<td>…</td>\n</tr>\n<tr>\n<td>xxx_image</td>\n<td>opacity 0.3535 439 426 1149 1533</td>\n</tr>\n<tr>\n<td>yyy_image</td>\n<td>none 1.0 0 0 1 1</td>\n</tr>\n<tr>\n<td>zzz_image</td>\n<td>opacity 0.3987 368 695 1092 2139 opacity 0.351…</td>\n</tr>\n<tr>\n<td>…</td>\n<td>…</td>\n</tr>\n</tbody>\n</table>\n<p>If I don't predict study_level, does the problem arise?<br>\nHas anyone else encountered <code>submission scoring error</code>?</p>\n<p>p.s. There are few submissions for testing, so I post this question. 😹</p>",
      "rawMarkdown": "Hi, I have some issues for submission.\n\nFor testing, I predict only `image_level` PredictionString like this:\n\n| id | PredictionString |\n| --- | --- |\n| 00188a671292_study | negative 1 0 0 1 1 (default string) |\n| ...| ... |\n| xxx_image | opacity 0.3535 439 426 1149 1533|\n| yyy_image | none 1.0 0 0 1 1 |\n| zzz_image | opacity 0.3987 368 695 1092 2139 opacity 0.351... |\n| ...| ... |\n\nIf I don't predict study_level, does the problem arise?\nHas anyone else encountered `submission scoring error`?\n\np.s. There are few submissions for testing, so I post this question. 😹",
      "votes": null
    },
    {
      "id": "1315360",
      "postDate": "05/19/2021 18:22:03",
      "content": "<p><a href=\"https://www.kaggle.com/piantic\" target=\"_blank\">@piantic</a> any luck with submission? It seems that for <strong>image-level</strong> or <strong>detection</strong>, the score is very low even though the bounding boxes seem fine. I've checked them visually both before and after <code>submission.csv</code></p>",
      "rawMarkdown": "piantic any luck with submission? It seems that for **image-level** or **detection**, the score is very low even though the bounding boxes seem fine. I've checked them visually both before and after `submission.csv `",
      "votes": null
    },
    {
      "id": "1315367",
      "postDate": "05/19/2021 18:30:02",
      "content": "<p>I don't know what went wrong, I added image-level predictions. The score only increases by 0.001.</p>",
      "rawMarkdown": "I don't know what went wrong, I added image-level predictions. The score only increases by 0.001.",
      "votes": null
    },
    {
      "id": "1315371",
      "postDate": "05/19/2021 18:34:45",
      "content": "<p>yes same for me</p>",
      "rawMarkdown": "yes same for me",
      "votes": null
    },
    {
      "id": "1315374",
      "postDate": "05/19/2021 18:37:17",
      "content": "<p>Thanks for comment!</p>\n<p>Instead of submission, I looked at the inference images for <code>image-level</code>. In my case, the score related to train <code>image-level</code> model is also pretty low. Current submission is mainly based on result from testing <code>student-level</code>. </p>\n<p>p.s. It seems the main reason of <code>submission scoring error</code> or not. 😓<br>\n<a href=\"https://www.kaggle.com/c/siim-covid19-detection/discussion/240411\" target=\"_blank\">https://www.kaggle.com/c/siim-covid19-detection/discussion/240411</a></p>\n<p><a href=\"https://www.kaggle.com/awsaf49\" target=\"_blank\">@awsaf49</a> </p>",
      "rawMarkdown": "Thanks for comment!\n\nInstead of submission, I looked at the inference images for `image-level`. In my case, the score related to train `image-level` model is also pretty low. Current submission is mainly based on result from testing `student-level`. \n\np.s. It seems the main reason of `submission scoring error` or not. 😓\nhttps://www.kaggle.com/c/siim-covid19-detection/discussion/240411\n\n@awsaf49",
      "votes": null
    },
    {
      "id": "1315387",
      "postDate": "05/19/2021 18:45:42",
      "content": "<p>It is still in the early stages of the competition, so I think we need to look at it more. However, as mentioned above, it is true that the current results are not good for <code>image-level</code>. 😂 <a href=\"https://www.kaggle.com/h053473666\" target=\"_blank\">@h053473666</a> </p>",
      "rawMarkdown": "It is still in the early stages of the competition, so I think we need to look at it more. However, as mentioned above, it is true that the current results are not good for `image-level`. 😂 @h053473666",
      "votes": null
    },
    {
      "id": "1317265",
      "postDate": "05/21/2021 09:02:03",
      "content": "<p>Same here. Yesterday I submitted for the first time and got this error :) not very pleasant. I'm leaving it for some days and then come back later. It seems there need to be lots of fixes here.</p>",
      "rawMarkdown": "Same here. Yesterday I submitted for the first time and got this error :) not very pleasant. I'm leaving it for some days and then come back later. It seems there need to be lots of fixes here.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1315360,
      "author_name": "awsaf49",
      "author_url": "",
      "post_date": "05/19/2021 18:22:03",
      "content": "<p><a href=\"https://www.kaggle.com/piantic\" target=\"_blank\">@piantic</a> any luck with submission? It seems that for <strong>image-level</strong> or <strong>detection</strong>, the score is very low even though the bounding boxes seem fine. I've checked them visually both before and after <code>submission.csv</code></p>",
      "votes": null,
      "replies": [
        {
          "id": 1315367,
          "author_name": "h053473666",
          "author_url": "",
          "post_date": "05/19/2021 18:30:02",
          "content": "<p>I don't know what went wrong, I added image-level predictions. The score only increases by 0.001.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1315371,
          "author_name": "awsaf49",
          "author_url": "",
          "post_date": "05/19/2021 18:34:45",
          "content": "<p>yes same for me</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1315374,
          "author_name": "piantic",
          "author_url": "",
          "post_date": "05/19/2021 18:37:17",
          "content": "<p>Thanks for comment!</p>\n<p>Instead of submission, I looked at the inference images for <code>image-level</code>. In my case, the score related to train <code>image-level</code> model is also pretty low. Current submission is mainly based on result from testing <code>student-level</code>. </p>\n<p>p.s. It seems the main reason of <code>submission scoring error</code> or not. 😓<br>\n<a href=\"https://www.kaggle.com/c/siim-covid19-detection/discussion/240411\" target=\"_blank\">https://www.kaggle.com/c/siim-covid19-detection/discussion/240411</a></p>\n<p><a href=\"https://www.kaggle.com/awsaf49\" target=\"_blank\">@awsaf49</a> </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1315387,
          "author_name": "piantic",
          "author_url": "",
          "post_date": "05/19/2021 18:45:42",
          "content": "<p>It is still in the early stages of the competition, so I think we need to look at it more. However, as mentioned above, it is true that the current results are not good for <code>image-level</code>. 😂 <a href=\"https://www.kaggle.com/h053473666\" target=\"_blank\">@h053473666</a> </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1317265,
      "author_name": "moeinshariatnia",
      "author_url": "",
      "post_date": "05/21/2021 09:02:03",
      "content": "<p>Same here. Yesterday I submitted for the first time and got this error :) not very pleasant. I'm leaving it for some days and then come back later. It seems there need to be lots of fixes here.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1314720": "Hi, I have some issues for submission.\n\nFor testing, I predict only `image_level` PredictionString like this:\n\n| id | PredictionString |\n| --- | --- |\n| 00188a671292_study | negative 1 0 0 1 1 (default string) |\n| ...| ... |\n| xxx_image | opacity 0.3535 439 426 1149 1533|\n| yyy_image | none 1.0 0 0 1 1 |\n| zzz_image | opacity 0.3987 368 695 1092 2139 opacity 0.351... |\n| ...| ... |\n\nIf I don't predict study_level, does the problem arise?\nHas anyone else encountered `submission scoring error`?\n\np.s. There are few submissions for testing, so I post this question. 😹",
    "1315360": "piantic any luck with submission? It seems that for **image-level** or **detection**, the score is very low even though the bounding boxes seem fine. I've checked them visually both before and after `submission.csv `",
    "1315367": "I don't know what went wrong, I added image-level predictions. The score only increases by 0.001.",
    "1315371": "yes same for me",
    "1315374": "Thanks for comment!\n\nInstead of submission, I looked at the inference images for `image-level`. In my case, the score related to train `image-level` model is also pretty low. Current submission is mainly based on result from testing `student-level`. \n\np.s. It seems the main reason of `submission scoring error` or not. 😓\nhttps://www.kaggle.com/c/siim-covid19-detection/discussion/240411\n\n@awsaf49",
    "1315387": "It is still in the early stages of the competition, so I think we need to look at it more. However, as mentioned above, it is true that the current results are not good for `image-level`. 😂 @h053473666",
    "1317265": "Same here. Yesterday I submitted for the first time and got this error :) not very pleasant. I'm leaving it for some days and then come back later. It seems there need to be lots of fixes here."
  },
  "source": "meta"
}