{
  "id": 230107,
  "title": "PLEASE HELP!!! Submission Scoring Error",
  "url": "/competitions/hubmap-kidney-segmentation/discussion/230107",
  "author_name": "",
  "post_date": "2021-04-02T01:27:30.973953200Z",
  "votes": null,
  "comment_count": 8,
  "views": 0,
  "content": "<p>I already several days is trying to submit something, but failed. I just wrote simple code where all mask is 1, so encoding in lines is \"1 widht*height\". And got \"Submission Scoring Error\" Why?:<br>\nimport glob<br>\nimport os<br>\nfrom tifffile import tifffile<br>\nimport gc</p>\n<p>DIR_IMAGES = '../input/hubmap-kidney-segmentation/test'<br>\nOUTPUT_FILE = './submission.csv'</p>\n<p>output_file_fid = open(OUTPUT_FILE, 'w')<br>\nline = 'id,predicted\\n'</p>\n<h1>print(line)</h1>\n<p>output_file_fid.write(line)</p>\n<p>files = glob.glob(DIR_IMAGES + '/*.tiff')<br>\nfor file_index in range(len(files)):<br>\n    file = files[file_index]<br>\n    file_base = os.path.basename(file)<br>\n    file_base_no_ext = os.path.splitext(file_base)[0]<br>\n    image_id = file_base_no_ext</p>\n<pre><code>image_b2 = tifffile.imread(DIR_IMAGES + '/' + image_id + '.tiff')\n\n\n\n#print(image_b2.shape)\nif len(image_b2.shape) != 3:\n    image_b1 = image_b2[0, 0, :, :, :]\nelse:\n    image_b1 = image_b2\ndel image_b2\ngc.collect()\n\n#print(image_b1.shape)\nif image_b1.shape[0] == 3:\n    image = image_b1.transpose((1, 2, 0))\nelse:\n    image = image_b1\ndel image_b1\ngc.collect()\n#print(image.shape)\n\nwidth = image.shape[1]\nheight = image.shape[0]\n\ndel image\ngc.collect()\n\narea = height * width\n\nprint(file_index, image_id, width, height)\n\nline = image_id + ',1 ' + str(area)  + '\\n'\n#print(line)\noutput_file_fid.write(line)\n</code></pre>\n<p>output_file_fid.close()</p>",
  "messages": [
    {
      "id": "1260265",
      "postDate": "04/02/2021 01:27:30",
      "content": "<p>I already several days is trying to submit something, but failed. I just wrote simple code where all mask is 1, so encoding in lines is \"1 widht*height\". And got \"Submission Scoring Error\" Why?:<br>\nimport glob<br>\nimport os<br>\nfrom tifffile import tifffile<br>\nimport gc</p>\n<p>DIR_IMAGES = '../input/hubmap-kidney-segmentation/test'<br>\nOUTPUT_FILE = './submission.csv'</p>\n<p>output_file_fid = open(OUTPUT_FILE, 'w')<br>\nline = 'id,predicted\\n'</p>\n<h1>print(line)</h1>\n<p>output_file_fid.write(line)</p>\n<p>files = glob.glob(DIR_IMAGES + '/*.tiff')<br>\nfor file_index in range(len(files)):<br>\n    file = files[file_index]<br>\n    file_base = os.path.basename(file)<br>\n    file_base_no_ext = os.path.splitext(file_base)[0]<br>\n    image_id = file_base_no_ext</p>\n<pre><code>image_b2 = tifffile.imread(DIR_IMAGES + '/' + image_id + '.tiff')\n\n\n\n#print(image_b2.shape)\nif len(image_b2.shape) != 3:\n    image_b1 = image_b2[0, 0, :, :, :]\nelse:\n    image_b1 = image_b2\ndel image_b2\ngc.collect()\n\n#print(image_b1.shape)\nif image_b1.shape[0] == 3:\n    image = image_b1.transpose((1, 2, 0))\nelse:\n    image = image_b1\ndel image_b1\ngc.collect()\n#print(image.shape)\n\nwidth = image.shape[1]\nheight = image.shape[0]\n\ndel image\ngc.collect()\n\narea = height * width\n\nprint(file_index, image_id, width, height)\n\nline = image_id + ',1 ' + str(area)  + '\\n'\n#print(line)\noutput_file_fid.write(line)\n</code></pre>\n<p>output_file_fid.close()</p>",
      "rawMarkdown": "I already several days is trying to submit something, but failed. I just wrote simple code where all mask is 1, so encoding in lines is \"1 widht*height\". And got \"Submission Scoring Error\" Why?:\nimport glob\nimport os\nfrom tifffile import tifffile\nimport gc\n\nDIR_IMAGES = '../input/hubmap-kidney-segmentation/test'\nOUTPUT_FILE = './submission.csv'\n\noutput_file_fid = open(OUTPUT_FILE, 'w')\nline = 'id,predicted\\n'\n#print(line)\noutput_file_fid.write(line)\n\nfiles = glob.glob(DIR_IMAGES + '/*.tiff')\nfor file_index in range(len(files)):\n    file = files[file_index]\n    file_base = os.path.basename(file)\n    file_base_no_ext = os.path.splitext(file_base)[0]\n    image_id = file_base_no_ext\n    \n    image_b2 = tifffile.imread(DIR_IMAGES + '/' + image_id + '.tiff')\n    \n    \n    \n    #print(image_b2.shape)\n    if len(image_b2.shape) != 3:\n        image_b1 = image_b2[0, 0, :, :, :]\n    else:\n        image_b1 = image_b2\n    del image_b2\n    gc.collect()\n        \n    #print(image_b1.shape)\n    if image_b1.shape[0] == 3:\n        image = image_b1.transpose((1, 2, 0))\n    else:\n        image = image_b1\n    del image_b1\n    gc.collect()\n    #print(image.shape)\n    \n    width = image.shape[1]\n    height = image.shape[0]\n    \n    del image\n    gc.collect()\n    \n    area = height * width\n    \n    print(file_index, image_id, width, height)\n    \n    line = image_id + ',1 ' + str(area)  + '\\n'\n    #print(line)\n    output_file_fid.write(line)\n\noutput_file_fid.close()",
      "votes": null
    },
    {
      "id": "1260392",
      "postDate": "04/02/2021 04:47:18",
      "content": "<p>Use rasterio to import the images. A couple of people have shared their code here. If you use tifffile, PIL, or skimage etc you will run into dimension issues. Some images will have shape <code>[1,1,W,H,3]</code> some have <code>[1,W,H,3]</code> some <code>[3,W,H]</code>. In the private test set there's another unknown shape which breaks your model… </p>\n<p><a href=\"https://www.kaggle.com/iafoss/256x256-images\" target=\"_blank\">https://www.kaggle.com/iafoss/256x256-images</a></p>",
      "rawMarkdown": "Use rasterio to import the images. A couple of people have shared their code here. If you use tifffile, PIL, or skimage etc you will run into dimension issues. Some images will have shape `[1,1,W,H,3]` some have `[1,W,H,3]` some `[3,W,H]`. In the private test set there's another unknown shape which breaks your model... \n\n[https://www.kaggle.com/iafoss/256x256-images](https://www.kaggle.com/iafoss/256x256-images)",
      "votes": null
    },
    {
      "id": "1261401",
      "postDate": "04/03/2021 03:48:10",
      "content": "<p>The rasterio does not help: I used it just to get image sizes, and then make some dumb submission. I still get \"Submission Scoring Error\". ANY IDEAS WHY !? Here is the code:<br>\n`import rasterio<br>\nimport glob<br>\nimport os<br>\nimport gc<br>\nimport numpy as np</p>\n<p>IMAGES_DIR = '../input/hubmap-kidney-segmentation/test'<br>\nOUTPUT_FILE = './submission.csv'</p>\n<p>output_file_fid = open(OUTPUT_FILE, 'w')<br>\nline = 'id,predicted\\n'<br>\noutput_file_fid.write(line)</p>\n<p>files = glob.glob(IMAGES_DIR + '/*.tiff')<br>\nfor file_index in range(len(files)):</p>\n<pre><code>file = files[file_index]\nfile_base = os.path.basename(file)\nfile_base_no_ext = os.path.splitext(file_base)[0]\nimage_id = file_base_no_ext\n\nfid = rasterio.open(file, 'r', num_threads='all_cpus')\n\nwidth = fid.shape[1]\nheight = fid.shape[0]\n\nprint(file_index, image_id, width, height)\n\nfid.close()\n\narea = height * width\nline = image_id + ',1 ' + str(area)  + '\\n'\nprint(line)\noutput_file_fid.write(line)\n</code></pre>\n<p>output_file_fid.close()`</p>",
      "rawMarkdown": "The rasterio does not help: I used it just to get image sizes, and then make some dumb submission. I still get \"Submission Scoring Error\". ANY IDEAS WHY !? Here is the code:\n`import rasterio\nimport glob\nimport os\nimport gc\nimport numpy as np\n\nIMAGES_DIR = '../input/hubmap-kidney-segmentation/test'\nOUTPUT_FILE = './submission.csv'\n\noutput_file_fid = open(OUTPUT_FILE, 'w')\nline = 'id,predicted\\n'\noutput_file_fid.write(line)\n\nfiles = glob.glob(IMAGES_DIR + '/*.tiff')\nfor file_index in range(len(files)):\n\n    \n    file = files[file_index]\n    file_base = os.path.basename(file)\n    file_base_no_ext = os.path.splitext(file_base)[0]\n    image_id = file_base_no_ext\n    \n    fid = rasterio.open(file, 'r', num_threads='all_cpus')\n    \n    width = fid.shape[1]\n    height = fid.shape[0]\n    \n    print(file_index, image_id, width, height)\n    \n    fid.close()\n    \n    area = height * width\n    line = image_id + ',1 ' + str(area)  + '\\n'\n    print(line)\n    output_file_fid.write(line)\n\noutput_file_fid.close()`",
      "votes": null
    },
    {
      "id": "1261411",
      "postDate": "04/03/2021 04:14:36",
      "content": "<p>all pixels are 1 : <code>line = image_id + ',1 ' + str(area)  + '\\n'</code> - does not work<br>\nonly pre-last pixel is 1: l<code>ine = image_id + ',' + str(area - 1) + ' ' + '1'  + '\\n'</code>- works<br>\nonly last pixel is not 1:   <code>line = image_id + ',1 ' + str(area - 1)  + '\\n'</code> - does not work</p>",
      "rawMarkdown": "all pixels are 1 : `line = image_id + ',1 ' + str(area)  + '\\n'` - does not work\nonly pre-last pixel is 1: l`ine = image_id + ',' + str(area - 1) + ' ' + '1'  + '\\n' `- works\nonly last pixel is not 1:   `line = image_id + ',1 ' + str(area - 1)  + '\\n'` - does not work",
      "votes": null
    },
    {
      "id": "1261418",
      "postDate": "04/03/2021 04:22:47",
      "content": "<p>Last 1000 pixels also works:<br>\nline = image_id + ',' + str(area - 999) + ' ' + str(1000)  + '\\n'<br>\nIt seems that the scoring algorithm can process only small objects </p>",
      "rawMarkdown": "Last 1000 pixels also works:\nline = image_id + ',' + str(area - 999) + ' ' + str(1000)  + '\\n'\nIt seems that the scoring algorithm can process only small objects",
      "votes": null
    },
    {
      "id": "1262041",
      "postDate": "04/03/2021 17:54:10",
      "content": "<p>Maxim, I think the problem arises because your RLE is \"1 \"   This means that, because it image starts at 0, not 1, so you RLE is asking to put a \"1\" at position , which is one past the end of the image.   Try changing to \"0 \".</p>",
      "rawMarkdown": "Maxim, I think the problem arises because your RLE is \"1 <area>\"   This means that, because it image starts at 0, not 1, so you RLE is asking to put a \"1\" at position <area>, which is one past the end of the image.   Try changing to \"0 <area>\".",
      "votes": null
    },
    {
      "id": "1262442",
      "postDate": "04/04/2021 09:19:17",
      "content": "<p>In description I see that pixels numbering starts fro 1. My hypothesis is that rle segment can not be longer that height of a image.</p>",
      "rawMarkdown": "In description I see that pixels numbering starts fro 1. My hypothesis is that rle segment can not be longer that height of a image.",
      "votes": null
    },
    {
      "id": "1262737",
      "postDate": "04/04/2021 17:00:31",
      "content": "<p>Maxim, since I have also been encountering \"submission scoring error\", I have tried some experiments where I bypass the model prediction and just write out the run-length-encoded results.   Here's what I've observed:</p>\n<ol>\n<li>RLE == \"0 1\"    submission succeeds</li>\n<li>RLE == \"0 2\"    submission succeeds</li>\n<li>RLE == \"0 npixels\"   where npixels is #rows * #cols    submission scoring error</li>\n</ol>\n<p>The last one contradicts my previous suggestion.<br>\nAlthough I have not tried it, I'm curious as to what \"0 npixels-1\" does.</p>",
      "rawMarkdown": "Maxim, since I have also been encountering \"submission scoring error\", I have tried some experiments where I bypass the model prediction and just write out the run-length-encoded results.   Here's what I've observed:\n1. RLE == \"0 1\"    submission succeeds\n1. RLE == \"0 2\"    submission succeeds\n1. RLE == \"0 npixels\"   where npixels is #rows * #cols    submission scoring error\n\nThe last one contradicts my previous suggestion.\nAlthough I have not tried it, I'm curious as to what \"0 npixels-1\" does.",
      "votes": null
    },
    {
      "id": "1262987",
      "postDate": "04/05/2021 01:23:25",
      "content": "<p>The 3. can have error because of scoring algorithm bug: probably because of too long of 1-pixel sequence.</p>",
      "rawMarkdown": "The 3. can have error because of scoring algorithm bug: probably because of too long of 1-pixel sequence.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1260392,
      "author_name": "erikdali",
      "author_url": "",
      "post_date": "04/02/2021 04:47:18",
      "content": "<p>Use rasterio to import the images. A couple of people have shared their code here. If you use tifffile, PIL, or skimage etc you will run into dimension issues. Some images will have shape <code>[1,1,W,H,3]</code> some have <code>[1,W,H,3]</code> some <code>[3,W,H]</code>. In the private test set there's another unknown shape which breaks your model… </p>\n<p><a href=\"https://www.kaggle.com/iafoss/256x256-images\" target=\"_blank\">https://www.kaggle.com/iafoss/256x256-images</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1261401,
          "author_name": "vedenev",
          "author_url": "",
          "post_date": "04/03/2021 03:48:10",
          "content": "<p>The rasterio does not help: I used it just to get image sizes, and then make some dumb submission. I still get \"Submission Scoring Error\". ANY IDEAS WHY !? Here is the code:<br>\n`import rasterio<br>\nimport glob<br>\nimport os<br>\nimport gc<br>\nimport numpy as np</p>\n<p>IMAGES_DIR = '../input/hubmap-kidney-segmentation/test'<br>\nOUTPUT_FILE = './submission.csv'</p>\n<p>output_file_fid = open(OUTPUT_FILE, 'w')<br>\nline = 'id,predicted\\n'<br>\noutput_file_fid.write(line)</p>\n<p>files = glob.glob(IMAGES_DIR + '/*.tiff')<br>\nfor file_index in range(len(files)):</p>\n<pre><code>file = files[file_index]\nfile_base = os.path.basename(file)\nfile_base_no_ext = os.path.splitext(file_base)[0]\nimage_id = file_base_no_ext\n\nfid = rasterio.open(file, 'r', num_threads='all_cpus')\n\nwidth = fid.shape[1]\nheight = fid.shape[0]\n\nprint(file_index, image_id, width, height)\n\nfid.close()\n\narea = height * width\nline = image_id + ',1 ' + str(area)  + '\\n'\nprint(line)\noutput_file_fid.write(line)\n</code></pre>\n<p>output_file_fid.close()`</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1261411,
          "author_name": "vedenev",
          "author_url": "",
          "post_date": "04/03/2021 04:14:36",
          "content": "<p>all pixels are 1 : <code>line = image_id + ',1 ' + str(area)  + '\\n'</code> - does not work<br>\nonly pre-last pixel is 1: l<code>ine = image_id + ',' + str(area - 1) + ' ' + '1'  + '\\n'</code>- works<br>\nonly last pixel is not 1:   <code>line = image_id + ',1 ' + str(area - 1)  + '\\n'</code> - does not work</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1261418,
          "author_name": "vedenev",
          "author_url": "",
          "post_date": "04/03/2021 04:22:47",
          "content": "<p>Last 1000 pixels also works:<br>\nline = image_id + ',' + str(area - 999) + ' ' + str(1000)  + '\\n'<br>\nIt seems that the scoring algorithm can process only small objects </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1262041,
      "author_name": "markalavin",
      "author_url": "",
      "post_date": "04/03/2021 17:54:10",
      "content": "<p>Maxim, I think the problem arises because your RLE is \"1 \"   This means that, because it image starts at 0, not 1, so you RLE is asking to put a \"1\" at position , which is one past the end of the image.   Try changing to \"0 \".</p>",
      "votes": null,
      "replies": [
        {
          "id": 1262442,
          "author_name": "vedenev",
          "author_url": "",
          "post_date": "04/04/2021 09:19:17",
          "content": "<p>In description I see that pixels numbering starts fro 1. My hypothesis is that rle segment can not be longer that height of a image.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1262737,
          "author_name": "markalavin",
          "author_url": "",
          "post_date": "04/04/2021 17:00:31",
          "content": "<p>Maxim, since I have also been encountering \"submission scoring error\", I have tried some experiments where I bypass the model prediction and just write out the run-length-encoded results.   Here's what I've observed:</p>\n<ol>\n<li>RLE == \"0 1\"    submission succeeds</li>\n<li>RLE == \"0 2\"    submission succeeds</li>\n<li>RLE == \"0 npixels\"   where npixels is #rows * #cols    submission scoring error</li>\n</ol>\n<p>The last one contradicts my previous suggestion.<br>\nAlthough I have not tried it, I'm curious as to what \"0 npixels-1\" does.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1262987,
          "author_name": "vedenev",
          "author_url": "",
          "post_date": "04/05/2021 01:23:25",
          "content": "<p>The 3. can have error because of scoring algorithm bug: probably because of too long of 1-pixel sequence.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1260265": "I already several days is trying to submit something, but failed. I just wrote simple code where all mask is 1, so encoding in lines is \"1 widht*height\". And got \"Submission Scoring Error\" Why?:\nimport glob\nimport os\nfrom tifffile import tifffile\nimport gc\n\nDIR_IMAGES = '../input/hubmap-kidney-segmentation/test'\nOUTPUT_FILE = './submission.csv'\n\noutput_file_fid = open(OUTPUT_FILE, 'w')\nline = 'id,predicted\\n'\n#print(line)\noutput_file_fid.write(line)\n\nfiles = glob.glob(DIR_IMAGES + '/*.tiff')\nfor file_index in range(len(files)):\n    file = files[file_index]\n    file_base = os.path.basename(file)\n    file_base_no_ext = os.path.splitext(file_base)[0]\n    image_id = file_base_no_ext\n    \n    image_b2 = tifffile.imread(DIR_IMAGES + '/' + image_id + '.tiff')\n    \n    \n    \n    #print(image_b2.shape)\n    if len(image_b2.shape) != 3:\n        image_b1 = image_b2[0, 0, :, :, :]\n    else:\n        image_b1 = image_b2\n    del image_b2\n    gc.collect()\n        \n    #print(image_b1.shape)\n    if image_b1.shape[0] == 3:\n        image = image_b1.transpose((1, 2, 0))\n    else:\n        image = image_b1\n    del image_b1\n    gc.collect()\n    #print(image.shape)\n    \n    width = image.shape[1]\n    height = image.shape[0]\n    \n    del image\n    gc.collect()\n    \n    area = height * width\n    \n    print(file_index, image_id, width, height)\n    \n    line = image_id + ',1 ' + str(area)  + '\\n'\n    #print(line)\n    output_file_fid.write(line)\n\noutput_file_fid.close()",
    "1260392": "Use rasterio to import the images. A couple of people have shared their code here. If you use tifffile, PIL, or skimage etc you will run into dimension issues. Some images will have shape `[1,1,W,H,3]` some have `[1,W,H,3]` some `[3,W,H]`. In the private test set there's another unknown shape which breaks your model... \n\n[https://www.kaggle.com/iafoss/256x256-images](https://www.kaggle.com/iafoss/256x256-images)",
    "1261401": "The rasterio does not help: I used it just to get image sizes, and then make some dumb submission. I still get \"Submission Scoring Error\". ANY IDEAS WHY !? Here is the code:\n`import rasterio\nimport glob\nimport os\nimport gc\nimport numpy as np\n\nIMAGES_DIR = '../input/hubmap-kidney-segmentation/test'\nOUTPUT_FILE = './submission.csv'\n\noutput_file_fid = open(OUTPUT_FILE, 'w')\nline = 'id,predicted\\n'\noutput_file_fid.write(line)\n\nfiles = glob.glob(IMAGES_DIR + '/*.tiff')\nfor file_index in range(len(files)):\n\n    \n    file = files[file_index]\n    file_base = os.path.basename(file)\n    file_base_no_ext = os.path.splitext(file_base)[0]\n    image_id = file_base_no_ext\n    \n    fid = rasterio.open(file, 'r', num_threads='all_cpus')\n    \n    width = fid.shape[1]\n    height = fid.shape[0]\n    \n    print(file_index, image_id, width, height)\n    \n    fid.close()\n    \n    area = height * width\n    line = image_id + ',1 ' + str(area)  + '\\n'\n    print(line)\n    output_file_fid.write(line)\n\noutput_file_fid.close()`",
    "1261411": "all pixels are 1 : `line = image_id + ',1 ' + str(area)  + '\\n'` - does not work\nonly pre-last pixel is 1: l`ine = image_id + ',' + str(area - 1) + ' ' + '1'  + '\\n' `- works\nonly last pixel is not 1:   `line = image_id + ',1 ' + str(area - 1)  + '\\n'` - does not work",
    "1261418": "Last 1000 pixels also works:\nline = image_id + ',' + str(area - 999) + ' ' + str(1000)  + '\\n'\nIt seems that the scoring algorithm can process only small objects",
    "1262041": "Maxim, I think the problem arises because your RLE is \"1 <area>\"   This means that, because it image starts at 0, not 1, so you RLE is asking to put a \"1\" at position <area>, which is one past the end of the image.   Try changing to \"0 <area>\".",
    "1262442": "In description I see that pixels numbering starts fro 1. My hypothesis is that rle segment can not be longer that height of a image.",
    "1262737": "Maxim, since I have also been encountering \"submission scoring error\", I have tried some experiments where I bypass the model prediction and just write out the run-length-encoded results.   Here's what I've observed:\n1. RLE == \"0 1\"    submission succeeds\n1. RLE == \"0 2\"    submission succeeds\n1. RLE == \"0 npixels\"   where npixels is #rows * #cols    submission scoring error\n\nThe last one contradicts my previous suggestion.\nAlthough I have not tried it, I'm curious as to what \"0 npixels-1\" does.",
    "1262987": "The 3. can have error because of scoring algorithm bug: probably because of too long of 1-pixel sequence."
  },
  "source": "meta"
}