{
  "id": 226930,
  "title": "Need help with submission",
  "url": "/competitions/hpa-single-cell-image-classification/discussion/226930",
  "author_name": "Amit D",
  "post_date": "2021-03-18T08:30:06.621000",
  "votes": 5,
  "comment_count": 18,
  "views": 0,
  "content": "<p>Hello,</p>\n<p>I am trying to submit predictions for this competition, but unfortunately, all the submissions' status is \"Submission Scoring Error\".</p>\n<p>Now, I am out of idea as to what is probably the mistake here. I have uploaded the submission csv as a dataset. Can someone please help in understanding what might be the issue in the CSV data?</p>\n<p><a href=\"https://www.kaggle.com/amdaga/hpasubmissioncsv\" target=\"_blank\">submission.csv</a></p>\n<p>Disclaimer: I am a newbie in ML and Kaggle platform, so currently I am trying to connect the dots and improve my understanding. So far, this competition has been a good learning experience.</p>\n<p><strong>UPDATE</strong><br>\nI was able to solve this issue by doing two things:</p>\n<ul>\n<li>Removed test image resizing that was being done before segmenting and saving the cell masks.</li>\n<li>Size-wise segmentation i.e. images of different sizes will not be segmented in the same batch.</li>\n</ul>",
  "messages": [
    {
      "id": 1243465,
      "postDate": "2021-03-18T08:30:06.620Z",
      "content": "<p>Hello,</p>\n<p>I am trying to submit predictions for this competition, but unfortunately, all the submissions' status is \"Submission Scoring Error\".</p>\n<p>Now, I am out of idea as to what is probably the mistake here. I have uploaded the submission csv as a dataset. Can someone please help in understanding what might be the issue in the CSV data?</p>\n<p><a href=\"https://www.kaggle.com/amdaga/hpasubmissioncsv\" target=\"_blank\">submission.csv</a></p>\n<p>Disclaimer: I am a newbie in ML and Kaggle platform, so currently I am trying to connect the dots and improve my understanding. So far, this competition has been a good learning experience.</p>\n<p><strong>UPDATE</strong><br>\nI was able to solve this issue by doing two things:</p>\n<ul>\n<li>Removed test image resizing that was being done before segmenting and saving the cell masks.</li>\n<li>Size-wise segmentation i.e. images of different sizes will not be segmented in the same batch.</li>\n</ul>",
      "rawMarkdown": "Hello,\n\nI am trying to submit predictions for this competition, but unfortunately, all the submissions' status is \"Submission Scoring Error\".\n\nNow, I am out of idea as to what is probably the mistake here. I have uploaded the submission csv as a dataset. Can someone please help in understanding what might be the issue in the CSV data?\n\n[submission.csv](https://www.kaggle.com/amdaga/hpasubmissioncsv)\n\nDisclaimer: I am a newbie in ML and Kaggle platform, so currently I am trying to connect the dots and improve my understanding. So far, this competition has been a good learning experience.\n\n**UPDATE**\nI was able to solve this issue by doing two things:\n- Removed test image resizing that was being done before segmenting and saving the cell masks.\n- Size-wise segmentation i.e. images of different sizes will not be segmented in the same batch.",
      "votes": 5
    },
    {
      "id": 1243638,
      "postDate": "2021-03-18T11:36:50.687Z",
      "content": "<p>Hi there.</p>\n<p>This competiton is a <strong>kernels only</strong> competition which means the <strong><code>submission.csv</code></strong> file has to be generated within a Kaggle kernel.</p>\n<p>You will then be able to “submit” this file. But what your really doing is submitting your kernel. The competition will then rerun your kernel with the private test data swapped in for the public test data. This should generate a much different csv file (bigger, different images, etc.)</p>\n<p>There are some examples of how to submit a sample baseline in my <strong><a href=\"https://www.kaggle.com/dschettler8845/sample-submission-on-public-test-data-only\" target=\"_blank\">notebook here</a></strong>.</p>\n<p>I hope this helps and good luck!</p>",
      "rawMarkdown": "Hi there.\n\nThis competiton is a **kernels only** competition which means the **`submission.csv`** file has to be generated within a Kaggle kernel.\n\nYou will then be able to “submit” this file. But what your really doing is submitting your kernel. The competition will then rerun your kernel with the private test data swapped in for the public test data. This should generate a much different csv file (bigger, different images, etc.)\n\nThere are some examples of how to submit a sample baseline in my **[notebook here](https://www.kaggle.com/dschettler8845/sample-submission-on-public-test-data-only)**.\n\nI hope this helps and good luck!",
      "votes": 1,
      "replies": [
        {
          "id": 1243676,
          "postDate": "2021-03-18T12:11:55.360Z",
          "content": "<p>Darien, Thank you for responding.</p>\n<p>I have tried submitting my kernel, which generates a predicted submission.csv based on the files present in the test folder when running the notebook. That did not work.</p>\n<p>I had shared submission.csv in this thread to get input if there are any obvious mistakes that I might be making.</p>",
          "rawMarkdown": "Darien, Thank you for responding.\n\nI have tried submitting my kernel, which generates a predicted submission.csv based on the files present in the test folder when running the notebook. That did not work.\n\nI had shared submission.csv in this thread to get input if there are any obvious mistakes that I might be making."
        },
        {
          "id": 1243844,
          "postDate": "2021-03-18T14:24:37.903Z",
          "content": "<p>Hi Amit, unfortunately, we won't be able to tell from just the <strong><code>submission.csv</code></strong> file. </p>\n<p>You will need to be calculating the <strong><code>submission.csv</code></strong> file using the test images present in the competition dataset test folder. If you share a link to your notebook I can take a look.</p>\n<p>Otherwise, I would recommend that you start from the top (something simple) and follow a rigorous procedure by introducing more and more complexity.</p>\n<hr>\n<p>Start by submitting a baseline submission (i.e. predict the same thing for every example) and verify that this works. If this isn't working for you, then please share the notebook (as it won't contain any private information)  and I can help.</p>\n<p>If that works, start increasing complexity and adding things in and keep trying to submit. At some point, something will break and your submission will start to fail. This should help you identify where the error is and will allow you/us to be able to solve it.</p>\n<hr>\n<p>Hope this helps! :)</p>",
          "rawMarkdown": "Hi Amit, unfortunately, we won't be able to tell from just the **`submission.csv`** file. \n\nYou will need to be calculating the **`submission.csv`** file using the test images present in the competition dataset test folder. If you share a link to your notebook I can take a look.\n\nOtherwise, I would recommend that you start from the top (something simple) and follow a rigorous procedure by introducing more and more complexity.\n\n---\n\nStart by submitting a baseline submission (i.e. predict the same thing for every example) and verify that this works. If this isn't working for you, then please share the notebook (as it won't contain any private information)  and I can help.\n\nIf that works, start increasing complexity and adding things in and keep trying to submit. At some point, something will break and your submission will start to fail. This should help you identify where the error is and will allow you/us to be able to solve it.\n\n---\n\nHope this helps! :)",
          "votes": 1
        },
        {
          "id": 1243895,
          "postDate": "2021-03-18T15:02:25.963Z",
          "content": "<p>Hi Darien, that's right. I should have started with a baseline prediction and then try to improve on it. I will try that and see if it works for me.</p>\n<p>Also, I have made my notebook public. Here's the <a href=\"https://www.kaggle.com/amdaga/hpa-competition-submission/\" target=\"_blank\">link</a>. (I also need to improve my notebook documentation. I will try to improve that as well).</p>",
          "rawMarkdown": "Hi Darien, that's right. I should have started with a baseline prediction and then try to improve on it. I will try that and see if it works for me.\n\nAlso, I have made my notebook public. Here's the [link](https://www.kaggle.com/amdaga/hpa-competition-submission/). (I also need to improve my notebook documentation. I will try to improve that as well).\n",
          "votes": 1
        }
      ]
    },
    {
      "id": 1268264,
      "postDate": "2021-04-09T08:49:27.003Z",
      "content": "<p>Hii <a href=\"https://www.kaggle.com/amdaga\" target=\"_blank\">@amdaga</a>, a comment on the updates:</p>\n<p>I tried removing the resize previous to segmentation (I had an image resizing of 512,512 px). The problem is that computation time has risen to the point I can no longer submint due 'Notebook Timeout'. I believe due to full image processing. The results are better, but can handle 9h of run time.</p>\n<p>Did this happen to you?</p>",
      "rawMarkdown": "Hii @amdaga, a comment on the updates:\n\nI tried removing the resize previous to segmentation (I had an image resizing of 512,512 px). The problem is that computation time has risen to the point I can no longer submint due 'Notebook Timeout'. I believe due to full image processing. The results are better, but can handle 9h of run time.\n\nDid this happen to you?",
      "replies": [
        {
          "id": 1268289,
          "postDate": "2021-04-09T09:21:23.417Z",
          "content": "<p>Using the Cellsegmentor with full size image is slow. especially post processing is slow.  But there are solution to improve speed even with full size image.</p>\n<p><a href=\"https://www.kaggle.com/linshokaku/faster-hpa-cell-segmentation/comments\" target=\"_blank\">https://www.kaggle.com/linshokaku/faster-hpa-cell-segmentation/comments</a></p>\n<p>The solution of deoxy is at leas 5 times faster. It has a effect on LB score of -0.003 so i think the trade-off using that solution is good</p>",
          "rawMarkdown": "Using the Cellsegmentor with full size image is slow. especially post processing is slow.  But there are solution to improve speed even with full size image.\n\nhttps://www.kaggle.com/linshokaku/faster-hpa-cell-segmentation/comments\n\nThe solution of deoxy is at leas 5 times faster. It has a effect on LB score of -0.003 so i think the trade-off using that solution is good",
          "votes": 1
        },
        {
          "id": 1268301,
          "postDate": "2021-04-09T09:31:46.787Z",
          "content": "<p>Thanks again Luca!! </p>\n<p>So how does the procedure work exactly?<br>\nI see you found the key in with the <code>cv2.resize(mask,(2048,2048),interpolation=cv2.INTER_NEAREST)</code><br>\nSo it seems like you are resizing the images first to (512,512) first, for later resizing to original size…</p>\n<p>What I don't get is when to do it and why…</p>",
          "rawMarkdown": "Thanks again Luca!! \n\nSo how does the procedure work exactly?\nI see you found the key in with the `cv2.resize(mask,(2048,2048),interpolation=cv2.INTER_NEAREST)`\nSo it seems like you are resizing the images first to (512,512) first, for later resizing to original size...\n\nWhat I don't get is when to do it and why...\n"
        }
      ]
    },
    {
      "id": 1266972,
      "postDate": "2021-04-08T08:03:52.903Z",
      "content": "<p>Did you find the reason for the error?</p>",
      "rawMarkdown": "Did you find the reason for the error?"
    },
    {
      "id": 1244232,
      "postDate": "2021-03-18T19:41:43.577Z",
      "content": "<p>Hi, I think your calculating the prediction string just for the public test data. When your are submitting your notbooke sample_submission.csv is going to have more images. You have to take care of that. Now you see 559 images in sample_submission.csv. After submitting there are going to be 1800 ( I guess). </p>\n<p>while f'{image_id}_{cell_id}' in X_test_dict:</p>\n<p>produces prediction for public test data. So for the rest of data you have to take the Prediction string in sample_submission.csv</p>\n<p>I hope it is clear now</p>",
      "rawMarkdown": "Hi, I think your calculating the prediction string just for the public test data. When your are submitting your notbooke sample_submission.csv is going to have more images. You have to take care of that. Now you see 559 images in sample_submission.csv. After submitting there are going to be 1800 ( I guess). \n\nwhile f'{image_id}_{cell_id}' in X_test_dict:\n\nproduces prediction for public test data. So for the rest of data you have to take the Prediction string in sample_submission.csv\n\nI hope it is clear now",
      "replies": [
        {
          "id": 1244578,
          "postDate": "2021-03-19T04:51:19.850Z",
          "content": "<p>Hello, this is what I am doing right now.</p>\n<ol>\n<li>Read sample_submission.csv to get the test image ids</li>\n<li>Generate cell masks for the test image ids</li>\n<li>Segment test images using cell masks</li>\n<li>Clear cuda cache</li>\n<li>Load the pre-trained model</li>\n<li>Run prediction for test cells</li>\n<li>Prepare submission data from the predictions</li>\n<li>Save submission data in a csv <strong>submission.csv</strong></li>\n</ol>\n<p>The notebook assumes that private/public test data will be present in sample_submission.csv and the same has been used for the above steps.</p>\n<p>Also, all of my submissions (even those older than 9 hours) are still being shown with the loading icon.</p>",
          "rawMarkdown": "Hello, this is what I am doing right now.\n\n1. Read sample_submission.csv to get the test image ids\n2. Generate cell masks for the test image ids\n3. Segment test images using cell masks\n4. Clear cuda cache\n5. Load the pre-trained model\n6. Run prediction for test cells\n7. Prepare submission data from the predictions\n8. Save submission data in a csv **submission.csv**\n\nThe notebook assumes that private/public test data will be present in sample_submission.csv and the same has been used for the above steps.\n\nAlso, all of my submissions (even those older than 9 hours) are still being shown with the loading icon.",
          "votes": 1
        },
        {
          "id": 1249502,
          "postDate": "2021-03-23T10:47:33.933Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/amdaga\" target=\"_blank\">@amdaga</a> </p>\n<p>I'm following the same procedure but I have trouble importing the hpacellsegmentator and the pycocotools APIs to process the test data. As you know there is no internet available so I downloaded the .zip containing the APIs and I loaded them in my notebook with the '+ Add Data' tool.<br>\nHowever, when I try importing the functions, it won't let me. I've tried these lines:</p>\n<p>!pip install pycocotools --no-index --find-links=file:///kaggle/input/pycocotools/pycocotools-window-master/PythonAPI</p>\n<p>import hpacellseg.cellsegmentator as cellsegmentator</p>\n<p>None of them work.. How do you import the segmentator??</p>\n<p>Thank you in advance :)</p>",
          "rawMarkdown": "Hi @amdaga \n\nI'm following the same procedure but I have trouble importing the hpacellsegmentator and the pycocotools APIs to process the test data. As you know there is no internet available so I downloaded the .zip containing the APIs and I loaded them in my notebook with the '+ Add Data' tool.\nHowever, when I try importing the functions, it won't let me. I've tried these lines:\n\n!pip install pycocotools --no-index --find-links=file:///kaggle/input/pycocotools/pycocotools-window-master/PythonAPI\n\nimport hpacellseg.cellsegmentator as cellsegmentator\n\nNone of them work.. How do you import the segmentator??\n\nThank you in advance :)"
        },
        {
          "id": 1249570,
          "postDate": "2021-03-23T11:18:59.223Z",
          "content": "<p>hi,</p>\n<p>l imported <a href=\"https://www.kaggle.com/marzellt/pycocotools\" target=\"_blank\">https://www.kaggle.com/marzellt/pycocotools</a> (Add Data)</p>\n<p>!pip install ../input/pycocotools/pycocotools-2.0.0</p>\n<p>same with Cellsegmentor</p>\n<p><a href=\"https://www.kaggle.com/rdizzl3/hpacellsegmentatormaster\" target=\"_blank\">https://www.kaggle.com/rdizzl3/hpacellsegmentatormaster</a></p>\n<p>!pip install ../input/hpacellsegmentatormaster/HPA-Cell-Segmentation-master</p>\n<p>I think thats the easiest way</p>",
          "rawMarkdown": "hi,\n\nl imported https://www.kaggle.com/marzellt/pycocotools (Add Data)\n\n!pip install ../input/pycocotools/pycocotools-2.0.0\n\nsame with Cellsegmentor\n\nhttps://www.kaggle.com/rdizzl3/hpacellsegmentatormaster\n\n!pip install ../input/hpacellsegmentatormaster/HPA-Cell-Segmentation-master\n\nI think thats the easiest way",
          "votes": 2
        },
        {
          "id": 1249577,
          "postDate": "2021-03-23T11:32:47.707Z",
          "content": "<p><a href=\"https://www.kaggle.com/glopezzz\" target=\"_blank\">@glopezzz</a> The solution provided by <a href=\"https://www.kaggle.com/lucamtb\" target=\"_blank\">@lucamtb</a> worked for me as well. It is indeed the simplest way.</p>",
          "rawMarkdown": "@glopezzz The solution provided by @lucamtb worked for me as well. It is indeed the simplest way.",
          "votes": 1
        },
        {
          "id": 1249588,
          "postDate": "2021-03-23T11:46:51.117Z",
          "content": "<p>Awesome guys!! Thank you!! :)</p>",
          "rawMarkdown": "Awesome guys!! Thank you!! :)"
        },
        {
          "id": 1249594,
          "postDate": "2021-03-23T11:53:49.500Z",
          "content": "<p>I have one problem. When I add pycocotools, I get this directory <code>../input/pycocotools/null</code><br>\nIt seems like it's not loading correctly, although in the right menu I do see the directory and the files…<br>\nDo you know what's going on?</p>\n<p>Im also getting this error when downloading the nuclei and cell model:</p>\n<pre><code>NUC_MODEL = \"./nuclei-model.pth\"\nCELL_MODEL = \"./cell-model.pth\"\n</code></pre>\n<p>I get this error:<br>\n<code>URLError: &lt;urlopen error [Errno -3] Temporary failure in name resolution&gt;</code></p>\n<p>☹️</p>",
          "rawMarkdown": "I have one problem. When I add pycocotools, I get this directory `../input/pycocotools/null`\nIt seems like it's not loading correctly, although in the right menu I do see the directory and the files...\nDo you know what's going on?\n\nIm also getting this error when downloading the nuclei and cell model:\n```\nNUC_MODEL = \"./nuclei-model.pth\"\nCELL_MODEL = \"./cell-model.pth\"\n```\n\nI get this error:\n`URLError: <urlopen error [Errno -3] Temporary failure in name resolution>`\n\n☹️"
        },
        {
          "id": 1249687,
          "postDate": "2021-03-23T13:00:43.207Z",
          "content": "<p>You can use </p>\n<p><a href=\"https://www.kaggle.com/daishu/hpacellsegmodel\" target=\"_blank\">https://www.kaggle.com/daishu/hpacellsegmodel</a></p>\n<p>or</p>\n<p><a href=\"https://www.kaggle.com/rdizzl3/hpacellsegmentatormodelweights\" target=\"_blank\">https://www.kaggle.com/rdizzl3/hpacellsegmentatormodelweights</a></p>",
          "rawMarkdown": "You can use \n\nhttps://www.kaggle.com/daishu/hpacellsegmodel\n\nor\n\nhttps://www.kaggle.com/rdizzl3/hpacellsegmentatormodelweights",
          "votes": 1
        },
        {
          "id": 1264696,
          "postDate": "2021-04-06T11:27:18.770Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/lucamtb\" target=\"_blank\">@lucamtb</a>,<br>\nMay I ask you what do you mean by \" your calculating the prediction string just for the public test data\". We should make prediction for images in test folder and write results into <code>submission.csv</code> file. For now we have public test set, but later, during final evaluation step it will be replaced by another one. But all of those images will be in the test folder of the HPA dataset and code have to make predictions for all images in the test folder. Do I understand properly?</p>",
          "rawMarkdown": "Hi @lucamtb,\nMay I ask you what do you mean by \" your calculating the prediction string just for the public test data\". We should make prediction for images in test folder and write results into `submission.csv` file. For now we have public test set, but later, during final evaluation step it will be replaced by another one. But all of those images will be in the test folder of the HPA dataset and code have to make predictions for all images in the test folder. Do I understand properly?"
        },
        {
          "id": 1265661,
          "postDate": "2021-04-07T05:26:01.693Z",
          "content": "<p>Yes. You understand it right. We have to make prediction for images in the test folder.</p>",
          "rawMarkdown": "Yes. You understand it right. We have to make prediction for images in the test folder.",
          "votes": 1
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1243638,
      "author_name": "Darien Schettler",
      "author_url": "",
      "post_date": "2021-03-18T11:36:50.687000",
      "content": "<p>Hi there.</p>\n<p>This competiton is a <strong>kernels only</strong> competition which means the <strong><code>submission.csv</code></strong> file has to be generated within a Kaggle kernel.</p>\n<p>You will then be able to “submit” this file. But what your really doing is submitting your kernel. The competition will then rerun your kernel with the private test data swapped in for the public test data. This should generate a much different csv file (bigger, different images, etc.)</p>\n<p>There are some examples of how to submit a sample baseline in my <strong><a href=\"https://www.kaggle.com/dschettler8845/sample-submission-on-public-test-data-only\" target=\"_blank\">notebook here</a></strong>.</p>\n<p>I hope this helps and good luck!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1243676,
          "author_name": "Amit D",
          "author_url": "",
          "post_date": "2021-03-18T12:11:55.360000",
          "content": "<p>Darien, Thank you for responding.</p>\n<p>I have tried submitting my kernel, which generates a predicted submission.csv based on the files present in the test folder when running the notebook. That did not work.</p>\n<p>I had shared submission.csv in this thread to get input if there are any obvious mistakes that I might be making.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1243844,
          "author_name": "Darien Schettler",
          "author_url": "",
          "post_date": "2021-03-18T14:24:37.903000",
          "content": "<p>Hi Amit, unfortunately, we won't be able to tell from just the <strong><code>submission.csv</code></strong> file. </p>\n<p>You will need to be calculating the <strong><code>submission.csv</code></strong> file using the test images present in the competition dataset test folder. If you share a link to your notebook I can take a look.</p>\n<p>Otherwise, I would recommend that you start from the top (something simple) and follow a rigorous procedure by introducing more and more complexity.</p>\n<hr>\n<p>Start by submitting a baseline submission (i.e. predict the same thing for every example) and verify that this works. If this isn't working for you, then please share the notebook (as it won't contain any private information)  and I can help.</p>\n<p>If that works, start increasing complexity and adding things in and keep trying to submit. At some point, something will break and your submission will start to fail. This should help you identify where the error is and will allow you/us to be able to solve it.</p>\n<hr>\n<p>Hope this helps! :)</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1243895,
          "author_name": "Amit D",
          "author_url": "",
          "post_date": "2021-03-18T15:02:25.963000",
          "content": "<p>Hi Darien, that's right. I should have started with a baseline prediction and then try to improve on it. I will try that and see if it works for me.</p>\n<p>Also, I have made my notebook public. Here's the <a href=\"https://www.kaggle.com/amdaga/hpa-competition-submission/\" target=\"_blank\">link</a>. (I also need to improve my notebook documentation. I will try to improve that as well).</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1268264,
      "author_name": "glopezzz",
      "author_url": "",
      "post_date": "2021-04-09T08:49:27.003000",
      "content": "<p>Hii <a href=\"https://www.kaggle.com/amdaga\" target=\"_blank\">@amdaga</a>, a comment on the updates:</p>\n<p>I tried removing the resize previous to segmentation (I had an image resizing of 512,512 px). The problem is that computation time has risen to the point I can no longer submint due 'Notebook Timeout'. I believe due to full image processing. The results are better, but can handle 9h of run time.</p>\n<p>Did this happen to you?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1268289,
          "author_name": "LucaMTB",
          "author_url": "",
          "post_date": "2021-04-09T09:21:23.417000",
          "content": "<p>Using the Cellsegmentor with full size image is slow. especially post processing is slow.  But there are solution to improve speed even with full size image.</p>\n<p><a href=\"https://www.kaggle.com/linshokaku/faster-hpa-cell-segmentation/comments\" target=\"_blank\">https://www.kaggle.com/linshokaku/faster-hpa-cell-segmentation/comments</a></p>\n<p>The solution of deoxy is at leas 5 times faster. It has a effect on LB score of -0.003 so i think the trade-off using that solution is good</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1268301,
          "author_name": "glopezzz",
          "author_url": "",
          "post_date": "2021-04-09T09:31:46.787000",
          "content": "<p>Thanks again Luca!! </p>\n<p>So how does the procedure work exactly?<br>\nI see you found the key in with the <code>cv2.resize(mask,(2048,2048),interpolation=cv2.INTER_NEAREST)</code><br>\nSo it seems like you are resizing the images first to (512,512) first, for later resizing to original size…</p>\n<p>What I don't get is when to do it and why…</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1266972,
      "author_name": "Zaakcii Ru",
      "author_url": "",
      "post_date": "2021-04-08T08:03:52.903000",
      "content": "<p>Did you find the reason for the error?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1244232,
      "author_name": "LucaMTB",
      "author_url": "",
      "post_date": "2021-03-18T19:41:43.577000",
      "content": "<p>Hi, I think your calculating the prediction string just for the public test data. When your are submitting your notbooke sample_submission.csv is going to have more images. You have to take care of that. Now you see 559 images in sample_submission.csv. After submitting there are going to be 1800 ( I guess). </p>\n<p>while f'{image_id}_{cell_id}' in X_test_dict:</p>\n<p>produces prediction for public test data. So for the rest of data you have to take the Prediction string in sample_submission.csv</p>\n<p>I hope it is clear now</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1244578,
          "author_name": "Amit D",
          "author_url": "",
          "post_date": "2021-03-19T04:51:19.850000",
          "content": "<p>Hello, this is what I am doing right now.</p>\n<ol>\n<li>Read sample_submission.csv to get the test image ids</li>\n<li>Generate cell masks for the test image ids</li>\n<li>Segment test images using cell masks</li>\n<li>Clear cuda cache</li>\n<li>Load the pre-trained model</li>\n<li>Run prediction for test cells</li>\n<li>Prepare submission data from the predictions</li>\n<li>Save submission data in a csv <strong>submission.csv</strong></li>\n</ol>\n<p>The notebook assumes that private/public test data will be present in sample_submission.csv and the same has been used for the above steps.</p>\n<p>Also, all of my submissions (even those older than 9 hours) are still being shown with the loading icon.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1249502,
          "author_name": "glopezzz",
          "author_url": "",
          "post_date": "2021-03-23T10:47:33.933000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/amdaga\" target=\"_blank\">@amdaga</a> </p>\n<p>I'm following the same procedure but I have trouble importing the hpacellsegmentator and the pycocotools APIs to process the test data. As you know there is no internet available so I downloaded the .zip containing the APIs and I loaded them in my notebook with the '+ Add Data' tool.<br>\nHowever, when I try importing the functions, it won't let me. I've tried these lines:</p>\n<p>!pip install pycocotools --no-index --find-links=file:///kaggle/input/pycocotools/pycocotools-window-master/PythonAPI</p>\n<p>import hpacellseg.cellsegmentator as cellsegmentator</p>\n<p>None of them work.. How do you import the segmentator??</p>\n<p>Thank you in advance :)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1249570,
          "author_name": "LucaMTB",
          "author_url": "",
          "post_date": "2021-03-23T11:18:59.223000",
          "content": "<p>hi,</p>\n<p>l imported <a href=\"https://www.kaggle.com/marzellt/pycocotools\" target=\"_blank\">https://www.kaggle.com/marzellt/pycocotools</a> (Add Data)</p>\n<p>!pip install ../input/pycocotools/pycocotools-2.0.0</p>\n<p>same with Cellsegmentor</p>\n<p><a href=\"https://www.kaggle.com/rdizzl3/hpacellsegmentatormaster\" target=\"_blank\">https://www.kaggle.com/rdizzl3/hpacellsegmentatormaster</a></p>\n<p>!pip install ../input/hpacellsegmentatormaster/HPA-Cell-Segmentation-master</p>\n<p>I think thats the easiest way</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1249577,
          "author_name": "Amit D",
          "author_url": "",
          "post_date": "2021-03-23T11:32:47.707000",
          "content": "<p><a href=\"https://www.kaggle.com/glopezzz\" target=\"_blank\">@glopezzz</a> The solution provided by <a href=\"https://www.kaggle.com/lucamtb\" target=\"_blank\">@lucamtb</a> worked for me as well. It is indeed the simplest way.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1249588,
          "author_name": "glopezzz",
          "author_url": "",
          "post_date": "2021-03-23T11:46:51.117000",
          "content": "<p>Awesome guys!! Thank you!! :)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1249594,
          "author_name": "glopezzz",
          "author_url": "",
          "post_date": "2021-03-23T11:53:49.500000",
          "content": "<p>I have one problem. When I add pycocotools, I get this directory <code>../input/pycocotools/null</code><br>\nIt seems like it's not loading correctly, although in the right menu I do see the directory and the files…<br>\nDo you know what's going on?</p>\n<p>Im also getting this error when downloading the nuclei and cell model:</p>\n<pre><code>NUC_MODEL = \"./nuclei-model.pth\"\nCELL_MODEL = \"./cell-model.pth\"\n</code></pre>\n<p>I get this error:<br>\n<code>URLError: &lt;urlopen error [Errno -3] Temporary failure in name resolution&gt;</code></p>\n<p>☹️</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1249687,
          "author_name": "LucaMTB",
          "author_url": "",
          "post_date": "2021-03-23T13:00:43.207000",
          "content": "<p>You can use </p>\n<p><a href=\"https://www.kaggle.com/daishu/hpacellsegmodel\" target=\"_blank\">https://www.kaggle.com/daishu/hpacellsegmodel</a></p>\n<p>or</p>\n<p><a href=\"https://www.kaggle.com/rdizzl3/hpacellsegmentatormodelweights\" target=\"_blank\">https://www.kaggle.com/rdizzl3/hpacellsegmentatormodelweights</a></p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1264696,
          "author_name": "Anton Popov",
          "author_url": "",
          "post_date": "2021-04-06T11:27:18.770000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/lucamtb\" target=\"_blank\">@lucamtb</a>,<br>\nMay I ask you what do you mean by \" your calculating the prediction string just for the public test data\". We should make prediction for images in test folder and write results into <code>submission.csv</code> file. For now we have public test set, but later, during final evaluation step it will be replaced by another one. But all of those images will be in the test folder of the HPA dataset and code have to make predictions for all images in the test folder. Do I understand properly?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1265661,
          "author_name": "LucaMTB",
          "author_url": "",
          "post_date": "2021-04-07T05:26:01.693000",
          "content": "<p>Yes. You understand it right. We have to make prediction for images in the test folder.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1243465": "Hello,\n\nI am trying to submit predictions for this competition, but unfortunately, all the submissions' status is \"Submission Scoring Error\".\n\nNow, I am out of idea as to what is probably the mistake here. I have uploaded the submission csv as a dataset. Can someone please help in understanding what might be the issue in the CSV data?\n\n[submission.csv](https://www.kaggle.com/amdaga/hpasubmissioncsv)\n\nDisclaimer: I am a newbie in ML and Kaggle platform, so currently I am trying to connect the dots and improve my understanding. So far, this competition has been a good learning experience.\n\n**UPDATE**\nI was able to solve this issue by doing two things:\n- Removed test image resizing that was being done before segmenting and saving the cell masks.\n- Size-wise segmentation i.e. images of different sizes will not be segmented in the same batch.",
    "1243638": "Hi there.\n\nThis competiton is a **kernels only** competition which means the **`submission.csv`** file has to be generated within a Kaggle kernel.\n\nYou will then be able to “submit” this file. But what your really doing is submitting your kernel. The competition will then rerun your kernel with the private test data swapped in for the public test data. This should generate a much different csv file (bigger, different images, etc.)\n\nThere are some examples of how to submit a sample baseline in my **[notebook here](https://www.kaggle.com/dschettler8845/sample-submission-on-public-test-data-only)**.\n\nI hope this helps and good luck!",
    "1268264": "Hii @amdaga, a comment on the updates:\n\nI tried removing the resize previous to segmentation (I had an image resizing of 512,512 px). The problem is that computation time has risen to the point I can no longer submint due 'Notebook Timeout'. I believe due to full image processing. The results are better, but can handle 9h of run time.\n\nDid this happen to you?",
    "1266972": "Did you find the reason for the error?",
    "1244232": "Hi, I think your calculating the prediction string just for the public test data. When your are submitting your notbooke sample_submission.csv is going to have more images. You have to take care of that. Now you see 559 images in sample_submission.csv. After submitting there are going to be 1800 ( I guess). \n\nwhile f'{image_id}_{cell_id}' in X_test_dict:\n\nproduces prediction for public test data. So for the rest of data you have to take the Prediction string in sample_submission.csv\n\nI hope it is clear now"
  }
}