{
  "id": 546361,
  "title": "Submissions fail without error",
  "url": "/competitions/czii-cryo-et-object-identification/discussion/546361",
  "author_name": "Mart Last",
  "post_date": "2024-11-15T09:33:46.059000",
  "votes": 0,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi all,</p>\n<p>I was hoping (in a sportsmanlike way of course) that others might also have run in to this issue and that someone could perhaps help me out.</p>\n<p>My competition submissions fail, but the logs indicate that the run was succesfull (\"Successfully ran in 988.4s\") and what output I can see in the notebook is also the usual. My code works on the test data in Kaggle and with internet access disabled. The notebook run is also indicated as succesful, it's just the 'Competition submission' that fails:</p>\n<p><strong>\"Notebook threw exception\"</strong><br>\n*Your notebook hit an unhandled error while rerunning your code. Note that the hidden dataset can be larger/smaller/different than the public dataset *</p>\n<p>I guess it might relate to how I load in the data: </p>\n<pre><code>datasets = [os.path.basename(f)  f  glob.glob()]\n\n tag  datasets:\n    volume_path = \n   ...\n</code></pre>\n<p>but if that were the case, I would expect the error not after ~15 minutes but right at the start.</p>\n<p>I've made the notebook public in case anyone wanted to have a look: <a href=\"https://www.kaggle.com/code/martlast/241114-mart-1/notebook?scriptVersionId=207515404\" target=\"_blank\">https://www.kaggle.com/code/martlast/241114-mart-1/notebook?scriptVersionId=207515404</a></p>\n<p>Any suggestions would be much appreciated.</p>\n<p>Best,<br>\nMart</p>",
  "messages": [
    {
      "id": 3046802,
      "postDate": "2024-11-15T21:59:09.127Z",
      "content": "<p>Check that the file exist before you open it in case there are some non-zarrs in there.</p>\n<pre><code>        os.path.isdir(volume_path):\n           ...\n</code></pre>",
      "rawMarkdown": "Check that the file exist before you open it in case there are some non-zarrs in there.\n\n```python\n       if os.path.isdir(volume_path):\n           ...\n\n``` ",
      "replies": [
        {
          "id": 3048724,
          "postDate": "2024-11-18T09:50:22.977Z",
          "content": "<p>Okay yes, pretty obvious in hindsight, thank you :)</p>",
          "rawMarkdown": "Okay yes, pretty obvious in hindsight, thank you :)"
        }
      ]
    },
    {
      "id": 3046375,
      "postDate": "2024-11-15T12:33:04.630Z",
      "content": "<p>if you think it is problem with reading file you should</p>\n<pre><code>    :\n   data = just    nothing\n\nmake  dummy submission   point(,,)   particle  tomo \n</code></pre>\n<p>if this has no error, then the error occurs when you try to run you model on the fail (and not at file reading)</p>",
      "rawMarkdown": "if you think it is problem with reading file you should\n\n```\nfor each file in directory:\n   data = just read and do nothing\n\nmake a dummy submission with one point(0,0,0) for each particle and tomo file\n\n```\n\nif this has no error, then the error occurs when you try to run you model on the fail (and not at file reading)",
      "replies": [
        {
          "id": 3046796,
          "postDate": "2024-11-15T21:54:05.537Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 3046262,
      "postDate": "2024-11-15T09:33:46.060Z",
      "content": "<p>Hi all,</p>\n<p>I was hoping (in a sportsmanlike way of course) that others might also have run in to this issue and that someone could perhaps help me out.</p>\n<p>My competition submissions fail, but the logs indicate that the run was succesfull (\"Successfully ran in 988.4s\") and what output I can see in the notebook is also the usual. My code works on the test data in Kaggle and with internet access disabled. The notebook run is also indicated as succesful, it's just the 'Competition submission' that fails:</p>\n<p><strong>\"Notebook threw exception\"</strong><br>\n*Your notebook hit an unhandled error while rerunning your code. Note that the hidden dataset can be larger/smaller/different than the public dataset *</p>\n<p>I guess it might relate to how I load in the data: </p>\n<pre><code>datasets = [os.path.basename(f)  f  glob.glob()]\n\n tag  datasets:\n    volume_path = \n   ...\n</code></pre>\n<p>but if that were the case, I would expect the error not after ~15 minutes but right at the start.</p>\n<p>I've made the notebook public in case anyone wanted to have a look: <a href=\"https://www.kaggle.com/code/martlast/241114-mart-1/notebook?scriptVersionId=207515404\" target=\"_blank\">https://www.kaggle.com/code/martlast/241114-mart-1/notebook?scriptVersionId=207515404</a></p>\n<p>Any suggestions would be much appreciated.</p>\n<p>Best,<br>\nMart</p>",
      "rawMarkdown": "Hi all,\n\nI was hoping (in a sportsmanlike way of course) that others might also have run in to this issue and that someone could perhaps help me out.\n\nMy competition submissions fail, but the logs indicate that the run was succesfull (\"Successfully ran in 988.4s\") and what output I can see in the notebook is also the usual. My code works on the test data in Kaggle and with internet access disabled. The notebook run is also indicated as succesful, it's just the 'Competition submission' that fails:\n \n**\"Notebook threw exception\"**\n*Your notebook hit an unhandled error while rerunning your code. Note that the hidden dataset can be larger/smaller/different than the public dataset *\n\nI guess it might relate to how I load in the data: \n```python\ndatasets = [os.path.basename(f) for f in glob.glob(\"/kaggle/input/czii-cryo-et-object-identification/test/static/ExperimentRuns/*\")]\n\nfor tag in datasets:\n    volume_path = f\"/kaggle/input/czii-cryo-et-object-identification/test/static/ExperimentRuns/{tag}/VoxelSpacing10.000/denoised.zarr\"\n   ...\n```\n\nbut if that were the case, I would expect the error not after ~15 minutes but right at the start.\n\nI've made the notebook public in case anyone wanted to have a look: https://www.kaggle.com/code/martlast/241114-mart-1/notebook?scriptVersionId=207515404\n\nAny suggestions would be much appreciated.\n\nBest,\nMart\n\n\n\n"
    },
    {
      "id": 3046797,
      "postDate": "2024-11-15T21:54:32.523Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 3046802,
      "author_name": "David List",
      "author_url": "",
      "post_date": "2024-11-15T21:59:09.127000",
      "content": "<p>Check that the file exist before you open it in case there are some non-zarrs in there.</p>\n<pre><code>        os.path.isdir(volume_path):\n           ...\n</code></pre>",
      "votes": 0,
      "replies": [
        {
          "id": 3048724,
          "author_name": "Mart Last",
          "author_url": "",
          "post_date": "2024-11-18T09:50:22.977000",
          "content": "<p>Okay yes, pretty obvious in hindsight, thank you :)</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3046375,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2024-11-15T12:33:04.630000",
      "content": "<p>if you think it is problem with reading file you should</p>\n<pre><code>    :\n   data = just    nothing\n\nmake  dummy submission   point(,,)   particle  tomo \n</code></pre>\n<p>if this has no error, then the error occurs when you try to run you model on the fail (and not at file reading)</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3046796,
          "author_name": "",
          "author_url": "",
          "post_date": "2024-11-15T21:54:05.537000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3046797,
      "author_name": "",
      "author_url": "",
      "post_date": "2024-11-15T21:54:32.523000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3046802": "Check that the file exist before you open it in case there are some non-zarrs in there.\n\n```python\n       if os.path.isdir(volume_path):\n           ...\n\n``` ",
    "3046375": "if you think it is problem with reading file you should\n\n```\nfor each file in directory:\n   data = just read and do nothing\n\nmake a dummy submission with one point(0,0,0) for each particle and tomo file\n\n```\n\nif this has no error, then the error occurs when you try to run you model on the fail (and not at file reading)",
    "3046262": "Hi all,\n\nI was hoping (in a sportsmanlike way of course) that others might also have run in to this issue and that someone could perhaps help me out.\n\nMy competition submissions fail, but the logs indicate that the run was succesfull (\"Successfully ran in 988.4s\") and what output I can see in the notebook is also the usual. My code works on the test data in Kaggle and with internet access disabled. The notebook run is also indicated as succesful, it's just the 'Competition submission' that fails:\n \n**\"Notebook threw exception\"**\n*Your notebook hit an unhandled error while rerunning your code. Note that the hidden dataset can be larger/smaller/different than the public dataset *\n\nI guess it might relate to how I load in the data: \n```python\ndatasets = [os.path.basename(f) for f in glob.glob(\"/kaggle/input/czii-cryo-et-object-identification/test/static/ExperimentRuns/*\")]\n\nfor tag in datasets:\n    volume_path = f\"/kaggle/input/czii-cryo-et-object-identification/test/static/ExperimentRuns/{tag}/VoxelSpacing10.000/denoised.zarr\"\n   ...\n```\n\nbut if that were the case, I would expect the error not after ~15 minutes but right at the start.\n\nI've made the notebook public in case anyone wanted to have a look: https://www.kaggle.com/code/martlast/241114-mart-1/notebook?scriptVersionId=207515404\n\nAny suggestions would be much appreciated.\n\nBest,\nMart\n\n\n\n",
    "3046797": ""
  }
}