{
  "id": 177345,
  "title": "Possible Workaround for Submission Bug / Question on Code Competition",
  "url": "/competitions/lyft-motion-prediction-autonomous-vehicles/discussion/177345",
  "author_name": "",
  "post_date": "2020-08-25T15:23:04.629878800Z",
  "votes": null,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Currently there is a problem with submitting and freezing browsers due to l5kit. </p>\n<p>I thought about introducing a bool that is True when commiting but False when submitting and rerunning on the private test set, which would be handy anyways to not waste GPU and is used in many code competitions for faster deployment. </p>\n<p>By this I am able to submit the submission.csv file as l5kit is only installed when submitting, which you are otherwise not able to, but the bool does not change when rerunning on private test set. </p>\n<p>I tried 3 things. Theoretically if the testset changes these should change aswell.: </p>\n<p>1 Length of sample submission.csv should change when running on private -&gt; does not</p>\n<p><code>IS_COMMIT = np.array((len(sub) == 71122))</code></p>\n<p>2 First value of sample submission.csv may change -&gt; does not</p>\n<p><code>IS_COMMIT = (sub.iloc[0, 0] == 1578606007801600134)</code></p>\n<p>3 Number of agents in the test folder may change -&gt; does not</p>\n<p><code>IS_COMMIT = np.array((len(os.listdir(\"../input/lyft-motion-prediction-autonomous-vehicles/scenes/test.zarr/agents\")) == 4431))</code></p>\n<p>Do you have an idea why these do not change when submitting to the competition?</p>",
  "messages": [
    {
      "id": "985247",
      "postDate": "08/25/2020 15:23:04",
      "content": "<p>Currently there is a problem with submitting and freezing browsers due to l5kit. </p>\n<p>I thought about introducing a bool that is True when commiting but False when submitting and rerunning on the private test set, which would be handy anyways to not waste GPU and is used in many code competitions for faster deployment. </p>\n<p>By this I am able to submit the submission.csv file as l5kit is only installed when submitting, which you are otherwise not able to, but the bool does not change when rerunning on private test set. </p>\n<p>I tried 3 things. Theoretically if the testset changes these should change aswell.: </p>\n<p>1 Length of sample submission.csv should change when running on private -&gt; does not</p>\n<p><code>IS_COMMIT = np.array((len(sub) == 71122))</code></p>\n<p>2 First value of sample submission.csv may change -&gt; does not</p>\n<p><code>IS_COMMIT = (sub.iloc[0, 0] == 1578606007801600134)</code></p>\n<p>3 Number of agents in the test folder may change -&gt; does not</p>\n<p><code>IS_COMMIT = np.array((len(os.listdir(\"../input/lyft-motion-prediction-autonomous-vehicles/scenes/test.zarr/agents\")) == 4431))</code></p>\n<p>Do you have an idea why these do not change when submitting to the competition?</p>",
      "rawMarkdown": "Currently there is a problem with submitting and freezing browsers due to l5kit. \n\nI thought about introducing a bool that is True when commiting but False when submitting and rerunning on the private test set, which would be handy anyways to not waste GPU and is used in many code competitions for faster deployment. \n\nBy this I am able to submit the submission.csv file as l5kit is only installed when submitting, which you are otherwise not able to, but the bool does not change when rerunning on private test set. \n\nI tried 3 things. Theoretically if the testset changes these should change aswell.: \n\n1 Length of sample submission.csv should change when running on private -> does not\n\n`IS_COMMIT = np.array((len(sub) == 71122))`\n\n2 First value of sample submission.csv may change -> does not\n\n`IS_COMMIT = (sub.iloc[0, 0] == 1578606007801600134)`\n\n3 Number of agents in the test folder may change -> does not\n\n`IS_COMMIT = np.array((len(os.listdir(\"../input/lyft-motion-prediction-autonomous-vehicles/scenes/test.zarr/agents\")) == 4431))`\n\nDo you have an idea why these do not change when submitting to the competition?",
      "votes": null
    },
    {
      "id": "985266",
      "postDate": "08/25/2020 15:38:17",
      "content": "<p>Probably because for this competition the test.csv contains both the public and the private part, and they don't replace it during submission.</p>\n<p>You can try one more thing: generate a hash from a file. Something like this:</p>\n<pre><code>import hashlib\n\ndef file_as_bytes(file):\n    with file:\n        return file.read()\n\nmask_file = '/kaggle/input/lyft-motion-prediction-autonomous-vehicles/scenes/mask.npz'\n\nprint(hashlib.md5(file_as_bytes(open(mask_file, 'rb'))).hexdigest())\n</code></pre>\n<p>This should output <code>5fe176ab4bd80f4e6db43d7793ab77f7</code> for the mask file we have access in the notebooks.</p>",
      "rawMarkdown": "Probably because for this competition the test.csv contains both the public and the private part, and they don't replace it during submission.\n\nYou can try one more thing: generate a hash from a file. Something like this:\n\n```\nimport hashlib\n\ndef file_as_bytes(file):\n    with file:\n        return file.read()\n\nmask_file = '/kaggle/input/lyft-motion-prediction-autonomous-vehicles/scenes/mask.npz'\n    \nprint(hashlib.md5(file_as_bytes(open(mask_file, 'rb'))).hexdigest())\n```\n\nThis should output `5fe176ab4bd80f4e6db43d7793ab77f7` for the mask file we have access in the notebooks.",
      "votes": null
    },
    {
      "id": "985270",
      "postDate": "08/25/2020 15:41:34",
      "content": "<p>Ah okay so Code Competition here only means to restrict computational ressources for the final submission. <br>\nThanks I will try.</p>",
      "rawMarkdown": "Ah okay so Code Competition here only means to restrict computational ressources for the final submission. \nThanks I will try.",
      "votes": null
    },
    {
      "id": "985271",
      "postDate": "08/25/2020 15:43:01",
      "content": "<p>I think it is fixed now..</p>\n<p><em>Update</em> I was a bit optimistic :(</p>",
      "rawMarkdown": "I think it is fixed now..\n\n*Update* I was a bit optimistic :(",
      "votes": null
    },
    {
      "id": "985279",
      "postDate": "08/25/2020 15:53:16",
      "content": "<p>Hash did not work. I guess we just have to wait for the fix.<br>\nBut it would be good to figure this out by itself to not have to waste GPU for the commits. </p>",
      "rawMarkdown": "Hash did not work. I guess we just have to wait for the fix.\nBut it would be good to figure this out by itself to not have to waste GPU for the commits.",
      "votes": null
    },
    {
      "id": "985282",
      "postDate": "08/25/2020 15:59:56",
      "content": "<p>On top of this bug, if you add l5kit it is brake somehow the access to the cuda :) The GPU is enabled, your quota is decreasing, but your script is running on CPU</p>\n<pre><code>import torch\ntorch.cuda.is_available()\n</code></pre>",
      "rawMarkdown": "On top of this bug, if you add l5kit it is brake somehow the access to the cuda :) The GPU is enabled, your quota is decreasing, but your script is running on CPU\n\n```\nimport torch\ntorch.cuda.is_available()\n```",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 985266,
      "author_name": "pestipeti",
      "author_url": "",
      "post_date": "08/25/2020 15:38:17",
      "content": "<p>Probably because for this competition the test.csv contains both the public and the private part, and they don't replace it during submission.</p>\n<p>You can try one more thing: generate a hash from a file. Something like this:</p>\n<pre><code>import hashlib\n\ndef file_as_bytes(file):\n    with file:\n        return file.read()\n\nmask_file = '/kaggle/input/lyft-motion-prediction-autonomous-vehicles/scenes/mask.npz'\n\nprint(hashlib.md5(file_as_bytes(open(mask_file, 'rb'))).hexdigest())\n</code></pre>\n<p>This should output <code>5fe176ab4bd80f4e6db43d7793ab77f7</code> for the mask file we have access in the notebooks.</p>",
      "votes": null,
      "replies": [
        {
          "id": 985270,
          "author_name": "jpbremer",
          "author_url": "",
          "post_date": "08/25/2020 15:41:34",
          "content": "<p>Ah okay so Code Competition here only means to restrict computational ressources for the final submission. <br>\nThanks I will try.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 985271,
          "author_name": "pestipeti",
          "author_url": "",
          "post_date": "08/25/2020 15:43:01",
          "content": "<p>I think it is fixed now..</p>\n<p><em>Update</em> I was a bit optimistic :(</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 985279,
          "author_name": "jpbremer",
          "author_url": "",
          "post_date": "08/25/2020 15:53:16",
          "content": "<p>Hash did not work. I guess we just have to wait for the fix.<br>\nBut it would be good to figure this out by itself to not have to waste GPU for the commits. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 985282,
          "author_name": "pestipeti",
          "author_url": "",
          "post_date": "08/25/2020 15:59:56",
          "content": "<p>On top of this bug, if you add l5kit it is brake somehow the access to the cuda :) The GPU is enabled, your quota is decreasing, but your script is running on CPU</p>\n<pre><code>import torch\ntorch.cuda.is_available()\n</code></pre>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "985247": "Currently there is a problem with submitting and freezing browsers due to l5kit. \n\nI thought about introducing a bool that is True when commiting but False when submitting and rerunning on the private test set, which would be handy anyways to not waste GPU and is used in many code competitions for faster deployment. \n\nBy this I am able to submit the submission.csv file as l5kit is only installed when submitting, which you are otherwise not able to, but the bool does not change when rerunning on private test set. \n\nI tried 3 things. Theoretically if the testset changes these should change aswell.: \n\n1 Length of sample submission.csv should change when running on private -> does not\n\n`IS_COMMIT = np.array((len(sub) == 71122))`\n\n2 First value of sample submission.csv may change -> does not\n\n`IS_COMMIT = (sub.iloc[0, 0] == 1578606007801600134)`\n\n3 Number of agents in the test folder may change -> does not\n\n`IS_COMMIT = np.array((len(os.listdir(\"../input/lyft-motion-prediction-autonomous-vehicles/scenes/test.zarr/agents\")) == 4431))`\n\nDo you have an idea why these do not change when submitting to the competition?",
    "985266": "Probably because for this competition the test.csv contains both the public and the private part, and they don't replace it during submission.\n\nYou can try one more thing: generate a hash from a file. Something like this:\n\n```\nimport hashlib\n\ndef file_as_bytes(file):\n    with file:\n        return file.read()\n\nmask_file = '/kaggle/input/lyft-motion-prediction-autonomous-vehicles/scenes/mask.npz'\n    \nprint(hashlib.md5(file_as_bytes(open(mask_file, 'rb'))).hexdigest())\n```\n\nThis should output `5fe176ab4bd80f4e6db43d7793ab77f7` for the mask file we have access in the notebooks.",
    "985270": "Ah okay so Code Competition here only means to restrict computational ressources for the final submission. \nThanks I will try.",
    "985271": "I think it is fixed now..\n\n*Update* I was a bit optimistic :(",
    "985279": "Hash did not work. I guess we just have to wait for the fix.\nBut it would be good to figure this out by itself to not have to waste GPU for the commits.",
    "985282": "On top of this bug, if you add l5kit it is brake somehow the access to the cuda :) The GPU is enabled, your quota is decreasing, but your script is running on CPU\n\n```\nimport torch\ntorch.cuda.is_available()\n```"
  },
  "source": "meta"
}