{
  "id": 241690,
  "title": "Test Soundscape files and submission.csv not found (fixed)",
  "url": "/competitions/birdclef-2021/discussion/241690",
  "author_name": "Tobias Bonnesen",
  "post_date": "2021-05-25T17:13:33.319000",
  "votes": 0,
  "comment_count": 20,
  "views": 0,
  "content": "<p>[Problem is fixed]</p>\n<p>As title says, test Soundscape files and submission.csv not found</p>\n<p>I have my notebook, which does both training and evaluation in one go. <br>\nAs far as I can tell, I'm hooked up to the actual BirdCLEF 2021 competition data:</p>\n<p><img src=\"https://i.imgur.com/g9XZfol.png\" alt=\"From a normal &quot;Run all&quot; run\"></p>\n<p>Above picture is the data, after a normal \"Run all\" run. </p>\n<p>I'm using below code, to get a hold of the test set data:<br>\n<img src=\"https://i.imgur.com/DJLOuKR.png\" alt=\"\"></p>\n<p>It's code adapted from here: <a href=\"https://www.kaggle.com/stefankahl/birdclef2021-sample-submission\" target=\"_blank\">here\n</a></p>\n<p>I finish off by saving csv file and printing this:<br>\n<img src=\"https://i.imgur.com/HRDjxCX.png\" alt=\"\"></p>\n<p>When I \"Save and Run All\" it, of output of interest, I get the following:<br>\n<img src=\"https://i.imgur.com/zVywcuL.png\" alt=\"\"></p>\n<p>It's very clear to see in the first output in the picture, stemming from<br>\n<code>print(os.listdir('../input/birdclef-2021/test_soundscapes'))\n    time.sleep(5)\n    print(os.listdir('../input/birdclef-2021/test_soundscapes')[-1])\n    time.sleep(5)</code><br>\nthat it succesfully finds the test folder at the path I'm providing it with, as the csv files in that folder is listed, yet there's no ogg files to be found. (Even though we're in a Save and Run All mode)</p>\n<p>So that's at least 1 thing that's wrong. But another thing is also, that the code is setup to still predict on the training soundscapes if the test soundscapes are not found, and save the results to the csv file. We can see in some of the last output in the above pic, the format for the csv file. Yet, when I go to \"Submit to Competition\" it says \"No Output Files Found\". So, not even the succesfully created training set data is found. </p>\n<p>Now, it doesn't seem like there's any code throwing errors, or anything like that. Is it possible that the code is somehow \"breaking\" submission rules and is thus disregarded or something? </p>\n<p>I'm having the following commands in the start of the notebook:</p>\n<p><img src=\"https://i.imgur.com/LWljWSI.png\" alt=\"\"><br>\nand they all install succesfully.</p>\n<p>Could it be because I'm \"Using internet\" in order to do the pip commands? And that it's not allowed? But as far as I understand, it should still run and finish, and create the csv. It's first when I try to submit the csv that it would warn me. </p>\n<p>Thank you very much in advance for your help! </p>",
  "messages": [
    {
      "id": 1324268,
      "postDate": "2021-05-26T18:47:56.097Z",
      "content": "<p>How did you fixed it?  </p>",
      "rawMarkdown": "How did you fixed it?  ",
      "votes": 1,
      "replies": [
        {
          "id": 1325369,
          "postDate": "2021-05-27T17:36:55.610Z",
          "content": "<p>Tbh, I don't know. I made some conditional code unconditional (i.e. removed boolean flag that had to be true for it to run, but it was always set to true) and thus unindented the code. And then I threw several to_csv() calls in, to different takes on the path. So, maybe it was a bug in the indentation level of the code or writing to the wrong path. I will look at it really close later and find out for sure, for now I'm just happy that I can submit 😌</p>",
          "rawMarkdown": "Tbh, I don't know. I made some conditional code unconditional (i.e. removed boolean flag that had to be true for it to run, but it was always set to true) and thus unindented the code. And then I threw several to_csv() calls in, to different takes on the path. So, maybe it was a bug in the indentation level of the code or writing to the wrong path. I will look at it really close later and find out for sure, for now I'm just happy that I can submit 😌"
        }
      ]
    },
    {
      "id": 1322822,
      "postDate": "2021-05-25T17:51:04.057Z",
      "content": "<p>internet connection has to be disabled for submission.  And your notebook can only access the test data in the submission run, not in the commit run.<br>\nLook at the example submission notebooks shared by host or some participants.</p>",
      "rawMarkdown": "internet connection has to be disabled for submission.  And your notebook can only access the test data in the submission run, not in the commit run.\nLook at the example submission notebooks shared by host or some participants.",
      "votes": 1,
      "replies": [
        {
          "id": 1323668,
          "postDate": "2021-05-26T11:45:58.340Z",
          "content": "<p>Thanks for replying! I feel like I'm following the steps from here: <a href=\"https://www.kaggle.com/stefankahl/birdclef2021-sample-submission\" target=\"_blank\">https://www.kaggle.com/stefankahl/birdclef2021-sample-submission</a> exactly as they're written. </p>\n<p>But I'm still stuck.</p>\n<p>1) Within the notebook, I disable the internet, then press Save Version and select \"Save and Run All (Commit)\". </p>\n<p>2) I then go to Code -&gt; Your Work -&gt; Press the notebook --&gt; Press the three dots --&gt; Press Submit to Competition. </p>\n<p>3) Then below image comes up. It says no csv file found (which I don't understand, because when I run the notebook from within the notebook through \"Run All\", a submission.csv file is created - even if that's not based on the test dataset). But other than that, I can't press the Submit buttom either. And I suppose that it's this buttom I have to press, to make a submission run? </p>\n<p><img src=\"https://i.imgur.com/viTlQCs.png\" alt=\"\"> </p>",
          "rawMarkdown": "Thanks for replying! I feel like I'm following the steps from here: https://www.kaggle.com/stefankahl/birdclef2021-sample-submission exactly as they're written. \n\nBut I'm still stuck.\n\n1) Within the notebook, I disable the internet, then press Save Version and select \"Save and Run All (Commit)\". \n\n2) I then go to Code -> Your Work -> Press the notebook --> Press the three dots --> Press Submit to Competition. \n\n3) Then below image comes up. It says no csv file found (which I don't understand, because when I run the notebook from within the notebook through \"Run All\", a submission.csv file is created - even if that's not based on the test dataset). But other than that, I can't press the Submit buttom either. And I suppose that it's this buttom I have to press, to make a submission run? \n\n![](https://i.imgur.com/viTlQCs.png) "
        },
        {
          "id": 1323671,
          "postDate": "2021-05-26T11:48:00.553Z",
          "content": "<p>do you perform step 2 after step 1 has completed?  The csv file it is looking for should be the output of your committed version.  You need to select the submission file if you have more output than just the submission file.</p>",
          "rawMarkdown": "do you perform step 2 after step 1 has completed?  The csv file it is looking for should be the output of your committed version.  You need to select the submission file if you have more output than just the submission file."
        },
        {
          "id": 1323679,
          "postDate": "2021-05-26T11:51:26.500Z",
          "content": "<p>Yes. \"Eval 1 - No Internet\" e.g. is the run I just made, with internet turned off, and I waited until it said \"Done\". Might it be because it detected that no submission.csv file was generated in the \"test run\" (when pressing Save and Run all), thus it's preventing me from pressing \"Submit\" because it believes that no csv file would come in the real run? </p>\n<p>It's just weird, because the submission.csv file is generated when pressing \"Run all\" within the kernel (i.e. not when saving) (you can see it in the previously uploaded pic here, that the submission.csv file is indeed under the working dir)</p>",
          "rawMarkdown": "Yes. \"Eval 1 - No Internet\" e.g. is the run I just made, with internet turned off, and I waited until it said \"Done\". Might it be because it detected that no submission.csv file was generated in the \"test run\" (when pressing Save and Run all), thus it's preventing me from pressing \"Submit\" because it believes that no csv file would come in the real run? \n\nIt's just weird, because the submission.csv file is generated when pressing \"Run all\" within the kernel (i.e. not when saving) (you can see it in the previously uploaded pic here, that the submission.csv file is indeed under the working dir)\n\n\n"
        },
        {
          "id": 1323689,
          "postDate": "2021-05-26T11:57:12.457Z",
          "content": "<p>Okay, so several things might be playing against me. When I originally made the thread, I had Internet on so that I could do pip installs necessary for the training step (audiomentations). So that might have been a problem back then. Now I've done so that I'm training in the kernel, then when doing the submission I'm ONLY evaluating. But this means that it can't find the model ofc. </p>\n<p>\"OSError: SavedModel file does not exist at: best_model.h5/{saved_model.pbtxt|saved_model.pb}\"</p>\n<p>So that might be my problem now :D Any smart way to upload the model, instead of having to add it as a \"dataset\" ? Would get cumbersome every time a new model is trained d; </p>\n<p>I will try and test these things out, e.g. upload the saved model to dataset, then try again. I'll keep you posted. Thanks for taking the time to reply! :-)</p>",
          "rawMarkdown": "Okay, so several things might be playing against me. When I originally made the thread, I had Internet on so that I could do pip installs necessary for the training step (audiomentations). So that might have been a problem back then. Now I've done so that I'm training in the kernel, then when doing the submission I'm ONLY evaluating. But this means that it can't find the model ofc. \n\n\"OSError: SavedModel file does not exist at: best_model.h5/{saved_model.pbtxt|saved_model.pb}\"\n\nSo that might be my problem now :D Any smart way to upload the model, instead of having to add it as a \"dataset\" ? Would get cumbersome every time a new model is trained d; \n\nI will try and test these things out, e.g. upload the saved model to dataset, then try again. I'll keep you posted. Thanks for taking the time to reply! :-)\n"
        },
        {
          "id": 1323785,
          "postDate": "2021-05-26T12:46:46.630Z",
          "content": "<p>Okay:<br>\n1) No internet<br>\n2) No pip installs<br>\n3) Both training and evaluating<br>\n4) Saved model succesfully retrieved<br>\n5) NO errors. </p>\n<p>Pressing \"Save version\" --&gt; Save and Run all (Commit) <br>\nGoing to Code --&gt; Your Work --&gt; BirdClef21 Thesis Training --&gt; Version 19. <br>\nI look at the log, no errors, it ends with this output: <br>\n<img src=\"https://i.imgur.com/Qr7qraO.png\" alt=\"\"><br>\nI'm printing the os.listdir and it's very clear that the submission.csv file is there. <br>\nYet when I press the three dots, Submit to Competition - it says no csv file found and I can't press submit. I'm at a loss d; </p>",
          "rawMarkdown": "Okay:\n1) No internet\n2) No pip installs\n3) Both training and evaluating\n4) Saved model succesfully retrieved\n5) NO errors. \n\nPressing \"Save version\" --> Save and Run all (Commit) \nGoing to Code --> Your Work --> BirdClef21 Thesis Training --> Version 19. \nI look at the log, no errors, it ends with this output: \n![](https://i.imgur.com/Qr7qraO.png)\nI'm printing the os.listdir and it's very clear that the submission.csv file is there. \nYet when I press the three dots, Submit to Competition - it says no csv file found and I can't press submit. I'm at a loss d; \n"
        },
        {
          "id": 1323801,
          "postDate": "2021-05-26T12:55:48.950Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 1323844,
          "postDate": "2021-05-26T13:15:46.580Z",
          "content": "<p>do you see submission.csv in the output of your version?  I am not discussing os.dir print, I am discussing the output as shown by kaggle UI.</p>",
          "rawMarkdown": "do you see submission.csv in the output of your version?  I am not discussing os.dir print, I am discussing the output as shown by kaggle UI."
        },
        {
          "id": 1323932,
          "postDate": "2021-05-26T14:08:58.753Z",
          "content": "<p>If you don't answer then I assume you don't need further help.  Good luck.</p>\n<p>PS.  I looked at your google doc, and I don't see the answer to my question there either.</p>",
          "rawMarkdown": "If you don't answer then I assume you don't need further help.  Good luck.\n\nPS.  I looked at your google doc, and I don't see the answer to my question there either.",
          "votes": 1
        },
        {
          "id": 1324143,
          "postDate": "2021-05-26T16:44:18.600Z",
          "content": "<p>Sorry for not answering - kept working on it and my hair turned gray. </p>\n<p>I started a new notebook, and only had the Sample Submission code in it from Stefan Kahl, and it worked. I could submit it and everything. Then I added all my training code to it. And just like that, it didn't work anymore. Now I've made a version with only evaluation step in it. And it doesn't work either. I think I can share the eval notebook with you, there's nothing groundbreaking in it that will affect the competition. </p>\n<p>When you ask if the submission.csv is in the output of the Kaggle UI, what are you referring to? Are you referring to an output section here? (see picture below) <br>\n<img src=\"https://i.imgur.com/2CctBC8.png\" alt=\"\"></p>\n<p>Because if so, then in this evaluation version, there's not such an output section no. In the training version, there was an output section, but it only showed Melspecs? </p>",
          "rawMarkdown": "Sorry for not answering - kept working on it and my hair turned gray. \n\nI started a new notebook, and only had the Sample Submission code in it from Stefan Kahl, and it worked. I could submit it and everything. Then I added all my training code to it. And just like that, it didn't work anymore. Now I've made a version with only evaluation step in it. And it doesn't work either. I think I can share the eval notebook with you, there's nothing groundbreaking in it that will affect the competition. \n\nWhen you ask if the submission.csv is in the output of the Kaggle UI, what are you referring to? Are you referring to an output section here? (see picture below) \n![](https://i.imgur.com/2CctBC8.png)\n\nBecause if so, then in this evaluation version, there's not such an output section no. In the training version, there was an output section, but it only showed Melspecs? \n"
        },
        {
          "id": 1324152,
          "postDate": "2021-05-26T16:49:04.430Z",
          "content": "<p>Sorry for you gray hair ;)</p>\n<p>Don't share anything privately with me as this is against the rules.  And if you did I will not open it for the same reasons.</p>\n<p>Yes, I am referring to the output section you point to.</p>\n<p>If there is no output section then it is because you don't save any file probably.</p>",
          "rawMarkdown": "Sorry for you gray hair ;)\n\nDon't share anything privately with me as this is against the rules.  And if you did I will not open it for the same reasons.\n\nYes, I am referring to the output section you point to.\n\nIf there is no output section then it is because you don't save any file probably.",
          "votes": 1
        },
        {
          "id": 1324171,
          "postDate": "2021-05-26T17:01:55.103Z",
          "content": "<p>It's okay.. actually, the grey hair strands were already there.. 😉</p>\n<p>Ah sorry! Thought it was only if it was a high ranking notebook. </p>\n<p>It's insane. I have a boolean value, \"TEST_SUBMISSION\", that, when true, makes it so only Stefans code here runs, in a cell: <a href=\"https://www.kaggle.com/stefankahl/birdclef2021-sample-submission\" target=\"_blank\">https://www.kaggle.com/stefankahl/birdclef2021-sample-submission</a></p>\n<p>And then I can submit it to the competition and everything works. But if I, in the same notebook, do any of my training or evaluation, and use the exact same code, in another cell and context, to save the result to a submission.csv file --- then it's not in the output. </p>\n<p>😂😭</p>",
          "rawMarkdown": "It's okay.. actually, the grey hair strands were already there.. 😉\n\nAh sorry! Thought it was only if it was a high ranking notebook. \n\nIt's insane. I have a boolean value, \"TEST_SUBMISSION\", that, when true, makes it so only Stefans code here runs, in a cell: [https://www.kaggle.com/stefankahl/birdclef2021-sample-submission](https://www.kaggle.com/stefankahl/birdclef2021-sample-submission)\n\nAnd then I can submit it to the competition and everything works. But if I, in the same notebook, do any of my training or evaluation, and use the exact same code, in another cell and context, to save the result to a submission.csv file --- then it's not in the output. \n\n😂😭"
        },
        {
          "id": 1324181,
          "postDate": "2021-05-26T17:08:06.030Z",
          "content": "<p>Thanks for the help at least! :-) I don't know what more there is to be done, other than really looking every character of code through and make sure that everything is as it's supposed to. I just feel like I haven't seen any indications in print statements etc. that anything was amiss. Oh well. </p>",
          "rawMarkdown": "Thanks for the help at least! :-) I don't know what more there is to be done, other than really looking every character of code through and make sure that everything is as it's supposed to. I just feel like I haven't seen any indications in print statements etc. that anything was amiss. Oh well. ",
          "votes": 1
        },
        {
          "id": 1324194,
          "postDate": "2021-05-26T17:21:36.390Z",
          "content": "<p>If there is no output then it is because your code breaks somewhere.  Good luck with debugging it.</p>",
          "rawMarkdown": "If there is no output then it is because your code breaks somewhere.  Good luck with debugging it."
        }
      ]
    },
    {
      "id": 1323850,
      "postDate": "2021-05-26T13:18:02.583Z",
      "content": "<p><a href=\"https://docs.google.com/document/d/1n1ee8vcIyKdzF0d-SVOeRXUxSmuJNbW0DtE8mxqJzvs/edit?usp=sharing\" target=\"_blank\">https://docs.google.com/document/d/1n1ee8vcIyKdzF0d-SVOeRXUxSmuJNbW0DtE8mxqJzvs/edit?usp=sharing</a></p>\n<p>Here's everything I'm doing described with screenshots! </p>",
      "rawMarkdown": "https://docs.google.com/document/d/1n1ee8vcIyKdzF0d-SVOeRXUxSmuJNbW0DtE8mxqJzvs/edit?usp=sharing\n\nHere's everything I'm doing described with screenshots! "
    },
    {
      "id": 1323809,
      "postDate": "2021-05-26T12:59:55.620Z",
      "content": "<p>ahh you are not allowed to submit because you train and evaluate (inference) on the same kernel and that violates the allowed time I guess, so try to separate training from inference </p>\n<p>see Overview/Code requirements </p>\n<blockquote>\n  <p>GPU Notebook &lt;= 3 hours run-time</p>\n</blockquote>",
      "rawMarkdown": "ahh you are not allowed to submit because you train and evaluate (inference) on the same kernel and that violates the allowed time I guess, so try to separate training from inference \n\nsee Overview/Code requirements \n> GPU Notebook <= 3 hours run-time",
      "replies": [
        {
          "id": 1323835,
          "postDate": "2021-05-26T13:11:13.013Z",
          "content": "<p>Thanks for replying Ioannis, but I'm running a very simple model, 25 epochs - it takes less than 10 minutes to run both training and eval :-) </p>",
          "rawMarkdown": "Thanks for replying Ioannis, but I'm running a very simple model, 25 epochs - it takes less than 10 minutes to run both training and eval :-) "
        }
      ]
    },
    {
      "id": 1323795,
      "postDate": "2021-05-26T12:53:13.540Z",
      "content": "<p>I'd suggest you to fork and copy a public inference kernel, e.g. from kkiller's  (do not to forget to upvote if you make use someone else work)<br>\n-change accordingly with your model def &amp; weights - press  \"Save and Run All (Commit)\" with internet disabled and when it finishes check the Output section - normally you will see the <code>submission.csv</code> file</p>",
      "rawMarkdown": "I'd suggest you to fork and copy a public inference kernel, e.g. from kkiller's  (do not to forget to upvote if you make use someone else work)\n-change accordingly with your model def & weights - press  \"Save and Run All (Commit)\" with internet disabled and when it finishes check the Output section - normally you will see the `submission.csv` file\n"
    },
    {
      "id": 1322783,
      "postDate": "2021-05-25T17:13:33.320Z",
      "content": "<p>[Problem is fixed]</p>\n<p>As title says, test Soundscape files and submission.csv not found</p>\n<p>I have my notebook, which does both training and evaluation in one go. <br>\nAs far as I can tell, I'm hooked up to the actual BirdCLEF 2021 competition data:</p>\n<p><img src=\"https://i.imgur.com/g9XZfol.png\" alt=\"From a normal &quot;Run all&quot; run\"></p>\n<p>Above picture is the data, after a normal \"Run all\" run. </p>\n<p>I'm using below code, to get a hold of the test set data:<br>\n<img src=\"https://i.imgur.com/DJLOuKR.png\" alt=\"\"></p>\n<p>It's code adapted from here: <a href=\"https://www.kaggle.com/stefankahl/birdclef2021-sample-submission\" target=\"_blank\">here\n</a></p>\n<p>I finish off by saving csv file and printing this:<br>\n<img src=\"https://i.imgur.com/HRDjxCX.png\" alt=\"\"></p>\n<p>When I \"Save and Run All\" it, of output of interest, I get the following:<br>\n<img src=\"https://i.imgur.com/zVywcuL.png\" alt=\"\"></p>\n<p>It's very clear to see in the first output in the picture, stemming from<br>\n<code>print(os.listdir('../input/birdclef-2021/test_soundscapes'))\n    time.sleep(5)\n    print(os.listdir('../input/birdclef-2021/test_soundscapes')[-1])\n    time.sleep(5)</code><br>\nthat it succesfully finds the test folder at the path I'm providing it with, as the csv files in that folder is listed, yet there's no ogg files to be found. (Even though we're in a Save and Run All mode)</p>\n<p>So that's at least 1 thing that's wrong. But another thing is also, that the code is setup to still predict on the training soundscapes if the test soundscapes are not found, and save the results to the csv file. We can see in some of the last output in the above pic, the format for the csv file. Yet, when I go to \"Submit to Competition\" it says \"No Output Files Found\". So, not even the succesfully created training set data is found. </p>\n<p>Now, it doesn't seem like there's any code throwing errors, or anything like that. Is it possible that the code is somehow \"breaking\" submission rules and is thus disregarded or something? </p>\n<p>I'm having the following commands in the start of the notebook:</p>\n<p><img src=\"https://i.imgur.com/LWljWSI.png\" alt=\"\"><br>\nand they all install succesfully.</p>\n<p>Could it be because I'm \"Using internet\" in order to do the pip commands? And that it's not allowed? But as far as I understand, it should still run and finish, and create the csv. It's first when I try to submit the csv that it would warn me. </p>\n<p>Thank you very much in advance for your help! </p>",
      "rawMarkdown": "[Problem is fixed]\n\nAs title says, test Soundscape files and submission.csv not found\n\nI have my notebook, which does both training and evaluation in one go. \nAs far as I can tell, I'm hooked up to the actual BirdCLEF 2021 competition data:\n\n![From a normal \"Run all\" run](https://i.imgur.com/g9XZfol.png)\n\nAbove picture is the data, after a normal \"Run all\" run. \n\nI'm using below code, to get a hold of the test set data:\n![](https://i.imgur.com/DJLOuKR.png)\n\nIt's code adapted from here: [here\n](https://www.kaggle.com/stefankahl/birdclef2021-sample-submission)\n\nI finish off by saving csv file and printing this:\n![](https://i.imgur.com/HRDjxCX.png)\n\nWhen I \"Save and Run All\" it, of output of interest, I get the following:\n![](https://i.imgur.com/zVywcuL.png)\n\nIt's very clear to see in the first output in the picture, stemming from\n`    print(os.listdir('../input/birdclef-2021/test_soundscapes'))\n    time.sleep(5)\n    print(os.listdir('../input/birdclef-2021/test_soundscapes')[-1])\n    time.sleep(5)`\nthat it succesfully finds the test folder at the path I'm providing it with, as the csv files in that folder is listed, yet there's no ogg files to be found. (Even though we're in a Save and Run All mode)\n\nSo that's at least 1 thing that's wrong. But another thing is also, that the code is setup to still predict on the training soundscapes if the test soundscapes are not found, and save the results to the csv file. We can see in some of the last output in the above pic, the format for the csv file. Yet, when I go to \"Submit to Competition\" it says \"No Output Files Found\". So, not even the succesfully created training set data is found. \n\nNow, it doesn't seem like there's any code throwing errors, or anything like that. Is it possible that the code is somehow \"breaking\" submission rules and is thus disregarded or something? \n\nI'm having the following commands in the start of the notebook:\n\n![](https://i.imgur.com/LWljWSI.png)\nand they all install succesfully.\n\nCould it be because I'm \"Using internet\" in order to do the pip commands? And that it's not allowed? But as far as I understand, it should still run and finish, and create the csv. It's first when I try to submit the csv that it would warn me. \n\nThank you very much in advance for your help! \n\n\n\n\n"
    }
  ],
  "comments": [
    {
      "id": 1324268,
      "author_name": "CPMP",
      "author_url": "",
      "post_date": "2021-05-26T18:47:56.097000",
      "content": "<p>How did you fixed it?  </p>",
      "votes": 1,
      "replies": [
        {
          "id": 1325369,
          "author_name": "Tobias Bonnesen",
          "author_url": "",
          "post_date": "2021-05-27T17:36:55.610000",
          "content": "<p>Tbh, I don't know. I made some conditional code unconditional (i.e. removed boolean flag that had to be true for it to run, but it was always set to true) and thus unindented the code. And then I threw several to_csv() calls in, to different takes on the path. So, maybe it was a bug in the indentation level of the code or writing to the wrong path. I will look at it really close later and find out for sure, for now I'm just happy that I can submit 😌</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1322822,
      "author_name": "CPMP",
      "author_url": "",
      "post_date": "2021-05-25T17:51:04.057000",
      "content": "<p>internet connection has to be disabled for submission.  And your notebook can only access the test data in the submission run, not in the commit run.<br>\nLook at the example submission notebooks shared by host or some participants.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1323668,
          "author_name": "Tobias Bonnesen",
          "author_url": "",
          "post_date": "2021-05-26T11:45:58.340000",
          "content": "<p>Thanks for replying! I feel like I'm following the steps from here: <a href=\"https://www.kaggle.com/stefankahl/birdclef2021-sample-submission\" target=\"_blank\">https://www.kaggle.com/stefankahl/birdclef2021-sample-submission</a> exactly as they're written. </p>\n<p>But I'm still stuck.</p>\n<p>1) Within the notebook, I disable the internet, then press Save Version and select \"Save and Run All (Commit)\". </p>\n<p>2) I then go to Code -&gt; Your Work -&gt; Press the notebook --&gt; Press the three dots --&gt; Press Submit to Competition. </p>\n<p>3) Then below image comes up. It says no csv file found (which I don't understand, because when I run the notebook from within the notebook through \"Run All\", a submission.csv file is created - even if that's not based on the test dataset). But other than that, I can't press the Submit buttom either. And I suppose that it's this buttom I have to press, to make a submission run? </p>\n<p><img src=\"https://i.imgur.com/viTlQCs.png\" alt=\"\"> </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1323671,
          "author_name": "CPMP",
          "author_url": "",
          "post_date": "2021-05-26T11:48:00.553000",
          "content": "<p>do you perform step 2 after step 1 has completed?  The csv file it is looking for should be the output of your committed version.  You need to select the submission file if you have more output than just the submission file.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1323679,
          "author_name": "Tobias Bonnesen",
          "author_url": "",
          "post_date": "2021-05-26T11:51:26.500000",
          "content": "<p>Yes. \"Eval 1 - No Internet\" e.g. is the run I just made, with internet turned off, and I waited until it said \"Done\". Might it be because it detected that no submission.csv file was generated in the \"test run\" (when pressing Save and Run all), thus it's preventing me from pressing \"Submit\" because it believes that no csv file would come in the real run? </p>\n<p>It's just weird, because the submission.csv file is generated when pressing \"Run all\" within the kernel (i.e. not when saving) (you can see it in the previously uploaded pic here, that the submission.csv file is indeed under the working dir)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1323689,
          "author_name": "Tobias Bonnesen",
          "author_url": "",
          "post_date": "2021-05-26T11:57:12.457000",
          "content": "<p>Okay, so several things might be playing against me. When I originally made the thread, I had Internet on so that I could do pip installs necessary for the training step (audiomentations). So that might have been a problem back then. Now I've done so that I'm training in the kernel, then when doing the submission I'm ONLY evaluating. But this means that it can't find the model ofc. </p>\n<p>\"OSError: SavedModel file does not exist at: best_model.h5/{saved_model.pbtxt|saved_model.pb}\"</p>\n<p>So that might be my problem now :D Any smart way to upload the model, instead of having to add it as a \"dataset\" ? Would get cumbersome every time a new model is trained d; </p>\n<p>I will try and test these things out, e.g. upload the saved model to dataset, then try again. I'll keep you posted. Thanks for taking the time to reply! :-)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1323785,
          "author_name": "Tobias Bonnesen",
          "author_url": "",
          "post_date": "2021-05-26T12:46:46.630000",
          "content": "<p>Okay:<br>\n1) No internet<br>\n2) No pip installs<br>\n3) Both training and evaluating<br>\n4) Saved model succesfully retrieved<br>\n5) NO errors. </p>\n<p>Pressing \"Save version\" --&gt; Save and Run all (Commit) <br>\nGoing to Code --&gt; Your Work --&gt; BirdClef21 Thesis Training --&gt; Version 19. <br>\nI look at the log, no errors, it ends with this output: <br>\n<img src=\"https://i.imgur.com/Qr7qraO.png\" alt=\"\"><br>\nI'm printing the os.listdir and it's very clear that the submission.csv file is there. <br>\nYet when I press the three dots, Submit to Competition - it says no csv file found and I can't press submit. I'm at a loss d; </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1323801,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-05-26T12:55:48.950000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1323844,
          "author_name": "CPMP",
          "author_url": "",
          "post_date": "2021-05-26T13:15:46.580000",
          "content": "<p>do you see submission.csv in the output of your version?  I am not discussing os.dir print, I am discussing the output as shown by kaggle UI.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1323932,
          "author_name": "CPMP",
          "author_url": "",
          "post_date": "2021-05-26T14:08:58.753000",
          "content": "<p>If you don't answer then I assume you don't need further help.  Good luck.</p>\n<p>PS.  I looked at your google doc, and I don't see the answer to my question there either.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1324143,
          "author_name": "Tobias Bonnesen",
          "author_url": "",
          "post_date": "2021-05-26T16:44:18.600000",
          "content": "<p>Sorry for not answering - kept working on it and my hair turned gray. </p>\n<p>I started a new notebook, and only had the Sample Submission code in it from Stefan Kahl, and it worked. I could submit it and everything. Then I added all my training code to it. And just like that, it didn't work anymore. Now I've made a version with only evaluation step in it. And it doesn't work either. I think I can share the eval notebook with you, there's nothing groundbreaking in it that will affect the competition. </p>\n<p>When you ask if the submission.csv is in the output of the Kaggle UI, what are you referring to? Are you referring to an output section here? (see picture below) <br>\n<img src=\"https://i.imgur.com/2CctBC8.png\" alt=\"\"></p>\n<p>Because if so, then in this evaluation version, there's not such an output section no. In the training version, there was an output section, but it only showed Melspecs? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1324152,
          "author_name": "CPMP",
          "author_url": "",
          "post_date": "2021-05-26T16:49:04.430000",
          "content": "<p>Sorry for you gray hair ;)</p>\n<p>Don't share anything privately with me as this is against the rules.  And if you did I will not open it for the same reasons.</p>\n<p>Yes, I am referring to the output section you point to.</p>\n<p>If there is no output section then it is because you don't save any file probably.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1324171,
          "author_name": "Tobias Bonnesen",
          "author_url": "",
          "post_date": "2021-05-26T17:01:55.103000",
          "content": "<p>It's okay.. actually, the grey hair strands were already there.. 😉</p>\n<p>Ah sorry! Thought it was only if it was a high ranking notebook. </p>\n<p>It's insane. I have a boolean value, \"TEST_SUBMISSION\", that, when true, makes it so only Stefans code here runs, in a cell: <a href=\"https://www.kaggle.com/stefankahl/birdclef2021-sample-submission\" target=\"_blank\">https://www.kaggle.com/stefankahl/birdclef2021-sample-submission</a></p>\n<p>And then I can submit it to the competition and everything works. But if I, in the same notebook, do any of my training or evaluation, and use the exact same code, in another cell and context, to save the result to a submission.csv file --- then it's not in the output. </p>\n<p>😂😭</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1324181,
          "author_name": "Tobias Bonnesen",
          "author_url": "",
          "post_date": "2021-05-26T17:08:06.030000",
          "content": "<p>Thanks for the help at least! :-) I don't know what more there is to be done, other than really looking every character of code through and make sure that everything is as it's supposed to. I just feel like I haven't seen any indications in print statements etc. that anything was amiss. Oh well. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1324194,
          "author_name": "CPMP",
          "author_url": "",
          "post_date": "2021-05-26T17:21:36.390000",
          "content": "<p>If there is no output then it is because your code breaks somewhere.  Good luck with debugging it.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1323850,
      "author_name": "Tobias Bonnesen",
      "author_url": "",
      "post_date": "2021-05-26T13:18:02.583000",
      "content": "<p><a href=\"https://docs.google.com/document/d/1n1ee8vcIyKdzF0d-SVOeRXUxSmuJNbW0DtE8mxqJzvs/edit?usp=sharing\" target=\"_blank\">https://docs.google.com/document/d/1n1ee8vcIyKdzF0d-SVOeRXUxSmuJNbW0DtE8mxqJzvs/edit?usp=sharing</a></p>\n<p>Here's everything I'm doing described with screenshots! </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1323809,
      "author_name": "Ioannis M",
      "author_url": "",
      "post_date": "2021-05-26T12:59:55.620000",
      "content": "<p>ahh you are not allowed to submit because you train and evaluate (inference) on the same kernel and that violates the allowed time I guess, so try to separate training from inference </p>\n<p>see Overview/Code requirements </p>\n<blockquote>\n  <p>GPU Notebook &lt;= 3 hours run-time</p>\n</blockquote>",
      "votes": 0,
      "replies": [
        {
          "id": 1323835,
          "author_name": "Tobias Bonnesen",
          "author_url": "",
          "post_date": "2021-05-26T13:11:13.013000",
          "content": "<p>Thanks for replying Ioannis, but I'm running a very simple model, 25 epochs - it takes less than 10 minutes to run both training and eval :-) </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1323795,
      "author_name": "Ioannis M",
      "author_url": "",
      "post_date": "2021-05-26T12:53:13.540000",
      "content": "<p>I'd suggest you to fork and copy a public inference kernel, e.g. from kkiller's  (do not to forget to upvote if you make use someone else work)<br>\n-change accordingly with your model def &amp; weights - press  \"Save and Run All (Commit)\" with internet disabled and when it finishes check the Output section - normally you will see the <code>submission.csv</code> file</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1324268": "How did you fixed it?  ",
    "1322822": "internet connection has to be disabled for submission.  And your notebook can only access the test data in the submission run, not in the commit run.\nLook at the example submission notebooks shared by host or some participants.",
    "1323850": "https://docs.google.com/document/d/1n1ee8vcIyKdzF0d-SVOeRXUxSmuJNbW0DtE8mxqJzvs/edit?usp=sharing\n\nHere's everything I'm doing described with screenshots! ",
    "1323809": "ahh you are not allowed to submit because you train and evaluate (inference) on the same kernel and that violates the allowed time I guess, so try to separate training from inference \n\nsee Overview/Code requirements \n> GPU Notebook <= 3 hours run-time",
    "1323795": "I'd suggest you to fork and copy a public inference kernel, e.g. from kkiller's  (do not to forget to upvote if you make use someone else work)\n-change accordingly with your model def & weights - press  \"Save and Run All (Commit)\" with internet disabled and when it finishes check the Output section - normally you will see the `submission.csv` file\n",
    "1322783": "[Problem is fixed]\n\nAs title says, test Soundscape files and submission.csv not found\n\nI have my notebook, which does both training and evaluation in one go. \nAs far as I can tell, I'm hooked up to the actual BirdCLEF 2021 competition data:\n\n![From a normal \"Run all\" run](https://i.imgur.com/g9XZfol.png)\n\nAbove picture is the data, after a normal \"Run all\" run. \n\nI'm using below code, to get a hold of the test set data:\n![](https://i.imgur.com/DJLOuKR.png)\n\nIt's code adapted from here: [here\n](https://www.kaggle.com/stefankahl/birdclef2021-sample-submission)\n\nI finish off by saving csv file and printing this:\n![](https://i.imgur.com/HRDjxCX.png)\n\nWhen I \"Save and Run All\" it, of output of interest, I get the following:\n![](https://i.imgur.com/zVywcuL.png)\n\nIt's very clear to see in the first output in the picture, stemming from\n`    print(os.listdir('../input/birdclef-2021/test_soundscapes'))\n    time.sleep(5)\n    print(os.listdir('../input/birdclef-2021/test_soundscapes')[-1])\n    time.sleep(5)`\nthat it succesfully finds the test folder at the path I'm providing it with, as the csv files in that folder is listed, yet there's no ogg files to be found. (Even though we're in a Save and Run All mode)\n\nSo that's at least 1 thing that's wrong. But another thing is also, that the code is setup to still predict on the training soundscapes if the test soundscapes are not found, and save the results to the csv file. We can see in some of the last output in the above pic, the format for the csv file. Yet, when I go to \"Submit to Competition\" it says \"No Output Files Found\". So, not even the succesfully created training set data is found. \n\nNow, it doesn't seem like there's any code throwing errors, or anything like that. Is it possible that the code is somehow \"breaking\" submission rules and is thus disregarded or something? \n\nI'm having the following commands in the start of the notebook:\n\n![](https://i.imgur.com/LWljWSI.png)\nand they all install succesfully.\n\nCould it be because I'm \"Using internet\" in order to do the pip commands? And that it's not allowed? But as far as I understand, it should still run and finish, and create the csv. It's first when I try to submit the csv that it would warn me. \n\nThank you very much in advance for your help! \n\n\n\n\n"
  }
}