{
  "id": 480686,
  "title": "Seeking tips for debugging for submission",
  "url": "/competitions/hms-harmful-brain-activity-classification/discussion/480686",
  "author_name": "",
  "post_date": "2024-02-29T15:45:08.856022500Z",
  "votes": null,
  "comment_count": 12,
  "views": 0,
  "content": "<p>EDIT: I finally figured it out. I was appending the segment information onto the eeg_id for the final label. Thanks everyone</p>\n<p>I am looking for help on how to debug my notebook for the private data. My code breaks somewhere and I get the error \"cannot find submission.csv\". This was very confusing at first because I could find my submission.csv in the output of my notebook run. I found another forum message that stated the private data does a different run. </p>\n<p>Does the data persist? If so, which data?</p>\n<p>Any tips would be appreciated. Thanks.</p>",
  "messages": [
    {
      "id": "2674917",
      "postDate": "02/29/2024 15:45:08",
      "content": "<p>EDIT: I finally figured it out. I was appending the segment information onto the eeg_id for the final label. Thanks everyone</p>\n<p>I am looking for help on how to debug my notebook for the private data. My code breaks somewhere and I get the error \"cannot find submission.csv\". This was very confusing at first because I could find my submission.csv in the output of my notebook run. I found another forum message that stated the private data does a different run. </p>\n<p>Does the data persist? If so, which data?</p>\n<p>Any tips would be appreciated. Thanks.</p>",
      "rawMarkdown": "EDIT: I finally figured it out. I was appending the segment information onto the eeg_id for the final label. Thanks everyone\n\nI am looking for help on how to debug my notebook for the private data. My code breaks somewhere and I get the error \"cannot find submission.csv\". This was very confusing at first because I could find my submission.csv in the output of my notebook run. I found another forum message that stated the private data does a different run. \n\nDoes the data persist? If so, which data?\n\nAny tips would be appreciated. Thanks.",
      "votes": null
    },
    {
      "id": "2675018",
      "postDate": "02/29/2024 17:01:08",
      "content": "<p>Change the test path to train data and execute it locally. You'll see why crashes.</p>",
      "rawMarkdown": "Change the test path to train data and execute it locally. You'll see why crashes.",
      "votes": null
    },
    {
      "id": "2675109",
      "postDate": "02/29/2024 17:56:52",
      "content": "<p>Thank you but it still ran</p>",
      "rawMarkdown": "Thank you but it still ran",
      "votes": null
    },
    {
      "id": "2675170",
      "postDate": "02/29/2024 18:36:54",
      "content": "<p>I eventually had written it to be modular. Any other tips, please?</p>",
      "rawMarkdown": "I eventually had written it to be modular. Any other tips, please?",
      "votes": null
    },
    {
      "id": "2675173",
      "postDate": "02/29/2024 18:39:13",
      "content": "<p>It inferenced train folder without error but can't inference hidden test?</p>",
      "rawMarkdown": "It inferenced train folder without error but can't inference hidden test?",
      "votes": null
    },
    {
      "id": "2675204",
      "postDate": "02/29/2024 18:50:06",
      "content": "<p>That’s what it seems like</p>",
      "rawMarkdown": "That’s what it seems like",
      "votes": null
    },
    {
      "id": "2675211",
      "postDate": "02/29/2024 18:53:03",
      "content": "<p>I'm referencing the test.csv file. Is that a mistake?</p>",
      "rawMarkdown": "I'm referencing the test.csv file. Is that a mistake?",
      "votes": null
    },
    {
      "id": "2675229",
      "postDate": "02/29/2024 19:03:08",
      "content": "<p>No, but local test folder is one sample only. Hidden test is many samples. I suspect the problem is with multiple samples management. So that's why I suggested debugging by running inference on train samples, cause there are many.</p>",
      "rawMarkdown": "No, but local test folder is one sample only. Hidden test is many samples. I suspect the problem is with multiple samples management. So that's why I suggested debugging by running inference on train samples, cause there are many.",
      "votes": null
    },
    {
      "id": "2675862",
      "postDate": "03/01/2024 06:26:44",
      "content": "<p>I had the same error when I saved the cache from the dataset in addition to the submission.csv file to the output directory. Deleting everything except submission.csv from output solved the problem in my case.</p>",
      "rawMarkdown": "I had the same error when I saved the cache from the dataset in addition to the submission.csv file to the output directory. Deleting everything except submission.csv from output solved the problem in my case.",
      "votes": null
    },
    {
      "id": "2677168",
      "postDate": "03/02/2024 01:40:33",
      "content": "<p>I copy/pasted the original test to give myself a second test. It still works locally but not in the cloud. On the positive side, I'm using numba with the GPU to speed up my feature generation.</p>",
      "rawMarkdown": "I copy/pasted the original test to give myself a second test. It still works locally but not in the cloud. On the positive side, I'm using numba with the GPU to speed up my feature generation.",
      "votes": null
    },
    {
      "id": "2677169",
      "postDate": "03/02/2024 01:41:00",
      "content": "<ul>\n<li>which is to say my script execution has gone from 6 hours to 10 minutes with even more features.</li>\n</ul>",
      "rawMarkdown": "which is to say my script execution has gone from 6 hours to 10 minutes with even more features.",
      "votes": null
    },
    {
      "id": "2677170",
      "postDate": "03/02/2024 01:42:39",
      "content": "<p>Thank you Ivan. I'm using the shutil library to do the same. Locally, pandas overwrites the old file but I'll try deleting submission.csv while I cleanup other files.</p>",
      "rawMarkdown": "Thank you Ivan. I'm using the shutil library to do the same. Locally, pandas overwrites the old file but I'll try deleting submission.csv while I cleanup other files.",
      "votes": null
    },
    {
      "id": "2678058",
      "postDate": "03/02/2024 15:17:43",
      "content": "<p>After working with python for 12 years, I finally learned that eval(\"123_0\") = 1230</p>",
      "rawMarkdown": "After working with python for 12 years, I finally learned that eval(\"123_0\") = 1230",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2675018,
      "author_name": "sacuscreed",
      "author_url": "",
      "post_date": "02/29/2024 17:01:08",
      "content": "<p>Change the test path to train data and execute it locally. You'll see why crashes.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2675109,
          "author_name": "andrewmatte",
          "author_url": "",
          "post_date": "02/29/2024 17:56:52",
          "content": "<p>Thank you but it still ran</p>",
          "votes": null,
          "replies": [
            {
              "id": 2675170,
              "author_name": "andrewmatte",
              "author_url": "",
              "post_date": "02/29/2024 18:36:54",
              "content": "<p>I eventually had written it to be modular. Any other tips, please?</p>",
              "votes": null,
              "replies": []
            },
            {
              "id": 2675173,
              "author_name": "sacuscreed",
              "author_url": "",
              "post_date": "02/29/2024 18:39:13",
              "content": "<p>It inferenced train folder without error but can't inference hidden test?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2675204,
                  "author_name": "andrewmatte",
                  "author_url": "",
                  "post_date": "02/29/2024 18:50:06",
                  "content": "<p>That’s what it seems like</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2675211,
                      "author_name": "andrewmatte",
                      "author_url": "",
                      "post_date": "02/29/2024 18:53:03",
                      "content": "<p>I'm referencing the test.csv file. Is that a mistake?</p>",
                      "votes": null,
                      "replies": [
                        {
                          "id": 2675229,
                          "author_name": "sacuscreed",
                          "author_url": "",
                          "post_date": "02/29/2024 19:03:08",
                          "content": "<p>No, but local test folder is one sample only. Hidden test is many samples. I suspect the problem is with multiple samples management. So that's why I suggested debugging by running inference on train samples, cause there are many.</p>",
                          "votes": null,
                          "replies": [
                            {
                              "id": 2677168,
                              "author_name": "andrewmatte",
                              "author_url": "",
                              "post_date": "03/02/2024 01:40:33",
                              "content": "<p>I copy/pasted the original test to give myself a second test. It still works locally but not in the cloud. On the positive side, I'm using numba with the GPU to speed up my feature generation.</p>",
                              "votes": null,
                              "replies": [
                                {
                                  "id": 2677169,
                                  "author_name": "andrewmatte",
                                  "author_url": "",
                                  "post_date": "03/02/2024 01:41:00",
                                  "content": "<ul>\n<li>which is to say my script execution has gone from 6 hours to 10 minutes with even more features.</li>\n</ul>",
                                  "votes": null,
                                  "replies": []
                                }
                              ]
                            },
                            {
                              "id": 2678058,
                              "author_name": "andrewmatte",
                              "author_url": "",
                              "post_date": "03/02/2024 15:17:43",
                              "content": "<p>After working with python for 12 years, I finally learned that eval(\"123_0\") = 1230</p>",
                              "votes": null,
                              "replies": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2675862,
      "author_name": "ivanilyushchenko",
      "author_url": "",
      "post_date": "03/01/2024 06:26:44",
      "content": "<p>I had the same error when I saved the cache from the dataset in addition to the submission.csv file to the output directory. Deleting everything except submission.csv from output solved the problem in my case.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2677170,
          "author_name": "andrewmatte",
          "author_url": "",
          "post_date": "03/02/2024 01:42:39",
          "content": "<p>Thank you Ivan. I'm using the shutil library to do the same. Locally, pandas overwrites the old file but I'll try deleting submission.csv while I cleanup other files.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2674917": "EDIT: I finally figured it out. I was appending the segment information onto the eeg_id for the final label. Thanks everyone\n\nI am looking for help on how to debug my notebook for the private data. My code breaks somewhere and I get the error \"cannot find submission.csv\". This was very confusing at first because I could find my submission.csv in the output of my notebook run. I found another forum message that stated the private data does a different run. \n\nDoes the data persist? If so, which data?\n\nAny tips would be appreciated. Thanks.",
    "2675018": "Change the test path to train data and execute it locally. You'll see why crashes.",
    "2675109": "Thank you but it still ran",
    "2675170": "I eventually had written it to be modular. Any other tips, please?",
    "2675173": "It inferenced train folder without error but can't inference hidden test?",
    "2675204": "That’s what it seems like",
    "2675211": "I'm referencing the test.csv file. Is that a mistake?",
    "2675229": "No, but local test folder is one sample only. Hidden test is many samples. I suspect the problem is with multiple samples management. So that's why I suggested debugging by running inference on train samples, cause there are many.",
    "2675862": "I had the same error when I saved the cache from the dataset in addition to the submission.csv file to the output directory. Deleting everything except submission.csv from output solved the problem in my case.",
    "2677168": "I copy/pasted the original test to give myself a second test. It still works locally but not in the cloud. On the positive side, I'm using numba with the GPU to speed up my feature generation.",
    "2677169": "which is to say my script execution has gone from 6 hours to 10 minutes with even more features.",
    "2677170": "Thank you Ivan. I'm using the shutil library to do the same. Locally, pandas overwrites the old file but I'll try deleting submission.csv while I cleanup other files.",
    "2678058": "After working with python for 12 years, I finally learned that eval(\"123_0\") = 1230"
  },
  "source": "meta"
}