{
  "id": 403552,
  "title": "Submission Error ~ Output Look Correct?",
  "url": "/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/403552",
  "author_name": "backpack",
  "post_date": "2023-04-23T18:17:28.175000",
  "votes": 0,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Hello, I got submission scoring error (Your notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected). </p>\n<p>I did scan through the test folder and tried to include all test files.</p>\n<p>I also print out the generated submission file as below, does it look correct (format, number of row etc)?</p>\n<p>output_combined:                        Id  StartHesitation  Turn  Walking<br>\n0            003f117e14_0              0.2   0.3     0.08<br>\n1            003f117e14_1              0.2   0.3     0.08<br>\n2            003f117e14_2              0.7   0.4     0.10<br>\n3            003f117e14_3              0.7   0.4     0.10<br>\n4            003f117e14_4              0.7   0.4     0.10<br>\n…                   …              …   …      …<br>\n281683  02ab235146_281683              0.2   0.4     0.07<br>\n281684  02ab235146_281684              0.2   0.4     0.07<br>\n281685  02ab235146_281685              0.2   0.4     0.07<br>\n281686  02ab235146_281686              0.2   0.4     0.07<br>\n281687  02ab235146_281687              0.2   0.4     0.07</p>\n<p>[286370 rows x 4 columns]</p>",
  "messages": [
    {
      "id": 2232023,
      "postDate": "2023-04-23T22:57:19.757Z",
      "content": "<p>I also struggled to make a first correct submission. <br>\nThe output should look somewhat like this:<br>\n    Id  StartHesitation Turn    Walking<br>\n0    02ab235146_0    1.419214e-12    2.032387e-08    5.861471e-18<br>\n1    02ab235146_1    2.269881e-18    2.042996e-13    1.929627e-21<br>\n2    02ab235146_2    1.221375e-17    1.342052e-10    8.594881e-19<br>\n3    02ab235146_3    1.334140e-18    4.714775e-11    9.597851e-23<br>\n4    02ab235146_4    1.841961e-18    6.536569e-12    2.878523e-23<br>\n…    … … … …<br>\n286365    003f117e14_4677 4.201266e-05    4.687012e-05    3.718816e-05<br>\n286366    003f117e14_4678 4.114008e-05    4.690264e-05    3.579163e-05<br>\n286367    003f117e14_4679 4.034276e-05    4.730092e-05    4.052546e-05<br>\n286368    003f117e14_4680 4.053275e-05    4.488072e-05    3.556220e-05<br>\n286369    003f117e14_4681 4.854446e-05    4.875891e-05    4.324796e-05</p>\n<p>The main difference is that the id contains the Id plus the row number per file (instead of id + index number of the entire array). </p>",
      "rawMarkdown": "I also struggled to make a first correct submission. \nThe output should look somewhat like this:\n\tId\tStartHesitation\tTurn\tWalking\n0\t02ab235146_0\t1.419214e-12\t2.032387e-08\t5.861471e-18\n1\t02ab235146_1\t2.269881e-18\t2.042996e-13\t1.929627e-21\n2\t02ab235146_2\t1.221375e-17\t1.342052e-10\t8.594881e-19\n3\t02ab235146_3\t1.334140e-18\t4.714775e-11\t9.597851e-23\n4\t02ab235146_4\t1.841961e-18\t6.536569e-12\t2.878523e-23\n...\t...\t...\t...\t...\n286365\t003f117e14_4677\t4.201266e-05\t4.687012e-05\t3.718816e-05\n286366\t003f117e14_4678\t4.114008e-05\t4.690264e-05\t3.579163e-05\n286367\t003f117e14_4679\t4.034276e-05\t4.730092e-05\t4.052546e-05\n286368\t003f117e14_4680\t4.053275e-05\t4.488072e-05\t3.556220e-05\n286369\t003f117e14_4681\t4.854446e-05\t4.875891e-05\t4.324796e-05\n\nThe main difference is that the id contains the Id plus the row number per file (instead of id + index number of the entire array). ",
      "replies": [
        {
          "id": 2232034,
          "postDate": "2023-04-23T23:23:07.370Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 2232158,
          "postDate": "2023-04-24T04:51:05.383Z",
          "content": "<p>I made a slight change and re-run the code below. This is what I can see, it is similar to what you posted. But I still get the submission scoring error.</p>\n<p>output.to_csv('submission.csv',index=False)<br>\nprint(\"output:\",output)<br>\noutput:                        Id  StartHesitation  Turn  Walking<br>\n0            003f117e14_0              0.2   0.3     0.08<br>\n1            003f117e14_1              0.2   0.3     0.08<br>\n2            003f117e14_2              0.7   0.4     0.10<br>\n3            003f117e14_3              0.7   0.4     0.10<br>\n4            003f117e14_4              0.7   0.4     0.10<br>\n…                   …              …   …      …<br>\n286365  02ab235146_281683              0.2   0.4     0.07<br>\n286366  02ab235146_281684              0.2   0.4     0.07<br>\n286367  02ab235146_281685              0.2   0.4     0.07<br>\n286368  02ab235146_281686              0.2   0.4     0.07<br>\n286369  02ab235146_281687              0.2   0.4     0.07</p>\n<p>[286370 rows x 4 columns]</p>",
          "rawMarkdown": "I made a slight change and re-run the code below. This is what I can see, it is similar to what you posted. But I still get the submission scoring error.\n\n\n\noutput.to_csv('submission.csv',index=False)\nprint(\"output:\",output)\noutput:                        Id  StartHesitation  Turn  Walking\n0            003f117e14_0              0.2   0.3     0.08\n1            003f117e14_1              0.2   0.3     0.08\n2            003f117e14_2              0.7   0.4     0.10\n3            003f117e14_3              0.7   0.4     0.10\n4            003f117e14_4              0.7   0.4     0.10\n...                   ...              ...   ...      ...\n286365  02ab235146_281683              0.2   0.4     0.07\n286366  02ab235146_281684              0.2   0.4     0.07\n286367  02ab235146_281685              0.2   0.4     0.07\n286368  02ab235146_281686              0.2   0.4     0.07\n286369  02ab235146_281687              0.2   0.4     0.07\n\n[286370 rows x 4 columns]",
          "replies": [
            {
              "id": 2232197,
              "postDate": "2023-04-24T05:40:43.063Z",
              "rawMarkdown": "",
              "isDeleted": true
            },
            {
              "id": 2232258,
              "postDate": "2023-04-24T06:26:41.963Z",
              "content": "<p>I am confused. I downloaded the \"sample_submission\" file and check the last row id in it is \"02ab235146_281687<br>\n\",  the last row ID generated by my submission matches it. If not \"02ab235146_281687\", what is the correct id? I find there are 281688 rows in file \"02ab235146.csv\"</p>",
              "rawMarkdown": "I am confused. I downloaded the \"sample_submission\" file and check the last row id in it is \"02ab235146_281687\n\",  the last row ID generated by my submission matches it. If not \"02ab235146_281687\", what is the correct id? I find there are 281688 rows in file \"02ab235146.csv\""
            },
            {
              "id": 2232302,
              "postDate": "2023-04-24T07:13:52.797Z",
              "content": "<p>I'm sorry, you're absolutely right. I deleted my previous comment to avoid further confusion.<br>\nI believe the format of your file is correct. Are you sure you are creating predictions for all possible files in test/defog and test/tdcsfog?<br>\nI mean using something like: </p>\n<pre><code>submission = []\nfor folder in [\"defog\", \"tdcsfog\"]:       \n    for file in glob.glob(str(PATH/f\"test/{folder}/*.csv\")):\n        df = pd.read_csv(file)\n        ....\n</code></pre>",
              "rawMarkdown": "I'm sorry, you're absolutely right. I deleted my previous comment to avoid further confusion.\nI believe the format of your file is correct. Are you sure you are creating predictions for all possible files in test/defog and test/tdcsfog?\nI mean using something like: \n```\nsubmission = []\nfor folder in [\"defog\", \"tdcsfog\"]:       \n    for file in glob.glob(str(PATH/f\"test/{folder}/*.csv\")):\n        df = pd.read_csv(file)\n        ....\n```"
            },
            {
              "id": 2234151,
              "postDate": "2023-04-25T00:40:34.733Z",
              "content": "<p>I use something like<br>\n\"\"\"<br>\ntdcsfog_test_root = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/tdcsfog/'<br>\ntdcsfog_test_names = []<br>\nfor dirname, _, filenames in os.walk(tdcsfog_test_root):<br>\n--    for filename in filenames:<br>\n  ----      if filename.endswith('.csv'):<br>\n        ------    tdcsfog_test_names.append(filename)<br>\n\"\"\"<br>\nprint(\"tdcsfog_test_names:\",tdcsfog_test_names)<br>\ntdcsfog_test_names: ['003f117e14.csv']</p>",
              "rawMarkdown": "I use something like\n\"\"\"\ntdcsfog_test_root = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/tdcsfog/'\ntdcsfog_test_names = []\nfor dirname, _, filenames in os.walk(tdcsfog_test_root):\n--    for filename in filenames:\n  ----      if filename.endswith('.csv'):\n        ------    tdcsfog_test_names.append(filename)\n\"\"\"\nprint(\"tdcsfog_test_names:\",tdcsfog_test_names)\ntdcsfog_test_names: ['003f117e14.csv']\n\n"
            }
          ]
        }
      ]
    },
    {
      "id": 2231841,
      "postDate": "2023-04-23T18:17:28.177Z",
      "content": "<p>Hello, I got submission scoring error (Your notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected). </p>\n<p>I did scan through the test folder and tried to include all test files.</p>\n<p>I also print out the generated submission file as below, does it look correct (format, number of row etc)?</p>\n<p>output_combined:                        Id  StartHesitation  Turn  Walking<br>\n0            003f117e14_0              0.2   0.3     0.08<br>\n1            003f117e14_1              0.2   0.3     0.08<br>\n2            003f117e14_2              0.7   0.4     0.10<br>\n3            003f117e14_3              0.7   0.4     0.10<br>\n4            003f117e14_4              0.7   0.4     0.10<br>\n…                   …              …   …      …<br>\n281683  02ab235146_281683              0.2   0.4     0.07<br>\n281684  02ab235146_281684              0.2   0.4     0.07<br>\n281685  02ab235146_281685              0.2   0.4     0.07<br>\n281686  02ab235146_281686              0.2   0.4     0.07<br>\n281687  02ab235146_281687              0.2   0.4     0.07</p>\n<p>[286370 rows x 4 columns]</p>",
      "rawMarkdown": "Hello, I got submission scoring error (Your notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected). \n\n\nI did scan through the test folder and tried to include all test files.\n\nI also print out the generated submission file as below, does it look correct (format, number of row etc)?\n\noutput_combined:                        Id  StartHesitation  Turn  Walking\n0            003f117e14_0              0.2   0.3     0.08\n1            003f117e14_1              0.2   0.3     0.08\n2            003f117e14_2              0.7   0.4     0.10\n3            003f117e14_3              0.7   0.4     0.10\n4            003f117e14_4              0.7   0.4     0.10\n...                   ...              ...   ...      ...\n281683  02ab235146_281683              0.2   0.4     0.07\n281684  02ab235146_281684              0.2   0.4     0.07\n281685  02ab235146_281685              0.2   0.4     0.07\n281686  02ab235146_281686              0.2   0.4     0.07\n281687  02ab235146_281687              0.2   0.4     0.07\n\n[286370 rows x 4 columns]"
    }
  ],
  "comments": [
    {
      "id": 2232023,
      "author_name": "Ignacio Oguiza",
      "author_url": "",
      "post_date": "2023-04-23T22:57:19.757000",
      "content": "<p>I also struggled to make a first correct submission. <br>\nThe output should look somewhat like this:<br>\n    Id  StartHesitation Turn    Walking<br>\n0    02ab235146_0    1.419214e-12    2.032387e-08    5.861471e-18<br>\n1    02ab235146_1    2.269881e-18    2.042996e-13    1.929627e-21<br>\n2    02ab235146_2    1.221375e-17    1.342052e-10    8.594881e-19<br>\n3    02ab235146_3    1.334140e-18    4.714775e-11    9.597851e-23<br>\n4    02ab235146_4    1.841961e-18    6.536569e-12    2.878523e-23<br>\n…    … … … …<br>\n286365    003f117e14_4677 4.201266e-05    4.687012e-05    3.718816e-05<br>\n286366    003f117e14_4678 4.114008e-05    4.690264e-05    3.579163e-05<br>\n286367    003f117e14_4679 4.034276e-05    4.730092e-05    4.052546e-05<br>\n286368    003f117e14_4680 4.053275e-05    4.488072e-05    3.556220e-05<br>\n286369    003f117e14_4681 4.854446e-05    4.875891e-05    4.324796e-05</p>\n<p>The main difference is that the id contains the Id plus the row number per file (instead of id + index number of the entire array). </p>",
      "votes": 0,
      "replies": [
        {
          "id": 2232034,
          "author_name": "",
          "author_url": "",
          "post_date": "2023-04-23T23:23:07.370000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2232158,
          "author_name": "backpack",
          "author_url": "",
          "post_date": "2023-04-24T04:51:05.383000",
          "content": "<p>I made a slight change and re-run the code below. This is what I can see, it is similar to what you posted. But I still get the submission scoring error.</p>\n<p>output.to_csv('submission.csv',index=False)<br>\nprint(\"output:\",output)<br>\noutput:                        Id  StartHesitation  Turn  Walking<br>\n0            003f117e14_0              0.2   0.3     0.08<br>\n1            003f117e14_1              0.2   0.3     0.08<br>\n2            003f117e14_2              0.7   0.4     0.10<br>\n3            003f117e14_3              0.7   0.4     0.10<br>\n4            003f117e14_4              0.7   0.4     0.10<br>\n…                   …              …   …      …<br>\n286365  02ab235146_281683              0.2   0.4     0.07<br>\n286366  02ab235146_281684              0.2   0.4     0.07<br>\n286367  02ab235146_281685              0.2   0.4     0.07<br>\n286368  02ab235146_281686              0.2   0.4     0.07<br>\n286369  02ab235146_281687              0.2   0.4     0.07</p>\n<p>[286370 rows x 4 columns]</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2232197,
              "author_name": "",
              "author_url": "",
              "post_date": "2023-04-24T05:40:43.063000",
              "content": "",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2232258,
              "author_name": "backpack",
              "author_url": "",
              "post_date": "2023-04-24T06:26:41.963000",
              "content": "<p>I am confused. I downloaded the \"sample_submission\" file and check the last row id in it is \"02ab235146_281687<br>\n\",  the last row ID generated by my submission matches it. If not \"02ab235146_281687\", what is the correct id? I find there are 281688 rows in file \"02ab235146.csv\"</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2232302,
              "author_name": "Ignacio Oguiza",
              "author_url": "",
              "post_date": "2023-04-24T07:13:52.797000",
              "content": "<p>I'm sorry, you're absolutely right. I deleted my previous comment to avoid further confusion.<br>\nI believe the format of your file is correct. Are you sure you are creating predictions for all possible files in test/defog and test/tdcsfog?<br>\nI mean using something like: </p>\n<pre><code>submission = []\nfor folder in [\"defog\", \"tdcsfog\"]:       \n    for file in glob.glob(str(PATH/f\"test/{folder}/*.csv\")):\n        df = pd.read_csv(file)\n        ....\n</code></pre>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2234151,
              "author_name": "backpack",
              "author_url": "",
              "post_date": "2023-04-25T00:40:34.733000",
              "content": "<p>I use something like<br>\n\"\"\"<br>\ntdcsfog_test_root = '/kaggle/input/tlvmc-parkinsons-freezing-gait-prediction/test/tdcsfog/'<br>\ntdcsfog_test_names = []<br>\nfor dirname, _, filenames in os.walk(tdcsfog_test_root):<br>\n--    for filename in filenames:<br>\n  ----      if filename.endswith('.csv'):<br>\n        ------    tdcsfog_test_names.append(filename)<br>\n\"\"\"<br>\nprint(\"tdcsfog_test_names:\",tdcsfog_test_names)<br>\ntdcsfog_test_names: ['003f117e14.csv']</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2232023": "I also struggled to make a first correct submission. \nThe output should look somewhat like this:\n\tId\tStartHesitation\tTurn\tWalking\n0\t02ab235146_0\t1.419214e-12\t2.032387e-08\t5.861471e-18\n1\t02ab235146_1\t2.269881e-18\t2.042996e-13\t1.929627e-21\n2\t02ab235146_2\t1.221375e-17\t1.342052e-10\t8.594881e-19\n3\t02ab235146_3\t1.334140e-18\t4.714775e-11\t9.597851e-23\n4\t02ab235146_4\t1.841961e-18\t6.536569e-12\t2.878523e-23\n...\t...\t...\t...\t...\n286365\t003f117e14_4677\t4.201266e-05\t4.687012e-05\t3.718816e-05\n286366\t003f117e14_4678\t4.114008e-05\t4.690264e-05\t3.579163e-05\n286367\t003f117e14_4679\t4.034276e-05\t4.730092e-05\t4.052546e-05\n286368\t003f117e14_4680\t4.053275e-05\t4.488072e-05\t3.556220e-05\n286369\t003f117e14_4681\t4.854446e-05\t4.875891e-05\t4.324796e-05\n\nThe main difference is that the id contains the Id plus the row number per file (instead of id + index number of the entire array). ",
    "2231841": "Hello, I got submission scoring error (Your notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected). \n\n\nI did scan through the test folder and tried to include all test files.\n\nI also print out the generated submission file as below, does it look correct (format, number of row etc)?\n\noutput_combined:                        Id  StartHesitation  Turn  Walking\n0            003f117e14_0              0.2   0.3     0.08\n1            003f117e14_1              0.2   0.3     0.08\n2            003f117e14_2              0.7   0.4     0.10\n3            003f117e14_3              0.7   0.4     0.10\n4            003f117e14_4              0.7   0.4     0.10\n...                   ...              ...   ...      ...\n281683  02ab235146_281683              0.2   0.4     0.07\n281684  02ab235146_281684              0.2   0.4     0.07\n281685  02ab235146_281685              0.2   0.4     0.07\n281686  02ab235146_281686              0.2   0.4     0.07\n281687  02ab235146_281687              0.2   0.4     0.07\n\n[286370 rows x 4 columns]"
  }
}