{
  "id": 431531,
  "title": "submission format error question",
  "url": "/competitions/asl-fingerspelling/discussion/431531",
  "author_name": "",
  "post_date": "2023-08-14T00:43:06.957449500Z",
  "votes": 1,
  "comment_count": 7,
  "views": 0,
  "content": "<p>I am currently experiencing a scoring error when trying to use a model.tflite and inference_args.json that I load from my local machine that I then use in a notebook. Was wondering if I misunderstood how much actually needs to be in the notebook aside from the saving of the submission.zip (?)</p>\n<p>Here is the code:</p>\n<pre><code>\n! submission.  \n</code></pre>\n<p>Error appears as:</p>\n<blockquote>\n  <p>Submission Scoring Error<br>\n  Your notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips</p>\n</blockquote>\n<p>Here is what I changed my code to and now I have a 'Notebook Running' that has been doing so for a while</p>\n<pre><code> shutil\n\nshutil.copy(,)\nshutil.copy(,)\n\n! submission.   \n</code></pre>\n<p>any advice would help, thanks in advance.</p>",
  "messages": [
    {
      "id": "2389300",
      "postDate": "08/14/2023 00:43:06",
      "content": "<p>I am currently experiencing a scoring error when trying to use a model.tflite and inference_args.json that I load from my local machine that I then use in a notebook. Was wondering if I misunderstood how much actually needs to be in the notebook aside from the saving of the submission.zip (?)</p>\n<p>Here is the code:</p>\n<pre><code>\n! submission.  \n</code></pre>\n<p>Error appears as:</p>\n<blockquote>\n  <p>Submission Scoring Error<br>\n  Your notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips</p>\n</blockquote>\n<p>Here is what I changed my code to and now I have a 'Notebook Running' that has been doing so for a while</p>\n<pre><code> shutil\n\nshutil.copy(,)\nshutil.copy(,)\n\n! submission.   \n</code></pre>\n<p>any advice would help, thanks in advance.</p>",
      "rawMarkdown": "I am currently experiencing a scoring error when trying to use a model.tflite and inference_args.json that I load from my local machine that I then use in a notebook. Was wondering if I misunderstood how much actually needs to be in the notebook aside from the saving of the submission.zip (?)\n\nHere is the code:\n```python\n# save sample submission zip to working and we should be set\n!zip submission.zip '/kaggle/input/sample-local-submission/output/tflite_models/tutorial_model.tflite' '/kaggle/input/sample-local-submission/output/inference_args/tutorial.json'\n```\n\nError appears as:\n>Submission Scoring Error\nYour notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips\n\nHere is what I changed my code to and now I have a 'Notebook Running' that has been doing so for a while\n```python\nimport shutil\n\nshutil.copy(\"/kaggle/input/sample-local-submission/output/tflite_models/tutorial_model.tflite\",\"./model.tflite\")\nshutil.copy(\"/kaggle/input/sample-local-submission/output/inference_args/tutorial.json\",\"./inference_args.json\")\n\n!zip submission.zip  './model.tflite' './inference_args.json'\n```\n\nany advice would help, thanks in advance.",
      "votes": null
    },
    {
      "id": "2389305",
      "postDate": "08/14/2023 00:54:44",
      "content": "<p>For your non-notebook code, the error happens because of the file name.  Your notebook code should be fine.</p>",
      "rawMarkdown": "For your non-notebook code, the error happens because of the file name.  Your notebook code should be fine.",
      "votes": null
    },
    {
      "id": "2389307",
      "postDate": "08/14/2023 01:05:54",
      "content": "<p>I'm getting the same Submission Scoring Error as you. I just now discovered I'm returning 0 rows, shape (0,59), in rare occurrences.  I'm  just now refreshing another model to see if this fixes it, I plan on returning a tensor with one character, shape (1,59) when this occurs.  Fingers crossed!!!  I'll let you know if it fixes it in an hour.  The things that I've done so far to avoid scoring errors are:</p>\n<ol>\n<li>putting this at the beginning of your preprocessing to remove blank inputs<br>\n x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)<br>\n x= x[None]</li>\n<li>making sure that you're returning a float data type and not an integer.</li>\n<li>making sure you're returning the shape of (x, 59).  So removing the pad, sos, and eos tokens before returning.</li>\n<li>Stopped using tf.lite.OpsSet.SELECT_TF_OPS or any other additional supported ops when converting the tflite model, use only tflite runtime.  This meant that i had to bring in the multiheadattention source code and stop using this particular function, tf.linalg.band_part, within it, which was being used to create the causal attention mask.</li>\n</ol>",
      "rawMarkdown": "I'm getting the same Submission Scoring Error as you. I just now discovered I'm returning 0 rows, shape (0,59), in rare occurrences.  I'm  just now refreshing another model to see if this fixes it, I plan on returning a tensor with one character, shape (1,59) when this occurs.  Fingers crossed!!!  I'll let you know if it fixes it in an hour.  The things that I've done so far to avoid scoring errors are:\n1. putting this at the beginning of your preprocessing to remove blank inputs\n     x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)\n     x= x[None]\n2. making sure that you're returning a float data type and not an integer.\n3. making sure you're returning the shape of (x, 59).  So removing the pad, sos, and eos tokens before returning.\n4. Stopped using tf.lite.OpsSet.SELECT_TF_OPS or any other additional supported ops when converting the tflite model, use only tflite runtime.  This meant that i had to bring in the multiheadattention source code and stop using this particular function, tf.linalg.band_part, within it, which was being used to create the causal attention mask.",
      "votes": null
    },
    {
      "id": "2389397",
      "postDate": "08/14/2023 03:18:46",
      "content": "<p>You were totally correct, it just finished scoring after 3hrs</p>",
      "rawMarkdown": "You were totally correct, it just finished scoring after 3hrs",
      "votes": null
    },
    {
      "id": "2389399",
      "postDate": "08/14/2023 03:19:37",
      "content": "<p>My issue was related to the naming of the files so my updated code worked. Best of luck with your situation and thanks for the 4 additional points to reference in case I hit this err again</p>",
      "rawMarkdown": "My issue was related to the naming of the files so my updated code worked. Best of luck with your situation and thanks for the 4 additional points to reference in case I hit this err again",
      "votes": null
    },
    {
      "id": "2389449",
      "postDate": "08/14/2023 04:20:01",
      "content": "<p>Dude!  I didn't realize I had to have the right file name for my model!  I followed your example above to rename my model.  I'd never gotten past 2 mins of scoring and now i got to 7 mins. So that's something!!! Thanks for asking the question.</p>",
      "rawMarkdown": "Dude!  I didn't realize I had to have the right file name for my model!  I followed your example above to rename my model.  I'd never gotten past 2 mins of scoring and now i got to 7 mins. So that's something!!! Thanks for asking the question.",
      "votes": null
    },
    {
      "id": "2389451",
      "postDate": "08/14/2023 04:20:50",
      "content": "<p>Thank you!  This allowed me to get past initialization in the scoring!  Now i need to figure out why it's breaking after 7 mins.</p>",
      "rawMarkdown": "Thank you!  This allowed me to get past initialization in the scoring!  Now i need to figure out why it's breaking after 7 mins.",
      "votes": null
    },
    {
      "id": "2404639",
      "postDate": "08/23/2023 12:01:34",
      "content": "<p>hi, I'm wondering if returning (0, 59) is valid or not?</p>",
      "rawMarkdown": "hi, I'm wondering if returning (0, 59) is valid or not?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2389305,
      "author_name": "baohaoliao",
      "author_url": "",
      "post_date": "08/14/2023 00:54:44",
      "content": "<p>For your non-notebook code, the error happens because of the file name.  Your notebook code should be fine.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2389397,
          "author_name": "clarksaben",
          "author_url": "",
          "post_date": "08/14/2023 03:18:46",
          "content": "<p>You were totally correct, it just finished scoring after 3hrs</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2389451,
          "author_name": "andyatkinson",
          "author_url": "",
          "post_date": "08/14/2023 04:20:50",
          "content": "<p>Thank you!  This allowed me to get past initialization in the scoring!  Now i need to figure out why it's breaking after 7 mins.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2389307,
      "author_name": "andyatkinson",
      "author_url": "",
      "post_date": "08/14/2023 01:05:54",
      "content": "<p>I'm getting the same Submission Scoring Error as you. I just now discovered I'm returning 0 rows, shape (0,59), in rare occurrences.  I'm  just now refreshing another model to see if this fixes it, I plan on returning a tensor with one character, shape (1,59) when this occurs.  Fingers crossed!!!  I'll let you know if it fixes it in an hour.  The things that I've done so far to avoid scoring errors are:</p>\n<ol>\n<li>putting this at the beginning of your preprocessing to remove blank inputs<br>\n x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)<br>\n x= x[None]</li>\n<li>making sure that you're returning a float data type and not an integer.</li>\n<li>making sure you're returning the shape of (x, 59).  So removing the pad, sos, and eos tokens before returning.</li>\n<li>Stopped using tf.lite.OpsSet.SELECT_TF_OPS or any other additional supported ops when converting the tflite model, use only tflite runtime.  This meant that i had to bring in the multiheadattention source code and stop using this particular function, tf.linalg.band_part, within it, which was being used to create the causal attention mask.</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 2389399,
          "author_name": "clarksaben",
          "author_url": "",
          "post_date": "08/14/2023 03:19:37",
          "content": "<p>My issue was related to the naming of the files so my updated code worked. Best of luck with your situation and thanks for the 4 additional points to reference in case I hit this err again</p>",
          "votes": null,
          "replies": [
            {
              "id": 2389449,
              "author_name": "andyatkinson",
              "author_url": "",
              "post_date": "08/14/2023 04:20:01",
              "content": "<p>Dude!  I didn't realize I had to have the right file name for my model!  I followed your example above to rename my model.  I'd never gotten past 2 mins of scoring and now i got to 7 mins. So that's something!!! Thanks for asking the question.</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 2404639,
          "author_name": "dangnh0611",
          "author_url": "",
          "post_date": "08/23/2023 12:01:34",
          "content": "<p>hi, I'm wondering if returning (0, 59) is valid or not?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2389300": "I am currently experiencing a scoring error when trying to use a model.tflite and inference_args.json that I load from my local machine that I then use in a notebook. Was wondering if I misunderstood how much actually needs to be in the notebook aside from the saving of the submission.zip (?)\n\nHere is the code:\n```python\n# save sample submission zip to working and we should be set\n!zip submission.zip '/kaggle/input/sample-local-submission/output/tflite_models/tutorial_model.tflite' '/kaggle/input/sample-local-submission/output/inference_args/tutorial.json'\n```\n\nError appears as:\n>Submission Scoring Error\nYour notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips\n\nHere is what I changed my code to and now I have a 'Notebook Running' that has been doing so for a while\n```python\nimport shutil\n\nshutil.copy(\"/kaggle/input/sample-local-submission/output/tflite_models/tutorial_model.tflite\",\"./model.tflite\")\nshutil.copy(\"/kaggle/input/sample-local-submission/output/inference_args/tutorial.json\",\"./inference_args.json\")\n\n!zip submission.zip  './model.tflite' './inference_args.json'\n```\n\nany advice would help, thanks in advance.",
    "2389305": "For your non-notebook code, the error happens because of the file name.  Your notebook code should be fine.",
    "2389307": "I'm getting the same Submission Scoring Error as you. I just now discovered I'm returning 0 rows, shape (0,59), in rare occurrences.  I'm  just now refreshing another model to see if this fixes it, I plan on returning a tensor with one character, shape (1,59) when this occurs.  Fingers crossed!!!  I'll let you know if it fixes it in an hour.  The things that I've done so far to avoid scoring errors are:\n1. putting this at the beginning of your preprocessing to remove blank inputs\n     x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)\n     x= x[None]\n2. making sure that you're returning a float data type and not an integer.\n3. making sure you're returning the shape of (x, 59).  So removing the pad, sos, and eos tokens before returning.\n4. Stopped using tf.lite.OpsSet.SELECT_TF_OPS or any other additional supported ops when converting the tflite model, use only tflite runtime.  This meant that i had to bring in the multiheadattention source code and stop using this particular function, tf.linalg.band_part, within it, which was being used to create the causal attention mask.",
    "2389397": "You were totally correct, it just finished scoring after 3hrs",
    "2389399": "My issue was related to the naming of the files so my updated code worked. Best of luck with your situation and thanks for the 4 additional points to reference in case I hit this err again",
    "2389449": "Dude!  I didn't realize I had to have the right file name for my model!  I followed your example above to rename my model.  I'd never gotten past 2 mins of scoring and now i got to 7 mins. So that's something!!! Thanks for asking the question.",
    "2389451": "Thank you!  This allowed me to get past initialization in the scoring!  Now i need to figure out why it's breaking after 7 mins.",
    "2404639": "hi, I'm wondering if returning (0, 59) is valid or not?"
  },
  "source": "meta"
}