{
  "id": 393242,
  "title": "Model constraints check",
  "url": "/competitions/asl-signs/discussion/393242",
  "author_name": "",
  "post_date": "2023-03-08T15:31:21.977900100Z",
  "votes": 1,
  "comment_count": 6,
  "views": 0,
  "content": "<blockquote>\n  <p>Your model must also perform inference with less than 100 milliseconds of latency per video on average and use less than 40 MB of storage space. Expect to see approximately 40,000 videos in the test set. We allow an additional 10 minute buffer for loading the data and miscellaneous overhead.</p>\n</blockquote>\n<p>Hello everyone. How do I check that my solution and model is ok with these conditions? </p>",
  "messages": [
    {
      "id": "2173711",
      "postDate": "03/08/2023 15:31:21",
      "content": "<blockquote>\n  <p>Your model must also perform inference with less than 100 milliseconds of latency per video on average and use less than 40 MB of storage space. Expect to see approximately 40,000 videos in the test set. We allow an additional 10 minute buffer for loading the data and miscellaneous overhead.</p>\n</blockquote>\n<p>Hello everyone. How do I check that my solution and model is ok with these conditions? </p>",
      "rawMarkdown": ">Your model must also perform inference with less than 100 milliseconds of latency per video on average and use less than 40 MB of storage space. Expect to see approximately 40,000 videos in the test set. We allow an additional 10 minute buffer for loading the data and miscellaneous overhead.\n\nHello everyone. How do I check that my solution and model is ok with these conditions?",
      "votes": null
    },
    {
      "id": "2173805",
      "postDate": "03/08/2023 16:47:29",
      "content": "<p>A Kaggle staff member said earlier this week that they hope to have an evaluation script out by the end of the week that we can use.  However, in the mean time…</p>\n<ul>\n<li>If your submissions are getting close to 75mins to score then you're getting close to the latency limit.</li>\n<li>Unzipped size of the TFLite model is a pretty good proxy for the size constraint.</li>\n</ul>",
      "rawMarkdown": "A Kaggle staff member said earlier this week that they hope to have an evaluation script out by the end of the week that we can use.  However, in the mean time...\n\n- If your submissions are getting close to 75mins to score then you're getting close to the latency limit.\n- Unzipped size of the TFLite model is a pretty good proxy for the size constraint.",
      "votes": null
    },
    {
      "id": "2174343",
      "postDate": "03/09/2023 04:40:59",
      "content": "<p><a href=\"https://www.tensorflow.org/lite/performance/measurement\" target=\"_blank\">https://www.tensorflow.org/lite/performance/measurement</a><br>\n<a href=\"https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#parameters\" target=\"_blank\">https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#parameters</a></p>\n<p><br>\n</p>\n<p>memory is now \"irrelevant\" as requirement has been updated to use less than \"40 MB of storage\" </p>",
      "rawMarkdown": "https://www.tensorflow.org/lite/performance/measurement\nhttps://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#parameters\n\n~~tflite benchmark tool works the best for me.~~\n~~it measures peak and overall memory~~\n\nmemory is now \"irrelevant\" as requirement has been updated to use less than \"40 MB of storage\"",
      "votes": null
    },
    {
      "id": "2174344",
      "postDate": "03/09/2023 04:42:54",
      "content": "<p>\"Unzipped size of the TFLite model is a pretty good proxy for the size constraint.\"</p>\n<p></p>\n<p>memory is now \"irrelevant\" as requirement has been updated to  use less than \"40 MB of storage\" </p>",
      "rawMarkdown": "\"Unzipped size of the TFLite model is a pretty good proxy for the size constraint.\"\n\n~~my concern is variable size input. e.g. processing a 500 length video will not consume the same memory as  10 length video~~\n\nmemory is now \"irrelevant\" as requirement has been updated to  use less than \"40 MB of storage\"",
      "votes": null
    },
    {
      "id": "2174923",
      "postDate": "03/09/2023 14:13:05",
      "content": "<p>Sorry, still do not really get it, how should I check storage size. Could you explain please?</p>",
      "rawMarkdown": "Sorry, still do not really get it, how should I check storage size. Could you explain please?",
      "votes": null
    },
    {
      "id": "2175315",
      "postDate": "03/09/2023 19:10:03",
      "content": "<p>You can use command line tools like <code>du</code> or Python's <code>os.path.getsize</code>.</p>",
      "rawMarkdown": "You can use command line tools like `du` or Python's `os.path.getsize`.",
      "votes": null
    },
    {
      "id": "2175351",
      "postDate": "03/09/2023 19:45:28",
      "content": "<p>Do I need to get a size of the model? Also, what's about the solution speed, how should I check it?</p>",
      "rawMarkdown": "Do I need to get a size of the model? Also, what's about the solution speed, how should I check it?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2173805,
      "author_name": "andrewrrose",
      "author_url": "",
      "post_date": "03/08/2023 16:47:29",
      "content": "<p>A Kaggle staff member said earlier this week that they hope to have an evaluation script out by the end of the week that we can use.  However, in the mean time…</p>\n<ul>\n<li>If your submissions are getting close to 75mins to score then you're getting close to the latency limit.</li>\n<li>Unzipped size of the TFLite model is a pretty good proxy for the size constraint.</li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 2174344,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "03/09/2023 04:42:54",
          "content": "<p>\"Unzipped size of the TFLite model is a pretty good proxy for the size constraint.\"</p>\n<p></p>\n<p>memory is now \"irrelevant\" as requirement has been updated to  use less than \"40 MB of storage\" </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2174343,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/09/2023 04:40:59",
      "content": "<p><a href=\"https://www.tensorflow.org/lite/performance/measurement\" target=\"_blank\">https://www.tensorflow.org/lite/performance/measurement</a><br>\n<a href=\"https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#parameters\" target=\"_blank\">https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#parameters</a></p>\n<p><br>\n</p>\n<p>memory is now \"irrelevant\" as requirement has been updated to use less than \"40 MB of storage\" </p>",
      "votes": null,
      "replies": [
        {
          "id": 2174923,
          "author_name": "manwithaflower",
          "author_url": "",
          "post_date": "03/09/2023 14:13:05",
          "content": "<p>Sorry, still do not really get it, how should I check storage size. Could you explain please?</p>",
          "votes": null,
          "replies": [
            {
              "id": 2175315,
              "author_name": "sohier",
              "author_url": "",
              "post_date": "03/09/2023 19:10:03",
              "content": "<p>You can use command line tools like <code>du</code> or Python's <code>os.path.getsize</code>.</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2175351,
                  "author_name": "manwithaflower",
                  "author_url": "",
                  "post_date": "03/09/2023 19:45:28",
                  "content": "<p>Do I need to get a size of the model? Also, what's about the solution speed, how should I check it?</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2173711": ">Your model must also perform inference with less than 100 milliseconds of latency per video on average and use less than 40 MB of storage space. Expect to see approximately 40,000 videos in the test set. We allow an additional 10 minute buffer for loading the data and miscellaneous overhead.\n\nHello everyone. How do I check that my solution and model is ok with these conditions?",
    "2173805": "A Kaggle staff member said earlier this week that they hope to have an evaluation script out by the end of the week that we can use.  However, in the mean time...\n\n- If your submissions are getting close to 75mins to score then you're getting close to the latency limit.\n- Unzipped size of the TFLite model is a pretty good proxy for the size constraint.",
    "2174343": "https://www.tensorflow.org/lite/performance/measurement\nhttps://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#parameters\n\n~~tflite benchmark tool works the best for me.~~\n~~it measures peak and overall memory~~\n\nmemory is now \"irrelevant\" as requirement has been updated to use less than \"40 MB of storage\"",
    "2174344": "\"Unzipped size of the TFLite model is a pretty good proxy for the size constraint.\"\n\n~~my concern is variable size input. e.g. processing a 500 length video will not consume the same memory as  10 length video~~\n\nmemory is now \"irrelevant\" as requirement has been updated to  use less than \"40 MB of storage\"",
    "2174923": "Sorry, still do not really get it, how should I check storage size. Could you explain please?",
    "2175315": "You can use command line tools like `du` or Python's `os.path.getsize`.",
    "2175351": "Do I need to get a size of the model? Also, what's about the solution speed, how should I check it?"
  },
  "source": "meta"
}