{
  "id": 412008,
  "title": "How long should scoring take?",
  "url": "/competitions/birdclef-2023/discussion/412008",
  "author_name": "",
  "post_date": "2023-05-21T23:31:12.224837700Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I finally got stuff working well enough to make a submission.</p>\n<p>My notebook finished running in about a minute (too slow - need to work on that…) - but I've been waiting for the \"BirdCLEF 2023 scoring…\" task for 45 minutes now.</p>\n<p>Is this normal?  </p>\n<p>Thanks for any thoughts!</p>",
  "messages": [
    {
      "id": "2268694",
      "postDate": "05/21/2023 23:31:12",
      "content": "<p>I finally got stuff working well enough to make a submission.</p>\n<p>My notebook finished running in about a minute (too slow - need to work on that…) - but I've been waiting for the \"BirdCLEF 2023 scoring…\" task for 45 minutes now.</p>\n<p>Is this normal?  </p>\n<p>Thanks for any thoughts!</p>",
      "rawMarkdown": "I finally got stuff working well enough to make a submission.\n\nMy notebook finished running in about a minute (too slow - need to work on that...) - but I've been waiting for the \"BirdCLEF 2023 scoring...\" task for 45 minutes now.\n\nIs this normal?  \n\nThanks for any thoughts!",
      "votes": null
    },
    {
      "id": "2268734",
      "postDate": "05/22/2023 00:49:36",
      "content": "<p>It appears my notebook ran successfully - but after about 90 minutes of \"scoring\" my entry failed with:</p>\n<p>\"Your notebook requested more memory (RAM) than is available. \"</p>\n<p>Anyone else having trouble submitting?</p>",
      "rawMarkdown": "It appears my notebook ran successfully - but after about 90 minutes of \"scoring\" my entry failed with:\n\n \"Your notebook requested more memory (RAM) than is available. \"\n\nAnyone else having trouble submitting?",
      "votes": null
    },
    {
      "id": "2268762",
      "postDate": "05/22/2023 01:38:29",
      "content": "<p>Sounds like you have a memory \"leak\" - perhaps you're keeping references to the sound file data, so it accumulates as you go along? One simple way of checking is to run your notebook against 200 copies of the single test soundscape we're supplied and see what happens with memory usage as this runs.</p>\n<p>Submission scoring time is a major part of the challenge. I think most single models take roughly an hour to run, but that obviously depends on the complexity of the model.</p>",
      "rawMarkdown": "Sounds like you have a memory \"leak\" - perhaps you're keeping references to the sound file data, so it accumulates as you go along? One simple way of checking is to run your notebook against 200 copies of the single test soundscape we're supplied and see what happens with memory usage as this runs.\n\nSubmission scoring time is a major part of the challenge. I think most single models take roughly an hour to run, but that obviously depends on the complexity of the model.",
      "votes": null
    },
    {
      "id": "2268775",
      "postDate": "05/22/2023 02:00:43",
      "content": "<p>OK - I can see I misunderstood things a little.  I was under the incorrect impression that the initial submissions were just against the public soundscape (and thought that the 2nd task that got spun-up was just scoring the CSV).  </p>\n<p>Feel silly now….I've got some testing to do….</p>\n<p>Thank you for your help on this!</p>",
      "rawMarkdown": "OK - I can see I misunderstood things a little.  I was under the incorrect impression that the initial submissions were just against the public soundscape (and thought that the 2nd task that got spun-up was just scoring the CSV).  \n\nFeel silly now....I've got some testing to do....\n\nThank you for your help on this!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2268734,
      "author_name": "richolson",
      "author_url": "",
      "post_date": "05/22/2023 00:49:36",
      "content": "<p>It appears my notebook ran successfully - but after about 90 minutes of \"scoring\" my entry failed with:</p>\n<p>\"Your notebook requested more memory (RAM) than is available. \"</p>\n<p>Anyone else having trouble submitting?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2268762,
          "author_name": "dsosnoski",
          "author_url": "",
          "post_date": "05/22/2023 01:38:29",
          "content": "<p>Sounds like you have a memory \"leak\" - perhaps you're keeping references to the sound file data, so it accumulates as you go along? One simple way of checking is to run your notebook against 200 copies of the single test soundscape we're supplied and see what happens with memory usage as this runs.</p>\n<p>Submission scoring time is a major part of the challenge. I think most single models take roughly an hour to run, but that obviously depends on the complexity of the model.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2268775,
              "author_name": "richolson",
              "author_url": "",
              "post_date": "05/22/2023 02:00:43",
              "content": "<p>OK - I can see I misunderstood things a little.  I was under the incorrect impression that the initial submissions were just against the public soundscape (and thought that the 2nd task that got spun-up was just scoring the CSV).  </p>\n<p>Feel silly now….I've got some testing to do….</p>\n<p>Thank you for your help on this!</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2268694": "I finally got stuff working well enough to make a submission.\n\nMy notebook finished running in about a minute (too slow - need to work on that...) - but I've been waiting for the \"BirdCLEF 2023 scoring...\" task for 45 minutes now.\n\nIs this normal?  \n\nThanks for any thoughts!",
    "2268734": "It appears my notebook ran successfully - but after about 90 minutes of \"scoring\" my entry failed with:\n\n \"Your notebook requested more memory (RAM) than is available. \"\n\nAnyone else having trouble submitting?",
    "2268762": "Sounds like you have a memory \"leak\" - perhaps you're keeping references to the sound file data, so it accumulates as you go along? One simple way of checking is to run your notebook against 200 copies of the single test soundscape we're supplied and see what happens with memory usage as this runs.\n\nSubmission scoring time is a major part of the challenge. I think most single models take roughly an hour to run, but that obviously depends on the complexity of the model.",
    "2268775": "OK - I can see I misunderstood things a little.  I was under the incorrect impression that the initial submissions were just against the public soundscape (and thought that the 2nd task that got spun-up was just scoring the CSV).  \n\nFeel silly now....I've got some testing to do....\n\nThank you for your help on this!"
  },
  "source": "meta"
}