{
  "id": 498023,
  "title": "Metric patch complete",
  "url": "/competitions/uspto-explainable-ai/discussion/498023",
  "author_name": "",
  "post_date": "2024-04-26T16:09:07.422548900Z",
  "votes": 15,
  "comment_count": 2,
  "views": 0,
  "content": "<p><a href=\"https://www.kaggle.com/competitions/uspto-explainable-ai/discussion/497907#2777320\" target=\"_blank\">As noted here</a>, the metric wasn't working as intended. The root cause turned out to be that at some point prior to launch the metric notebook lost access to the <code>whoosh_utils</code> utility script. I'm not sure exactly how that happened; most likely a process error on my part, but I've resolved that issue. </p>\n<p>New test submissions have successfully finished scoring.</p>\n<p>Thanks to everyone who reported the issue and provided clean demonstration cases, and apologies to everyone who lost time dealing with this.</p>",
  "messages": [
    {
      "id": "2777364",
      "postDate": "04/26/2024 16:09:07",
      "content": "<p><a href=\"https://www.kaggle.com/competitions/uspto-explainable-ai/discussion/497907#2777320\" target=\"_blank\">As noted here</a>, the metric wasn't working as intended. The root cause turned out to be that at some point prior to launch the metric notebook lost access to the <code>whoosh_utils</code> utility script. I'm not sure exactly how that happened; most likely a process error on my part, but I've resolved that issue. </p>\n<p>New test submissions have successfully finished scoring.</p>\n<p>Thanks to everyone who reported the issue and provided clean demonstration cases, and apologies to everyone who lost time dealing with this.</p>",
      "rawMarkdown": "[As noted here](https://www.kaggle.com/competitions/uspto-explainable-ai/discussion/497907#2777320), the metric wasn't working as intended. The root cause turned out to be that at some point prior to launch the metric notebook lost access to the `whoosh_utils` utility script. I'm not sure exactly how that happened; most likely a process error on my part, but I've resolved that issue. \n\nNew test submissions have successfully finished scoring.\n\nThanks to everyone who reported the issue and provided clean demonstration cases, and apologies to everyone who lost time dealing with this.",
      "votes": null
    },
    {
      "id": "2778971",
      "postDate": "04/27/2024 13:23:18",
      "content": "<p>we all been there, we all done that. Thanks for fixing!</p>",
      "rawMarkdown": "we all been there, we all done that. Thanks for fixing!",
      "votes": null
    },
    {
      "id": "2780008",
      "postDate": "04/28/2024 01:21:08",
      "content": "<p><a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> what is the best way to check the query time and choice of the kaggle environment to verify =&gt; now i am facing score exception, so need to replicate submission environment.</p>\n<blockquote>\n  <p><strong>The metric notebook must finish running your queries in 60 minutes</strong>, not including the time required for loading the whoosh index. Note that the metric uses four Whoosh searchers in parallel. The time required to start the metric notebook and download the data it uses does not count towards the 60 minutes.</p>\n</blockquote>\n<hr>\n<blockquote>\n  <p>train_index A Whoosh text search index equivalent in size and setup to the index the metric will use to evaluate submitted queries. Only includes patents published on or after 1975. The subset of patents covered by the actual metric index will not be disclosed even to your submission notebook. </p>\n</blockquote>\n<hr>\n<p>is the  train_index_patent_ids.json =&gt; # of patents in the hidden set are close to 7k or train_index in hidden size close to 6GB? </p>",
      "rawMarkdown": "sohier what is the best way to check the query time and choice of the kaggle environment to verify => now i am facing score exception, so need to replicate submission environment.\n\n> **The metric notebook must finish running your queries in 60 minutes**, not including the time required for loading the whoosh index. Note that the metric uses four Whoosh searchers in parallel. The time required to start the metric notebook and download the data it uses does not count towards the 60 minutes.\n\n---\n\n> train_index A Whoosh text search index equivalent in size and setup to the index the metric will use to evaluate submitted queries. Only includes patents published on or after 1975. The subset of patents covered by the actual metric index will not be disclosed even to your submission notebook. \n\n---\n\nis the  train_index_patent_ids.json => # of patents in the hidden set are close to 7k or train_index in hidden size close to 6GB?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2778971,
      "author_name": "valentinwerner",
      "author_url": "",
      "post_date": "04/27/2024 13:23:18",
      "content": "<p>we all been there, we all done that. Thanks for fixing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2780008,
      "author_name": "seshurajup",
      "author_url": "",
      "post_date": "04/28/2024 01:21:08",
      "content": "<p><a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> what is the best way to check the query time and choice of the kaggle environment to verify =&gt; now i am facing score exception, so need to replicate submission environment.</p>\n<blockquote>\n  <p><strong>The metric notebook must finish running your queries in 60 minutes</strong>, not including the time required for loading the whoosh index. Note that the metric uses four Whoosh searchers in parallel. The time required to start the metric notebook and download the data it uses does not count towards the 60 minutes.</p>\n</blockquote>\n<hr>\n<blockquote>\n  <p>train_index A Whoosh text search index equivalent in size and setup to the index the metric will use to evaluate submitted queries. Only includes patents published on or after 1975. The subset of patents covered by the actual metric index will not be disclosed even to your submission notebook. </p>\n</blockquote>\n<hr>\n<p>is the  train_index_patent_ids.json =&gt; # of patents in the hidden set are close to 7k or train_index in hidden size close to 6GB? </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2777364": "[As noted here](https://www.kaggle.com/competitions/uspto-explainable-ai/discussion/497907#2777320), the metric wasn't working as intended. The root cause turned out to be that at some point prior to launch the metric notebook lost access to the `whoosh_utils` utility script. I'm not sure exactly how that happened; most likely a process error on my part, but I've resolved that issue. \n\nNew test submissions have successfully finished scoring.\n\nThanks to everyone who reported the issue and provided clean demonstration cases, and apologies to everyone who lost time dealing with this.",
    "2778971": "we all been there, we all done that. Thanks for fixing!",
    "2780008": "sohier what is the best way to check the query time and choice of the kaggle environment to verify => now i am facing score exception, so need to replicate submission environment.\n\n> **The metric notebook must finish running your queries in 60 minutes**, not including the time required for loading the whoosh index. Note that the metric uses four Whoosh searchers in parallel. The time required to start the metric notebook and download the data it uses does not count towards the 60 minutes.\n\n---\n\n> train_index A Whoosh text search index equivalent in size and setup to the index the metric will use to evaluate submitted queries. Only includes patents published on or after 1975. The subset of patents covered by the actual metric index will not be disclosed even to your submission notebook. \n\n---\n\nis the  train_index_patent_ids.json => # of patents in the hidden set are close to 7k or train_index in hidden size close to 6GB?"
  },
  "source": "meta"
}