{
  "id": 507567,
  "title": "Why did my baseline sub score zero?",
  "url": "/competitions/uspto-explainable-ai/discussion/507567",
  "author_name": "Khoi Nguyen",
  "post_date": "2024-05-26T11:23:22.191000",
  "votes": 6,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Since the bug seems to be mostly solved I decided to give this another try, however when I try to make a few simple submissions they all scored zero. Here's what I did:</p>\n<ul>\n<li>Take the title of the <code>target_0</code> patient, remove all the stop words.</li>\n<li>Make the query: <code>ti(word1 AND word2 ...)</code> for all remaining words in the title that is not a stop word.   </li>\n</ul>\n<p>I got about 0.04 in my local validation which makes sense since that query should get all patents with title similar to <code>target_0</code> and maybe another one or two if I'm lucky. However in the public test it scored zero. Did I miss something about the metric or did the test index intentionally constructed to be consist of patents with very similar titles?</p>",
  "messages": [
    {
      "id": 2837221,
      "postDate": "2024-05-26T11:23:22.193Z",
      "content": "<p>Since the bug seems to be mostly solved I decided to give this another try, however when I try to make a few simple submissions they all scored zero. Here's what I did:</p>\n<ul>\n<li>Take the title of the <code>target_0</code> patient, remove all the stop words.</li>\n<li>Make the query: <code>ti(word1 AND word2 ...)</code> for all remaining words in the title that is not a stop word.   </li>\n</ul>\n<p>I got about 0.04 in my local validation which makes sense since that query should get all patents with title similar to <code>target_0</code> and maybe another one or two if I'm lucky. However in the public test it scored zero. Did I miss something about the metric or did the test index intentionally constructed to be consist of patents with very similar titles?</p>",
      "rawMarkdown": "Since the bug seems to be mostly solved I decided to give this another try, however when I try to make a few simple submissions they all scored zero. Here's what I did:\n- Take the title of the `target_0` patient, remove all the stop words.\n- Make the query: `ti(word1 AND word2 ...)` for all remaining words in the title that is not a stop word.   \n\nI got about 0.04 in my local validation which makes sense since that query should get all patents with title similar to `target_0` and maybe another one or two if I'm lucky. However in the public test it scored zero. Did I miss something about the metric or did the test index intentionally constructed to be consist of patents with very similar titles?",
      "votes": 5
    },
    {
      "id": 2838764,
      "postDate": "2024-05-27T07:47:06.437Z",
      "content": "<p>You seem to miss<code>:</code></p>",
      "rawMarkdown": "You seem to miss` :`",
      "votes": 2,
      "replies": [
        {
          "id": 2839183,
          "postDate": "2024-05-27T12:18:27.550Z",
          "content": "<p>It was just a typo, I didn't miss it in the sub and still got zero 🤔</p>",
          "rawMarkdown": "It was just a typo, I didn't miss it in the sub and still got zero 🤔",
          "replies": [
            {
              "id": 2839231,
              "postDate": "2024-05-27T12:47:00.207Z",
              "content": "<p>Then your local validation is probably optimistic.</p>",
              "rawMarkdown": "Then your local validation is probably optimistic."
            },
            {
              "id": 2853053,
              "postDate": "2024-06-03T15:17:32.930Z",
              "content": "<p>Good catch.</p>",
              "rawMarkdown": "Good catch."
            }
          ]
        }
      ]
    },
    {
      "id": 2899944,
      "postDate": "2024-07-02T00:41:43.920Z",
      "content": "<p>I have also encountered this problem, I run the official baseline of 0.17, but my Leaderboard score is also 0, may I ask you to find the reason?@Ahmet Erdem</p>",
      "rawMarkdown": "I have also encountered this problem, I run the official baseline of 0.17, but my Leaderboard score is also 0, may I ask you to find the reason?@Ahmet Erdem"
    },
    {
      "id": 2858343,
      "postDate": "2024-06-06T12:37:59.247Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2838764,
      "author_name": "Ahmet Erdem",
      "author_url": "",
      "post_date": "2024-05-27T07:47:06.437000",
      "content": "<p>You seem to miss<code>:</code></p>",
      "votes": 2,
      "replies": [
        {
          "id": 2839183,
          "author_name": "Khoi Nguyen",
          "author_url": "",
          "post_date": "2024-05-27T12:18:27.550000",
          "content": "<p>It was just a typo, I didn't miss it in the sub and still got zero 🤔</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2839231,
              "author_name": "Ahmet Erdem",
              "author_url": "",
              "post_date": "2024-05-27T12:47:00.207000",
              "content": "<p>Then your local validation is probably optimistic.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2853053,
              "author_name": "Austin T O'Donoghue",
              "author_url": "",
              "post_date": "2024-06-03T15:17:32.930000",
              "content": "<p>Good catch.</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2899944,
      "author_name": "Alannikos",
      "author_url": "",
      "post_date": "2024-07-02T00:41:43.920000",
      "content": "<p>I have also encountered this problem, I run the official baseline of 0.17, but my Leaderboard score is also 0, may I ask you to find the reason?@Ahmet Erdem</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2858343,
      "author_name": "",
      "author_url": "",
      "post_date": "2024-06-06T12:37:59.247000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2837221": "Since the bug seems to be mostly solved I decided to give this another try, however when I try to make a few simple submissions they all scored zero. Here's what I did:\n- Take the title of the `target_0` patient, remove all the stop words.\n- Make the query: `ti(word1 AND word2 ...)` for all remaining words in the title that is not a stop word.   \n\nI got about 0.04 in my local validation which makes sense since that query should get all patents with title similar to `target_0` and maybe another one or two if I'm lucky. However in the public test it scored zero. Did I miss something about the metric or did the test index intentionally constructed to be consist of patents with very similar titles?",
    "2838764": "You seem to miss` :`",
    "2899944": "I have also encountered this problem, I run the official baseline of 0.17, but my Leaderboard score is also 0, may I ask you to find the reason?@Ahmet Erdem",
    "2858343": ""
  }
}