{
  "id": 607805,
  "title": "What are the tools allowed",
  "url": "/competitions/acm-icaif-25-ai-agentic-retrieval-grand-challenge/discussion/607805",
  "author_name": "",
  "post_date": "2025-09-16T08:31:17.223025400Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I am wondering what the restrictions are on the type of tools we can use. There are 5 types of tools</p>\n<ol>\n<li>Open-sourced LLM deployed on a local machine</li>\n<li>Open-sourced LLM deployed on  LLM through API</li>\n<li>Commercial LLM from API</li>\n<li>Open-sourced tools such as sentence transformer, spaCy, NLTK, etc</li>\n<li>Commercially available tools such as text encoder eg: text-embedding-3-large</li>\n</ol>",
  "messages": [
    {
      "id": "3289563",
      "postDate": "09/16/2025 08:31:17",
      "content": "<p>I am wondering what the restrictions are on the type of tools we can use. There are 5 types of tools</p>\n<ol>\n<li>Open-sourced LLM deployed on a local machine</li>\n<li>Open-sourced LLM deployed on  LLM through API</li>\n<li>Commercial LLM from API</li>\n<li>Open-sourced tools such as sentence transformer, spaCy, NLTK, etc</li>\n<li>Commercially available tools such as text encoder eg: text-embedding-3-large</li>\n</ol>",
      "rawMarkdown": "I am wondering what the restrictions are on the type of tools we can use. There are 5 types of tools\n1. Open-sourced LLM deployed on a local machine\n2. Open-sourced LLM deployed on  LLM through API\n3.  Commercial LLM from API\n4. Open-sourced tools such as sentence transformer, spaCy, NLTK, etc\n5. Commercially available tools such as text encoder eg: text-embedding-3-large",
      "votes": null
    },
    {
      "id": "3290024",
      "postDate": "09/17/2025 01:48:10",
      "content": "<p>We recommend using an open-source LLM to ensure reproducibility and to better manage costs.<br>\nThe Databricks free trial provides a $400 credit, which is sufficient to participate in this challenge.</p>\n<p>While we cannot prohibit the use of commercial LLMs, if their use hinders the verification of reproducibility, it may adversely impact the final ranking.</p>",
      "rawMarkdown": "We recommend using an open-source LLM to ensure reproducibility and to better manage costs.\nThe Databricks free trial provides a $400 credit, which is sufficient to participate in this challenge.\n\nWhile we cannot prohibit the use of commercial LLMs, if their use hinders the verification of reproducibility, it may adversely impact the final ranking.",
      "votes": null
    },
    {
      "id": "3290034",
      "postDate": "09/17/2025 02:24:48",
      "content": "<p>What is the process to verify the reproducibility?<br>\nIn your example, you used gpt-4o-mini. Does it mean OpenAI models are fine in the reproducibility verification?</p>",
      "rawMarkdown": "What is the process to verify the reproducibility?\nIn your example, you used gpt-4o-mini. Does it mean OpenAI models are fine in the reproducibility verification?",
      "votes": null
    },
    {
      "id": "3291080",
      "postDate": "09/19/2025 01:00:16",
      "content": "<p>In our baseline, <code>gpt-4o-mini</code> is used only to convert the open-source LLM’s output into a structured format.<br>\nIt does not significantly affect the results.<br>\nHowever, reliance on commercial LLMs will be considered when judging eligibility for an oral presentation at ICAIF 2025.</p>",
      "rawMarkdown": "In our baseline, `gpt-4o-mini` is used only to convert the open-source LLM’s output into a structured format.\nIt does not significantly affect the results.\nHowever, reliance on commercial LLMs will be considered when judging eligibility for an oral presentation at ICAIF 2025.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3290024,
      "author_name": "jihoonkwon",
      "author_url": "",
      "post_date": "09/17/2025 01:48:10",
      "content": "<p>We recommend using an open-source LLM to ensure reproducibility and to better manage costs.<br>\nThe Databricks free trial provides a $400 credit, which is sufficient to participate in this challenge.</p>\n<p>While we cannot prohibit the use of commercial LLMs, if their use hinders the verification of reproducibility, it may adversely impact the final ranking.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3290034,
          "author_name": "yimingqian1",
          "author_url": "",
          "post_date": "09/17/2025 02:24:48",
          "content": "<p>What is the process to verify the reproducibility?<br>\nIn your example, you used gpt-4o-mini. Does it mean OpenAI models are fine in the reproducibility verification?</p>",
          "votes": null,
          "replies": [
            {
              "id": 3291080,
              "author_name": "jihoonkwon",
              "author_url": "",
              "post_date": "09/19/2025 01:00:16",
              "content": "<p>In our baseline, <code>gpt-4o-mini</code> is used only to convert the open-source LLM’s output into a structured format.<br>\nIt does not significantly affect the results.<br>\nHowever, reliance on commercial LLMs will be considered when judging eligibility for an oral presentation at ICAIF 2025.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3289563": "I am wondering what the restrictions are on the type of tools we can use. There are 5 types of tools\n1. Open-sourced LLM deployed on a local machine\n2. Open-sourced LLM deployed on  LLM through API\n3.  Commercial LLM from API\n4. Open-sourced tools such as sentence transformer, spaCy, NLTK, etc\n5. Commercially available tools such as text encoder eg: text-embedding-3-large",
    "3290024": "We recommend using an open-source LLM to ensure reproducibility and to better manage costs.\nThe Databricks free trial provides a $400 credit, which is sufficient to participate in this challenge.\n\nWhile we cannot prohibit the use of commercial LLMs, if their use hinders the verification of reproducibility, it may adversely impact the final ranking.",
    "3290034": "What is the process to verify the reproducibility?\nIn your example, you used gpt-4o-mini. Does it mean OpenAI models are fine in the reproducibility verification?",
    "3291080": "In our baseline, `gpt-4o-mini` is used only to convert the open-source LLM’s output into a structured format.\nIt does not significantly affect the results.\nHowever, reliance on commercial LLMs will be considered when judging eligibility for an oral presentation at ICAIF 2025."
  },
  "source": "meta"
}