Hi Kagglers, This is my first competition,and as a Kaggle newbie, I feel like a super nut. I'd appreciate your guidance.
\nBelow are the statements from Overview and Data.
\n\n\ntest.csv A subset of nearest_neighbors.csv that will cover 2,500 patents in the hidden dataset.
\nThis leaderboard is calculated with approximately 28% of the test data. The final results will be based on the other 72%, so the final standings may be different.
\n
1) If my interpretation is correct, 28%x2,500 patents is being run for current leaderboard public score, and 72%x2,500 patents will be run for leaderboard private score, which is the final eligible score for this competition. Can anyone confirm?
\n2) If that's true, when I submit my notebook now, it should be scored within (28/72)x9hours ( NOT (28/100)x9hours) in order that the same notebook should be run within 9 hours for the final private score. Can anyone confirm?
\n\n\nCPU Notebook <= 9 hours run-time
\n
\n GPU Notebook <= 9 hours run-timeThe metric notebook must finish running your queries in 60 minutes, not including the time required for loading the whoosh index. Note that the metric uses four Whoosh searchers in parallel. The time required to start the metric notebook and download the data it uses does not count towards the 60 minutes.
\n
3) Does this 60 minutes limit start counting from the point that the whoosh index has been loaded?
", "messages": [ { "id": "2900413", "postDate": "07/02/2024 09:03:41", "content": "Hi Kagglers, This is my first competition,and as a Kaggle newbie, I feel like a super nut. I'd appreciate your guidance.
\nBelow are the statements from Overview and Data.
\n\n\ntest.csv A subset of nearest_neighbors.csv that will cover 2,500 patents in the hidden dataset.
\nThis leaderboard is calculated with approximately 28% of the test data. The final results will be based on the other 72%, so the final standings may be different.
\n
1) If my interpretation is correct, 28%x2,500 patents is being run for current leaderboard public score, and 72%x2,500 patents will be run for leaderboard private score, which is the final eligible score for this competition. Can anyone confirm?
\n2) If that's true, when I submit my notebook now, it should be scored within (28/72)x9hours ( NOT (28/100)x9hours) in order that the same notebook should be run within 9 hours for the final private score. Can anyone confirm?
\n\n\nCPU Notebook <= 9 hours run-time
\n
\n GPU Notebook <= 9 hours run-timeThe metric notebook must finish running your queries in 60 minutes, not including the time required for loading the whoosh index. Note that the metric uses four Whoosh searchers in parallel. The time required to start the metric notebook and download the data it uses does not count towards the 60 minutes.
\n
3) Does this 60 minutes limit start counting from the point that the whoosh index has been loaded?
", "rawMarkdown": "Hi Kagglers, This is my first competition,and as a Kaggle newbie, I feel like a super nut. I'd appreciate your guidance.\n\nBelow are the statements from Overview and Data.\n\n\n>test.csv A subset of nearest_neighbors.csv that will cover 2,500 patents in the hidden dataset.\n\n>This leaderboard is calculated with approximately 28% of the test data. The final results will be based on **the other 72%**, so the final standings may be different.\n\n\n1) If my interpretation is correct, 28%x2,500 patents is being run for current leaderboard public score, and 72%x2,500 patents will be run for leaderboard private score, which is the final eligible score for this competition. Can anyone confirm?\n\n2) If that's true, when I submit my notebook now, it should be scored within (28/72)x9hours ( NOT (28/100)x9hours) in order that the same notebook should be run within 9 hours for the final private score. Can anyone confirm?\n\n\n>CPU Notebook <= 9 hours run-time\n>GPU Notebook <= 9 hours run-time\n\n>The metric notebook must finish running your queries in 60 minutes, not including the time required for loading the whoosh index. Note that the metric uses four Whoosh searchers in parallel. The time required to start the metric notebook and download the data it uses does not count towards the 60 minutes.\n\n\n3) Does this 60 minutes limit start counting from the point that the whoosh index has been loaded?", "votes": null }, { "id": "2903954", "postDate": "07/04/2024 05:38:36", "content": "I didn't quite understand the 2nd question.
\nYes, the test.csv and sample_submission.csv files contain 2500 patents for which you need to compose 2500 of your requests in submission.csv.
\nYou have <=9 hours to compile submission.csv for 2500 patents. After that, within 1 hour, a search for neighbouring patents will be performed.
\nThe public leaderboard shows the result of 28% (2500 * 0.28) test data, when the competition closes you will be shown 72% (2500 * 0.72) result. (notebooks will not be restarted).
\nhttps://www.kaggle.com/competitions/uspto-explainable-ai/discussion/497547#2775330
", "rawMarkdown": "I didn't quite understand the 2nd question.\n## 1. **How many patents are in the submission?**\nYes, the test.csv and sample_submission.csv files contain 2500 patents for which you need to compose 2500 of your requests in submission.csv.\n## 2. **Operating time of the notebook**\nYou have <=9 hours to compile submission.csv for 2500 patents. After that, within 1 hour, a search for neighbouring patents will be performed.\n## 3. **Notebook evaluation**\nThe public leaderboard shows the result of 28% (2500 * 0.28) test data, when the competition closes you will be shown 72% (2500 * 0.72) result. (notebooks will not be restarted).\n## 4. **Link to previous discussion**\nhttps://www.kaggle.com/competitions/uspto-explainable-ai/discussion/497547#2775330", "votes": null }, { "id": "2904061", "postDate": "07/04/2024 06:53:35", "content": "@qurusx Many thanks! So far, I have submitted the submission.csv containing only the patents listed in the test.csv along with their queries, because I thought that test.csv in my Notebook will be replaced with the set of 2,500 patents owned by the host during the public scoring.
\nI have one follow-up question: if each participant chooses their own 2,500 patents, and the final score is also based on the set chosen by each participant, is it guaranteed that the scores can be compared fairly? (Or choosing proper 2,500 patents from nearest_neighbors.csv is part of this competition's requirement?) I thought there were 2,500 patents prepared by the host, and the public score was being calculated based on 28% of them…
\n\n", "rawMarkdown": "qurusx Many thanks! So far, I have submitted the submission.csv containing only the patents listed in the test.csv along with their queries, because I thought that test.csv in my Notebook will be replaced with the set of 2,500 patents owned by the host during the public scoring.\n\nI have one follow-up question: if each participant chooses their own 2,500 patents, and the final score is also based on the set chosen by each participant, is it guaranteed that the scores can be compared fairly? (Or choosing proper 2,500 patents from nearest_neighbors.csv is part of this competition's requirement?) I thought there were 2,500 patents prepared by the host, and the public score was being calculated based on 28% of them...\n\n>Submission File\n>For each publication_number in the test set, you must generate a boolean query that yields all 50 of the target patent IDs specified in **test.csv**. Your submission file must include a header and have the following format:", "votes": null }, { "id": "2905669", "postDate": "07/05/2024 05:29:51", "content": "Submission File
\n
\n For each publication_number in the test set, you must generate a boolean query that yields all 50 of the target patent IDs specified in test.csv. Your submission file must include a header and have the following format:
All data /kaggle/input/uspto-explainable-ai for all participants is the same, unchangeable and set at the beginning of the competition. test.csv['publication_number'] = sample_submission.csv['publication_number'].
\nIn test.csv, the neighbours are the 50 neighbours for each patent to be found using your queries.
\nJust at submission time, the 10 patents in these csv files are replaced by 2500 unknown patents, which your notebook processes again.
@gowillgo @qurusx Thank you very much for your questions and answers. Can you confirm whether our notebook will run again after the competition is over? I'm a little worried about that
", "rawMarkdown": "gowillgo @qurusx Thank you very much for your questions and answers. Can you confirm whether our notebook will run again after the competition is over? I'm a little worried about that", "votes": null }, { "id": "2929610", "postDate": "07/20/2024 08:09:03", "content": "@alannikos Considering the runtime on submission, I figure they run with 100% of the dataset but only show the result with 28% of the dataset. So, I don't think the Notebook will be run again after the deadline. Hope this helps.
", "rawMarkdown": "alannikos Considering the runtime on submission, I figure they run with 100% of the dataset but only show the result with 28% of the dataset. So, I don't think the Notebook will be run again after the deadline. Hope this helps.", "votes": null } ], "comments": [ { "id": 2903954, "author_name": "qurusx", "author_url": "", "post_date": "07/04/2024 05:38:36", "content": "I didn't quite understand the 2nd question.
\nYes, the test.csv and sample_submission.csv files contain 2500 patents for which you need to compose 2500 of your requests in submission.csv.
\nYou have <=9 hours to compile submission.csv for 2500 patents. After that, within 1 hour, a search for neighbouring patents will be performed.
\nThe public leaderboard shows the result of 28% (2500 * 0.28) test data, when the competition closes you will be shown 72% (2500 * 0.72) result. (notebooks will not be restarted).
\nhttps://www.kaggle.com/competitions/uspto-explainable-ai/discussion/497547#2775330
", "votes": null, "replies": [ { "id": 2904061, "author_name": "gowillgo", "author_url": "", "post_date": "07/04/2024 06:53:35", "content": "@qurusx Many thanks! So far, I have submitted the submission.csv containing only the patents listed in the test.csv along with their queries, because I thought that test.csv in my Notebook will be replaced with the set of 2,500 patents owned by the host during the public scoring.
\nI have one follow-up question: if each participant chooses their own 2,500 patents, and the final score is also based on the set chosen by each participant, is it guaranteed that the scores can be compared fairly? (Or choosing proper 2,500 patents from nearest_neighbors.csv is part of this competition's requirement?) I thought there were 2,500 patents prepared by the host, and the public score was being calculated based on 28% of them…
\n\n", "votes": null, "replies": [ { "id": 2905669, "author_name": "qurusx", "author_url": "", "post_date": "07/05/2024 05:29:51", "content": "Submission File
\n
\n For each publication_number in the test set, you must generate a boolean query that yields all 50 of the target patent IDs specified in test.csv. Your submission file must include a header and have the following format:
All data /kaggle/input/uspto-explainable-ai for all participants is the same, unchangeable and set at the beginning of the competition. test.csv['publication_number'] = sample_submission.csv['publication_number'].
\nIn test.csv, the neighbours are the 50 neighbours for each patent to be found using your queries.
\nJust at submission time, the 10 patents in these csv files are replaced by 2500 unknown patents, which your notebook processes again.
@gowillgo @qurusx Thank you very much for your questions and answers. Can you confirm whether our notebook will run again after the competition is over? I'm a little worried about that
", "votes": null, "replies": [ { "id": 2929610, "author_name": "gowillgo", "author_url": "", "post_date": "07/20/2024 08:09:03", "content": "@alannikos Considering the runtime on submission, I figure they run with 100% of the dataset but only show the result with 28% of the dataset. So, I don't think the Notebook will be run again after the deadline. Hope this helps.
", "votes": null, "replies": [] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2900413": "Hi Kagglers, This is my first competition,and as a Kaggle newbie, I feel like a super nut. I'd appreciate your guidance.\n\nBelow are the statements from Overview and Data.\n\n\n>test.csv A subset of nearest_neighbors.csv that will cover 2,500 patents in the hidden dataset.\n\n>This leaderboard is calculated with approximately 28% of the test data. The final results will be based on **the other 72%**, so the final standings may be different.\n\n\n1) If my interpretation is correct, 28%x2,500 patents is being run for current leaderboard public score, and 72%x2,500 patents will be run for leaderboard private score, which is the final eligible score for this competition. Can anyone confirm?\n\n2) If that's true, when I submit my notebook now, it should be scored within (28/72)x9hours ( NOT (28/100)x9hours) in order that the same notebook should be run within 9 hours for the final private score. Can anyone confirm?\n\n\n>CPU Notebook <= 9 hours run-time\n>GPU Notebook <= 9 hours run-time\n\n>The metric notebook must finish running your queries in 60 minutes, not including the time required for loading the whoosh index. Note that the metric uses four Whoosh searchers in parallel. The time required to start the metric notebook and download the data it uses does not count towards the 60 minutes.\n\n\n3) Does this 60 minutes limit start counting from the point that the whoosh index has been loaded?", "2903954": "I didn't quite understand the 2nd question.\n## 1. **How many patents are in the submission?**\nYes, the test.csv and sample_submission.csv files contain 2500 patents for which you need to compose 2500 of your requests in submission.csv.\n## 2. **Operating time of the notebook**\nYou have <=9 hours to compile submission.csv for 2500 patents. After that, within 1 hour, a search for neighbouring patents will be performed.\n## 3. **Notebook evaluation**\nThe public leaderboard shows the result of 28% (2500 * 0.28) test data, when the competition closes you will be shown 72% (2500 * 0.72) result. (notebooks will not be restarted).\n## 4. **Link to previous discussion**\nhttps://www.kaggle.com/competitions/uspto-explainable-ai/discussion/497547#2775330", "2904061": "qurusx Many thanks! So far, I have submitted the submission.csv containing only the patents listed in the test.csv along with their queries, because I thought that test.csv in my Notebook will be replaced with the set of 2,500 patents owned by the host during the public scoring.\n\nI have one follow-up question: if each participant chooses their own 2,500 patents, and the final score is also based on the set chosen by each participant, is it guaranteed that the scores can be compared fairly? (Or choosing proper 2,500 patents from nearest_neighbors.csv is part of this competition's requirement?) I thought there were 2,500 patents prepared by the host, and the public score was being calculated based on 28% of them...\n\n>Submission File\n>For each publication_number in the test set, you must generate a boolean query that yields all 50 of the target patent IDs specified in **test.csv**. Your submission file must include a header and have the following format:", "2905669": "All data ` /kaggle/input/uspto-explainable-ai` for all participants is the same, unchangeable and set at the beginning of the competition. test.csv['publication_number'] = sample_submission.csv['publication_number'].\nIn test.csv, the neighbours are the 50 neighbours for each patent to be found using your queries.\nJust at submission time, the 10 patents in these csv files are replaced by 2500 unknown patents, which your notebook processes again.", "2929247": "gowillgo @qurusx Thank you very much for your questions and answers. Can you confirm whether our notebook will run again after the competition is over? I'm a little worried about that", "2929610": "alannikos Considering the runtime on submission, I figure they run with 100% of the dataset but only show the result with 28% of the dataset. So, I don't think the Notebook will be run again after the deadline. Hope this helps." }, "source": "meta" }