\n","metadata":{}},{"cell_type":"markdown","source":"**```\nThis seems like an interesting competition. Let's dive in and try to understand what this competition is about.\n```**","metadata":{}},{"cell_type":"markdown","source":"**Goal of the competition**\n\n>The goal of this competition is to **generate Boolean search queries** that effectively **characterize collections of patent documents**. You are challenged to create a **query generation model** that, given an input set of related patents, outputs a Boolean query that returns the same set of patent documents.","metadata":{}},{"cell_type":"markdown","source":"**My Understanding** : We're here to build a model that would efficiently predict teh required query to find a specific set of data","metadata":{}},{"cell_type":"markdown","source":"**From Description** : \n\n> Inventions are legally protected by patents. Governments grant patents to inventors, offering exclusive rights for a defined period in exchange for public disclosure to foster innovation in various fields. But **before an inventor can obtain a patent, a patent professional must assess** whether the invention meets the necessary criteria*. **AI-powered search tools** could help patent professionals streamline these tasks.\n\n> **When using search tools, patent professionals receive certain information on documents in the result set.** This information may include text and metadata snippets (such as the classification term(s)) that played a significant role in selecting included information, as well as quantitative measures such as similarity scores. However, this provided information may not always fully explain why the specific documents in the result set were returned. **Patent professionals are most familiar with leveraging and reading Boolean search expressions to determine whether they have sufficiently searched the patent space.**\n\n> Your work will help translate result sets from AI and other search tools into the language of patent professionals. By combining the benefits of AI with the familiarity of the Boolean search system with which patent professionals are most familiar, you can help make the patent search process more efficient, effective, and explainable.","metadata":{}},{"cell_type":"markdown","source":"
\n
Looking into the data","metadata":{}},{"cell_type":"markdown","source":"**Dataset description**\n> Interpretability is a challenge for many applications of machine learning. AI tools can help patent professionals review new applications, but it's difficult to determine if a given set of results is adequate. Given the prevalence of Boolean-based search engines for patent documents, **access to Boolean search expressions that effectively characterize result sets from the AI tools would help patent professionals assess whether they have everything they need to complete their review.** Your objective in this competition is to **generate a query suitable for traditional patent search tools that returns a specific set of similar patents.**","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/uspto-explainable-ai","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:29:15.004745Z","iopub.execute_input":"2024-04-26T00:29:15.005458Z","iopub.status.idle":"2024-04-26T00:29:16.075438Z","shell.execute_reply.started":"2024-04-26T00:29:15.005425Z","shell.execute_reply":"2024-04-26T00:29:16.073782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"That seems like there's a lot of files here! Let's go through them one by one!","metadata":{}},{"cell_type":"markdown","source":"
Patent data
\n \n > Patent text from [Google Patents Public Data by IFI CLAIMS Patent Services and Google](https://console.cloud.google.com/marketplace/product/google_patents_public_datasets/google-patents-public-data?pli=1). The competition data is current through July, 2023.\n \n> This folder contains patent data in **patent_data/year_month.parquet** format ","metadata":{}},{"cell_type":"code","source":"import pandas as pd\npatent_data_file = \"/kaggle/input/uspto-explainable-ai/patent_data/1838_4.parquet\"\ndf = pd.read_parquet(patent_data_file)\ndisplay(df.head(5))\n\nprint(f\"Columns in each patent data : {df.columns}\")","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:50:11.890378Z","iopub.execute_input":"2024-04-26T00:50:11.890795Z","iopub.status.idle":"2024-04-26T00:50:12.487965Z","shell.execute_reply.started":"2024-04-26T00:50:11.890742Z","shell.execute_reply":"2024-04-26T00:50:12.486629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So there are five columns in each patent data : \n\n* **publication_number** - The patent identifier.\n* **title** - The text of the patent's title.\n* **abstract** - The text of the patent's abstract.\n* **claims** - The text of the patent's claims.\n* **description** - The text of the patent's full description.","metadata":{}},{"cell_type":"code","source":"# How many files are there?\nimport os\nfiles = os.listdir(\"/kaggle/input/uspto-explainable-ai/patent_data\")\nprint(f\"Total files : {len(files)}\")","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:29:17.181128Z","iopub.execute_input":"2024-04-26T00:29:17.181959Z","iopub.status.idle":"2024-04-26T00:29:17.284713Z","shell.execute_reply.started":"2024-04-26T00:29:17.181910Z","shell.execute_reply":"2024-04-26T00:29:17.283594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
Damn! There are data available from 1790 to 2023.
Understanding : This folder contains data for the available months from the year 1790 to 2023
\n","metadata":{}},{"cell_type":"markdown","source":"
patent_metadata.parquet
","metadata":{}},{"cell_type":"markdown","source":"Metadata for the most recent patents in each patent family.","metadata":{}},{"cell_type":"code","source":"metadata = pd.read_parquet(\"/kaggle/input/uspto-explainable-ai/patent_metadata.parquet\")\nmetadata.head()","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:54:39.881642Z","iopub.execute_input":"2024-04-26T00:54:39.882047Z","iopub.status.idle":"2024-04-26T00:55:26.208633Z","shell.execute_reply.started":"2024-04-26T00:54:39.882013Z","shell.execute_reply":"2024-04-26T00:55:26.206952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So there are five columns in each patent data : \n\n* **publication_number** - The patent identifier.\n* **publication_date** - The date the patent was published.\n* **filing_date**- The date the patent was filed.\n* **family_id** - An identifier for the patent family.\n* **cpc_codes** - A list of the [cooperative patent classifcation codes covering](https://en.wikipedia.org/wiki/Cooperative_Patent_Classification) the patent.","metadata":{}},{"cell_type":"code","source":"metadata.shape","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:31:49.761074Z","iopub.execute_input":"2024-04-26T00:31:49.762039Z","iopub.status.idle":"2024-04-26T00:31:49.768854Z","shell.execute_reply.started":"2024-04-26T00:31:49.762001Z","shell.execute_reply":"2024-04-26T00:31:49.767624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"metadata.columns","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:33:36.469048Z","iopub.execute_input":"2024-04-26T00:33:36.469777Z","iopub.status.idle":"2024-04-26T00:33:36.476138Z","shell.execute_reply.started":"2024-04-26T00:33:36.469742Z","shell.execute_reply":"2024-04-26T00:33:36.475195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for col in metadata.columns:\n try:\n print(f\"Total unique values of column {col} : f{metadata[col].nunique()}\")\n except:\n print(f\"Can't find unique values of column {col}\")\n ","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:34:48.815504Z","iopub.execute_input":"2024-04-26T00:34:48.816393Z","iopub.status.idle":"2024-04-26T00:35:03.792860Z","shell.execute_reply.started":"2024-04-26T00:34:48.816358Z","shell.execute_reply":"2024-04-26T00:35:03.791745Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
nearest_neighbors.csv
","metadata":{}},{"cell_type":"code","source":"nn = pd.read_csv(\"/kaggle/input/uspto-explainable-ai/nearest_neighbors.csv\",nrows=100)\nnn.head()","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:29:46.621185Z","iopub.execute_input":"2024-04-26T00:29:46.621470Z","iopub.status.idle":"2024-04-26T00:29:46.659125Z","shell.execute_reply.started":"2024-04-26T00:29:46.621446Z","shell.execute_reply":"2024-04-26T00:29:46.658137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* publication_number\n* neighbor_[N] - The Nth nearest neighbor of the patent, based on embeddings from the [Google Patents Research Data BigQuery dataset by IFI CLAIMS Patent Services and Google](https://console.cloud.google.com/marketplace/product/google_patents_public_datasets/google-patents-research-data). The competition data is current through July, 2023.","metadata":{}},{"cell_type":"markdown","source":"
Note : this might be the most important piece of the dataset
Contains the publication number of the 50 neighbor of the patent
","metadata":{}},{"cell_type":"markdown","source":"
Train index
","metadata":{}},{"cell_type":"markdown","source":"A [Whoosh](https://whoosh.readthedocs.io/en/latest/intro.html) text search index equivalent in size and setup to the index the metric will use to evaluate submitted queries. Only includes patents published on or after 1975. The subset of patents covered by the actual metric index will not be disclosed even to your submission notebook. The following fields are searchable:\n\n* **ti:** title.\n* **ab**: abstract.\n* **clm**: claims.\n* **detd**: description.\n* **cpc**: cpc codes.","metadata":{}},{"cell_type":"markdown","source":"**ToDO : More on this later**","metadata":{}},{"cell_type":"markdown","source":"
train_index_patent_ids.json
\n A list of the patents included in the Whoosh index","metadata":{}},{"cell_type":"code","source":"tr_idx = pd.read_json(\"/kaggle/input/uspto-explainable-ai/train_index_patent_ids.json\")\ntr_idx","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:50:18.634554Z","iopub.execute_input":"2024-04-26T00:50:18.635108Z","iopub.status.idle":"2024-04-26T00:50:18.788019Z","shell.execute_reply.started":"2024-04-26T00:50:18.635062Z","shell.execute_reply":"2024-04-26T00:50:18.786727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
test.csv
","metadata":{}},{"cell_type":"code","source":"test = pd.read_csv(\"/kaggle/input/uspto-explainable-ai/test.csv\")\ntest.head()","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:38:08.172604Z","iopub.execute_input":"2024-04-26T00:38:08.172989Z","iopub.status.idle":"2024-04-26T00:38:08.207448Z","shell.execute_reply.started":"2024-04-26T00:38:08.172958Z","shell.execute_reply":"2024-04-26T00:38:08.206480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
A subset of nearest_neighbors.csv that will cover 2,500 patents in the hidden dataset.
You'll need to predict which query you need to filter these neighbors
","metadata":{}},{"cell_type":"markdown","source":"
sample_submission.csv
","metadata":{}},{"cell_type":"markdown","source":"> A sample submission file in the correct format.","metadata":{}},{"cell_type":"code","source":"sub = pd.read_csv(\"/kaggle/input/uspto-explainable-ai/sample_submission.csv\")\nsub.head()","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:39:56.015167Z","iopub.execute_input":"2024-04-26T00:39:56.016123Z","iopub.status.idle":"2024-04-26T00:39:56.033429Z","shell.execute_reply.started":"2024-04-26T00:39:56.016086Z","shell.execute_reply":"2024-04-26T00:39:56.032472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
For each publication number, we need to submit what query was needed to filter those neighbor patents
","metadata":{}},{"cell_type":"markdown","source":"
\n
\n
Whoosh","metadata":{}},{"cell_type":"markdown","source":"One of the most important part of this competition will be to understand [whoosh](https://whoosh.readthedocs.io/en/latest/)! Let's dive a little bit into it.","metadata":{}},{"cell_type":"markdown","source":"> Whoosh is a fast, featureful full-text indexing and searching library implemented in pure Python. Programmers can use it to easily add search functionality to their applications and websites. Every part of how Whoosh works can be extended or replaced to meet your needs exactly.","metadata":{}},{"cell_type":"markdown","source":"> **Some of Whoosh’s features include**:\n\n * Pythonic API.\n\n * Pure-Python. No compilation or binary packages needed, no mysterious crashes.\n\n * Fielded indexing and search.\n\n * Fast indexing and retrieval – faster than any other pure-Python, scoring, full-text search solution I know of.\n\n * Pluggable scoring algorithm (including BM25F), text analysis, storage, posting format, etc.\n\n * Powerful query language.\n\n * Pure Python spell-checker (as far as I know, the only one).\n[Source](https://pypi.org/project/Whoosh/)\n\n\nOrganizers published [this notebook](https://www.kaggle.com/code/sohier/basic-whoosh-search-demo) to give us an idea about whoosh","metadata":{}},{"cell_type":"markdown","source":"We can use whoosh to load our train indexes","metadata":{}},{"cell_type":"code","source":"import whoosh_utils","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:48:52.144036Z","iopub.execute_input":"2024-04-26T00:48:52.145063Z","iopub.status.idle":"2024-04-26T00:49:07.202513Z","shell.execute_reply.started":"2024-04-26T00:48:52.145008Z","shell.execute_reply":"2024-04-26T00:49:07.201331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Basic setup","metadata":{}},{"cell_type":"code","source":"train_idx = whoosh_utils.load_index('/kaggle/input/uspto-explainable-ai/train_index')\nsearcher = whoosh_utils.get_searcher(train_idx)\nqp = whoosh_utils.get_query_parser()","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:51:04.935303Z","iopub.execute_input":"2024-04-26T00:51:04.935699Z","iopub.status.idle":"2024-04-26T00:52:33.167437Z","shell.execute_reply.started":"2024-04-26T00:51:04.935671Z","shell.execute_reply":"2024-04-26T00:52:33.166290Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Run a simple query**","metadata":{}},{"cell_type":"markdown","source":"If you remember, in the train index : \n* **ti:** title.\n* **ab**: abstract.\n* **clm**: claims.\n* **detd**: description.\n* **cpc**: cpc codes.","metadata":{}},{"cell_type":"code","source":"query = 'ti:balloons OR ti:string'\nresult = whoosh_utils.execute_query(query, qp, searcher)[:5]\nresult","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:54:25.918553Z","iopub.execute_input":"2024-04-26T00:54:25.919010Z","iopub.status.idle":"2024-04-26T00:54:25.932349Z","shell.execute_reply.started":"2024-04-26T00:54:25.918976Z","shell.execute_reply":"2024-04-26T00:54:25.931092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So we're here searching for indexes with title ballons or string.\nLet's look at these patent numbers","metadata":{}},{"cell_type":"code","source":"metadata[metadata['publication_number'].isin(result)]","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:58:34.486125Z","iopub.execute_input":"2024-04-26T00:58:34.486999Z","iopub.status.idle":"2024-04-26T00:58:35.724990Z","shell.execute_reply.started":"2024-04-26T00:58:34.486959Z","shell.execute_reply":"2024-04-26T00:58:35.723864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Dang! Now we have to look through the patent_data parquet files!","metadata":{}},{"cell_type":"code","source":"sample = pd.read_parquet(\"/kaggle/input/uspto-explainable-ai/patent_data/\"+\"2011_3.parquet\")\nsample[sample['publication_number'] == \"US-2011056083-A1\"]","metadata":{"execution":{"iopub.status.busy":"2024-04-26T00:58:49.206629Z","iopub.execute_input":"2024-04-26T00:58:49.207635Z","iopub.status.idle":"2024-04-26T00:58:58.960289Z","shell.execute_reply.started":"2024-04-26T00:58:49.207592Z","shell.execute_reply":"2024-04-26T00:58:58.959032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Okay!So here we've actually got a patent with title **\"Degradable string for string trimmer\"** that contains the **\"string\"** text we searched with in title!","metadata":{}},{"cell_type":"markdown","source":"
Summary : \nWhat happens when you run this code?
\n\n```\nquery = 'ti:balloons OR ti:string'\nresult = whoosh_utils.execute_query(query, qp, searcher)[:5]\nresult\n```\n
Answer : \n It finds all the patents with \"ballons\" or \"string\" in the title
\n\n**You can think it like how you find a file in your file explorer, except it can find a file from a giant storage of files with different criterias selected, it's like finding a needle in a haystack!**","metadata":{}},{"cell_type":"markdown","source":"There are some other examples in the Notebook! You can look at them until your head starts spinning :P","metadata":{}},{"cell_type":"markdown","source":"
\n
\n
Evaluation","metadata":{}},{"cell_type":"markdown","source":" Submissions are evaluated using mean average precision at 50 (mAP@50) between the patents your queries retrieve and the provided, related patent set.","metadata":{}},{"cell_type":"markdown","source":"What does it mean?\n* You'll be predicting a query. They will run that query and pulll out the matching publication numbers\n* There's a ground truth for each test id\n* Then the mAP@50 is taken\n* For more reading on understanding mAP - [resource](https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52)","metadata":{}},{"cell_type":"markdown","source":"
\n
\n
Let's make a simple submission","metadata":{}},{"cell_type":"code","source":"test = pd.read_csv(\"/kaggle/input/uspto-explainable-ai/test.csv\")\ntest.head()","metadata":{"execution":{"iopub.status.busy":"2024-04-26T02:03:46.731484Z","iopub.execute_input":"2024-04-26T02:03:46.732349Z","iopub.status.idle":"2024-04-26T02:03:46.780347Z","shell.execute_reply.started":"2024-04-26T02:03:46.732312Z","shell.execute_reply":"2024-04-26T02:03:46.779550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pd.read_csv(\"/kaggle/input/uspto-explainable-ai/sample_submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2024-04-26T02:36:50.285726Z","iopub.execute_input":"2024-04-26T02:36:50.286161Z","iopub.status.idle":"2024-04-26T02:36:50.295385Z","shell.execute_reply.started":"2024-04-26T02:36:50.286130Z","shell.execute_reply":"2024-04-26T02:36:50.293615Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub['query'] = 'ti:the AND ti:engine'","metadata":{"execution":{"iopub.status.busy":"2024-04-26T02:36:52.384875Z","iopub.execute_input":"2024-04-26T02:36:52.385243Z","iopub.status.idle":"2024-04-26T02:36:52.395784Z","shell.execute_reply.started":"2024-04-26T02:36:52.385217Z","shell.execute_reply":"2024-04-26T02:36:52.394533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub.head()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub.to_csv(\"submission.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2024-04-26T02:36:54.981629Z","iopub.execute_input":"2024-04-26T02:36:54.982599Z","iopub.status.idle":"2024-04-26T02:36:54.988825Z","shell.execute_reply.started":"2024-04-26T02:36:54.982564Z","shell.execute_reply":"2024-04-26T02:36:54.987514Z"},"trusted":true},"execution_count":null,"outputs":[]}]}