...

The CLAIMS Global Patent Database data is delivered as a separate record for each patent publication, with data merged from multiple sources. The sources include the following:

• DocDB DOCDB and legal status from the EPO
• Bibliographic and full text files from national patent offices
• Assignment information for US Translated bibliographic data for Japan from Patent Abstracts of Japan
• Data from national registers for AU and EP
• Extensions and associated drug names for AU and US

New raw data is loaded as soon as published and includes both new records as well as changes.

The loading of PDFs and other attachments generally occurs after the XML as they are delivered to us separately. In order to make the XML available as quickly as possible, we do not constrain delivery of the text until the images arrive. Attachments and images are usually available within 24 hours of the XML.

Because of the unpredictable, large amount of updated records delivered by DocDB DOCDB (which can cause delays in processing national office data), we are more selective in processing DocDB DOCDB data by effectively spreading the weekly loads over a full week instead of by processing the data when we it is received the data. Doing so allows us to make sure that the US, EP, and PCT files (in particular) files covering newly published documents are processed in a timely manner.

The data we receive from patent offices may differ from what is available through their portals as they are separate products with different maintenance procedures and schedules. As a result, it may be helpful to consult the /legalstatus service if a more current source is needed.

In the following sections, we include information about data volumes and frequency of updates for the main raw data sources.

Weekly

...

Updates

...

Data Volume and Type Per Delivery

The following table shows the average volume of records processed in each data delivery from the main raw data sources. It also indicates some of the elements that are commonly processed.

A: all first publications contain this element (excludes corrections, search reports, WO equivalents/republications, etc.)

S: some publications contain this element, data is loaded when available

A - all first publications contain this element (excludes corrections, search reports, WO equivalents/republications, etc.)

S - some publications contain this element, data is loaded when available

...

DOCDB Amended Records

DocDB DOCDB Amended Records (percentage of documents having updated elements for the given fields)

Non-Weekly Updates

Architecture

Two important scripts are used to manage the ongoing data update process.

• apgupd: The main update daemon along with the sub-script apgup is used to check for and download new or updated data from the CLAIMS Direct primary data warehouse.
• aidxd: This, along with the sub-script aidx (alexandria_index), is responsible for keeping the optional SOLR index updated with data synced from the individual client PostgreSQL database.

Update Process In Detail

The main components involved in content updates for Claims Direct client instances consist of the client PostgreSQL database proper, the remote web client, and the server-side web service end points (CDWS), as shown in the diagram below:

...

Content is processed on the IFI CLAIMS primary instance based on the concept of load source. These load sources are particular data feeds from issuing authorities. Load sources can include new, complete documents, updated complete documents or partial document updates. As these load sources are processed into the primary data warehouse, they are stamped with a load-id (an identifier used to group sets of documents together) and are immediately made available for client download.

...

All meta data pertaining to updates to both PostgreSQL and SOLR are contained in the PostgreSQL schema reporting. For further details, see design documentation.

PostgreSQL Updates

apgup (alexandria_update) is the mechanism through which client instances communicate with the IFI primary instance and is the only method of obtaining new or updated content for the client PostgreSQL database. You will need authentication credentials from IFI CLAIMS in order to communicate with the IFI primary server.

Detailed Usage (apgup)

apgup [ Options ... ]

  --help           print this usage and exit
  --update         update local database
    --continue     continue processing available load-ids
    --interval=i   number of seconds to sleep before continuing (default=10)
    --die_on_error Break out of continue loop if any error conditions arise
                     (default=false, i.e., continue trying to update)
  --check          check for available updates but don't proceed to actually update
    --limit=n      limit display of available updates to n (default=10)
  --stop           trigger a gracefull exit while in continuous mode
  --maxloadid=i    force a specific max-client-load-id value
  --noindex        don't trigger index of new load-id(s)

  Required Authorization Arguments
  --------
  --user=s         basic authorization name
  --password=s     basic authorization password

  Optional Update Arguments
  --------
  --url            Alexandria services url
  --update_method  method name for requesting server updates
  --status_method  method name for getting/setting session status
  --check_method   method name for checking available updates
  --schema         schema name for temporary work tables
  --force          force the update even if it is redundant
  --tmp            temporary directory for update processing
  --facility=s     logging facility (default=apgup)

  Optional Database Argument
  --------
  --pgdbname=conf  default database connection name (alexandria)

SOLR Indexing

Indexing into SOLR is controlled by an indexing daemon: aidxd. This daemon probes PostgreSQL for available load-id(s) to index. This "queue" is represented by the table reporting.t_client_index_process. When processing is successfully completed into PostgreSQL, apgup registers a new, index-ready load-id. The indexing daemon recognizes this as an available load-id and begins the indexing process for that particular load-id.

Detailed Usage (aidxd)

...