Installation

Hardware

As CLAIMS Direct Solr is a pre-configured, bundled distribution of Apache Solr, it can be deployed on any number of nodes (individual instances). A group of nodes function to expose a collection. Further, multiple collections could be searched across the distribution.


There are many scenarios for a CLAIMS Direct deployment that range from indexing the entire content of CLAIMS Direct XML to the sparse indexing of certain fields and ranges of publication dates for application-specific usage. There could also be specific QoS requirements: minimum supported queries per second, average response time et al. All of these factors play a role in planning for a CLAIMS Direct Solr deployment. Generally speaking, a full index with the entire content of CLAIMS Direct XML requires, at a minimum:

The ZooKeeper quorum could be placed together on Solr search servers or, optionally, you could break out the ZooKeeper configuration into an additional 3 separate servers.

The following diagram represents the minimum architecture required to support a full CLAIMS Direct index.

Installation

All CLAIMS Direct software is to be installed from the applicable repository.

Installing ZooKeeper

sudo yum install alexandria-solr-zookeeper

alexandria-solr-zookeeper comes pre-configured to support up to a 5-node ZooKeeper quorum. You should install the alexandria-solr-zookeeper on at least 3 nodes.

Configuring ZooKeeper

The main configuration file for ZooKeeper can be found as /opt/alexandria/zookeeper/conf/zoo.cfg. For each node, the dataDir  and server entries need to reflect the node number and quorum members respectively.

The format of the server entry is as follows:

• server.x 
x being the node number. This corresponds to a myid file located under dataDir for each node.
• IP address
  The IP address of the node
• Port 1
  Port to connect followers to the leader
• Port 2
  Port is used for leader election

The ports should not be changed.

A typical configuration for a 3-node quorum follows.

Node 1

dataDir=/var/lib/alexandria/zookeeper-data/zk1
# ...
server.1=10.234.1.91:3181:4180
server.2=10.234.1.92:3181:4180
server.3=10.234.1.93:3181:4180

Node 2

dataDir=/var/lib/alexandria/zookeeper-data/zk2
# ...
server.1=10.234.1.91:3181:4180
server.2=10.234.1.92:3181:4180
server.3=10.234.1.93:3181:4180

Node 3

dataDir=/var/lib/alexandria/zookeeper-data/zk3
# ...
server.1=10.234.1.91:3181:4180
server.2=10.234.1.92:3181:4180
server.3=10.234.1.93:3181:4180

Enable and Start ZooKeeper

The following should be done on all ZooKeeper nodes.

# enable automatic startup
sudo systemctl enable alexandria-solr-zookeeper

# start service
sudo systemctl start alexandria-solr-zookeeper

# check status
sudo systemctl status alexandria-solr-zookeeper

Installing SolrCloud

On all Solr nodes:

sudo yum install alexandria-solr-cloud

Configuring SolrCloud

The following table lists the applicable nodes and their functions in a typical 8-node environment:

Initial Template Node

On node-1, edit /opt/alexandria/solr/etc/solr-alexandria-vars. 

Enable and Start Solr

The following should be done on all Solr nodes.

# enable automatic startup
systemctl enable alexandria-solr-cloud

# start service
systemctl status alexandria-solr-cloud

# check status
systemctl status alexandria-solr-cloud

Viewing Solr and ZooKeeper Status

After starting ZooKeeper and Solr, you can visit the internal Solr web page: http://10.234.1.91:8983. Under Cloud → Nodes all available nodes should be visible as well as Cloud → ZK Status showing all ZooKeepers.

Bootstrapping the ZooKeeper Quorum

On the template node (node-1):

cd /opt/alexandria/solr/

# upload Solr configuration files
./bin/alexandria-solr-zookeeper-init
 Bootstrapping 10.234.1.91 ...
  -> name: acdidx-v3.0.0
  -> directory: /var/lib/alexandria/solr-data/conf

Creating the Solr alexandria Collection

cd /opt/alexandria/solr/

# Create the alexandria collection
./bin/alexandria-solr-cloud-admin alexandria
 
Creating (action=CREATE) on http://10.234.1.91:8983/solr/admin/collections ...
  -> name=alexandria
  -> numShards=8
  -> replicationFactor=2
  -> maxShardsPerNode=1
  -> collection.config=acdidx-v3.0.0
  -> property.config=solrconfig-alexandria.xml