Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Gliffy Diagram
namePrimary Architecture
pagePin45


Currently, the delivery of a fully populated CLAIMS Direct index requires the above SOLR and ZooKeeper configuration (8 SOLR servers + 3 ZooKeepers). Load balancers and web servers are required only if CLAIMS Direct Web Services (CDWS) will be installed as well. A customized deployment with select data to index is currently not offered fully populated. With a custom configuration, complete indexing will need to be done at the installation site.

...

NameUsed By
javaZooKeeper, SOLR and various support tools
wgetConfiguration tools (bootstrap-*.sh)
lsofStart/stop scripts (solrctl/zookeeperctl)

...


The configuration script setup.sh assumes that each node in the cluster will have the same directory structure. For example, if you download to a machine and unpack the archive into path /cdsolr, the full path to the package will be /cdsolr/alexandria-solr-v2.1.2-distribution. Each node must have the path /cdsolr available for deployment. You are free to choose any mount point or path as long as they are uniform across all nodes in the cluster and as long as the mount point or path for each SOLR node has at least 1TB of available disk space.

...

IP
Function
Description
10.234.1.91node-1: * SOLRbasic SOLR node
10.234.1.92node-2: * SOLR 
10.234.1.93node-3: * SOLR 
10.234.1.94node-4: * SOLR 
10.234.1.95node-5: * SOLR 
10.234.1.96node-6: * SOLR 
10.234.1.97node-7: * SOLR 
10.234.1.98node-8: * SOLR 

The following variables should be configured in solr-alexandria-vars:

...

IP
Function
Description
10.234.1.91node-1: * ZooKeeperbasic ZooKeeper node
10.234.1.92node-2: * ZooKeeper 
10.234.1.93node-3: * ZooKeeper 

The following variables should be configured in solr-alexandria-vars:

Variable
Value
Description
ALEXANDRIA_SOLR_ZK_NODES10.234.1.91,10.234.1.92,10.234.1.93Comma-delimited list of IP addresses
ALEXANDRIA_SOLR_ZK_HOST10.234.1.91First configured node for SolrCloud configuration bootstrap

...


Preparing Deployment

After ZooKeeper and SolrCloud node configuration is complete, the following prerequisites need to be installed and configured on each node:

PrerequisiteDescription
java version 8 (although 7 will work, this install has only been tested with 8)An up-to-date java install is required
public key ssh access for ALEXANDRIA_SOLR_CLOUD_USERIn order to facilitate configuration and index deployment as well as being able to manage start/stop scripts in one location.
create location that is read/write accessible by ALEXANDRIA_SOLR_CLOUD_USER to hold the programs, configurations, and index 
Info
titleNote

If you don't want to or can't enable public key access, the setup.sh script will error and exit. You will then need to follow the manual instructions below.

...

VariableDescription
ALEXANDRIA_SOLR_ZK_NODESComma-separated list of nodes on which ZooKeeper will run
ALEXANDRIA_SOLR_ZK_PORTZooKeeper listening port
ALEXANDRIA_SOLR_ZK_CONFIG_DIRMain configuration directory for SOLR
ALEXANDRIA_SOLR_ZK_CONFIG_NAMEConfiguration name

...


In addition, a valid solr.xml is generated from the template: solrxml.in. The following variables are used:

...

The next section in setup.sh deploys the code and configuration to all nodes. If you haven't enabled public key ssh access, the following commands need to be issued manually for each IP address in ALEXANDRIA_SOLR_CLOUD_NODES and ALEXANDRIA_SOLR_ZK_NODES:

Code Block
languagetextbash
rsync -va * $ALEXANDRIA_SOLR_USER@$IPADDRESS:$ALEXANDRIA_SOLR_HOME
# e.g.
rsync -va * asolr@10.234.1.91:/home/asolr/alexandria-solr-v2.1.2-distribution

 


Example Walk-Through

Assumptions:

8 SOLR nodesIP: 10.234.1.91-98 (as01-08)
3 ZooKeeper nodesIP 10.234.1.91-93 (shared resources with SOLR)
1 processing node10.234.1.99 (ap01)
deployment userasolr
deployment location/home/asolr

...


Create user on all machines

Code Block
languagetextbash
# as root on all machines 10.234.1.91-99
useradd -m asolr
passwd asolr # create your own password

...


Enable public key ssh access

Code Block
languagetextbash
# as user asolr on 10.234.1.99 (ap01)
cd ~/.ssh
ssh-keygen
# for each node from 10.234.1.91 -> .98
ssh-copy-id -i id_rsa.pub asolr@10.234.1.91
ssh-copy-id -i id_rsa.pub asolr@10.234.1.92
ssh-copy-id -i id_rsa.pub asolr@10.234.1.93
etc ...

 


Download package onto ap01 (10.234.1.99) and unpack into /home/asolr

...

VariableValue
ALEXANDRIA_SOLR_CLOUD_NUMSHARDS

8

ALEXANDRIA_SOLR_CLOUD_NODES

one line, no spaces:
10.234.1.91,10.234.1.92,10.234.1.93,10.234.1.94,
10.234.1.95,10.234.1.96,10.234.1.97,10.234.1.98 

ALEXANDRIA_SOLR_CLOUD_USERasolr
ALEXANDRIA_SOLR_URLhttp://10.234.1.91:$ALEXANDRIA_SOLR_PORT/$ALEXANDRIA_SOLR_CONTEXT
ALEXANDRIA_SOLR_ZK_NODES10.234.1.91,10.234.1.92,10.234.1.93
ALEXANDRIA_SOLR_ZK_HOST10.234.1.91

...


Run setup.sh

Code Block
languagetext
$ ./setup.sh 
Initializing ZooKeeper 3-node quorum configuration ...
  * -> zoo-1.cfg
  * -> zoo-2.cfg
  * -> zoo-3.cfg
Initializing solr.xml ... done
Testing ssh access to solr:
  * -> node: asolr@10.234.1.91 ... ok
  * -> node: asolr@10.234.1.92 ... ok
  * -> node: asolr@10.234.1.93 ... ok
  * -> node: asolr@10.234.1.94 ... ok
  * -> node: asolr@10.234.1.95 ... ok
  * -> node: asolr@10.234.1.96 ... ok
  * -> node: asolr@10.234.1.97 ... ok
  * -> node: asolr@10.234.1.98 ... ok
Testing ssh access to zookeeper:
  * -> node: asolr@10.234.1.91 ... ok
  * -> node: asolr@10.234.1.92 ... ok
  * -> node: asolr@10.234.1.93 ... ok
Deploying SOLR ...
  * -> asolr@10.234.1.91:/home/asolr ... ok
  * -> asolr@10.234.1.92:/home/asolr ... ok
  * -> asolr@10.234.1.93:/home/asolr ... ok
  * -> asolr@10.234.1.94:/home/asolr ... ok
  * -> asolr@10.234.1.95:/home/asolr ... ok
  * -> asolr@10.234.1.96:/home/asolr ... ok
  * -> asolr@10.234.1.97:/home/asolr ... ok
  * -> asolr@10.234.1.98:/home/asolr ... ok
Deploying ZooKeeper ...
  * -> asolr@10.234.1.91:/home/asolr ... ok
  * -> asolr@10.234.1.92:/home/asolr ... ok
  * -> asolr@10.234.1.93:/home/asolr ... ok

 


Starting and Bootstrapping the ZooKeeper Quorum

Code Block
languagetextbash
./zookeeperctl start
Starting zookeeper ... STARTED
Starting zookeeper ... STARTED
Starting zookeeper ... STARTED


# check the quorum is running:
./zookeeperctl status
Mode: follower
Mode: follower
Mode: leader


# install SOLR configuration files
./bootstrap-zookeeper.sh 
Bootstrapping 10.234.1.91 ...
  -> name: cdidx-v2.1.2
  -> directory: /home/asolr/alexandria-solr-v2.1.2-distribution/conf-2.1.2

Starting and Bootstrapping the SOLR Collection

Code Block
languagetextbash
 $ ./solrctl start
Initializing solr.xml ... done
asolr@10.234.1.91: SOLR ( start ) ...Waiting up to 30 seconds to see Solr running on port 8080 [/]  
Started Solr server on port 8080 (pid=10520). Happy searching!
### etc ...
 
# check cluster status
./solrctl status
asolr@10.234.1.91: SOLR ( status ) ...
Found 1 Solr nodes: 
Solr process 11273 running on port 8080
{
  "solr_home":"/home/asolr/alexandria-solr-v2.1.2distribution",
  "version":"5.3.1 1703449 - noble - 2015-09-17 01:48:15",
  "startTime":"2016-02-23T08:18:36.059Z",
  "uptime":"0 days, 0 hours, 1 minutes, 9 seconds",
  "memory":"63.6 MB (%6.5) of 981.4 MB",
  "cloud":{
    "ZooKeeper":"10.234.1.91:2181,10.234.1.92:2181,10.234.1.93:2181",
    "liveNodes":"8",
    "collections":"0"}}
### etc.

# create the collection
./bootstrap-alexandria.sh
 
Creating (action=CREATE) on http://10.234.1.91:8080/alexandria-v2.1/admin/collections ...
  -> name=alexandria
  -> numShards=8
  -> replicationFactor=1
  -> maxShardsPerNode=1
  -> collection.config=cdidx-v2.1.2
  -> property.config=solrconfig-alexandria.xml
  HTTP/1.1 200 OK
  Content-Type: application/xml; charset=UTF-8
  Transfer-Encoding: chunked
<?xml version="1.0" encoding="UTF-8"?>
<response>
 <lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">3136</int>
 </lst>
  # etc ..
</response> 

...


SOLR Interface

At this point, the ZooKeeper quorum and all SOLR nodes should be running. You can now visit http://10.234.1.91:8080/alexandria-v2.1/old.html

Stopping SOLR Cluster and ZooKeeper Quorum

Code Block
languagetextbash
./solrctl stop
asolr@10.234.1.91: SOLR ( stop ) ...
# etc.
 
./zookeeperctl stop 

...