High Availability of IBM Security Directory Server Using Heartbeat v1

High Availability of IBM Security Directory
Server using Heartbeat

A highly available authentication system
Prabir Meher
IBM India Software Lab, Pune
2014/02/20, 1.0
Abstract:
The purpose of this article is to demonstrate the use of ISDS (IBM Security Directory Server), formerly known
as ITDS (IBM Tivoli Directory Server), using Heartbeat for creating a highly available authentication system
with fail-over mechanism. High availability is very critical for enterprise authentication services since
consolidating any service on a particular server is not at all reliable. Depending on a single server eventually
creates a "single point of failure" (SPOF), which can break the entire organization's authentication system.
You will see here one method of creating a reliable authentication server using IBM Security Directory Server,
which can be adapted by many different applications. We will use the Heartbeat package from the Linux HA
project (see the "Resources" section) to design a highly available authentication system using IBM Security
Directory Server.
Table of Contents
1Introduction .............................................................................................................................................4
2Environment setup...................................................................................................................................5
3About IBM Security Directory Server (ISDS).........................................................................................5
4About Heartbeat.......................................................................................................................................5
5Prerequisite...............................................................................................................................................6
6IBM Security Directory Server replication..............................................................................................7
6.1Log in to Web Administration Tool..................................................................................................8
6.2WAT introduction page.....................................................................................................................8
6.3Adding servers to WAT....................................................................................................................9
6.4Log in to the ISDS instance to configure replication.....................................................................11
6.5Setting up the replication................................................................................................................11
6.5.1Check replication-related entries in ISDS using command line.............................................23
6.6Synchronize data between two replication servers.........................................................................23
6.6.1Exporting data.........................................................................................................................24
6.6.2Exporting data using salt and seed of the destination server..................................................24
How to obtain salt and seed of an ISDS server..........................................................................25
6.6.3Importing data.........................................................................................................................25
6.7Resume replication queue..............................................................................................................26
7Heartbeat setup.......................................................................................................................................28
7.1Installing Heartbeat........................................................................................................................28
7.2Configuring Heartbeat....................................................................................................................28
7.2.1The authkeys file.....................................................................................................................28
7.2.2The ha.cf file...........................................................................................................................29
7.2.3The haresources file................................................................................................................30
7.3Propagating configuration files to peer servers..............................................................................30
7.4Starting Heartbeat...........................................................................................................................31
7.4.1Network configuration before starting heartbeat....................................................................31
7.4.2Network configuration after starting heartbeat.......................................................................34
7.4.3Checking Heartbeat status......................................................................................................34
8Testing high-availability scenario..........................................................................................................35
8.1.1Checking the network configuration on ldaphost2.................................................................35
9DNS hack...............................................................................................................................................37
10Query ISDS server using virtual IP......................................................................................................37
11Resources.............................................................................................................................................38
Table of Figures
Figure 1: Directory operations under normal condition.............................................................................4
Figure 2: Directory operations under failover condition............................................................................5
Figure 3: WAT Console administration login.............................................................................................8
Figure 4: WAT console introduction page..................................................................................................8
Figure 5: Adding servers to WAT...............................................................................................................9
Figure 6: Adding servers to WAT(2)..........................................................................................................9
Figure 7: WAT servers list........................................................................................................................10
Figure 8: WAT Directory server login......................................................................................................11
Figure 9: WAT Introduction page.............................................................................................................12
Figure 10: Add replication subtree...........................................................................................................12
Figure 11: Browse subtree(1)...................................................................................................................13
Figure 12: Select replication suffix..........................................................................................................13

Figure 13: Browse subtree (2)..................................................................................................................13
Figure 14: Show topology........................................................................................................................14
Figure 15: Show topology for o=ibm,c=in...............................................................................................15
Figure 16: Replication pop-up menu.......................................................................................................15
Figure 17: Add peer server to replication topology.................................................................................16
Figure 18: Add credentials (1).................................................................................................................16
Figure 21: Select credential......................................................................................................................18
Figure 22: Add master server...................................................................................................................18
Figure 23: Add credential on peer server.................................................................................................19
Figure 24: Collect peer agreement information.......................................................................................19
Figure 25: Enter ldaphost1 credential information..................................................................................20
Figure 26: Replication setup success information...................................................................................23
Figure 27: Resume replication queue from ldaphost1 to ldaphost2.........................................................27
Figure 28: Resume replication queue from ldaphost2 to ldaphost1.........................................................27
Table of Listings
Listing 1: "idsilist" output from ldaphost1.................................................................................................6
Listing 2: "idsilist" output from ldaphost2.................................................................................................7
Listing 3: ISDS and admin server running on ldaphost1...........................................................................7
Listing 4: ISDS and admin server running on ldaphost2...........................................................................7
Listing 5: ISDS instance list.....................................................................................................................10
Listing 6: Information added to DIT after adding replication suffix.......................................................14
Listing 7: Bind credential.........................................................................................................................18
Listing 8: Supplier bind credential information.......................................................................................19
Listing 9: Information added to server ldaphost1....................................................................................21
Listing 10: Information added to server ldaphost2..................................................................................22
Listing 11: Exporting data with seed and salt..........................................................................................24
Listing 12: ISDS server's crypto salt........................................................................................................25
Listing 13: Copy the exported LDIF file to the target server...................................................................25
Listing 14: Importing data into the server................................................................................................26
Listing 15: View Heartbeat's document...................................................................................................28
Listing 16: The /etc/ha.d/authkeys file.....................................................................................................29
Listing 17: authkeys permission..............................................................................................................29
Listing 18: Sample ha.cf file....................................................................................................................29
Listing 19: Sample haresources file.........................................................................................................30
Listing 20: ha_propagate script location..................................................................................................30
Listing 21: ha_propagate..........................................................................................................................31
Listing 22: ifconfig output before starting heartbeat................................................................................32
Listing 23: Starting heartbeat...................................................................................................................32
Listing 24: log snippet from /var/log/ha-log on ldaphost1......................................................................33
Listing 25: ifconfig output after starting heartbeat..................................................................................34
Listing 26: Stopping heartbeat.................................................................................................................35
Listing 27: ifconfig output from ldaphost2 after stopping heartbeat on ldaphost1..................................35
Listing 28: log snippet from /var/log/ha-log on ldaphost2 after stopping heartbeat on ldaphost1..........36
Listing 29: /etc/hosts file from server ldaphost1 and ldaphost2..............................................................37
Listing 30: idsldapsearch using virtual IP/hostname...............................................................................38
1 Introduction
As organizations add applications and services, centralized authentication services can increase
security and decrease administrative tasks. However, being dependent on a single server eventually
creates a "single point of failure" (SPOF), which can break the entire organization's authentication
system. To overcome this SPOF, we will be discussing here how to configure IBM Security Directory
Server (ISDS) with Heartbeat to deliver a highly available authentication system.
In this article, we will demonstrate one method to create a reliable authentication server using ISDS,
which can be adapted by many organization-wide applications. We will use the Heartbeat package
from the Linux HA project.
Starting with two identical ISDS servers (peer-to-peer replication), several configurations can be used.
First, we could do a "cold standby" where the master ISDS server has a virtual IP and a running ISDS
instance. The secondary ISDS server sits idle. When the master server fails, the ISDS instance and
virtual IP move to the cold node (secondary server). This is a very simple setup to implement. See
Figure 1: Directory operations under normal condition. However, the data synchronization between the
master and secondary servers could be a problem. To solve that, we will configure the servers with live
ISDS instances running on both the servers. In this way, updates to the master server are immediately
replicated to the secondary server.
Figure 1: Directory operations under normal condition
Failure of the master ISDS server leaves our secondary ISDS server available to respond to client
queries. See Figure 2: Directory operations under failover condition
Figure 2: Directory operations under failover condition
2 Environment setup
The example described in this article is based on a setup that requires:
Two ISDS v6.3.1 Servers, installed on RHEL 6.5 64-bit, configured in a peer-to-peer or
master-master replication.
Server1 hostname ldaphost1.in.ibm.com - 192.168.56.20
Server2 hostname ldaphost2.in.ibm.com - 192.168.56.21
Heartbeat version 3.0.4 installed on both servers (i.e., ldaphost1 and ldaphost2)
Note: See the "Resources" section to download ISDS and Heartbeat packages.
3 About IBM Security Directory Server (ISDS)

IBM Security Directory Server is an enterprise identity management software that uses the
Lightweight Directory Access Protocol (LDAP) and provides a trusted identity data infrastructure for
authentication. See the "Resources" section to find out more about IBM Security Directory Server.
4 About Heartbeat
In an enterprise environment, certain servers (such as an authentication server) must always be up and
running for the business to keep functioning smoothly. These servers provide services that need to be
always available.
A cornerstone of any mission-critical service that always needs to be up with no downtime is being able
to transfer the service from one system to another gracefully. The magic that makes this possible is a
service called Heartbeat. Heartbeat is the main product of the High-Availability Linux project.
In this article, we have tested only an active/passive method with two ISDS peer servers, where the
active server provides the services and the passive server waits to take over in case the active server
goes down.
The best part of this method is that you do not need any hardware devices, which tend to be expensive,
to build a highly available authentication system.
5 Prerequisite
In this section, I assume that you have already installed ISDS on both systems (i.e., ldaphost1 and
ldaphost2).
I also assume that you have created an ISDS instance dsrdbm01 on both the servers, which runs on
port 389. If you have set up the instances properly, then you will see output from both servers similar
to the following two listings.
[root@ldaphost1 ~]# idsilist -a

Directory server instance(s):
-------------------------------------Instance 1:
Name: dsrdbm01
Version: 6.3.1
Location: /home/dsrdbm01
Description: IBM Security Directory Server Instance V6.3.1
IP Addresses: All available
Port: 389
Secure Port: 636
Admin Server Port: 3538
Admin Server Secure Port: 3539
Type: Directory Server
Listing 1: "idsilist" output from ldaphost1

Listing 1 shows sample output from ldaphost1.in.ibm.com. Listing 2 shows sample output from
ldaphost2.in.ibm.com.

-------------------------------------Instance 1:
Name: dsrdbm01
Version: 6.3.1
Port: 389
Secure Port: 636
Listing 2: "idsilist" output from ldaphost2

To configure the replication, both the instances on server ldaphost1 and ldaphost2 must be up and
running. To confirm that they are, type the following command on both servers:
[root@ldaphost1 ~]# netstat -tnlp | grep ibm
Listing 3shows sample output from ldaphost1:

tcp
0
0 :::389
:::*
tcp
0
0 :::3538
:::*
LISTEN
LISTEN
1755/ibmslapd
29489/ibmdiradm
LISTEN
LISTEN
14158/ibmslapd
4581/ibmdiradm
Listing 3: ISDS and admin server running on ldaphost1
Listing 4 shows sample output from ldaphost2:

tcp
0
0 :::389
:::*
tcp
0
0 :::3538
:::*
Listing 4: ISDS and admin server running on ldaphost2
6 IBM Security Directory Server replication

Replication is a technique used by directory servers to improve performance, availability, and
reliability. The replication process keeps the data in multiple directory servers synchronized.
We will set up a peer-to-peer replication using the Web Administration Tool (WAT), which uses a
graphical user interface to administer ISDS servers.
If you do not have WAT installed on your machine, see "Setting up Tivoli Directory Server replication
using the command line," an excellent article on developerWorks that presents a step-by-step method of
configuring replication in an ISDS server using command-line tools. It can be accessed through the link
provided below in "Resources."

For the current article, I have set up WAT on machine ldaphost1.in.ibm.com.
6.1 Log in to Web Administration Tool

You can log in to the Web Administration Tool (WAT) using the following url:
http://ldaphost1.in.ibm.com:12100/IDSWebApp/. You will need to change the hostname of the server
where you have deployed the WAT. If the above url doesn't work, try the direct login page:
http://ldaphost1.in.ibm.com:12100/IDSWebApp/IDSjsp/Login.jsp
Figure 3: WAT Console administration login

Use the default username superadmin and password secret to log in to the WAT.
6.2 WAT introduction page

After you have logged in to the WAT for the first time, click on Manage Console Servers to begin
adding the servers.
Figure 4: WAT console introduction page
6.3 Adding servers to WAT

Click on Add... to add your servers to the Web Administration Tool to manage from a remote location
or servers.
Figure 5: Adding servers to WAT
In the Server name field, enter a descriptive name for your ISDS instance. The best practice in creating
a server name is to use the hostname followed by the instance name (i.e., hostnameinstance_name). To use only the hostname, leave the field blank.
Hostname is the real server's name on which TDS instance is running, and the hostname must be
resolvable by your DNS, otherwise WAT will fail to contact the server. On the other hand, Server name
is for end user reference to give a meaningful name to the instance. Server name is confined to the
WAT portal only, whereas Hostname must be resolvable to a specific IP address.
Figure 6: Adding servers to WAT(2)
The panel in Figure 6 also prompts you for a Port number and an Administration port number. Use
the command line tool idsilist -a to find out these values. See, for example, the output in Listing
5.
-------------------------------------Instance 1:
Name: dsrdbm01
Version: 6.3.1
Port: 389
Secure Port: 636
Listing 5: ISDS instance list
After you have added each server, press OK. Add the rest of the servers that you want to administer
remotely using WAT.
After all the servers have been added, you can display the list in the Manage console servers page.
Figure 7: WAT servers list

Now click Logout to log out of the WAT.
6.4 Log in to the ISDS instance to configure replication

You can now log in to any of the ISDS servers that you added in the "Adding servers to WAT" section.
Here we will log in to the instance dsrdbm01 on server ldaphost1.
The LDAP Server Name drop-down field will pop out a list of all the servers that you have added.
See Figure 8: WAT Directory server login.
Select ldaphost1-dsrdbm01 as the LDAP Server Name and enter the ISDS administrative user
ID (in most cases, it is cn=root) and password, which was configured when the ISDS instance
dsrdbm01 was created on hostldaphost1.
Note: If you do not know the administrative user ID and password, you cannot log in to WAT as a
privileged user and cannot configure the replication. Please refer to the command line tool idsdnpw
for more information on how to configure the administrative DN and password for an ISDS instance.
Figure 8: WAT Directory server login
6.5 Setting up the replication

When you click Loginsee Figure 8you are presented with the introduction page of the Web
Administration Tool, where you can administer your ISDS instance from a local or remote system. See
Figure 9: WAT Introduction page.
Figure 9: WAT Introduction page

Near the top of Figure 9, two values have been outlined in red. The left box signifies the physical
server name and the port on which the ISDS instance is running along with the server name in brackets,
which you configured in the "Adding servers to WAT" section. The right box indicates the current
logged-in user, which in this case is cn=root (ISDS root administrator). Click on Manage
Replication to configure the replication for a particular suffix or subtree. See Figure 10: Add
replication subtree
Figure 10: Add replication subtree

Click on Add subtree... to add the suffix or subtree that you want to replicate. You are presented with
a screen similar to Figure 11: Browse subtree(1) to add your subtree.
Now click on Browse... to navigate through the

Directory Information Tree (DIT) of your ISDS
server.
Accept the default value in the Master server
referral LDAP URL field.
Figure 11: Browse subtree(1)
After clicking on OK, as shown in Figure 11: Browse subtree(1), you are presented with a screen
similar to Figure 12: Select replication suffix, on which to choose the suffix or subtree to replicate.
Figure 12: Select replication suffix

Click the radio button on the left to choose the suffix that you want to replicate. Click on Select.
You are redirected to a page similar to Figure 13: Browse subtree (2). This
screen resembles the one displayed in Figure 11, except that the Subtree
DN value is filled in with the selected subtree name: in this case,
o=ibm,c=in.
Next, click on the OK button to continue. WAT adds the following three
replication stanzas (shown in Listing 6: Information added to DIT after
adding replication suffix) to the DIT of instance dsrdbm01, which is
Figure 13: Browse subtree
running on host ldaphost1.in.ibm.com.
(2)
o=ibm,c=in
objectclass=organization
objectclass=top
objectclass=ibm-replicationcontext
o=ibm
ibm-replicareferralurl=ldap://ldaphost1.in.ibm.com:389
ibm-replicaGroup=default,o=ibm,c=in
ibm-replicagroup=default
objectclass=ibm-replicagroup
objectclass=top
cn=ldaphost1.in.ibm.com:389,ibm-replicaGroup=default,o=ibm,c=in
objectclass=ibm-replicasubentry
objectclass=top
ibm-replicaserverid=de8e2ec0-bfa2-1032-950c-8d69c0d06df0
ibm-replicationserverismaster=TRUE
cn=ldaphost1.in.ibm.com:389
Listing 6: Information added to DIT after adding replication suffix

Refer to "Check replication-related entries in ISDS using command line" to find out the Listing 6
information for your server.
After clicking on OK in Figure 13, you see the following screen:
Figure 14: Show topology

Notice that o=ibm,c=in has been added successfully as a replication subtree. The ISDS instance
dsrdbm01, which is running on port 389 on server ldaphost1.in.ibm.com, will act as a master
server for this subtree in the replication topology.
Click on Show topology to see a screen similar to that in Figure 15: Show topology for o=ibm,c=in:
Figure 15: Show topology for o=ibm,c=in

Click on the Topology for selected subtree button to
get a pop-up menu, as shown in Figure 16: Replication
pop-up menu.
This menu allows you to add a peer master or a replica
or even manage your gateway servers.
Since we are setting up a peer-to-peer replication
topology,we need to add a master server under the
server ldaphost1.in.ibm.com which is also a master
server for the suffix o=ibm, c=in.
Click on Add master to add another peer server to
Figure 16: Replication pop-up menu
ldaphost1.in.ibm.com. After clicking on Add
master, you are presented with a screen similar to that in Figure 17: Add peer server to replication
topology:
Figure 17: Add peer server to replication topology

Click on the drop-down list for the field Server hostname:port to select your peer master server. Click
on Get server ID to get the server ID of ldaphost2.in.ibm.com. You will now need bind credentials
which your server ldaphost1.in.ibm.com will use to bind to the server ldaphost2.in.ibm.com
and make the necessary changes during replication. If any entry information on server
ldaphost1.in.ibm.com changes by any client, then the same operation will be replicated to server
ldaphost2.in.ibm.com using these bind credentials.
Click Select as shown Figure 17. You will see a screen similar to Figure 18: Add credentials (1). No
credentials have yet been created; therefore, the Select credential field is empty. Select the replication
suffix (o=ibm,c=in) radio button under Location for credentials and click on Add credentials.
Figure 18: Add credentials (1)
You now see a screen similar to Figure 19: Add credentials (2), where you need to enter a CN
(Common Name) under which your bind credentials will be stored. Click on Next to continue.

You now see a screen similar to Figure 20: Add credentials (3), where you need to enter the bind DN
and a password. The DN will be used to authenticate the replication changes from server
ldaphost1.in.ibm.com to ldaphost2.in.ibm.com. Optionally, you can also enter a description for the
bind DN.

Click Finish on Figure 20: Add credentials (3). You now see a screen similar to Figure 21: Select
credential.
Figure 21: Select credential

A stanza has been added to your DIT under DN ibm-replicaGroup=default,O=IBM,C=IN as
shown in Listing 7: Bind credential.
cn=bindcreds,ibm-replicaGroup=default,O=IBM,C=IN
replicacredentials=manager
description=Bind to replication server ldaphost2
objectclass=ibm-replicationcredentials
objectclass=ibm-replicationcredentialssimple
objectclass=top
replicabinddn=cn=manager
cn=bindcreds
Listing 7: Bind credential
Refer to "Check replication-related entries in ISDS using command line" to find out the Listing 6
information for your server.
Click OK on Figure 21. You see a screen similar to that shown in Figure 22: Add master server.
Figure 22: Add master server
Click on Additional to configure the peer replication server (ldaphost2.in.ibm.com). You are presented
with a new screen. Scroll down to the Consumer section and add the details as shown in Figure 23:
Add credential on peer server.
Figure 23: Add credential on peer server

Click the Add credential information on consumer check box and type in the admin DN for the peer
replication server. In this case, the admin DN is cn=root. Enter the password root .
Click OK. The following supplier bind credential information, as shown in Listing 8: Supplier bind
credential information, for server ldaphost1.in.ibm.com will be added to server ldaphost2.in.ibm.com.
cn=Supplier1397407251698, cn=configuration
cn=Supplier1397407251698
ibm-slapdmasterdn=cn=manager
ibm-slapdmasterpw={AES256}eKP6VmxELqJHSNAlGYWUkw==
ibm-slapdreplicasubtree=O=IBM, C=IN
objectclass=ibm-slapdconfigentry
objectclass=ibm-slapdsupplier
objectclass=top
Listing 8: Supplier bind credential information

You now see a screen similar to Figure 24: Collect peer agreement information.
Figure 24: Collect peer agreement information

Click OK.You are asked to enter the credential information to be added to server
ldaphost1.in.ibm.com. See Figure 25: Enter ldaphost1 credential information.
Figure 25: Enter ldaphost1 credential information

Enter all the information requested in Figure 25. Click OK.
The following stanzas have now been added to server ldaphost1.in.ibm.com (see Listing 9: Information
added to server ldaphost1):
cn=Supplier1397416499356, cn=configuration
cn=Supplier1397416499356
ibm-slapdmasterdn=cn=manager
ibm-slapdmasterpw={AES256}mU4i4ChMyT7OytFazN+hIA==
ibm-slapdreplicasubtree=O=IBM, C=IN
objectclass=ibm-slapdconfigentry
objectclass=ibm-slapdsupplier
objectclass=top
cn=ldaphost2.in.ibm.com:389,ibm-replicaGroup=default,O=IBM,C=IN
objectclass=top
ibm-replicaserverid=7cf882c0-46f0-1033-8038-b508ef5ab1d7
cn=ldaphost1.in.ibm.com:389,cn=ldaphost2.in.ibm.com:389,ibmreplicaGroup=default,O=IBM,C=IN
ibm-replicamethod=1
ibm-replicaconsumerid=de8e2ec0-bfa2-1032-950c-8d69c0d06df0
ibm-replicationonhold=TRUE
ibm-replicacredentialsdn=cn=bindcreds,ibm-replicaGroup=default,O=IBM,C=IN
ibm-replicaurl=ldap://ldaphost1.in.ibm.com:389
objectclass=ibm-replicationagreement
objectclass=top
ibm-replicamethod=1
ibm-replicaconsumerid=7cf882c0-46f0-1033-8038-b508ef5ab1d7
objectclass=top
Listing 9: Information added to server ldaphost1
In addition, the following information has been added to server ldaphost2.in.ibm.com (Listing 10:
Information added to server ldaphost2):
o=ibm,c=in
objectclass=organization
objectclass=top
objectclass=ibm-replicationcontext
o=ibm
ibm-replicareferralurl=ldap://ldaphost1.in.ibm.com:389
ibm-replicaGroup=default,o=ibm,c=in
ibm-replicagroup=default
objectclass=ibm-replicagroup
objectclass=top
cn=ldaphost1.in.ibm.com:389,ibm-replicaGroup=default,o=ibm,c=in
objectclass=top
ibm-replicaserverid=de8e2ec0-bfa2-1032-950c-8d69c0d06df0
cn=ldaphost2.in.ibm.com:389,ibm-replicaGroup=default,O=IBM,C=IN
objectclass=top
ibm-replicaserverid=7cf882c0-46f0-1033-8038-b508ef5ab1d7
ibm-replicamethod=1
ibm-replicaconsumerid=de8e2ec0-bfa2-1032-950c-8d69c0d06df0
objectclass=top
cn=ldaphost2.in.ibm.com:389,cn=ldaphost1.in.ibm.com:389,ibm-replicaGroup=default
ibm-replicamethod=1
ibm-replicaconsumerid=7cf882c0-46f0-1033-8038-b508ef5ab1d7
objectclass=top
cn=bindcreds,ibm-replicaGroup=default,O=IBM,C=IN
replicacredentials=manager
description=Bind to replication server ldaphost2
objectclass=ibm-replicationcredentials
objectclass=ibm-replicationcredentialssimple
objectclass=top
replicabinddn=cn=manager
cn=bindcreds
Listing 10: Information added to server ldaphost2
Next, you are presented with a screen similar to that in Figure 26: Replication setup success
information.
Figure 26: Replication setup success information

At this point, you have successfully configured the peer-to-peer replication between host
ldaphost1.in.ibm.com and ldaphost2.in.ibm.com.
Note: If you already have data under suffix o=ibm,c=in, then you need to synchronize both servers
manually. When WAT is used to set up the replication, WAT doesn't synchronize the data between the
hosts but replicates only the configuration information essential for replication to work. By default, the
replication queue is in suspended mode. You need to resume the queue manually to allow the changes
made to both the servers to be replicated. Please refer to the section "Resume replication queue" to
find out how to resume the queue using WAT.
6.5.1 Check replication-related entries in ISDS using command line

Sometimes it is useful to check the replication-related entries in ISDS to find out the basic reason why
the replication is not working: whether an replication entry is missing or whether you have input the
replication hostname incorrectly.
Use the following command to find out all the replication related entries in ISDS.
idsldapsearch -h <hostname/IP> -p <port> -D <adminDN> -w <adminDN_PW> -s sub -b ""
objectclass=ibm-repl*
For example,
idsldapsearch -h ldaphost1.in.ibm.com -p 389 -D cn=root -w root -s sub -b ""
Note: You don't have to specify the port if it's 389, because 389 is the default port for LDAP. If the port
number is other than 389, then you must specify the port or else you'll receive an error message.
Use the following command to find out the replication-related entries within a subtree. For example, to
find out the replication-related entries under suffix "O=IBM,C=IN"
idsldapsearch -h ldaphost1.in.ibm.com -D cn=root -w root -s sub -b "O=IBM,C=IN"
6.6 Synchronize data between two replication servers

Suppose you already have data under the suffix o=ibm,c=in. After OK on Figure 26: Replication
setup success information is clicked, any changes made to or under suffix o=ibm, c=in will be
replicated to server ldaphost2.in.ibm.com. But what about the data that was there under o=ibm, c=in
on server ldaphost1.in.ibm.com before the replication was set up? No need to worry. All you have to
do is to export the data under o=ibm,c=in from server ldaphost1.in.ibm.com into the server
Note: For better performance, synchronize the directory servers that are taking part in the replication
cryptographically. See "Resources" for a link to Appendix J in the IBM Security Directory Server
Administration Guide.
6.6.1 Exporting data

You can export the data from a particular suffix using the command line tool idsdb2ldif. This will
create an LDIF file that will contain all the data under a particular suffix.
Syntax:
idsdb2ldif -I <instance_name> -o <outfile.ldif>
Note: If your server instances are not synchronized cryptographically, then exported data (in this case
<outfile.ldif> from server ldaphost1.in.ibm.com) cannot be imported into another server
ldaphost2.in.ibm.com. In that case, you need to export the data from server ldaphost1.in.ibm.com using
the salt and seed of the destination server ldaphost2.in.ibm.com. See "How to obtain salt and seed of
an ISDS server" to get the salt value (-t); the seed value should be known to the person who
originally created the ISDS instance.
6.6.2 Exporting data using salt and seed of the destination server
After you have obtained the seed and salt value of your destination serverin this case,
ldaphost2.in.ibm.comperform the following command to export the data from server
ldapohost1.in.ibm.com.
Syntax:
idsdb2ldif -I <instance_name> -k <seed_value> -t <salt_value> -o
<outfile.ldif>
For example, see Listing 11: Exporting data with seed and salt.
[root@ldaphost1 sbin]# pwd
/opt/ibm/ldap/V6.3.1/sbin
[root@ldaphost1 sbin]# ./idsdb2ldif -I dsrdbm01 -k abc0123456789 -t 'umU"mYJ4Oct0'
-o dsrdbm01_fullbackup.ldif
GLPCTL113I Largest core file size creation limit for the process (in bytes):
'0'(Soft limit) and '-1'(Hard limit).
GLPCTL119I Maximum Data Segment(Kbytes) soft ulimit for the process is -1 and the
prescribed minimum is 262144.
GLPCTL119I Maximum File Size(512 bytes block) soft ulimit for the process is -1 and
the prescribed minimum is 2097152.
GLPCTL122I Maximum Open Files soft ulimit for the process is 1024 and the
GLPCTL122I Maximum Stack Size(Kbytes) soft ulimit for the process is 10240 and the
GLPCTL119I Maximum Virtual Memory(Kbytes) soft ulimit for the process is -1 and the
GLPSRV221I Replication of security attributes feature is disabled.
GLPSRV200I Initializing primary database and its connections.
GLPD2L011I 61 entries have been successfully exported from the directory.
Listing 11: Exporting data with seed and salt
How to obtain salt and seed of an ISDS server
Obtain the encryption salt value by performing the following search:

ldapsearch -D <adminDN> -w <adminPw> -b "cn=crypto,cn=localhost"
objectclass=* ibm-slapdCryptoSalt
For example, to obtain the salt value of ldaphost2.in.ibm.com, execute the following command:
[root@ldaphost1 ~]# idsldapsearch -h ldaphost2.in.ibm.com -L -D cn=root -w root -s
base -b cn=crypto,cn=localhost objectclass=* ibm-slapdCryptoSalt
dn: cn=crypto,cn=localhost
ibm-slapdCryptoSalt: umU"mYJ4Oct0
Listing 12: ISDS server's crypto salt
Note: For security purposes, the encryption seed value is not stored anywhere in the server. While
creating the ISDS instance, be sure to make note of it so that you can remember it later.
6.6.3 Importing data

You can import an LDIF file into an ISDS server using a command line tool called idsldif2db.
Now you can copy the newly generated LDIF file to the destination server ldaphost2.in.ibm.com so that
you can import it into the ISDS instance.
Remote copy the LDIF file as shown in Listing 13: Copy the exported LDIF file to the target server (on
a Linux system):
[root@ldaphost1 ~]# scp dsrdbm01_fullbackup.ldif ldaphost2.in.ibm.com:~
The authenticity of host 'ldaphost2.in.ibm.com (192.168.56.21)' can't be
established.
RSA key fingerprint is ab:20:99:59:74:25:dc:96:20:47:e5:ae:7e:9c:a5:90.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ldaphost2.in.ibm.com' (RSA) to the list of known hosts.
root@ldaphost2.in.ibm.com's password:
dsrdbm01_fullbackup.ldif
100%
44KB 43.7KB/s
00:00
[root@ldaphost1 ~]#
Listing 13: Copy the exported LDIF file to the target server
On the destination server, ldaphost2.in.ibm.com, execute the command shown in Listing 14: Importing
data into the server to import the LDIF file.
[root@ldaphost2 sbin]# ./idsldif2db -I dsrdbm01 -i ~/dsrdbm01_fullbackup.ldif
GLPCTL113I Largest core file size creation limit for the process (in bytes):
'0'(Soft limit) and '-1'(Hard limit).
GLPCTL119I Maximum Data Segment(Kbytes) soft ulimit for the process is -1 and the
GLPCTL119I Maximum File Size(512 bytes block) soft ulimit for the process is -1 and
the prescribed minimum is 2097152.
GLPCTL122I Maximum Open Files soft ulimit for the process is 1024 and the
GLPCTL122I Maximum Stack Size(Kbytes) soft ulimit for the process is 10240 and the
GLPCTL119I Maximum Virtual Memory(Kbytes) soft ulimit for the process is -1 and the
GLPCOM022I The database plugin is successfully loaded from libback-config.so.
GLPSRV221I Replication of security attributes feature is disabled.
GLPSRV200I Initializing primary database and its connections.
GLPRPL137I Restricted Access to the replication topology is set to false.
GLPRDB052E Entry CN=IBMPOLICIES already exists.
GLPRDB052E Entry IBM-REPLICAGROUP=DEFAULT,CN=IBMPOLICIES already exists.
GLPRDB052E Entry globalGroupName=GlobalAdminGroup,cn=ibmpolicies already exists.
GLPRDB052E Entry cn=pwdpolicy,cn=ibmpolicies already exists.
GLPRDB052E Entry CN=REPLICATION,CN=IBMPOLICIES already exists.
GLPRDB052E Entry o=ibm,c=in already exists.
GLPRDB052E Entry ibm-replicaGroup=default,o=ibm,c=in already exists.
GLPRDB052E Entry cn=ldaphost1.in.ibm.com:389,ibm-replicaGroup=default,o=ibm,c=in
already exists.
GLPRDB052E Entry cn=bindcreds,ibm-replicaGroup=default,O=IBM,C=IN already exists.
GLPRDB052E Entry cn=ldaphost2.in.ibm.com:389,ibm-replicaGroup=default,O=IBM,C=IN
already exists.
GLPRDB052E Entry cn=ldaphost1.in.ibm.com:389,cn=ldaphost2.in.ibm.com:389,ibmreplicaGroup=default,O=IBM,C=IN already exists.
GLPRDB052E Entry cn=ldaphost2.in.ibm.com:389,cn=ldaphost1.in.ibm.com:389,ibmreplicaGroup=default,O=IBM,C=IN already exists.
GLPRDB002W ldif2db: 49 entries have been successfully added out of 61 attempted.
Listing 14: Importing data into the server
Note: The ISDS instance must be stopped before importing the LDIF file; otherwise it will not allow
you to import the data into it. Also, you can ignore the error message GLPRDB052E shown in Listing
14, as the entries were already present on the destination server, ldaphost2.in.ibm.com. The
idsldif2db utility will not change these entries; it will simply skip them.
At this point, both the source server (ldaphost1.in.ibm.com) and the destination server
(ldaphost2.in.ibm.com) are perfectly in sync. Now you can go ahead and resume the replication queue
to allow both the servers to replicate data to each other.
6.7 Resume replication queue

Log in to the instance that is running on server ldapohost1.in.ibm.com and click on
Replication management on the left pane. Under it, click on Manage queues.
Figure 27: Resume replication queue from ldaphost1 to ldaphost2

Click on Suspend/resume to resume the replication queue. If ldaphost2.in.ibm.com is displayed in the
Replica column, the replication queue flow is from server ldaphost1.in.ibm.com to
Next, log in to the instance that is running on server ldaphost2.in.ibm.com and click on Replication
management on the left pane. Under it, click on Manage queues.
Figure 28: Resume replication queue from ldaphost2 to ldaphost1

Click on Suspend/resume to resume the replication queue. If ldaphost1.in.ibm.com is displayed in the
Replica column, the replication queue flow is from server ldaphost2.in.ibm.com to
Now both the servers are perfectly in sync and the replication queue is in Ready state. We will now
configure high availability (HA) for these servers.
7 Heartbeat setup
Heartbeat is very flexible and powerful. In this article, I have touched on only the basic active/passive
mode with two servers, where the active server (ldaphost1.in.ibm.com) is providing the service and the
passive server (ldaphost2.in.ibm.com) is waiting to take over if necessary.
7.1 Installing Heartbeat

Installing the Heartbeat package from the yum repository is always easier than building it from the
source. The Heartbeat package is available in the EPEL (Extra Package for Enterprise Linux)
repository. See "Resources" for download information.
Execute the following command to install Heartbeat:
yum install heartbeat --enablerepo=epel
7.2 Configuring Heartbeat

You need three configuration files to work with Heartbeat. Each goes into the directory /etc/ha.d/
authkeys
File containing keys for mutual node authentication
ha.cf
Main Heartbeat configuration file
haresources
Resource configuration file
The ha.cf and hareources files may be readable by everyone, but the authkeys file must not be.
The good news is that sample versions of these files may be found in the documentation directory. If
you installed Heartbeat using yum, then the following command will show you where they are on your
system:
rpm -q heartbeat -d
For example, see Listing 15: View Heartbeat's document.
[root@ldaphost1 ~]# rpm -q heartbeat -d
......
......
/usr/share/doc/heartbeat-3.0.4/authkeys
/usr/share/doc/heartbeat-3.0.4/ha.cf
/usr/share/doc/heartbeat-3.0.4/haresources
......
......
Listing 15: View Heartbeat's document
7.2.1 The authkeys file

The authkeys file must be owned by root and be set to chmod 600. The actual format of the authkeys
file is very simple; it consists of only two lines. The first line contains an auth directive with an
associated method ID number, and the second line identifies the authentication method and the key that
go with the auth directive.
Three authentication methods are supported: CRC, MD5, and SHA1. CRC doesn't accept a key. You
normally have only one authentication method listed in a CRC file. It adds no security, except from
packet corruption, and should be used only on physically secure networks. Of the remaining two,
SHA1 is usually considered to be the best, followed by MD5.
Listing 16: The /etc/ha.d/authkeys file shows an example. Make the key long as it will improve
security and you will not have to type it again.
auth 1
1 sha a9e2189843e9c3fb7c5a1bf81587fc11
Listing 16: The /etc/ha.d/authkeys file

Check the permission on the file /etc/ha.d/authkeys. If the permission is not 600, then change
it to 600 as shown in Listing 17: authkeys permission:
[root@ldaphost1 ~]# ls -lh /etc/ha.d/authkeys
-rw-r--r-- 1 root root 46 Apr 19 01:51 /etc/ha.d/authkeys
[root@ldaphost1 ~]# chmod 600 /etc/ha.d/authkeys
[root@ldaphost1 ~]# ls -lh /etc/ha.d/authkeys
-rw------- 1 root root 46 Apr 19 01:51 /etc/ha.d/authkeys
Listing 17: authkeys permission
7.2.2 The ha.cf file

This is the main Heartbeat configuration file. The content of this file should be same on both the
servers with a couple of exceptions.
Heartbeat ships with an example file in the documentation directory that is well worth a look. Also,
when creating your ha.cf file, the order in which things appear matters. Do not move them around!
debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility local0
bcast eth0
warntime 5
deadtime 15
initdead 60
keepalive 2
auto_failback on
node ldaphost1.in.ibm.com
node ldaphost2.in.ibm.com
Listing 18: Sample ha.cf file
The auto_failback on command tells Heartbeat to prefer ldaphost1.in.ibm.com. If

ldaphost1.in.ibm.com should go down, ldaphost2.in.ibm.com will take over the Virtual IP Address and
start the highly available services. When ldaphost1.in.ibm.com comes back online,
ldaphost2.in.ibm.com will transfer (automatically failback) the Virtual IP and continue serving the
clients without any downtime.

You should familiarize yourself with the sample ha.cf file that comes with the Heartbeat package.
Refer to the Listing 15: View Heartbeat's document to find out more about ha.cf options. Also see the
"Resources" section for a link to the official Heartbeat documentation.
7.2.3 The haresources file

This file should be exactly the same on both the servers.
ldaphost1.in.ibm.com IPaddr::192.168.56.252/24/eth0:1/192.168.56.255
Listing 19: Sample haresources file

ldaphost1.in.ibm.com is the hostname of our first or primary node, and
IPaddr::192.168.56.252/24/eth0:1/192.168.56.255 is the virtual IP address with a
subnet mask 24. It will be attached to an Ethernet interface eth0:1 with a broadcast address of
192.168.56.255. See the official Heartbeat documentation to learn more about the options available.
7.3 Propagating configuration files to peer servers

The Heartbeat package provides a script ha_propagate to copy the configuration files over to peer
servers that are listed in the ha.cf file with a node directive. By default, the script ha_propagate copies
only the two files authkeys and ha.cf. However, I have tweaked my version of ha_propagate to copy
all the three files to the peer servers. See the "Resources" section to download the tweaked version of
the script.
If you want to find out where the default ha_propagate script is located, then execute the following
command:
# rpm -ql heartbeat | grep -i ha_propagate
/usr/share/heartbeat/ha_propagate
Listing 20: ha_propagate script location
Since we have created all the three files (authkeys, ha.cf, and haresources) in ldaphost1.in.ibm.com, we
need to copy them over to ldaphost2.in.ibm.com. Execute the downloaded script to copy the three
configuration files. See Listing 21: ha_propagate
[root@ldaphost1 heartbeat]# pwd

/usr/share/heartbeat
[root@ldaphost1 heartbeat]# ./ha_propagate
Propagating HA configuration files to node ldaphost2.in.ibm.com.
ha.cf
100% 259
0.3KB/s
00:00
authkeys
100%
47
0.1KB/s
00:00
haresources
100%
69
0.1KB/s
00:00
Setting HA startup configuration on node ldaphost2.in.ibm.com.
.....
.....
heartbeat
0:off
1:off
2:on
3:on
4:on
5:on
6:off
httpd
0:off
1:off
2:off
3:off
4:off
5:off
6:off
ip6tables
0:off
1:off
2:off
3:off
4:off
5:off
6:off
.....
.....
Listing 21: ha_propagate

When you execute the ha_propagate script, it may ask you for the password to the remote system to
which the configuration files are being copied. Provide the password to continue. In the above case,
since both the servers are configured for passwordless ssh, ha_propagate didn't prompt for any
password.
Note: ha_propagate uses scp to copy the files to the remote server. If you have set up passwordless
ssh, it will not ask you for the remote system's password.
7.4 Starting Heartbeat

Now that both the servers (ldaphost1 and ldaphost2) have the same Heartbeat configuration files, you
are almost ready to start the Heartbeat service on both.
7.4.1 Network configuration before starting heartbeat

Before starting Heartbeat, check your network configuration with the command line tool ifconfig.
See Listing 22: ifconfig output before starting heartbeat
[root@ldaphost1 ~]# ifconfig

eth0
Link encap:Ethernet HWaddr 08:00:27:63:97:31
inet addr:192.168.56.20 Bcast:192.168.56.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe63:9731/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27950 errors:0 dropped:0 overruns:0 frame:0
TX packets:23545 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5230588 (4.9 MiB) TX bytes:5613926 (5.3 MiB)
lo
Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
Listing 22: ifconfig output before starting heartbeat
Now start the Heartbeat service on server ldaphost1.in.ibm.com and ldaphost2.in.ibm.com.

[root@ldaphost1 ~]# service heartbeat start
Starting High-Availability services: INFO: Resource is stopped
Done.
[root@ldaphost2 ~]# service heartbeat start
Starting High-Availability services: INFO: Resource is stopped
Done.
Listing 23: Starting heartbeat

Next, check the log file /var/log/ha-log on the primary server, which is ldaphost1.in.ibm.com in
this case. Refer to Listing 18: Sample ha.cf file.
Apr 20 00:26:47 ldaphost1.in.ibm.com heartbeat: [30705]: info: Pacemaker support:

false
...........
Apr 20 00:26:47 ldaphost1.in.ibm.com heartbeat: [30705]: info: Configuration
validated. Starting heartbeat 3.0.4
Apr 20 00:26:47 ldaphost1.in.ibm.com heartbeat: [30706]: info: heartbeat: version
3.0.4
...........output omitted...........
....................................
Apr 20 00:26:49 ldaphost1.in.ibm.com heartbeat: [30706]: info: Local status now set
to: 'up'
Apr 20 00:26:49 ldaphost1.in.ibm.com heartbeat: [30706]: info: Link
ldaphost1.in.ibm.com:eth0 up.
Apr 20 00:26:57 ldaphost1.in.ibm.com heartbeat: [30706]: info: Link
ldaphost2.in.ibm.com:eth0 up.
Apr 20 00:26:57 ldaphost1.in.ibm.com heartbeat: [30706]: info: Status update for
node ldaphost2.in.ibm.com: status up
...........output omitted...........
....................................
ResourceManager(default)[30916]:
2014/04/20_00:27:17 info: Acquiring
resource group: ldaphost1.in.ibm.com
2014/04/20_00:27:21 info: Running
/etc/ha.d/resource.d/IPaddr 192.168.56.252/24/eth0:1/192.168.56.255 start
IPaddr(IPaddr_192.168.56.252)[31072]:
2014/04/20_00:27:24 INFO: Adding inet
address 192.168.56.252/24 with broadcast address 192.168.56.255 to device eth0
(with label eth0:1)
IPaddr(IPaddr_192.168.56.252)[31072]:
2014/04/20_00:27:24 INFO: Bringing device
eth0 up
...........output omitted...........
....................................
/usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.56.252)[31046]:
2014/04/20_00:27:25 INFO: Success
Apr 20 11:06:01 ldaphost1.in.ibm.com heartbeat: [30706]: info: No pkts missing from
ldaphost2.in.ibm.com!
Listing 24: log snippet from /var/log/ha-log on ldaphost1
7.4.2 Network configuration after starting heartbeat

eth0
inet6 addr: fe80::a00:27ff:fe63:9731/64 Scope:Link
eth0:1

lo

inet addr:127.0.0.1 Mask:255.0.0.0
Listing 25: ifconfig output after starting heartbeat
If you see the above ifconfig output, a new Ethernet alias eth0:1 has been added with an IP address
of 192.168.56.252 as specified in the file /etc/ha.d/haresources.
7.4.3 Checking Heartbeat status

You can check the status of the Heartbeat service from any server listed in the node directive as
specified in the ha.cf file. Use the command line tool /usr/bin/cl_status,which is provided by
the Heartbeat package. Refer to Listing 18: Sample ha.cf file.
Check the list of the servers that are configured to run heartbeat
[root@ldaphost1 ~]# cl_status listnodes
ldaphost2.in.ibm.com
ldaphost1.in.ibm.com
Check the heartbeat status on a particular server

[root@ldaphost1 ~]# cl_status hbstatus
Heartbeat is running on this machine.
Heartbeat is stopped on this machine.
[root@ldaphost1 ~]# cl_status nodestatus ldaphost1.in.ibm.com
active
dead
If you see the above output, since Heartbeat is running on ldaphost1, it is shown as being "active."
Server ldaphost2 is shown as being "dead."
Now start Heartbeat on server ldaphost2. See Listing 23.

Check the Heartbeat status on server ldaphost2:
Heartbeat is running on this machine.
active
8 Testing high-availability scenario

At this point, Heartbeat is configured on both the servers and is running successfully as we saw in the
section "Checking Heartbeat status."
In order to test the high-availability scenario, we can simulate a system failure by stopping Heartbeat
on one server (ldaphost1.in.ibm.com) so that the peer server (ldaphost2.in.ibm.com) will not receive
any Heartbeat response and will therefore assume that the system is dead.
Stop the Heartbeat service on server ldaphost1.in.ibm.com.
[root@ldaphost1 ~]# service heartbeat stop
Stopping High-Availability services: Done.
Listing 26: Stopping heartbeat
8.1.1 Checking the network configuration on ldaphost2

eth0
Link encap:Ethernet HWaddr 08:00:27:03:6B:CD
inet6 addr: fe80::a00:27ff:fe03:6bcd/64 Scope:Link
eth0:1
Link encap:Ethernet HWaddr 08:00:27:03:6B:CD

lo

inet addr:127.0.0.1 Mask:255.0.0.0
RX bytes:21318 (20.8 KiB) TX bytes:21318 (20.8 KiB)
Listing 27: ifconfig output from ldaphost2 after stopping heartbeat on ldaphost1
Check the log file /var/log/ha-log on server ldaphost2.in.ibm.com to see how the resources were
acquired. In this case, the IP alias eth0:1 will be destroyed on server ldaphost1.in.ibm.com and will be
created on the server ldaphost2.in.ibm.com.
Apr 20 13:02:18 ldaphost2.in.ibm.com heartbeat: [26755]: info: Received shutdown
notice from 'ldaphost1.in.ibm.com'.
Apr 20 13:02:18 ldaphost2.in.ibm.com heartbeat: [26755]: info: Resources being
acquired from ldaphost1.in.ibm.com.
........output omitted........
Apr 20 13:02:18 ldaphost2.in.ibm.com heartbeat: [27067]: info: local HA resource
acquisition completed (standby).
Apr 20 13:02:18 ldaphost2.in.ibm.com heartbeat: [26755]: info: Standby resource
acquisition done [foreign].
harc(default)[27093]:
2014/04/20_13:02:18 info: Running /etc/ha.d//rc.d/status
status
mach_down(default)[27110]:
2014/04/20_13:02:18 info: Taking over resource
group IPaddr::192.168.56.252/24/eth0:1/192.168.56.255
2014/04/20_13:02:18 info: Acquiring
resource group: ldaphost1.in.ibm.com
IPaddr::192.168.56.252/24/eth0:1/192.168.56.255
/usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.56.252)[27165]:
2014/04/20_13:02:19 INFO: Resource is stopped
2014/04/20_13:02:19 info: Running
/etc/ha.d/resource.d/IPaddr 192.168.56.252/24/eth0:1/192.168.56.255 start
IPaddr(IPaddr_192.168.56.252)[27293]:
2014/04/20_13:02:19 INFO: Adding inet
address 192.168.56.252/24 with broadcast address 192.168.56.255 to device eth0
(with label eth0:1)
IPaddr(IPaddr_192.168.56.252)[27293]:
2014/04/20_13:02:19 INFO: Bringing device
eth0 up
........output omitted........
Apr 20 13:02:19 ldaphost2.in.ibm.com heartbeat: [26755]: info: mach_down takeover
complete.
Apr 20 13:02:34 ldaphost2.in.ibm.com heartbeat: [26755]: WARN: node
ldaphost1.in.ibm.com: is dead
Apr 20 13:02:34 ldaphost2.in.ibm.com heartbeat: [26755]: info: Dead node
ldaphost1.in.ibm.com gave up resources.
Listing 28: log snippet from /var/log/ha-log on ldaphost2 after stopping heartbeat on ldaphost1
If you see output such as that in Listing 27, a new IP alias eth0:1 has been created and the same alias
has been destroyed on the server ldaphost1.in.ibm.com. Because of this floating IP address, your
services will never be down as long as one of the server nodes is running. Whenever the server
ldaphost1.in.ibm.com is up, resources from the secondary server ldaphost2.in.ibm.com will be acquired
again (i.e., IP alias eth0:1 will be destroyed on server ldaphost2.in.ibm.com and will be created on
primary server ldaphost1.in.ibm.com; the primary server will continue serving its clients).
Note: You need to remember that the ISDS client must be pointed to at its virtual IP address (i.e.,
192.168.56.252 here and not the physical IP address of the system), because this IP address will be
available at all times in case any of the physical servers goes down. Distributing the new IP address
(192.168.56.252) will not be a good idea, as it is very difficult to remember the IP addresses. Instead,
you should distribute a new hostname. At this point, however, you don't have a hostname for IP
address 192.168.56.252. Read on in the next section ("DNS hack") to solve this problem.
9 DNS hack
When you start Heartbeat, it creates a new IP alias with a new virtual IP address as specified in the
file /etc/ha.d/haresources. Refer to Listing 19: Sample haresources file.
Previously, we had two IP addresses: 192.168.56.20 with hostname ldaphost1.in.ibm.com and
192.168.56.21 with hostname ldaphost2.in.ibm.com. Now we have another IP address:
192.168.56.252, for which there is no DNS entry or any entry in file /etc/hosts (for Linux system).
You have two options:
You can tell your network administrator to add a new DNS entry for the new IP address (in this
case 192.168.56.252) with a new hostname, or
If the servers are serving a limited number of clients, then you can make use of the local hosts
file (/etc/hosts for Linux systems and %SystemRoot
%\System32\drivers\etc\hosts file for Windows systems) to resolve the IP addressto-hostname issue.
The first option is recommended for organizations having a large-scale deployment. However, for this
article, we are choosing the option 2 to simulate the behavior. See Listing 29: /etc/hosts file from server
ldaphost1 and ldaphost2
[root@ldaphost1 ~]# cat /etc/hosts
127.0.0.1
localhost localhost.localdomain localhost4
::1
192.168.56.20
ldaphost1.in.ibm.com ldaphost1
192.168.56.21
192.168.56.252 ldapserver.in.ibm.com ldapserver
[root@ldaphost1 ~]# ssh ldaphost2 'cat /etc/hosts'
127.0.0.1
::1
192.168.56.21
192.168.56.20
192.168.56.252 ldapserver.in.ibm.com ldapserver
localhost4.localdomain4
Listing 29: /etc/hosts file from server ldaphost1 and ldaphost2
10 Query ISDS server using virtual IP

Now all the clients should point to the ISDS server using a virtual IP 192.168.56.252 with a hostname
of ldapserver.in.ibm.com. You can perform the search operation in Listing 30: idsldapsearch using
virtual IP/hostnameto check whether everything is working as it should.
[root@ldaphost2 ~]# idsldapsearch -h ldapserver.in.ibm.com -s base objectclass=*

namingcontexts
namingcontexts=CN=SCHEMA
namingcontexts=CN=LOCALHOST
namingcontexts=CN=IBMPOLICIES
namingcontexts=O=SAMPLE
namingcontexts=O=IBM,C=IN
Listing 30: idsldapsearch using virtual IP/hostname
11 Resources
Learn
For assistance in installing IBM Security Directory Server, read the IBM Security Directory Server
Installation and Configuration Guide.
Visit the IBM Security Directory Server Version 6.3.1 information center,where you can find
information about installing, configuring, administering, and using the Directory Server.
Visit the official heartbeat documentation
Read "Setting up Tivoli Directory Server replication using the command line" for help with
configuring replication on an IBM Security Directory Server.
Get product and technologies
Get the IBM Security Directory Server Version 6.3.1 here.
Download the EPEL yum repository rpm package.
You can also download the source code for heartbeat at http://www.linux-ha.org/wiki/Downloads
Download the tweaked version of ha_propagate script.

High Availability of IBM Security Directory Server Using Heartbeat v1

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

High Availability of IBM Security Directory Server Using Heartbeat v1

Caricato da

Copyright:

Formati disponibili

High Availability of IBM Security Directory

Server using Heartbeat

Figure 12: Select replication suffix..........................................................................................................13

Listing 30: idsldapsearch using virtual IP/hostname...............................................................................38

Figure 1: Directory operations under normal condition

Figure 2: Directory operations under failover condition

3 About IBM Security Directory Server (ISDS)

[root@ldaphost1 ~]# idsilist -a

Listing 1: "idsilist" output from ldaphost1

[root@ldaphost2 ~]# idsilist -a

Listing 2: "idsilist" output from ldaphost2

Listing 3shows sample output from ldaphost1:

Listing 3: ISDS and admin server running on ldaphost1

Listing 4 shows sample output from ldaphost2:

Listing 4: ISDS and admin server running on ldaphost2

6 IBM Security Directory Server replication

provided below in "Resources."

6.1 Log in to Web Administration Tool

Figure 3: WAT Console administration login

6.2 WAT introduction page

Figure 4: WAT console introduction page

6.3 Adding servers to WAT

Figure 5: Adding servers to WAT

Figure 6: Adding servers to WAT(2)

Listing 5: ISDS instance list

Figure 7: WAT servers list

6.4 Log in to the ISDS instance to configure replication

Figure 8: WAT Directory server login

6.5 Setting up the replication

Figure 9: WAT Introduction page

Figure 10: Add replication subtree

Now click on Browse... to navigate through the

Figure 11: Browse subtree(1)

Figure 12: Select replication suffix

Listing 6: Information added to DIT after adding replication suffix

Figure 14: Show topology

Figure 15: Show topology for o=ibm,c=in

Figure 17: Add peer server to replication topology

Figure 18: Add credentials (1)

Figure 19: Add credentials (2)

Figure 20: Add credentials (3)

Figure 21: Select credential

Listing 7: Bind credential

Figure 22: Add master server

Figure 23: Add credential on peer server

Listing 8: Supplier bind credential information

Figure 24: Collect peer agreement information

Figure 25: Enter ldaphost1 credential information

Listing 9: Information added to server ldaphost1

Listing 10: Information added to server ldaphost2

Figure 26: Replication setup success information

6.5.1 Check replication-related entries in ISDS using command line

6.6 Synchronize data between two replication servers

6.6.1 Exporting data

Listing 11: Exporting data with seed and salt

How to obtain salt and seed of an ISDS server

Obtain the encryption salt value by performing the following search:

Listing 12: ISDS server's crypto salt

6.6.3 Importing data

Listing 14: Importing data into the server

6.7 Resume replication queue

Figure 27: Resume replication queue from ldaphost1 to ldaphost2

Figure 28: Resume replication queue from ldaphost2 to ldaphost1

configure high availability (HA) for these servers.