Configuring+HACMP+5 4

IBM STG Technical Conference
IBM Systems and Technology Group

Technical Conference
v
New Orleans, Louisiana

February 5 - 9, 2007
© 2007 IBM Corporation

Configuring HACMP 5.4

v
Bill Miller
Instructor - IBM Training
© 2007 IBM Corporation

Objectives
At the end of this presentation, you should be
able to:
• Describe the functions that HACMP provides
• Plan an HACMP implementation
• Configure HACMP to make an application
highly available
3 © 2007 IBM Corporation

What is HACMP for AIX?

• Software that attempts to maintain application
availability
– Application “fallover” performed if necessary
• Local and Remote “fallover” supported
• Relies on networking (IP/non-IP) for monitoring
• Systems access common storage (non-XD)
HACMP HACMP / XD
Site A Site B

HACMP Architecture
• Redundant Hardware / Duplicate systems
• Eliminate Single Points of Failure (SPOF)
IP VG
APP

How does it work?

• Application is up on one node
• Failure occurs causing the application to fallover
• Bring original node back up
• Fallback is at your discretion
IP VG
APP

Network Planning
• Multiple NICs per system in same VLAN/ broadcast domain
– Using same subnet mask, different subnets
– Addresses must be in /etc/hosts on each HACMP node
• IP address “moved” via IPAT
– IP Address Takeover
• Aliasing (recommended, requires Gratuitous ARP)
• Replacement (Hardware Address Takeover possible)
• Non-IP network(s) “required”
n1boot1 n1boot2 Non-IP n2boot1 n2boot2

en0 en1 en0 en1
n1adm Service n2adm

IP Addr

Network Configuration
• Use smitty chinet to place addresses on interfaces
– Common netmask, different subnets for each NIC pair
• Addressing shown is for IPAT via Aliasing
– Service address would have to be on one of the boot subnets for IPAT
via Replacement
• Create an Enhanced Concurrent Mode Volume Group for your non-
IP network
Netmask = 255.255.255.0
192.168.1.1 192.168.2.1 Non-IP 192.168.1.2 192.168.2.2

en0 en1 dhbvg en0 en1
10.1.10.1 10.1.10.5 10.1.10.2

Storage Planning
• Systems that will provide highly available application
must all have access to LUNs containing all
application data
– Enhanced Concurrent Mode Volume Groups allow for faster
disk takeover
• Unique names must be used for all shared LVM
components
• Pay attention to:
– PVIDs
– Multi-pathing software
– Driver/Firmware/Microcode/HACMP code levels
• Decide where software will be installed
– Non-shared disk (internal to each server)
– Shared disk (one place to update and maintain)

Storage Configuration
• Use standard AIX commands
– Can use C-SPOC once networking is defined
Create the VG on first node Import VG on other nodes
#mkvg -y‘appvg' -s'16' '-n' -V'100' '-C' hdisk3 #importvg -V100 -y appvg hdisk3
#mklv -y‘app10lv' -t'jfs2' appvg 10 #chvg -an appvg
#crfs -v jfs2 -d‘app10lv' -m'/app10‘ #varyoffvg appvg
#unmount /app10
#varyoffvg appvg
Netmask = 255.255.255.0
192.168.1.1 192.168.2.1 Non-IP 192.168.1.2 192.168.2.2

en0 en1 dhbvg en0 en1
10.1.10.1 10.1.10.5 10.1.10.2
appvg

Application Considerations
• All components of application must be started
and stopped via scripts
– No manual intervention
• Write and test scripts, then define to HACMP
• Strongly recommend HACMP Application
Monitors
• May need to implement pre/post-event scripts
• HACMP Smart Assists can help
– Only certain applications at this time

Cluster Creation
• Initialization and Standard Configuration Method is
recommended
• Multiple steps, on one node
– Create Cluster, Node and Networks
– Define Service Address(es)
– Define Application Server(s)
– Define Resource Group(s)
– Define resources in Resource Group(s)
– Verify and Synchronize to other nodes
– Start Cluster Services and verify resource acquisition
• Can use the Two-Node Configuration Assistant

– Not a replacement for knowing what it takes to make a
cluster

The Standard Configuration Method
Initialization and Standard Configuration

Move cursor to desired item and press Enter.
Configuration Assistants
Configure an HACMP Cluster and Nodes
Configure Resources to Make Highly Available
Configure HACMP Resource Groups
Verify and Synchronize HACMP Configuration

Display HACMP Configuration
HACMP Cluster Test Tool
F1=Help F2=Refresh F3=Cancel F8=Image

F9=Shell F10=Exit Enter=Do
What we’ll see, step-by-step

Start with Configure an HACMP Cluster and Nodes

Add Nodes to an HACMP Cluster

Configure Nodes to an HACMP Cluster (standard)
Type or select values in entry fields.

Press Enter AFTER making all desired changes.
[Entry Fields]
* Cluster Name [ibmcluster]
New Nodes (via selected communication paths) [n1boot1 n2boot1] +

Currently Configured Node(s)


Now Define Highly Available Resources




Start With Service Addresses

smit hacmp -> Initialization and Standard Configuration
Configure Service IP Labels/Addresses

Configure Application Servers
Configure Volume Groups, Logical Volumes and Filesystems
Configure Concurrent Volume Groups and Logical Volumes


Adding Service IP Label(s) Resource

Add a Service IP Label/Address

Change/Show a Service IP Label/Address
Remove Service IP Label(s)/Address(es)


Add your Service Label (1 of 2)

Add a Service IP Label/Address (standard)

[Entry Fields]
* IP Label/Address [] +
* Network Name [] +
+--------------------------------------------------------------------------+
¦ IP Label/Address ¦
¦ ¦
¦ Move cursor to desired item and press Enter. ¦
¦ ¦
¦ (none) ((none)) ¦
¦ n1adm (10.1.10.1) ¦
¦ n1adm (10.1.10.2) ¦
¦ xweb (10.1.10.5) ¦
¦ ¦
¦ ¦
¦ F1=Help F2=Refresh F3=Cancel ¦
¦ F8=Image F10=Exit Enter=Do ¦
¦ /=Find n=Find Next ¦
+--------------------------------------------------------------------------+

Add your Service Label (2 of 2)

Add a Service IP Label/Address (standard)

[Entry Fields]
* IP Label/Address [xweb] +
* Network Name [net_ether_01] +
F1=Help F2=Refresh F3=Cancel F4=List

F5=Reset F6=Command F7=Edit F8=Image
Repeat the process for every service address to be configured

Continue With Application Servers

smit hacmp -> Initialization and Standard Configuration

Configure Volume Groups, Logical Volumes and Filesystems
Configure Concurrent Volume Groups and Logical Volumes


Add your Application Server (1 of 2)

Add an Application Server

Change/Show an Application Server
Remove an Application Server


Add your Application Server (2 of 2)

Add Application Server

[Entry Fields]
* Server Name [xwebserver]
* Start Script [/usr/local/scripts/startxweb]
* Stop Script [/usr/local/scripts/stopxweb]

Repeat the process for every Application Server to be configured

Adding your Resource Group
Add a Resource Group

Change/Show a Resource Group
Remove a Resource Group
Change/Show Resources for a Resource Group (standard)


Setting Name, Nodes, and Policies
*Resource Group Name [xwebgroup]

*Participating Nodes(Default Node Priority) [n1 n2]
Startup Policy Online On Home Node O> +
Fallover Policy Fallover To NextPrio> +
Fallback Policy Fallback To Higher Pr> +

Repeat the process for every Resource Group to be configured

Adding Resources to your RG (1 of 2)


Change/Show a Resource Group
Remove a Resource Group
Change/Show Resources for a Resource Group (standard)
+----------------------------------------------------------------------+
¦ Select a Resource Group ¦
¦ ¦
¦ ¦
¦ xwebgroup ¦
| ywebgroup ¦
¦ ¦
¦ F8=Image F10=Exit Enter=Do ¦
¦ /=Find n=Find Next ¦
+----------------------------------------------------------------------+

Adding Resources to your RG (2 of 2)

Change/Show All Resources and Attributes for a
Resource Group
[Entry Fields]
Custom Resource Group Name xwebgroup
Participating Node Names (Default Node Priority) n1 n2
Startup Behavior Online On First Avail>

Fallover Behavior Fallover To Next Prio>
Fallback Behavior Fallback To Higher Pr>
Service IP Labels/Addresses [xweb] +

Application Servers [xwebserver] +
Volume Groups [xwebvg] +
Use forced varyon of volume groups, if necessary false +
Filesystems (empty is ALL for VGs specified) [] +
Repeat previous two steps for every configured Resource Group

Other important steps

• Configure non-IP network
– Configure the end-points of the network(s) and
synchronize
– This creates the network (monitoring will begin if
Cluster Services are active)
• Configure Persistent Address(es)
Both are access through Extended

Configuration menus

Synchronize and Test the Changes




Starting Cluster Services

# smit clstart
Start Cluster Services

[Entry Fields]
* Start now, on system restart or both now +
Start Cluster Services on these nodes [n1,n2] +
* Manage Resource Groups Automatically +
BROADCAST message at startup? true +
Startup Cluster Information Daemon? true +
Ignore verification errors? false +
Automatically correct errors found during Interactively +
cluster start?


Cluster Status
• Daemon status
# lssrc –ls clstrmgrES
• Look for ST_STABLE, may need to run multiple times to ensure
state is not changing
• Resource Group status
# clRGinfo
• Expect what state Resource Groups should be in, validate that
• Node status
– clstat
– WebSMIT
– SNMP
• Finally, read the logs (one of my favorites)

Verifying That Cluster Services Have Started

n1 # clcheck_server grpsvcs;print $?
n1 # clstat -a
1
clstat - HACMP Cluster Status Monitor Note: An rc=1 means cluster services is active
-------------------------------------
Cluster: ibmcluster (1156578448) usa # lssrc -ls clstrmgrES

Wed Aug 30 11:16:19 2006
State: UP Nodes: 2 Current state: ST_STABLE
SubState: STABLE
Node: n1 State: UP
Interface: n1boot1 (2) Address: 192.168.1.1
State: UP
Interface: n1boot2 (2) Address: 192.168.2.1 First three rules
State: UP
Interface: n1_hdisk5_01 (0) Address: 0.0.0.0 1. patience
State: UP
Interface: xweb (2) Address: 10.1.5.10 2. patience
State: UP
Resource Group: xwebgroup State: On line 3. patience
Also consider using the cldump command. This relies solely on SNMP to get the
current cluster status.
Stopping Cluster Services

# smit clstop
Stop Cluster Services
[Entry Fields]
* Stop now, on system restart or both now +
Stop Cluster Services on these nodes [usa] +
BROADCAST cluster shutdown? true +
* Select and Action on Resource Groups Bring Resource Groups> +
+--------------------------------------------------------------------------+
¦ Shutdown mode ¦
¦ ¦
¦ ¦
¦ Bring Resource Groups Offline ¦
¦ Move Resource Groups ¦
¦ Unmanage Resource Groups ¦
¦ ¦
F1¦ F8=Image F10=Exit Enter=Do ¦
F5¦ /=Find n=Find Next ¦
F9+--------------------------------------------------------------------------+
Verifying That Cluster Services Have Stopped

n1 # tail -2 /tmp/hacmp.out
clexit.rc : Normal termination of clstrmgrES. Restart now.
0513-059 The clstrmgrES Subsystem has been started. Subsystem PID is 483466.
n2 # clstat -a
clstat - HACMP Cluster Status Monitor
n1 # lssrc -ls clstrmgrES
-------------------------------------
Current state: ST_INIT
Cluster: ibmcluster (1156578448)
Wed Aug 30 10:44:20 2006
State: UP Nodes: 2
SubState: STABLE
Same three rules apply
Node: n1
Interface: usaboot1 (2)
State: DOWN
Address: 192.168.1.2
1. patience
Interface: usaboot2 (2)
State: DOWN
Address: 192.168.2.2
2. patience
State: DOWN 3. patience
n1 # tail -1 /tmp/clstrmgr.debug
Wed Aug 30 10:31:54 code is 0 - exhale our dying breath and count on the good graces of SRC to reincarnate us !
Summary
• It’s not easy, but it’s not brain surgery
• Proper planning is important
• Document your intent and keep the document
current
• Test, test, test
• Please fill out the evaluation and enjoy the

rest of the conference

Configuring+HACMP+5 4

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Configuring+HACMP+5 4

Caricato da

Copyright:

Formati disponibili

IBM STG Technical Conference

IBM Systems and Technology Group

New Orleans, Louisiana

© 2007 IBM Corporation

Configuring HACMP 5.4

© 2007 IBM Corporation

3 © 2007 IBM Corporation

What is HACMP for AIX?

4 © 2007 IBM Corporation

5 © 2007 IBM Corporation

How does it work?

6 © 2007 IBM Corporation

n1boot1 n1boot2 Non-IP n2boot1 n2boot2

n1adm Service n2adm

7 © 2007 IBM Corporation

192.168.1.1 192.168.2.1 Non-IP 192.168.1.2 192.168.2.2

10.1.10.1 10.1.10.5 10.1.10.2

8 © 2007 IBM Corporation

9 © 2007 IBM Corporation

192.168.1.1 192.168.2.1 Non-IP 192.168.1.2 192.168.2.2

10.1.10.1 10.1.10.5 10.1.10.2

10 © 2007 IBM Corporation

11 © 2007 IBM Corporation

• Can use the Two-Node Configuration Assistant

12 © 2007 IBM Corporation

The Standard Configuration Method

Initialization and Standard Configuration

Verify and Synchronize HACMP Configuration

F1=Help F2=Refresh F3=Cancel F8=Image

What we’ll see, step-by-step

13 © 2007 IBM Corporation

Add Nodes to an HACMP Cluster

Type or select values in entry fields.

New Nodes (via selected communication paths) [n1boot1 n2boot1] +

F1=Help F2=Refresh F3=Cancel F8=Image

14 © 2007 IBM Corporation

Now Define Highly Available Resources

Move cursor to desired item and press Enter.

Verify and Synchronize HACMP Configuration

F1=Help F2=Refresh F3=Cancel F8=Image

15 © 2007 IBM Corporation

Start With Service Addresses

Configure Resources to Make Highly Available

Move cursor to desired item and press Enter.

Configure Service IP Labels/Addresses

F1=Help F2=Refresh F3=Cancel F8=Image

16 © 2007 IBM Corporation

Adding Service IP Label(s) Resource

Move cursor to desired item and press Enter.

Add a Service IP Label/Address

F1=Help F2=Refresh F3=Cancel F8=Image

17 © 2007 IBM Corporation

Add your Service Label (1 of 2)

Type or select values in entry fields.

18 © 2007 IBM Corporation

Add your Service Label (2 of 2)

Type or select values in entry fields.

F1=Help F2=Refresh F3=Cancel F4=List

Repeat the process for every service address to be configured

Continue With Application Servers

Move cursor to desired item and press Enter.

Configure Service IP Labels/Addresses

F1=Help F2=Refresh F3=Cancel F8=Image