Avaya Linux Troubleshooting Guide

Application Note:
Troubleshooting Guide v1.0 for the Avaya

S8300, S8500, and S8700 Media Servers
ABSTRACT
This guide provides basic information about some of the commands, directories, and files
that can help you to troubleshoot the Linux-based servers that run Avaya Communication
Manager.
Application Note
February 2004
_____________________________________________________________________________________________
Shenoy; Miller; © 2004 Avaya Proprietary. 1
Feb 2004 All rights reserved.
Troubleshooting Guide v1.0 for the Avaya S8300, S8500,
and S8700 Media Servers
Table of Contents
1. Introduction ...........................................................................................................................5
2. Intended audience ..................................................................................................................5
3. References..............................................................................................................................5
4. Versions of this document.......................................................................................................5
5. Acronyms...............................................................................................................................5
6. Introduction to Linux .............................................................................................................5
6.1. Version numbers.................................................................................................................. 5
6.2. Shells and shell scripts ......................................................................................................... 6
6.2.1. Types of shells ................................................................................................ 6
6.2.2. Example of a shell command within Communication Manager ..... 6
7. Commonly used text editors within Linux ..............................................................................6
7.1. The vi editor......................................................................................................................... 6
7.1.1. Movement commands .................................. 7
7.1.2. Text entry commands .................................. 7
7.1.3. Cut, copy, paste, and undo commands ...................... 7
7.1.4. Search and replace commands ............................ 8
7.1.5. Save and quit commands................................ 8
7.2. The emacs editor.................................................................................................................. 8
7.2.1. General commands.................................... 9
7.2.2. Movement commands .................................. 9
7.2.3. Cutting and pasting commands ........................... 9
8. Commonly used Linux and Communication Manager commands ........................................10
8.1. Commands that are related to Communication Manager ................................................. 17
9. Commonly accessed directories and files for troubleshooting................................................24
9.1. The /etc/opt/ecs directory ...................................................................................................24
9.2. The /etc/hosts file ................................................................................................................24
9.3. The /etc/opt/defty directory ................................................................................................24
9.4. The/var/log/ecs directory ....................................................................................................25
9.5. The /var/log/messages file ...................................................................................................25
9.6. The /var/crash directory.....................................................................................................25
9.7. The /var/log/defty/dumps directory....................................................................................25
10. Core dumps and mini core dumps ........................................................................................26
_____________________________________________________________________________________________
Feb 2004 All Rights Reserved.
11. Introduction to commonly encountered entities within Communication Manager................26
12. Appendix A: Screenshots and additional information on Communication Manager commands
29
12.1. The /sbin/ifconfig command ...............................................................................................29
12.1.1. Avaya S8300 server ................................. 29
12.1.2. Avaya S8700 servers (Multi-Connect and IP-Connect) ......... 30
12.2. The almclear command ...................................................................................................... 31
12.3. The almdisplay command................................................................................................... 31
12.4. The fasttop command .........................................................................................................32
12.5. The modserv command ......................................................................................................33
12.5.1. Modem not connected ............................... 33
12.5.2. Modem administered by disabled ....................... 33
12.5.3. Modem administered and ready for access ................. 33
12.6. pingall .................................................................................................................................33
12.7. The restartcause command ................................................................................................34
12.8. The server command ..........................................................................................................35
12.8.1. Avaya S8700 server ................................. 35
12.8.2. Avaya S8300 and S8500 servers......................... 35
12.9. The statapp or systat command .........................................................................................36
12.9.2. Avaya S8300 server (with IA770 on-board messaging installed) .. 36
12.10. The statuslicense command ............................................................................................37
12.10.2. Avaya S8300 server ................................. 37
12.11. The swversion command ................................................................................................38
13. Appendix B: Server behavior on interchange .......................................................................40
14. Appendix C: Connecting to an S8300, S8500, or S8300 server from a Windows-based personal
computer.........................................................................................................................................43
14.1. To configure the remote PC for PPP modem connection ..................................................43
14.2. Use Windows for PPP modem connection .........................................................................43
14.3. Using telnet .........................................................................................................................44
14.4. Using ftp..............................................................................................................................45
14.4.1. Example FTP session using the commands listed above : ....... 45
15. Appendix D: A case study in which a segmentation fault causes switch reset........................48
15.1. Log in and connect to the system........................................................................................48
15.2. Examine the logs.................................................................................................................48
15.3. Analyze the problem...........................................................................................................48
_____________________________________________________________________________________________
_____________________________________________________________________________________________
1. Introduction
This guide provides basic information about some of the commands, directories, and files
that can help you to troubleshoot the Linux-based servers that run Avaya Communication
Manager.
2. Intended audience
This application note is for TSO, ITAC, and Business Partner technicians, and others who
must troubleshoot the S8300, the S8500, and the S8700 Media Servers.
3. References
The Linux Documentation Project: http://www.tldp.org/
The Avaya Support Web site: http://support.avaya.com/
4. Versions of this document

Version # Change
1.0 Original version
5. Acronyms
Acronym Meaning
CLI command line interface
FTP File Transfer Protocol
IPSI IP server interface
LSP local spare processor
MAP maintenance action process
MO drive magneto-optical drive
SAT system access terminal
SNMP Simple Network Management Protocol
6. Introduction to Linux
6.1. Version numbers
Linux is a freely distributed version of the Unix operating system. Each particular
CD-ROM distribution of Linux has a version number, such as “Red Hat Linux 8.0” or
“SuSE Linux Version 6.0.” These version numbers relate only to a particular distribution.
The Linux kernel, and the other components of Linux, generally have their own version
numbers. The media servers that run Communication Manager 2.0 use Version 2.4.20-AV9,
which is packaged with Linux 8.0.
_____________________________________________________________________________________________
6.2. Shells and shell scripts
The shell is a Linux utility that reads and runs the commands that you type. The shell also
schedules jobs, redirects input and output, and so on. A shell script is a file that contains a
series of shell commands that you run at the shell command prompt.
6.2.1. Types of shells

Many types of shells are available for Linux. Some basic aspects of these shells are the same,
such as the prompt, most of the commands, and so on. The primary difference is the
command language that the scripts within the shells use. Linux shells include the C shell, the
Bourne shell, and Bourne again shell. The Bourne again or bash shell is a variant of the
Bourne shell. The bash shell includes many advanced features such as job control, command
history, a vi-like or an emacs-like interface for command-line editing, and other extensions to
the Bourne shell. The bash shell is the default with Communication Manager.
6.2.2. Example of a shell command within Communication Manager

The following example shows the ls command with the –l option. You run this
command at the bash command prompt. The system displays this command prompt when
you log in to Communication Manager. The command prompt is in the format
login@servername, which in this example is init@saturn2.
init@saturn2> ls –l
total 32
-rw-r--r-- 1 inads susers 27916 Oct 6 11:22 ip2ext.jar
-rw-r--r-- 1 inads susers 244 Sep 29 16:59 mibs.env
When you run commands at the shell, you can invoke older commands or edit previous
commands that you ran at the shell. To invoke earlier commands, type the following
command at the Linux command line interface (CLI), and substitute the name of your text
editor:
set –o <name of text editor>
If you use vi as your text editor, type set –o vi. If you use emacs, type set –o
emacs.
7. Commonly used text editors within Linux
Knowing how to work with text editors is especially useful when you troubleshoot the
S8300, S8500, and S8700 servers The most popular Linux text editors are vi and emacs.
7.1. The vi editor
The vi editor is a Unix text editing program. To start vi, type vi <file name> at the
Unix prompt. If the file name does not exist, vi creates the file for you.
_____________________________________________________________________________________________
The three basic modes of operation in vi are the edit mode, the command mode, and the
visual mode. In this guide, we discuss the edit mode and the command mode. Use the edit
mode to enter data. Use the command mode to move the cursor and modify text. The vi
editor begins in the command mode. To change to the edit mode, use one of the text entry
commands. To change back to the command mode, press ESC. Press Return or Enter after
commands that begin with a colon ( : ), a slash (/), or a question mark (?).
7.1.1. Movement commands
Command Action
h Move the cursor left.
j Move the cursor down.
k Move the cursor up.
l Move the cursor right.
w Move the cursor forward one word.
B Move the cursor backward one word.
H Move the cursor to the top line on the screen.
L Move the cursor to the bottom line on the screen.
1G Move the cursor to the beginning of the file.
G Move the cursor to the end of the file.
Crtl-f Move forward by one screen.
Ctrl-b Move backward by one screen.
:<line#> Move to that line number. For example, : 8
means move to line 8.
Note: In most instances, you can use the cursor keys to move through a document as you
edit the text.
7.1.2. Text entry commands
Command Action
a Append text after the cursor.
A Append text at the end of the line.
i Insert text before the cursor.
I Insert text at the beginning of the line.
o Open the next line for text insertion.
O Open the previous line for text insertion.
R Overwrite the rest of line from the cursor.
r Replace one character.
s Replace one character and enter insert mode.
ESC Return to command mode.
7.1.3. Cut, copy, paste, and undo commands
_____________________________________________________________________________________________
Command Action
cw Replace a word.
c$ Replace to the end of the line.
dw Delete (cut) the word at the cursor.
dW Delete (cut) until the next white space.
dd Delete (cut) the current line.
d$ Delete to the end of the line.
p Paste text after the cursor.
P Paste text before the cursor.
yw Yank (copy) the word at the cursor.
yy Yank (copy) the current line.
u Undo the last change.
x Delete (cut) the character at the cursor.
7.1.4. Search and replace commands
Command Action
/<string> Search forward for
<string>.
?<string> Search backward for <
string >.
n Repeat the last search.
N Reverse the direction of
the last search.
:%s/<pat1>/<pat2>/g Replace all instances of
<pat1> with < pat2>.
7.1.5. Save and quit commands
Command Action
:w Save the file without quitting.
:q! Quit without saving the file.
:e! Return to the last saved version of the file.
:wq Save the file and quit.
:x Save the file and quit.
ZZ Save the file and quit.
7.2. The emacs editor

Within the emacs editor, C-x denotes a control character, such as CTRL-X. To type a
control character, you press and hold down CTRL, and then press the character key. M-x
_____________________________________________________________________________________________
denotes a meta character. To type a meta character, you press ESC, and then press the
character key. You can also press and hold down the meta key, and then press the character
key. On Sun keyboards, the Escape key is ESC (or CTRL-[). The meta key is next to the
spacebar, and is labeled with a small diamond.
7.2.1. General commands
Command Action
C-g Stop a partially typed or an executing command.
C-h t View the emacs tutorial.
C-l Redraw the screen.
C-r Search backward.
C-s Search forward.
C-x i Insert a file.
C-x s Save a file.
C-x u Undo.
C-x C-c Exit the emacs editor.
C-x C-f Change to a new file.
C-x C-w Write to a specified file.
M-x info View emacs info.
7.2.2. Movement commands
Command Action
C-b Move backward one character.
C-f Move forward one character.
C-n Move down one line.
C-p Move up one line.
C-v Move down one screen.
M-v Move up one screen.
7.2.3. Cutting and pasting commands
Command Action
DEL Delete the character before the cursor.
C-@ or Place a mark in text.
C-SPACE
C-d Delete the character under the cursor.
C-k Kill to the end of the line.
C-w Delete from the mark to the current position in
the text.
C-x DEL Kill to the beginning of the sentence.
C-y Yank the previous kill from the kill ring.
_____________________________________________________________________________________________
Command Action
M-DEL Kill to the beginning of the previous word.
M-d Kill to the end of the next word.
M-k Kill to the end of the sentence.
M-z <char> Kill to the next occurrence of <char>.
8. Commonly used Linux and Communication Manager

commands
Command Summary of use
cd Use the cd command to change the current directory. Type:
cd <name of directory>
You are always in a current directory, and can access any directories that
are hierarchically above or below the current directory, as long as you
have the right permissions.
Example: cd games
If the directory games is not located below the current directory, you must
provide the complete path name.
Example : cd /usr/games
To move up one directory, type:

cd ..
exit or Use exit or logout to end a user session. Both commands have
logout the same effect, which is to disconnect from your Linux–based
computer, or end a user session.
Note that leaving your account logged on can be a security concern.

Avaya recommends that you always promptly log out when you finish
using your account, especially if you are logged in as root.
find Use the find command to search for files on your file system.
For example, use find . / -name <file name> -print to

find the file called <file name> on your file system, starting with the
/ (root) directory. The value for < file name > can contain
wildcards, which you introduce with the special characters asterisk (*) or
question mark (?). Use the -print option to print the results to
_____________________________________________________________________________________________
standard output.
The find command has many powerful search options. For example,
you can search by date, size, permissions, or owner. Use the man find
command or the info find command to get more information on
the find command.
The following commands are more complex examples of how to use the
find command to accomplish some useful tasks:
find $HOME -name core -exec rm -f {}
This command starts at your home directory, and finds all files
with names that contain the string “core.” On each file, this
command then performs the action rm –f, which deletes the
file without confirmation. The braces symbol {} stands for the
file that the command finds.
Caution: The rm –f command permanently deletes all files that
match the criterion.
Caution: If you run this command at the highest root directory
(/) level in your system, this command permanently deletes all
files with names that contain “core.”
• find /dev -user "peter" |more
This command prints the file name for all devices that the user "peter"
owns. The |more option displays one page or screen at a time, and
waits for user input to display the next page. Since the default action of
find is to print the file name, you do not have to specify the print option
within the command.
grep Certain patterns in the log messages in /var/log/ecs can provide clues on
some obvious problems that can help you with basic troubleshooting.
Use the grep command to search the ecs log files.
In all of the following examples, we are in the directory /var/log/ecs,
and running the following command:
grep –i string *.log | more
Note that the –i option makes the search case insensitive. Replace
string above with one of the following sample patterns:
• interchange. Displays records in *.log files that include the

word interchange (interchange of the active and standby servers).
• CheckSlot. Helps you debug potential problems with IPSIs,
_____________________________________________________________________________________________
especially when fatal errors are reported.
• ALLOC_BUFF. Tells you when Communication Manager, as a
whole, runs out of buffers. This condition causes an interchange
on the duplicated S8700 servers that is equivalent to a
segmentation fault. ALLOC_BUFF failures, and analysis of the
aftermath, can provide much information on what takes place on
a particular system.
• EIP. Tells you when a segmentation fault occurred in a process.
A segmentation fault causes a restart of some type to occur in the
system, depending on the restart level of the process that
experienced the segmentation fault. Analysis of the segmentation
fault can uncover bugs in the code.
• FATAL. Indicates that an IPSI board entered FATAL FAULT.
This state means that all LAPD links, and therefore all calls, that
are associated with that board fail. Note that in a port network
with duplicated IPSIs, the presence of a FATAL in the log might
not indicate call failures on that port network.
• OVERLOAD (1 or 9). Indicates that the switch is running at
a very high occupancy, and is no longer handling off-hooks. Calls
are being lost if you see this message in the ecs logs.
Many other messages in the ecs log files provide similar help, but are too
numerous to discuss here.
kill Use kill to kill a process that is stopped. Type:
kill <pid>
where pid is the exact number of the process that you want to kill.
See the ps command below for how to find the PID of a process that
you want to kill.
Caution: Be careful when you use the kill command on a
Communication Manager platform. Killing certain processes can have
serious negative effects.
locate Use the locate command to find data files, programs, directories, and
objects that match your search. For example, type:
locate license.txt
The locate command shows you every related file or directory that
matches your search criterion. The result of the locate command can
be a long list. To narrow the search, type:
locate <search term> | more
_____________________________________________________________________________________________
The locate command works quickly, if the locate database is up to
date. The system automatically updates the locate database each night. So,
any new files or commands that were created in the system on the day
that you run locate might not appear in the output. If you have root
privileges, you can log in as root, and run the updatedb command to
update the locate database manually.
logv The logv command puts the selected output into a temporary file. The
logc logv command then puts you in the editor on that file, while the logc
logw command outputs the selected output to standard output. The logw
command watches the logs for a certain pattern until you enter ^C. The
following examples show the use of these commands.
• logv today pcd

This command gives you all the process manager messages from
the debug trace log. The debug trace log is the default for the
logv command.
• logv yesterday lm wd lxsys

This command merges and sorts the debug trace log (lm, for
logmanager), watchdog (wd), and /var/log/messages (lxsys, for
linux syslog) logs from yesterday.
• logc 20031001 > <filename>

This command puts the contents of the debug trace log from Oct
1, 2003 into the file <filename>.
• logw pro=
This command watches the debug trace log for software errors.
This command then prints the errors out as the errors occur until
you enter ^ C.
The following description shows the options that are available for the
logv, the logc, and the logw commands from the Linux CLI of a
server that is running Communication Manager:
logv [OPTIONS]... [LOGS...] [FILTERS]...
logc [OPTIONS]... [LOGS...] [FILTERS]...
logw [OPTIONS]... [LOGS...] [FILTERS]...
Options to modify command operation:

-? produce this message
-c cat the log instead
(set if called with name logc)
-ls list the names and the sizes of log files
-w watch the log instead
_____________________________________________________________________________________________
(set if called with name logw)
-v show the version of the command
-b remove blank lines
-l only the latest file in a log searched
(the default is all)
-s strip off the timestamp
(or other header) on each line
-clear clear the log files
Logs that can be displayed:

all all possible
lm logmanager debug trace (default)
lxboot Linux boot messages
lxcron Linux cron (scheduling) daemon
lxsys Linux syslog
lxsec Linux access security log
lxwtmp Linux login/logout/reboot log
lxx Linux file transfer log
cmds platform command history log
wd watchdog logs
Filters to display events that match a particular criterion. Multiple

applications and times can be given:
App filter events for a particular
application or process(for example, VFM,
audit,NW, ...)
time filter for a particular date or time
yyyy[mm[dd:[HH[MM[SS[mmm]]]]]]
today yesterday
ls Use ls <directory-name> to view a list of the files and
subdirectories in the directory named <directory-name>. If you
omit < directory-name >, the listing is for the current directory.
For example, if you are in the current directory named “games,” and you
type ls, the system lists all files and subdirectories that are in the games
directory.
Examples:
Type ls -ltr to see a list of all files and all directories that are located
in the current directory, sorted by the time of file creation.
This command is especially useful if you are looking for the latest logs in
/var/log/ecs. To view those logs, type ls /var/log/ecs –ltr.
Use the -a option with ls to list hidden files, such as .rc files. To list
hidden files, type ls –a, or ls –altr for a time-sorted list.
man Use the man command to find help for most commands. The man
_____________________________________________________________________________________________
command displays the manual pages, which tell you about all the other
commands. The man command has two forms:
man [section-number] page_name

man -k search_expression
The first form displays the page for a specific command. For example,
man ls displays the manual page for the ls command. Use the section
option to narrow the search to a particular part of the documentation.
In addition to the man command, Linux provides two other ways to get
help with a specific command:
<command_name> -?
<command_name> --help
more Use the more command to view the contents of a text file.
Type more <filename>
Note that if you use more with a nontext file type, the system displays a
list of unrecognizable characters.
ps Use the ps command to displays all current processes. This command is
also directly linked to issues with stopped processes, or "stopped jobs."
Type ps to list all of current processes that are running or stopped.
For example:
PID TTY STAT TIME COMMAND
23036 pl S 0:00 -csh
23070 pl R 0:00 vi
The number under PID is the process identification number.

Type ps –ef <process name> to look up the PID of the process
<process name>. See Section 11 of this document for information
on Communication Manager processes.
pwd Use the pwd command to view the full path name of the directory that
you are currently in. This command is especially useful when you forget
which directory that you changed to.
uname Use the uname command to print system information. To use the -a
option to include all information, such as kernel, nodename, and so on,
type:
uname -a
whatis Use the whatis command to determine what a specific command does.
At the command line, type:
whatis <command>
_____________________________________________________________________________________________
The result is usually a one-line summary of the command.
which Use the which command to show the path name to a command. Type:
which <command>
_____________________________________________________________________________________________
8.1. Commands that are related to Communication Manager
Command Brief description Caveats and frequently used
options
almcall Set the telephone
numbers to call the
INADS alarm receivers.
Enable or disable alarm
abbreviation on those
numbers, and the interval
between retries.
almdisplay Display server alarms.
almclear Clear server alarms. Typically used with the –a
option. Clears server alarms only.
Communication Manager alarms
must first be cleared within
Communication Manager before
this command actually clears the
alarms at the Linux shell.
almenable Enable and disable dial
out and SNMP alarm
origination.
almnotif Set "restart notification,”
“clear alarm notification,”
and "suspension
threshold."
almsnmpconf Administer SNMP
alarming to INADS.
almsuppress Suppress or unsuppress Use the -s option with the
alarm origination, or ((y)es) parameter to suppress
check the state of alarm alarm origination for 30 minutes.
suppression.
corevector Request or clear core
dump requests. The core
dump files are generated
prior to the execution of
the restart requested. Use
corevector -? for
detailed information.
dhelp List popular Linux
commands for the servers
that are running
Communication Manager
dsat Run a system access
_____________________________________________________________________________________________
options
terminal (SAT) session for
Communication Manager.
ftpserv Configure FTP service Use –f on/off to enable or
and access. disable FTP service. Use –a
on/off to enable or disable
anonymous FTP.
Note: This command works either
when you are logged in as root, or
when you use sudo to run the
command (for example, sudo
ftpserv –a on).
ipsiversion Get the version of the Use –a for all IPSIs. Use–c for
IPSIs that are in the a particular IPSI.
system.
loginreport Report on Linux logins. Use –a for all login reports. Use –sl
for successful Linux logins. Use –ul
for unsuccessful Linux logins. Use –sd
for successful CM logins. Use –ud
for unsuccessful CM logins. Use
–lo for login lockouts.
loadlicense Command line version of
the utility to load the
license file.
loadpwd Command line version of
the utility to load the
Avaya authentication file.
This file is also called the
password file.
locktrans Lock translations so that See also unlocktrans.
no save translations can
be performed during the
period of the lockout.
mo Perform maintenance that Use –c to check the MO drive.
is related to the magneto- Use –l to list the contents of the
optical (MO) drive on the MO drive. Use –g to get a file
S8700 platforms. from the drive.
patch_apply Apply an “installed” patch Communication Manager 1.1 and
to Communication early 1.2 systems require the sudo
Manager (see command before the
patch_install). patch_apply command, for
example, sudo patch_apply.
In servers that run the later 1.2
_____________________________________________________________________________________________
options
version and all 1.3 versions of
Communication Manager, the
sudo command is not required.
Also, with Communication
Manager 2.0, the
patch_install command no
longer exists. See
update_activate for more
information.
patch_info Provides information With Communication Manager 2.0,
about a patch that is the patch_info command no
already installed. longer exists. See update_info
for more information.
patch_install Installs the patch into the Communication Manager 1.1 and
system (see early 1.2 systems require the sudo
patch_apply). command before the
patch_install command,
for example, sudo
patch_install. In servers
that run the later 1.2 version and
all 1.3 versions of Communication
Manager, the sudo command is
not required. Also, with
Communication Manager 2.0, the
patch_install command no
longer exists. See
update_unpack for more
information.
patch_remove Script removes a Communication Manager 1.1 and
previously activated patch early 1.2 systems require the sudo
on the system. For a list command before the
of patches that are patch_remove command, for
installed in the system, use example, sudo
patch_show. patch_remove. In servers that
run the later 1.2 version and all 1.3
versions of Communication
patch_remove command no
longer exists. See
update_remove for more
information.
_____________________________________________________________________________________________
options
patch_show Lists the patches that are Communication Manager 2.0, the
installed on the system. patch_show command no
longer exists. See update_show
for more information.
patch_uninstall Script uninstalls a patch Communication Manager 1.1 and
from the target machine. early 1.2 systems require the sudo
For a list of patches that command before the
are installed in the system, patch_uninstall command,
use patch_show. for example, sudo
patch_uninstall. In servers
that run the later 1.2 version and
all 1.3 versions of Communication
patch_uninstall command
no longer exists. See
update_deactivate for
more information.
pingall With various options, Mostly applies to the S8700 and
provides the ability to the S8500 platforms.
ping certain interfaces.
productid Query or set the product Use -p to set the Communication
ID for Communication Manager product ID. Use -m to
Manager or Messaging set the Messaging product ID.
products.
resetipsi Reset one or more IPSI Use -a to reset all IPSIs. Use -c
boards. to reset a particular IPSI. This
command applies only to the S8700
and the S8500 platforms.
restartcause Displays the list of the
known restarts that
occurred on Definity© or
Communication Manager.
sat See dsat.
save_trans Equivalent of the
save_trans command
that you run from the
SAT.
serialnumber Responds with the serial Use -l to report the serial
number in the license file, number from the license file. Use
or detected in the network -q to force a query of IPSIs or
by the server. G700s for the serial numbers.
_____________________________________________________________________________________________
options
server Display server duplication Use -b busyout this server. Use

status if applicable, and -c for continuous looping of
the simultaneous display status updates. Use -i to
of status for each server, interchange active and standby
along with the overview servers. Use -r to release this
of server health. server if previously busied out. Use
-f (with –i) to force an
interchange if necessary.
setnic Primarily used on servers Use -q <interface> to query
that are running the current settings for
Communication Manager <interface >.
to set the speed and Caution: You can use setnic
duplex settings on the to query the interface
NIC of the server. Speed/Duplex settings. However,
if you use setnic to set the
Speed/Duplex of an interface, that
particular interface will be reset. Be
careful about doing this on a
customer system, especially on an
active server.
sudo Use to run some
commands as root
temporarily. This
command is usually used
by customers or service
logins that are not root.
start Usually used to start Use -a to start all processes.
Communication Manager Use –c for continuous update of
processes. output.
statapp Verify what See systat.
applications are currently
running on the servers.
statuslicense Check the current license Use -v (for verbose) to obtain
status. more information.
statusserver This command is similar See server.
to the server command
stop Stop all Communication Use -a to stop all processes.
Manager processes. Use –c for continuous update of
output. Use –f to force a
process to shut down if necessary.
swversion Check the current Use -r to check the standby
_____________________________________________________________________________________________
options
software version on the partition. Use -v to check the
active or the standby Communication Manager version
partitions. Note: You string. Use -l to check the
must be sroot to run Communication Manager load
portions of this command string. Use -a to check all fields.
successfully.
systat Get an update of the Use –c for continuous update of
processes that are output. Use -d to list all
currently running for applications that are not
Communication Manager. functioning. Use -D to exit
continuous update when no
processes are functioning. Use -U
to exit continuous update when all
processes are functioning. Use –l
to list the status of the applications
given. Use –s to show the status
of only the named applications (see
-l .) See also statapp.
telnetenable Configure the telnet Use -d to disable telnet service.
service for the IPSIs. Use –c [cab#carrier] to target an
IPSI board with cabinet number
and carrier ID. Use
-p <ip_address> to target an
IPSI board using its IP address.
testdupboard Test the duplication Use –s to run the short test.
memory card on the This option is the default.
S8700 server.
testinads Test the functionality of
alarm processing between
the system and INADS.
testled Test all the LEDs in the Use –a: to test LEDs on both
server. the server and the duplication
memory card LEDs
Use –d: to test the Trans LED
on the duplication memory card.
Use –s: to test the LEDs on the
server. This command applies only
to the S8700 and the S8500
platforms.
testmodem Tests the modem that is Use –s (short) to run the
connected to the server handshake and the offhook tests.
that is running Use –t to specify a certain
Communication Manager. component to be tested.
_____________________________________________________________________________________________
options
uname Print system information. Use –a to print all information,
such as the kernel, the node name,
and so on.
unlocktrans Unlock the “locked” See also locktrans.
translations. Translation
saves will now be
successful.
update_info Display information about Usage:
the specified update. This update_info <update-
command replaces the id>
patch_info command Note: In Communication Manager
that was used prior to 2.0, the term patch is changed to
Communication Manager software update.
2.0.
update_show Display information about Use –a: to display activated
the specified update. This updates only. Use –u: to display
command replaces the unpacked updates only. Use –h:
patch_show command to display help for this command.
that was used prior to
2.0.
update_unpack Unpack the software
update. This command is
the step prior to
“activating” a software
update. This command
replaces the
patch_install
command that was used
prior to Communication
Manager 2.0.
update_activate Activates a previously
unpacked update on the
server. This command
replaces the
patch_apply
Manager 2.0.
update_deactivate Deactivates a previously
activated update on the
server. This command
replaces the
_____________________________________________________________________________________________
options
patch_apply
Manager 2.0.
upate_remove Permanently removes a update_remove <update-
“deactivated” update from id> to remove the update. Use
the server. This command <update-id>
replaces the update_remove –a to
patch_uninstall remove all updates.
Manager 2.0.
vilog Run vi and open current
log files.
9. Commonly accessed directories and files for

troubleshooting
This section describes some useful files, and some common things to look for in log files
that can be quick indicators of problems.
9.1. The /etc/opt/ecs directory

This directory contains several important files:
• ecs.conf
This file is the configuration file for the switch. This file is essential for
Communication Manager Applications to run correctly. Flags that are set incorrectly
in this file can cause numerous problems in the switch.
• servers.conf
This file contains information on the IP addresses of the servers and the control
networks. This information is useful for troubleshooting possible network problems.
9.2. The /etc/hosts file
This file contains the IP addresses of all IPSIs, Cajun-family devices, and servers in the
system. This information is useful for troubleshooting possible network problems.
9.3. The /etc/opt/defty directory

This directory contains the translation files, which are named xln1 and xln2. The xln1 and
xln2 files are identical in all S8X00 servers, and are shadowed in the S8700 server. The
_____________________________________________________________________________________________
translation files store all the information that the switch uses with regard to stations, trunks,
and so on. Beginning with Communication Manager 2.0, the xln2 file is no longer used.
This directory also contains an lspList. This file is usually 0 bytes long, unless one or more
local spare processors (LSPs) are registered to this server. If LSPs are registered, this file
contains the IP address of the LSPs to which Communication Manager has tried to send the
translation files.
9.4. The/var/log/ecs directory

This directory that contains most of the log files that the switch generates. This directory
contains three very useful types of files.
• ecs log files
These log files are marked by the date on which the log files occur. These files
provide information from Communication Manager processes.
• Commandhistory
This file contains the history of commands that are issued on the server. This file
shows such things as when server interchanges were done, when patches were
applied, and when servers were started and stopped. Note that this file does not
record every command that is run at the Linux CLI.
• wdlog
This file is the watchdog log, the process in Communication Manager Applications
that watches over all other processes to ensure proper behavior. This log outputs
occupancy profiles on a per-process basis if the system is running at high occupancy.
9.5. The /var/log/messages file

This file contains more information about system behavior, including information on
modems, security, and traps.
9.6. The /var/crash directory

If the core-vector is set on a server that is running Communication Manager, a core dump is
generated. This directory contains core dumps on Linux-based servers. See Section 10,
“Core dumps and mini core dumps,” for some basic information about core dumps.
9.7. The /var/log/defty/dumps directory
_____________________________________________________________________________________________
If the core vector is not set on a server that is running Communication Manager, a mini core
dump might be generated. As the name suggests, the mini core dump is a smaller version of
the core dump. This directory contains core dumps on Linux-based servers. See Section 10,
“Core dumps and mini core dumps,” for some basic information about mini core dumps.
10. Core dumps and mini core dumps
A core dump is a file that contains a snapshot of the memory image of the server at the time
that the core dump is generated. A core dump is required to debug system failures in depth.
System failures can vary from a single process restart to a reload of the server. To generate a
core dump, you set a flag in the low-level maintenance monitor (LMM). This flag can be
enabled or disabled. When enabled, this flag can generate core dumps under various
conditions. On Linux-based servers, the /var/crash directory contains core dumps.
A mini core dump is usually generated without setting any flags. However, a mini core dump
generates less useful information than a core dump. On Linux-based servers, the
/var/log/defty/dump contains mini core dumps.
11. Introduction to commonly encountered entities within

This section describes some of the many processes that comprise Communication Manager.
All messages in the /var/log/ecs log files come from one of these entities. The following
list is not a complete list of all the processes, and these descriptions are very basic. However,
even a general idea of what these processes are, and how these processes communicate with
one another, can help you troubleshoot problems on the system.
• capro. The call-processing process for Communication Manager Applications.

This process is made up of many subprocesses, including the user manager, the
service dispatcher, and the connection manager. Each of these subprocesses
performs different functions.
• hmm. The high-level maintenance manager that handles all maintenance for
Communication Manager Applications. This process prints all process errors
(proc-errors), and is ultimately responsible for the initialization of the system. This
process also dispatches the maintenance action processes or maps that handle much
of the maintenance work, including the initmap. Note that the high-level
maintenance manager process is responsible for only printing proc-errors (process
errors).
• lmm. The low-level maintenance monitor. This bootstrap loader loads Oryx-Pecos
into memory and then passes control to it.
• prc_mgr. The process manager. This process is partially responsible for creating
and destroying all processes. The process manager ensures that the system comes
up correctly, and uses the high-level maintenance manager process and the initmap
process to manage what recovery actions to take when a process has a problem.
• gmm. The general maintenance manager. This process receives alarms directly
from platform processes, and ensures that alarms are created and reported
_____________________________________________________________________________________________
appropriately. For MultiVantage processes, the maintenance data manager spawns
an rma_mgr if an alarm is necessary.
• rma_mgr. The process that the MDM spawns to call out an alarm to the general
maintenance manager process. The rma_mgr process is created and immediately
destroyed by the prc_mgr after the process performs the call.
• arbiter. The process that makes decisions on server interchanges. This process
takes input from the packet control driver, the process manager, the general
maintenance manager, and the watchdog processes to make decisions on which
server is healthier. This process then uses that information to determine which
server should be active.
• watchd. The watchdog process that monitors all processes in the system, and
maintains heartbeats with both Communication Manager and the platform
processes. This process is responsible for stopping and starting processes when
necessary. This process monitors the entire system, while the initmap essentially
monitors only the processes that are related to Communication Manager.
• pcd. The packet control driver process provides the interface between the
Communication Manager application software and the PKTINT. All packet bus
communications that come up from the PKINT travel through the pcd (over IP for
Linux-based servers) to reach capro.
• lip. The link initiator process manages the pktint resource, and is the layer 2 LAPD
manager. This process deals only with managing the links, and is not concerned
with the data that travels through the links.
• gip. The gaz interface process creates all sockets for the user_mgr to communicate
with IP endpoints. One of the primary purposes of this process is to serve as the
resource manager for the CLAN board.
• fastmap. A maintenance action process (map) that the high-level maintenance
manager process spawns when high priority actions are required. The fastmap
process is usually allocated by the high-level maintenance manager process to handle
IPSI interchanges or various other SPECON recovery actions. This process has a
higher priority than the call processing process.
• border. The process that provides communications between platform processes and
Communication Manager processes. For example, the arbiter process requires the
border process to relay messages to the pcd.
• initmap. This process is spawned by the high-level maintenance manager process,
and is responsible for creating all permanent processes and maintaining sanity with
those processes. This process also notifies the system when the system reaches
normal_op (normal operation).
• ndm. The duplication manager process maintains shadowing between an active
server and a standby server. This process performs tests to ensure that the
duplication strategy of shadowing and filesync works as designed, and verifies that
the standby side is refreshed.
• pam. The permissions and access manager process prints all commands that are
entered in a SAT into the log file. This information is useful when troubleshooting
to help you determine what commands were issued on a system.
_____________________________________________________________________________________________
_____________________________________________________________________________________________
12. Appendix A: Screenshots and additional information on
Communication Manager commands
12.1. The /sbin/ifconfig command
Use the /sbin/ifconfig command to determine the IP addresses and the status of all
the Ethernet interfaces on the servers that run Communication Manager.
12.1.1. Avaya S8300 server
Interface name Function

eth0 S8300 services interface
eth1.0000 S8300 NIC or PROCR interface
ppp0 IP address for the modem connection
Interface eth1.0000 indicates the default VLAN, and eth1.4095 indicates the internal VLAN.
L0 is the loopback interface.
init@definity1> /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:04:0D:05:D9:F8
inet addr:192.11.13.6 Bcast:192.11.13.7 Mask:255.255.255.252
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2120 errors:0 dropped:0 overruns:0 frame:0
TX packets:1657 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
Interrupt:5 Base address:0x1080 Memory:d0f5e000-d0f5ec40
eth1 Link encap:Ethernet HWaddr 00:04:0D:05:D9:F2

Interrupt:5 Base address:0x10c0 Memory:d0f60000-d0f60c40
eth1.0000 Link encap:Ethernet HWaddr 00:04:0D:05:D9:F2

eth1.4095 Link encap:Ethernet HWaddr 00:04:0D:05:D9:F2

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.255.0.0
UP LOOPBACK RUNNING MTU:3924 Metric:1
_____________________________________________________________________________________________
ppp0 Link encap:Point-to-Point Protocol

inet addr:10.1.253.1 P-t-P:10.1.253.2 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
12.1.2. Avaya S8700 servers (Multi-Connect and IP-Connect)
Interface Multi-Connect IP-Connect

name
eth0 Control-Network A Control-Network A Customer LAN
eth1 S8700 services interface S8700 services interface
eth2 S8700 duplicated interface (or S8700 duplicated interface (or “dup”
“dup” interface) for this server interface) for this server
eth3 Control-Network B Unused
eth4 Customer LAN Unused
ppp0 IP address for the modem IP address for the modem
connection connection
To save space, the following example represents only a partial configuration from an S8700
Multi-Connect system.
init@8700a1> /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:02:B3:D0:2B:66
Interrupt:10 Base address:0xef00 Memory:e0a71000-e0a71c40
eth0:0 Link encap:Ethernet HWaddr 00:02:B3:D0:2B:66

Interrupt:10 Base address:0xef00 Memory:e0a71000-e0a71c40
eth1 Link encap:Ethernet HWaddr 00:02:B3:D0:2B:67

Interrupt:10 Base address:0xee80 Memory:e0a73000-e0a73c40
eth2 Link encap:Ethernet HWaddr 00:00:50:07:01:57

Interrupt:7 Base address:0xdf00 Memory:e0a75000-e0a75c40
_____________________________________________________________________________________________
Interrupt:11 Base address:0xde80 Memory:e0a77000-e0a77c40
eth3:0 Link encap:Ethernet HWaddr 00:00:50:07:01:58

Interrupt:11 Base address:0xde80 Memory:e0a77000-e0a77c40

Interrupt:5 Base address:0xdd80 Memory:e0a79000-e0a79c40
eth4:0 Link encap:Ethernet HWaddr 00:00:50:07:01:59

Interrupt:5 Base address:0xdd80 Memory:e0a79000-e0a79c40
lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:3924 Metric:1
ppp0 Link encap:Point-to-Point Protocol

inet addr:10.1.7.237 P-t-P:10.1.7.238 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
12.2. The almclear command

Use the almclear command to clear all outstanding server alarms. This command clears
only server alarms. If Communication Manager alarms exist that are unresolved within
Communication Manager, those alarms might be cleared temporarily when you run the
almclear command. These alarms persist, however, and reappear until you resolve these
alarms within Communication Manager. See Section 12.3, “The almdisplay command,” for
more information.
The following example shows the command to clear all server alarms.
init@grays-icc> almclear -a
All Server alarms cleared.
12.3. The almdisplay command
_____________________________________________________________________________________________
Use the almdisplay command to display messaging alarms, Communication Manager
alarms, and server alarms. A server alarms is generated at the Linux level, and is the only type
of alarm that you can clear with the almclear command. See Section 12.2, “The almclear
command,” for more information.
The following example shows the results of the almdisplay command.
init@chicc> almdisplay
MESSAGING ALARMS
================
No MESSAGING Alarms
CommunicaMgr ALARMS
===================
No CommunicaMgr Alarms
SERVER ALARMS
=============
No Server Alarms
12.4. The fasttop command

Use the fasttop command to view the occupancy numbers for the various Communication
Manager processes. The fasttop command is a customized version of the standard top
command that is available within Linux.
The following example shows the results of the fasttop command.
Sun Jan 18 12:27:31.917

processor at 851943 khz 5.00 seconds 34.19 % used 29.12 %
prio
Summary 108.29 seconds 14.00 % used 10.32 % prio
PID PPID %occ name pol prio memory
12822 12789 20.73 hmm FIFO 21 7036928
12827 12789 3.22 capro FIFO 17 133120000
29344 12789 2.28 map FIFO 4 3936256
12804 12789 1.75 pcd FIFO 22 7221248
12959 12789 1.51 mdm FIFO 6 3317760
12830 12789 1.49 dap FIFO 18 3756032
12828 12789 0.48 bdm FIFO 18 3665920
12808 12807 0.42 pcd FIFO 22 7221248
12988 12789 0.37 gip FIFO 20 7254016
2581 12789 0.25 map FIFO 4 3936256
_____________________________________________________________________________________________
3511 12789 0.22 tmr_mgr FIFO 23 3424256
3509 12789 0.16 tim FIFO 24 2895872
12.5. The modserv command

Use the modserv command to turn on or turn off modem access for Avaya Services.
Note that for releases prior to Communication Manager 2.0, you might have to use the
sudo version of this command, and thus type sudo modserv to turn modem access
on and off.
12.5.1. Modem not connected

The following example shows the current modem status when the modem is not connected.
craft@saturn2> modserv -v
Access through the modem is administered to disabled.
A modem has not been detected.
12.5.2. Modem administered but disabled

The following example shows the current modem status when the modem is disabled.
craft@saturn2> modserv -v
Access through the modem is administered to disabled.
Access through the modem is currently disabled.
12.5.3. Modem administered and ready for access

The following example shows the current modem status when the modem enabled.
init@saturn2> modserv -v
Modem access is administered for incoming calls.
12.6. pingall
Use the pingall command to ping all available interfaces in the S8X00 server view. The
pingall command is typically used on the S8500 server and the S8700 server to run a quick
ping of all available interfaces.
The following example shows the results of the pingall command for all network entities on
the remote side.
init@saturn2> pingall -a
Testing.
_____________________________________________________________________________________________
HostName IPAddress Status
saturn1-cnb 135.9.9.98 Pass
saturn1-cna 135.9.10.98 Pass
ipsi-A09b 135.9.9.9 Pass
ipsi-A09a 135.9.10.9 Pass
saturn1-dup 192.11.13.13 Pass
saturn1 135.9.49.98 Pass
12.7. The restartcause command

Use the restartcause command to view the time of the reset, and other important
details that were captured by the “cause” of the reset. The restartcause command
from the Linux CLI is similar in function to the display initcauses command that
is available from the Communication Manager system access terminal (SAT).
The following example shows the results of the restartcause command.
init@saturn2> restartcause
RESTART CAUSES
Cause Action Escalated Time
Internal Request 1 (WARM) no 11/14 6:02
Internal Request 2 (COLD_2) no 11/14 6:28
Initialized 4 (RELOAD) no 11/19 16:42
Craft Request 4 (RELOAD) no 11/19 16:46
Internal Request 4 (RELOAD) no 11/19 17:28
Internal Request 2 (COLD_2) no 11/20 22:38
Initialized 4 (RELOAD) no 12/01 16:12
Internal Request 4 (RELOAD) no 12/01 17:31
Software Request 4 (RELOAD) no 12/08 14:16
Cause Description
Internal Request The restart request was from an internal process as a result of an
interchange.
Software Request Communication Manager initiated the restart of the server.
_____________________________________________________________________________________________
Cause Description
Craft Request The server reset was initiated on demand by a user who logged in
from the SAT.
Initialized The server went through a power cycle or was intentionally shut
down.
Restart level Description

WARM Restarts all Communication Manager processes without reinitializing
data that is related to active calls. All stable calls are preserved.
COLD_2 Restarts all Communication Manager processes and preserves
translation data, but reinitializes the data space. All calls are dropped.
RELOAD Reloads Communication Manager, and reads in translations. All calls
are dropped.
12.8. The server command

Use the server command to display the status of the S8X00 servers.
The following output represents a set of ideal duplicated Avaya S8700 servers. Note that
Standby Refreshed ? is “yes,” Duplication link is “up,” no Major or Minor alarms exist, and
the control network is healthy. The entries in bold type are the entries of most interest for
troubleshooting.
init@saturn2> server
SERVER STATUS
Duplicated? yes
Standby Busied? no
Standby Refreshed? yes
Standby Shadowing: on
Duplication Link: up
Elapsed Time since Init/Interchange: 9d 21:36:05
saturn2 saturn1
Mode: Active Mode: Standby

Major Alarms: no Major Alarms: no
Minor Alarms: no Minor Alarms: no
Control Network: 5 / 5 Control Network: 5 / 5
Server Hardware: okay Server Hardware: okay
Processes: okay Processes: okay
12.8.2. Avaya S8300 and S8500 servers

The following output represents an S8300 simplex server or an S8500 simplex server. Note
that with simplex servers, “Duplicated” is set to “no,” and one of the servers displays the
status of “Not Ready.” The “Not Ready” status is normal and acceptable.
_____________________________________________________________________________________________
init@grays-icc> server
SERVER STATUS
Duplicated? no
Standby Busied? no
Standby Refreshed? no
Standby Shadowing: off
Duplication Link: init
Elapsed Time since Init/Interchange: 4d 02:56:34
grays-icc Remote
Mode: Active Mode: Not Ready

Major Alarms: no Major Alarms: no
Minor Alarms: no Minor Alarms: no
Control Network: 0 / 0 Control Network:
Server Hardware: okay Server Hardware:
Processes: okay Processes:
12.9. The statapp or systat command

The following example shows the results of the statapp command for the S8500 and
S8700 media servers.
init@saturn2> statapp
Watchdog 19/19 UP
TraceLogger 4/ 4 UP
ENV 0/ 1 OFF
LicenseServer 3/ 3 UP
INADSAlarmAgen 1/ 1 UP
G3AlarmAgent 1/ 1 UP
GMM 5/ 5 UP
SNMPManager 1/ 1 UP
arbiter 3/ 3 UP
filesyncd 9/ 9 UP
dupmgr 1/ 1 UP
MasterAgent 3/ 3 UP
MIB2Agent 1/ 1 UP
MVSubAgent 1/ 1 UP
SME 9/ 9 UP
CommunicaMgr 90/90 UP
12.9.2. Avaya S8300 server (with IA770 on-board

messaging installed)
_____________________________________________________________________________________________
The following example shows the results of the statapp command for the S8300 media
server with IA770 installed.
init@chicc> statapp
Watchdog 19/19 UP
TraceLogger 4/ 4 UP
slotmon 1/ 1 UP
ENV 0/ 1 OFF
LicenseServer 3/ 3 UP
INADSAlarmAgen 1/ 1 UP
G3AlarmAgent 1/ 1 UP
GMM 6/ 6 UP
SNMPManager 1/ 1 UP
arbiter 0/ 3 OFF
filesyncd 9/ 9 UP
dupmgr 0/ 1 OFF
MasterAgent 3/ 3 UP
MIB2Agent 1/ 1 UP
MVSubAgent 1/ 1 UP
SME 8/ 8 UP
CommunicaMgr 67/67 UP
Messaging 1/ 1 UP
12.10. The statuslicense command
The following example shows the results of the statuslicense command.
init@saturn2> statuslicense
CommunicaMgr License Mode: Normal
Network used for License: Carrier 02a
License Serial Number is 01DR08125609 on carrier 02a
Note that that the license serial number in the example is the serial number of the IPSI to
which this S8700 server can talk. In a system with multiple IPSIs, this IPSI is also known as
the “golden” IPSI.
init@chicc> statuslicense
CommunicaMgr License Mode: Normal
Network used for License: Carrier MGP
License Serial Number is 03J210708758 on carrier MGP
_____________________________________________________________________________________________
Note that that the license serial number in the example is the serial number of the gateway in
which this S8300 resides. The example shows the serial number of a G700 Media Gateway.
12.11. The swversion command
The following example shows the results of the swversion command.
init@saturn2> swversion
Operating system: Linux 2.4.20-AV9 i686 i686
Built: Nov 17 13:36 2003
Contains: 00.0.219.0
Reports as: R012x.00.0.219.0
Release String: S8700-012-00.0.219.0
UPDATES:
00.0.219.0-6227 activated The call preserving
super patch for 219
Translation Saved: Jan 16 12:00
License Installed: Jan 2 13:18
The swversion command produces the following output:
• Contains: 00.0.219.0
This output is the value of the Release string in the file /opt/ws/ENV.dat. This
value is the release value of the software itself, which is built in when the binaries are
compiled, and loaded during integration. Integrators refer to this value as the
"Definity load string."
• Reports as: R012x.00.0.219.0

This output is the value of the Version string in the file /etc/opt/ecs/ecs.conf, the
so-called "registry" of the server. This value is the official generic release name for
the software. Integrators refer to this value as the "Definity version string."
• Release String: S8500-012-00.0.219.0
This output is a concatenation of the following three information elements, with
hyphens as separators:
− ServerType, which is another entry in the file /etc/opt/ecs/ecs.conf. The following
table shows how the string that appears in that file maps into the displayed string:
_____________________________________________________________________________________________
Displayed
ServerType
string
isp2100 S8700
icc S8300
ibmx305 S8500
− Product string, which is also an entry in the file /opt/ws/ENV.dat. In this case,
012 indicates Communication Manager 2.0.
− Release string, which is the Definity load string that appears on the "Contains:"
line.
The information that is presented on the three lines is redundant. However, the
information does tell you whether the version of Communication Manager that is
running on the server is consistent with the /etc/opt/ecs/ecs.conf file that is created
or modified during configurations and upgrades. If "Contains:" and "Reports as:"
show different Definity load string components, the software is configured
incorrectly. In that case, you might want to examine the /opt/ws/ENV.dat and the
/etc/opt/ecs/ecs.conf files, to determine what underlies the inconsistency.
_____________________________________________________________________________________________
13. Appendix B: Server behavior on interchange
The following output is a portion of the logs from the file /var/log/ecs on the server when
a server interchange takes place. The standby server is being told to go active. The prc_mgr
pauses the necessary processes, and a warm start takes place on the server.
STANDBY TO ACTIVE TRANSITION:
Standby -> Told to go active -> WARM Restart -> Goes Active
20030113:151838676:2254:Arbiter(22656):MED:[Being told to Go Active!!!]

20030113:151838676:2255:Arbiter(22656):MED:[remot: gmm 0500, pcd 03/03,
dup 70, wd 01, actv 005]
20030113:151838676:2256:Arbiter(22656):MED:[State of other side is
Handoff]
20030113:151838676:2257:Arbiter(22656):MED:[IntchgMtce: NOREQ->COMEH]
20030113:151838676:2258:Arbiter(22656):MED:[STANDBY->ACTIVE :interchange
request]
20030113:151838676:2259:Arbiter(22656):MED:[State Transition: STANDBY to
ACTIVE ]
20030113:151838676:2260:Arbiter(22656):HIGH:[dupmem_rxoff called]
20030113:151838691:2261:Arbiter(22656):HIGH:[Interchange-Driven Warm
restart]
20030113:151838691:2262:Arbiter(22656):HIGH:[Setting 0x00000800, new SOH
= 0x00000b70]
20030113:151838691:2263:Arbiter(24336):MED:[Requesting a WARM restart
via border process]
20030113:151838692:2264:Arbiter(22656):HIGH:[dupmem_rxoff called]
20030113:151838707:2265:border(23741):HIGH:[platreq.c: pi is 4 ]
20030113:151838707:2266:prc_mgr(23032):HIGH:[restart_req: level=0481,
internal-requested]
20030113:151838707:2267:prc_mgr(23032):HIGH:[Escalation Levs:
SharedMem=0, DefRestartCnt=0]
20030113:151838708:2268:prc_mgr(23032):HIGH:[prc_mgr: restart_req:
internal-requested WARM: doing WARM]
20030113:151838708:2269:prc_mgr(23032):HIGH:[prc_mgr: restart_req: Upon
exit, DefRestartCnt=0]
20030113:151838708:2270:prc_mgr(23032):MED:[setRstLvl: sending
SIGDEFPAUSE to pid 23088 net_st, attributes=a0]
20030113:151838708:2271:prc_mgr(23032):MED:[setRstLvl: sending
SIGDEFPAUSE to pid 23086 mcp, attributes=a0]
Continues…
20030113:151838710:2302:prc_mgr(23032):HIGH:[rest_def: all processes

rstLevel set]
20030113:151838710:2303:tim(23734):HIGH:[sighandler: SIGDEFTERM
received]
20030113:151838710:2304:tim(23734):HIGH:[now calling exit]
20030113:151838710:2305:net_mgr(23733):HIGH:[sighandler: SIGDEFTERM
received]
20030113:151838710:2306:net_mgr(23733):HIGH:[now calling exit]
20030113:151838711:2307:phantom(23732):HIGH:[sighandler: SIGDEFTERM
received]
_____________________________________________________________________________________________
20030113:151838711:2308:phantom(23732):HIGH:[now calling exit]
The output continues on as all processes receive their pauses.
The following log is from the other server. This output shows what occurs when the active
server is told to go to standby.
ACTIVE TO STANDBY TRANSITION:
Active -> Told to go Standby -> Handoff -> COLD-2 Restart -> Warm
Restart -> Warm Restart -> Goes to Standby
20030113:151838015:3133:Arbiter(28117):MED:[local: gmm 0500, pcd 03/03,

dup 70, wd 01, actv 005]
20030113:151838015:3134:Arbiter(28117):MED:[IntchgMtce: NOREQ->GOWAY]
20030113:151838016:3135:Arbiter(28117):MED:[Preparing for Interchange
...]
20030113:151838016:3136:filesyncd(28160):MED:[fileset.c: performing
filesync for DUP]
filesync for DUP]
filesync for DUP]
filesync for DUP]
filesync for LSP]
20030113:151838674:3141:filesyncd(28160):MED:[fileset.c: this is a
primary.]
20030113:151838674:3142:border(29003):HIGH:[platreq.c: pathname = ADM_UM
pi is 8 ]
20030113:151838675:3143:border(29003):HIGH:[platreq.c: return 0 reached
]
20030113:151838675:3144:Arbiter(31998):MED:[fsync_request complete,
result=1]
dup 70, wd 01, actv 005]
20030113:151838676:3146:Arbiter(28117):MED:[ACTIVE ->Handoff:interchange
prepartion complete]
dup 70, wd 01, actv 005]
20030113:151838676:3148:Arbiter(28117):MED:[remot: gmm 0100, pcd 03/03,
dup 370, wd 01, actv 017]
20030113:151838676:3149:Arbiter(28117):MED:[planned_interchange: rsync
took 0.660478 seconds]
20030113:151838676:3150:Arbiter(28117):MED:[State Transition: ACTIVE to
Handoff]
20030113:151838692:3151:Arbiter(28117):MED:[TIMESYNC: ostime=80ee8,
tod=3e233b3e]
20030113:151838692:3152:Arbiter(28117):MED:[State of other side is
ACTIVE ]
20030113:151838692:3153:Arbiter(28117):MED:[Handoff->STANDBY:interchange
requested by CRAFT]
_____________________________________________________________________________________________
20030113:151838692:3154:Arbiter(28117):MED:[State Transition: Handoff to
STANDBY]
20030113:151838692:3155:Arbiter(28117):HIGH:[Cold-2 restart for New
Standby]
20030113:151838692:3156:Arbiter(28117):HIGH:[PDmShd = 0x2d000000]
20030113:151838692:3157:Arbiter(28117):HIGH:[Setting 0x00000800, new SOH
= 0x00000870]
20030113:151838693:3158:Arbiter(32011):MED:[Requesting a COLD-2 restart
via border process]
20030113:151838694:3159:border(29003):HIGH:[platreq.c: pi is 4 ]
20030113:151838694:3160:prc_mgr(28536):HIGH:[restart_req: level=0482,
internal-requested]
20030113:151838694:3161:prc_mgr(28536):HIGH:[Escalation Levs:
SharedMem=0, DefRestartCnt=0]
20030113:151838694:3162:Arbiter(32013):MED:[exec-ing STANDBY state
script]
20030113:151838695:3163:prc_mgr(28536):HIGH:[prc_mgr: restart_req:
internal-requested COLD_2: doing COLD_2]
When a server goes to standby, the normal restart scenario is Cold-2, Warm, Warm. The
following output represents one of the Warm requests:
20030113:151844883:3488:Arbiter(32088):MED:[Requesting a WARM restart
via border process]
_____________________________________________________________________________________________
14. Appendix C: Connecting to an S8300, an S8500, or an
S8300 server from a Windows-based personal computer
This appendix contains excerpts from the Maintenance Procedures document, 555-245-103.
The Maintenance Procedures document is available on the Avaya Support Web site:
http://support.avaya.com/
14.1. Configuring a remote computer that is running Windows 2000 or

Windows XP for PPP modem connection
To configure a Windows-based remote computer for PPP modem connection:
1. On the desktop of your computer, right-click My Network Places, and then click
Properties.
The system displays the Network and Dial-up Connections window.
2. Double-click the connection name that you made earlier.
Note: Depending on your system, the Connect window may appear. If the system
displays the Connect window, click Properties.
3. Click the Security tab.
4. Click Advanced (custom settings).
5. Select Show terminal window.
6. Click the Networking tab.
7. In the Components box, verify that Internet Protocol (TCP/IP) and Client for
Microsoft Networks are both selected.
8. Select Internet Protocol (TCP/IP), and click Properties.
9. Click Advanced.
10. Clear Use default gateway on remote network.
11. Click OK three times to exit and save the changes.
14.2. Using Windows 2000 or Windows XP for PPP modem connection

Note: Access to the system through a PPP modem connection may require RAS access and
ASG Mobile access.
To use Windows 2000 or Windows XP for PPP modem connection:

1. Return to the Network and Dial-up Connections window. Right-click the connection
that you just created.
2. Select Connect.
3. Leave the User Name, Password, and Domain fields blank. If the Dial field is blank,
enter the appropriate telephone number.
4. Click Dial. When the media server modem answers, the system displays the After
Dial Terminal window.
5. Log on to the LAN.
a. Enter your remote access login name and password.
_____________________________________________________________________________________________
b. When the Start PPP Now! message appears, click Done. The system
displays a small double-computer icon in the lower right portion of the
screen.
6. Double-click the double-computer icon. The system displays the Dialup Status box
for the connection.
7. Click the Details tab.
8. Note the Server IP address.
9. To open a telnet session to the Avaya S8X00 server, type telnet <ip address>,
where <ip address> is the IP address of the S8300 that you noted in the Dialup Status
box in Step 8.
10. Access SAT or use the CLI commands as needed.
14.3. Using telnet

telnet - user interface to the TELNET protocol
Syntax
telnet [host[port]]
Description
Use the telnet interface to communicate with another host by way of the TELNET protocol.
If telnet is invoked without arguments, telnet enters command mode, which is indicated by
the prompt, telnet>. In this mode, telnet accepts and runs the commands below. If
telnet is invoked with arguments, telnet performs an open command (see below) with those
arguments.
To issue telnet commands when in input mode, precede the commands with the telnet
escape sequence. Initially, the telnet escape sequence is the control character followed by a
right bracket, ^+] . When in command mode, use the normal terminal editing conventions.
Example
The following command connects your terminal to the host with IP address 10.3.4.10:
$ telnet 10.3.4.10
Options with the telnet command

telnet> ?
Commands may be abbreviated. Supported options are:

close close current connection
display display operating parameters
mode try to enter line-by-line or character-at-a-time mode
open connect to a site
quit exit telnet
send transmit special characters (send ? for more)
set set operating parameters (set ? for more)
unset unset operating parameters (unset ? for more)
_____________________________________________________________________________________________
status print status information
toggle toggle operating parameters (toggle ? for more)
slc change state of special characters (slc ? for more)
z suspend telnet
! invoke a subshell
environ change environment variables (environ ? for more)
? print help information
14.4. Using ftp

Basic commands
The following table includes some commonly used commands to establish and work within
an FTP session.
Command Action
open Initiate a session between the client computer and the remote
server.
nlist, dir, ls List the hierarchical organization of files on the remote server.
lls List the hierarchical organization of files on your local
computer.
cd Change directories on the remote computer either up or down.
lcd Change directories on your local computer either up or down.
pwd View the current directory and path on the remote server.
lpwd View the current directory and path on your local client
computer.
get Copy a file from the server to your local client computer.
put Copy a file from your client computer to the remote server.
mget Copy multiple files from the server to your local client
computer.
mput Copy multiple files from your client computer to the remote
server.
prompt Set interactive prompting. "On" is a safety feature that prompts
you for verification of each step of a multiple command. "Off"
allows the command to act unimpeded
ascii/binary Specify the type of file to transfer.
quit End the connection and end the session.
To view the full set of FTP commands, type help or ? at the FTP> prompt.
14.4.1. Example of an FTP session that uses the commands that are listed above
This example shows the how to use FTP to copy files from one computer to another.
Assumptions
Computer A is your personal computer, usually the laptop computer of a technician.
Computer B might be another computer, or the Communication Manager server that
can be reached from computer A over a LAN, a WAN, or the Internet.
_____________________________________________________________________________________________
You want to copy files Afile_1.exe, Afile_2.exe, and Afile_3.txt from computer A to
computer B. Here, Afile_1.exe might represent a software update file for
Communication Manager such as 00.0.219.0-6276.tar.gz.
You want to copy the file Bfile.exe from computer B, which might be a
Communication Manager server, to computer A, which might be the laptop
computer of a technician.
Computer B has the host name or the IP address <B_IP>.
The files on computer A are located in a directory called <A_ directory>.
The files on computer B are located in a directory called < B_directory>.
Command Description
Assume that you are already logged on to Computer A.
C:\> cd /<path>/<A_directory> On computer A, go to the
<A_ directory > directory.
C:\> ftp <B_IP> Set up an FTP session to computer B.
User: <your User-ID on computer B> Log on to the FTP session on
computer B. Enter your User Id at
the User prompt. If you are using
Anonymous FTP, enter
anonymous.
Password: <your password> Enter your password at the Password
prompt. If you are using Anonymous
FTP, enter your e-mail address.
ftp> cd <B_directory> Go to the <B- directory> directory on
computer B. If computer B is a media
server that is running Communication
Manager release 2.0 or later, use cd
to change to the /var/home/ftp/pub
directory. The /var/home/ftp
directory is not writable in 2.0 and
later releases.
ftp> ascii Specify an ASCII file type (used to
transfer text files only).
ftp> put Afile_3.txt Copy Afile_3.txt from computer A to
<B_ directory> on computer B.
ftp> binary Specify a binary file type. Note: To
transfer binary files such as
Communication Manager software
updates , or “patches,” you must be in
binary mode.
ftp> mput Afile*.exe Copy Afile_1.exe and Afile_2 to the
<B_ directory> on computer B.
mput Afile_1.exe? y Confirm the copy of Afile_1.exe.
mput Afile_2.exe? y Confirm the copy of Afile_2.exe.
ftp> get Bfile.exe Copy Bfile.exe from computer B to
<A_ directory> on computer A. You
_____________________________________________________________________________________________
can also use the command mget
Bfile.* if you want to copy
multiple files from Computer B to
Computer A.
ftp> bye Close the FTP session.
_____________________________________________________________________________________________
Appendix D: A case study in which a segmentation fault
causes switch reset
This appendix includes an example of how to use the information in this document to
determine the cause of a restart on an Avaya 8700 server.
15.1. Log in and connect to the system
To begin, log in to the system by way of a modem connection, over the LAN, or any other
means that is available. See Appendix C , “Connecting to an S8300, S8500, or S8300 server
from a Windows-based personal computer,” for how to set up a connection to a Avaya
server from a Windows-based personal computer. After you log in, run the
restartcause command from the bash. The system displays output that looks similar
to the output in Section 12.7 of this document. Match the time stamp of the customer
outage with the output on the screen to determine exactly what time the outage occurred.
15.2. Examine the logs
Change directories to /var/log/ecs. Run the ls –ltr command from the Linux shell or
the CLI to see a list of all the log files in that directory. The files are named with the time
stamp of when the logs began. Use this information to determine which log contains
information on the reset.
15.3. Analyze the problem
Use vi or emacs to open the file. See Section 7.1, “The vi editor,” for how to use vi. Use the
grep command to search for the keywords, as described in Section 8.4, “The grep
command.” In this example, the restart is the result of a software segmentation fault. Thus,
the EIP search string takes you to the location in the log file where the segmentation fault
occurred.
You now know that the switch reset was the result of a software problem, what process in
the software had the problem, and exactly what time the outage occurred. Based on this
information, the next step is to escalate this ticket to the next Tier for resolution. However,
before you send the ticket to Tier 3 or Tier 4, you must determine if a core file was
generated. Perform one of the following actions to determine if a core file was generated:
• Type corevector to see if the flag is turned on for traps.
• Change directories to /var/crash, as described in Section 10, “Core dumps and mini
core dumps.” Run the ls –ltr command in this directory to see when the latest
core files were generated. Determine if a core file exists with the same time stamp as
when the outage occurred.
Next, copy both the log file in question and the previous log file to the directory
/var/home/ftp. (For Communication Manager releases 2.0 and later, the directory is
/var/home/ftp/pub.) Finally, if a core file exists, copy the core file to /var/home/ftp.
Include this information, along with the messages log and the wdlog, with the ticket, and
send the ticket to Tier 3 or Tier 4.
_____________________________________________________________________________________________
© 2004 Avaya Inc. All rights reserved. All trademarks identified by the ®
or TM are registered trademarks or trademarks, respectively, of Avaya Inc.
All other trademarks are the property of their respective owners.
_____________________________________________________________________________________________
_____________________________________________________________________________________________

Avaya Linux Troubleshooting Guide

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Avaya Linux Troubleshooting Guide

Caricato da

Copyright:

Formati disponibili

Application Note:

Troubleshooting Guide v1.0 for the Avaya

4. Versions of this document

1.0 Original version

6.2.1. Types of shells

6.2.2. Example of a shell command within Communication Manager

7.1.1. Movement commands

7.1.2. Text entry commands

7.1.3. Cut, copy, paste, and undo commands

7.1.4. Search and replace commands

7.1.5. Save and quit commands

7.2. The emacs editor

7.2.1. General commands

7.2.2. Movement commands

7.2.3. Cutting and pasting commands

8. Commonly used Linux and Communication Manager

To move up one directory, type:

Note that leaving your account logged on can be a security concern.

For example, use find . / -name <file name> -print to

• find /dev -user "peter" |more

• interchange. Displays records in *.log files that include the

locate <search term> | more

• logv today pcd

• logv yesterday lm wd lxsys

• logc 20031001 > <filename>

Options to modify command operation:

Logs that can be displayed:

Filters to display events that match a particular criterion. Multiple

man [section-number] page_name

Type more <filename>

The number under PID is the process identification number.

server Display server duplication Use -b busyout this server. Use

9. Commonly accessed directories and files for

9.1. The /etc/opt/ecs directory

9.3. The /etc/opt/defty directory

9.4. The/var/log/ecs directory

• ecs log files

9.5. The /var/log/messages file

9.6. The /var/crash directory

9.7. The /var/log/defty/dumps directory

11. Introduction to commonly encountered entities within

• capro. The call-processing process for Communication Manager Applications.

12.1.1. Avaya S8300 server

Interface name Function

eth1 Link encap:Ethernet HWaddr 00:04:0D:05:D9:F2

eth1.0000 Link encap:Ethernet HWaddr 00:04:0D:05:D9:F2

eth1.4095 Link encap:Ethernet HWaddr 00:04:0D:05:D9:F2

lo Link encap:Local Loopback

ppp0 Link encap:Point-to-Point Protocol

12.1.2. Avaya S8700 servers (Multi-Connect and IP-Connect)

Interface Multi-Connect IP-Connect

eth0:0 Link encap:Ethernet HWaddr 00:02:B3:D0:2B:66

eth1 Link encap:Ethernet HWaddr 00:02:B3:D0:2B:67

eth2 Link encap:Ethernet HWaddr 00:00:50:07:01:57

eth3:0 Link encap:Ethernet HWaddr 00:00:50:07:01:58

eth4 Link encap:Ethernet HWaddr 00:00:50:07:01:59

eth4:0 Link encap:Ethernet HWaddr 00:00:50:07:01:59

lo Link encap:Local Loopback

ppp0 Link encap:Point-to-Point Protocol

12.2. The almclear command

12.3. The almdisplay command

12.4. The fasttop command

The following example shows the results of the fasttop command.

Sun Jan 18 12:27:31.917