ZXG10 iBSC Troubleshooting

ZTE University

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Troubleshooting procedure

Collect fault information

Fault analysis

Fault locating

Troubleshooting

Common methods to locate faults

Views of alarm, notification and operation log Analysis of status indicator Performance analysis Analysis of signaling tracing Analysis with instruments and meters Plugging/unplugging and pressing Isolation method Self-test method

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

NMS fault category

1 2
3 4 5 Platform faults Database faults Configuration and application faults Alarm application faults Performance application faults

Troubleshooting guide of NMS faults

Client can not connect server NetNumen M31 server startup failure Database startup faults Failure of observing status attribute in dynamic data management Failure of alarm box displaying audio/visual alarms Abnormal performance data report

Client can not connect server

Fault cause Troubleshooting

Network between server and client is disconnectted The client version differs from the server version Language setting in deploygsmomcr01.properties and deploy-010muep.properties differs from that on the server Disk is full

Use ping command to check whether the connection is OK Use ipconfig and netstat commands to check whether server IP address configuration is proper Check whether client version is consistent with server version Check whether language setting is consistent with that in server Use df kh to check disk space

NetNumen M31 server startup failure

Fault cause Troubleshooting

Server ini file doesnt have execution permission Java virtual machine is not started successfully Server profile is improper Oracle database can not start properly

Use ls l to check whether the ini file has executable permission Check whether compressed file under idk-solaris folder have been decompressed Check whether the server configuration file is correct Check whether it is successfully connected with Oracle database Check whether database server name, IP address and hostname configured in tnsnames.ora are consistent with actual value Use pkill -9 java to kill java process and restart the server

Database startup faults

Fault cause Troubleshooting

Database is already running Dtabase parameters are not set correctly, which cause insufficient resources needed for startup The system data file needed for startup is corrupt Use data file is corrupt when the database is opened

Execute sqlplus sys/oracle@gomcr as sysdba to check whether Oracle can be connected If the database does not properly start, check the returned result Check whether the disk is full by using command df k If the database starts properly but cannot be connected, check whether the database monitor has started

Failure of observing status attribute in dynamic data management

Fault cause Troubleshooting

Communication between foreground and background is improper Foreground and background data inconsistent

Check whether the communication link has been properly established between foreground and background Do synchronization for whole table to keep consistence for foreground and background data

Failure of alarm box displaying audio/visual alarms

Fault cause Troubleshooting

Probable reason is improper connection state in alarm box

Check the alarm box setting in NetNumen M31 alarm management, to check whether IP address/TCP port number of alarm box is consistent with actual value Chek whether ping can get response from the network about alarm box, preventing from network cable or IP address conflict Check whether the port number of alarm box has conflict with the port occupied by other processes Check whether there are the setting for this alarm box on NMS server and communication state is OK

Abnormal performance data report

Fault cause Troubleshooting

Link is disconnected between foreground and background Performance file reported from foreground is improper or performace data has not stored yet Nsf detect process is failed to connect foreground

Check physical link to ensure the foreground-background link is normal After logging in SBCX board, switch to corresponding measurement types directory to view whether original performance data zip file for granularity is missing check whether the data file for performance measurement is missing Check nsf service state after logining OMP Execute ps ef | grep nsfdetect check whether nsfdetect process exists View and print nsfdetect log

Alarm cases on NMS faults

After restarting iBSC OMP, background can not establish link with OMP Software version cannot be synchronized to NE and system prompts FTP download failed "License file invalid or missing" appears while configuring transceiver Wrong time zone configuration causes performance report issue NetNumen M31 upgrade fails

After restarting iBSC OMP,background can not establish link with OMP

Fault analysis

The reason is IP address of O&M board is not consistent with that of OMP at background while configuring data.

Troubleshooting

Change OMP IP address at background. Restart OMP after synchronizing data, and the fault is solved.

Software version cannot be synchronized to NE and system prompts FTP download failed
The reason is ftp server IP in deploygsmomcr01.properties is inconsistent with that of IMP communication.

Fault analysis

Troubleshooting

Modify ftpserver IP in deploygsmomcr01.properties and enable it consistent with IP address for IMP communication.

"License file invalid or missing" appears while configuring transceiver

Fault analysis

License file is invalid or missing.

Open Configuration Management view, License management appears in menu bar. Click View License in License Management, and go to Troubleshooting Update License interface. Select the path to save license, find license.dat. license.dat file is located under umssvr\deploy\runtime\,and click the Update button.

Wrong time zone configuration causes performance report issue

Time zone on client is not consistent with that on NetNumen(TM) M31 server. The system modifies the reported data according to time zone setting, so the time in performance report is not consistent with current time and performance data is displayed in two columns.

Fault analysis

Troubleshooting

Querying whether performance data is proper after modifying time zone.

NetNumen M31 upgrade fails

After upgrade fails, check the server and input echo $OMCHOME command, null string is returned, not the value of $OMCHOME. From this, it is determined that the installation fault is caused by wrong $OMCHOME setting. solaris has strict requirements on environment variable. The user shall not only modify /etc/profile file, also shall logout after performing /etc/profile command, and then use root user to login again.

Fault analysis

After adding $OMCHOME value into /etc/profile file Troubleshooting and making it effect, perform graphical installation process.

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Category of data and software loading faults

Fault from data configuration

Fault caused by settings of software version

Troubleshooting guide about data and software loading faults

Mismatched board type and version configuration Version file does not exist Version file check error IPA resource unavailable NetNumen M31 data synchronization faults

Mismatched board type and version configuration

Fault cause

Troubleshooting

Mismatch error between board version information configuration and board type causes impossible to download version Board type configuration error Access type error supported by the board

Check the version configuration and physical configuration of corresponding board, and correct corresponding errors

Version file does not exist

Fault cause Troubleshooting

Such a fault is usually caused by MP changeover or MP replacement

Check the data configuration to confirm whether the board configuration is correct Wait for a period of time (more than 15 minutes) and then check whether the board is running properly In software management on NMS, perform NE synchronization action for software, and upload the version file to OMP

Version file check error

Fault cause Troubleshooting

The check error on the reception end often results from the version file error occurred during the transmission

Download the version again from the server Contact the local ZTE office for the version file

IPA resource unavailable

Fault cause Troubleshooting

Data configuration of user plane IP is different from that for MSC and BSC AIPB board goes faulty DSP port configuration is incorrect on AIPB IPA resource is blocked manually

Check whether data configuration is consistent for MSC and iBSC and the address on user plane shall be connected View IPA resource state; view the state of AIPB board and DSP Check whether the ports correspond to DSP are correct Check whether IPA resource is blocked manually

NetNumen M31 data synchronization faults

Fault cause Troubleshooting

The link between NetNumen M31 and iBSC is disconnected iBSC is performing writing action for previous data synchronization

Check whether the link between NetNumen M31 and iBSC is normal by Ping packets Retry after waiting for a certain duration, and wait for the completion of writing action on previous synchronization

Alarm case on data and software loading faults

-------iBSC data failed to synchronize and download to NE

Fault analysis

Disconnected communication between server and OMP OMP doesn't add FTP user; while trying to login OMP through FTP, it is found that zte/zte can't login

Troubleshooting

Normal communication between server and OMP After logging into foreground OMP through zte/zte and adding users by addFtpUserInfo "zte","zte",3,3,0, it can synchronize with foreground

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Category of connection faults at interface A

1 2 3 4

Faulty physical transmission Signaling faults Trunkline fault at interface A

Fault caused by wrong data configuration

Troubleshooting Guide about Connection Faults at Interface A

Unstable No. 7 signaling BSC can not PING virtual address at interface A successfully Synchronization fault at interface address Office is available and BSC message can't reach MSC server

Unstable No. 7 signaling

Fault cause Troubleshooting

Transmission problems, including high bit error ratio of transmission equipments and bad contact of connectors The fault is possibly due to message overflow resulting from high signaling load

Check whether connections are reliable. Test the bit error ratio of transmission Check the signaling link load through performance measurement. Expand the link if the load is too high

BSC can not PING virtual address at interface A successfully

Fault cause Troubleshooting

This is caused by inconsistent ARP entries between MAC address of BSC interface board and virtual IP address at interface A in MSC Server NE

Manually clear wrong MAC or reset BSC IPI interface board

Synchronization fault at interface address

Fault cause Troubleshooting

Print all interface information on RPU, it is found that IPI interface is UP and another interface is configured as unknown unit interface address for IPI, its state is DOWN. Through analysis, it is acknowledged that old data is not deleted completely, leading to interface address incorrectly synchronized to interface board

Delete ZDB file on RPU and restart it

Office is available and BSC message can't reach MSC server

Fault cause Troubleshooting

SI0 location is inaccurate, the message transferred from exchange domain by BSC is implemented by keywords record in index SI0 location. If SI0 location is inaccurate, BSC may discard relevant messages Network appearance configuration of BSC is inconsistent with that of MSC Server

Use NetNumen M31 client to view whether "Original point code" that configures "SIO location" at interface A is correct Use NetNumen M31 client to view whether "network appearance" that configures "local office point" is consistent with network appearance setting on MSC Server. If inconsistent, you shall modify it to be consistent

Alarm case on connection faults at interface A

Coupling disconnection due to data configuration
Check process parameters on application server for local office point in background configuration management, "Application property" is configured as "server"; verify it with the engineer at CN side, this property for MSC Server is also configured as server", if the settings for both parties are same, coupling is impossible to connect

Fault analysis

Troubleshooting

After changing "application property" on application server at BSC side to "Client", the problem is solved

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Category of Connection Faults at Interface Gb

Faults caused by data configuration: such as wrong data configuration at Gb port

Fault caused by hardware problem: such as GIPB board damaged

Troubleshooting guide about connection Faults at interface Gb

Cannot ping iBSC successfully Cannot ping SGSN successfully

Cannot ping iBSC successfully

Fault cause Troubleshooting

Subnet mask is not configured as 255.255.255.255 while configuring iBSC virtual address Cross-network virtual address with iBSC, no static route or wrong static route configuration IPGB board uses wrong network ports

Configure RPU subnet mask for iBSC virtual address as 255.255.255.255 for interface configuration in IP related configuration Correctly configure static route Make network ports used by backplane of IPGB board matched with the ones which setting in NetNumen M31

Cannot ping SGSN successfully

Fault cause Troubleshooting

Configuration incomplete Subnet mask is not configured as 255.255.255.255 while configuring SGSN virtual address Cross-network virtual address with SGSN, no static route or wrong static route configuration

Port, loopback address, NSEI, local and remote IP port, and NSVC shall be configured in order Configure subnet mask for loopback address as 255.255.255.255 for interface configuration in protocol stack configuration Correctly configure static route in NetNumen M31

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Category of hardware faults

System faults Boards faults

System faults
Interconnected board link faults

Environmental faults

System fault

Startup faults

Clock faults

External interface link faults

Boards faults
OMP

GUIM

CMP UIMC

GUP2

Board
GIPI

CHUB

ICM GLI PSN

SBCX

Alarm Case on board faults

-------Instance of OMP restarting repeatedly

Fault analysis

It is found that OMP module type is configured as OMP and RPU. MP type is selected as OMP_SMP_CMP while creating OMP version of ini file

Troubleshooting

Re-create OMP version of ini file and select its MP type as OMP for later solution

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Category of CS service faults

Number Category of CS service faults

Access faults

Fault caused by frequent location update

Faults of service failure QoS fault

2 3 4

Handover faults Congestion faults Call drop faults Ground resource congestion Wireless resource congestion Call drop due to hardware problem Call drop due to data configuration and optimization

Voice faults

Faults from wireless environment

Faults from data configuration Faults from hardware issues

Troubleshooting guide about access faults(1)

-------Frequent Location Update
Fault cause Troubleshooting

When MS is powered on, it finds the network and performs a location update MS performs location update periodically MS is moved to a new location area

It needs to determine whether location areas are too small, and plan the location areas again if necessary It needs to adjust the radio coverage range, and if possible, increase the minimum access level of the cell Cooperate with MSC to modify the value of timer T3212

Troubleshooting guide about access faults(2)

-------Problem about High Priority Subscriber Access
Fault cause Troubleshooting

BSC provides many ways for the high priority subscriber to occupy resource first, which includes reserving a few TCH/F for high priority subscribers, and making the low priority subscriber handover to adjacent cell. The fault that call fails even for a high priority subscriber indicates the above measures are not taken effectively

In Other property of Cell params, open Use cell EMLPP threshold to configure some reserved channel for high-priority user, and set Reserved rate of eMLPP to configure user priority threshold of reserved channel Enable the cell parameters of allowing forced handover and allowing forced disconnection in Service process additional params

Alarm Case on access faults

-------Call disconnection due to different voice encryption modes at core network and radio ride

Fault analysis

By signaling trace SCCP protocol, it is found that unmatched encryption algorithm is prompted in connection error We verify and confirm with core network side that A5/1 voice encryption algorithm is opened and default iBSC parameters are closed

Troubleshooting

After modifying and using A5/1 encryption in Other property, the call is restored

Troubleshooting guide about handover faults

-------High Ratio of Handover Faults
Fault cause Troubleshooting

The handover failure of radio interface is related to the radio quality of the handover-in cell, for example, the handover-in cell has severe interference, hardware fault, or failure of obtaining the timing advance

Find out towards which cell the handover failure ratio is high Check whether the handover-in cell with handover failure has hardware faults in some carriers, improper power, or high assignment failure ratio Check whether the handover-in cell with handover failure has interference Check whether MARGIN settings for handover threshold window, slide window, and adjacent window are reasonable Check whether the adjacent relation is correct

Troubleshooting guide about congestion faults(1)

-------SDCCH congestion
Fault cause Troubleshooting

The location areas are too small Burst traffic (short messages for example) causes SDCCH congestion The traffic is too heavy and too few SDCCHs are configured Hardware fault of the carrier causes SDCCH congestion Interference on the radio interface causes congestion Periodical location updating timer T3212 timeout is too short

Adjust the location area size Enable the SDCCH dynamic conversion function Increase the number of SDCCHs in configuration Set MAX number of re-transmitting events times to reduce the number of random access retransmitted by MS Troubleshoot the hardware fault or replace the carrier to restore the number of available SDCCHs Check whether there is any TCH carrier with the same frequency and same color code as those of BCCH nearby

Troubleshooting guide about congestion faults(2)

-------IPA resource congestion
Fault cause Troubleshooting

By creating Performance Statistics at MAC Port and Performance Statistics at IP Port in background performance management, view transmitting & receiving bytes, its average rate, transmitted & received packet number, and discarded packet number View resource occupancy condition in IPA Management dynamically managed from background BSC data

Add IPI board Add AIPB board

Troubleshooting guide about call drop faults

-------Call drop due to interface
Fault cause Troubleshooting

Interference includes cochannel interference and adjacentchannel interference. When MS receives strong co-channel or adjacent-channel interference signals in the service cell, the signal quality degrades, which makes MS unable to accurately demodulate the BSIC of the adjacent cells or makes BTS unable to correctly receive the MS measurement report, thus the call quality becomes worse and call drop occurs

Interference can be avoided by modifying the BTS transmission power, downtilt of the antenna, or the frequency point plan of the cell concerned Use DTX, frequency-hopping technology, power control, and diversity Adjust the intra-cell handover parameters relevant with interference Adjust the handover threshold and cell reselection parameters to lower the cell traffic

Troubleshooting guide about voice faults

-------Echo problem
Fault cause Troubleshooting

Echo suppressor fault or a few trunks not connecting echo suppressors may all cause the echo problem The fault may be due to MS problems The fault may be due to BTS hardware problems The self loop of a few Ainterface circuits or inter-office trunks will cause that only ones own echo can be heard and the opposite partys voice can not be heard

Ensure whether the fault exists only in a few MSs Check whether the trunk self loop exists Check whether the echo suppressor of MSC works properly Decide which hardware of BTS is related to the fault through dialing test and signaling tracing, and replace relevant faulty hardware

Content

Overview NMS fault troubleshooting Data and software loading fault troubleshooting Connection fault troubleshooting at A interface Connection fault troubleshooting at Gb interface Hardware fault troubleshooting CS service fault troubleshooting PS service fault troubleshooting

Troubleshooting guide about PS service faults(1)

-------PDCH allocation faults

Fault cause

Troubleshooting

Insufficient radio channel Insufficient Abis transmission resource Poor UPPB DSP resource

Adjust wireless channel configuration and add PDCH channels Add transmission at Abis Adjust UPPB DSP or add UPPB boar

Troubleshooting guide about PS service faults(2)

-------MS PDP activation faults
Fault cause Troubleshooting

Network of the computer connected to MS isnt installed correctly The dial-up network setting is incorrect GPRS parameter setting of the MS is incorrect

Ensure the dial-up network is correctly installed in the network properties Check the dial-up network setting to ensure the data configurations (such as port number, rate setting etc) are correct Ensure the data configurations (such as APN, gateway setting) of MS are correct