Sei sulla pagina 1di 39

NokiaNetworks

WCDMA RAN, Rel. WCDMA


16, Operating Documentation,
Issue 03

Administering Recovery and


Unit Working State in IPA-
RNC
DN02143192
Issue 20
Approval Date 2015-06-30

AdministeringRecoveryandUnitWorkingStateinIPA-RNC

The information in this document applies solely to the hardware/software product (Product) specified
herein,andonlyasspecifiedherein.

ThisdocumentisintendedforusebyNokiaSolutionsandNetworks'customers(You)only,anditmaynot
beusedexceptforthepurposesdefinedintheagreementbetweenYouandNokiaSolutionsandNetworks
(Agreement) under which this document is distributed. No part of this document may be used, copied,
reproduced, modified or transmitted in any form or means without the prior written permission of Nokia
Solutions and Networks. If you have not entered into an Agreement applicable to the Product, or if that
Agreementhasexpiredorhasbeenterminated,YoumaynotusethisdocumentinanymannerandYou
areobligedtoreturnittoNokiaSolutionsandNetworksanddestroyordeleteanycopiesthereof.

The document has been prepared to be used by professional and properly trained personnel, and You
assumefullresponsibilitywhenusingit.NokiaSolutionsandNetworkswelcomeYourcommentsaspartof
theprocessofcontinuousdevelopmentandimprovementofthedocumentation.

This document and its contents are provided as a convenience to You. Any information or statements
concerningthesuitability,capacity,fitnessforpurposeorperformanceoftheProductaregivensolelyon
anasisandasavailablebasisinthisdocument,andNokiaSolutionsandNetworksreservestheright
tochangeanysuchinformationandstatementswithoutnotice.NokiaSolutionsandNetworkshasmadeall
reasonableeffortstoensurethatthecontentofthisdocumentisadequateandfreeofmaterialerrorsand
omissions, and Nokia Solutions and Networks will correct errors that You identify in this document. But,
NokiaSolutionsandNetworks'totalliabilityforanyerrorsinthedocumentisstrictlylimitedtothecorrection
ofsucherror(s).NokiaSolutionsandNetworksdoesnotwarrantthattheuseofthesoftwareintheProduct
willbeuninterruptedorerror-free.

NO WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO
ANY WARRANTY OF AVAILABILITY, ACCURACY, RELIABILITY, TITLE, NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, IS MADE IN RELATION TO THE
CONTENT OF THIS DOCUMENT. IN NO EVENT WILL NOKIA SOLUTIONS AND NETWORKS BE
LIABLE FOR ANY DAMAGES, INCLUDING BUT NOT LIMITED TO SPECIAL, DIRECT, INDIRECT,
INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED TO LOSS OF
PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY OR DATA THAT MAY
ARISEFROMTHEUSEOFTHISDOCUMENTORTHEINFORMATIONINIT,EVENINTHECASEOF
ERRORSINOROMISSIONSFROMTHISDOCUMENTORITSCONTENT.

ThisdocumentisNokiaSolutionsandNetworksproprietaryandconfidentialinformation,whichmaynotbe
distributed or disclosed to any third parties without the prior written consent of Nokia Solutions and
Networks.

Nokia is a registered trademark of Nokia Corporation. Other product names mentioned in this document
maybetrademarksoftheirrespectiveowners,andtheyarementionedforidentificationpurposesonly.

Copyright2016NokiaSolutionsandNetworks.Allrightsreserved.

f Important Notice on Product Safety


Thisproductmaypresentsafetyrisksduetolaser,electricity,heat,andothersourcesofdanger.

Only trained and qualified personnel may install, operate, maintain or otherwise handle this
productandonlyafterhavingcarefullyreadthesafetyinformationapplicabletothisproduct.

The safety information is provided in the Safety Information section in the Legal, Safety and
EnvironmentalInformationpartofthisdocumentordocumentationset.

Nokia Solutions and Networks is continually striving to reduce the adverse environmental effects of its
productsandservices.Wewouldliketoencourageyouasourcustomersanduserstojoinusinworking
towardsacleaner,saferenvironment.Pleaserecycleproductpackagingandfollowtherecommendations
forpoweruseandproperdisposalofourproductsandtheircomponents.

IfyoushouldhavequestionsregardingourEnvironmentalPolicyoranyoftheenvironmentalserviceswe
offer,pleasecontactusatNokiaSolutionsandNetworksforanyadditionalinformation.

2 DN02143192 Issue:20
AdministeringRecoveryandUnitWorkingStateinIPA-RNC

Table of Contents
Thisdocumenthas39pages

Summaryofchanges..................................................................... 6

1 Overviewofrecoveryandunitworkingstateadministration.......... 7
1.1 Unitworkingstates.........................................................................7
1.2 Restarts.......................................................................................... 9
1.3 Recoveryhistory.......................................................................... 10

2 Managingunitworkingstate........................................................ 15
2.1 Interrogatingunitworkingstateandstatus.................................. 15
2.2 Activatingswitchover....................................................................17
2.3 Configuringrevertiveswitchover.................................................. 18
2.4 Takingaunitoutofuse................................................................ 19
2.5 Takingaunitbackintouse...........................................................23
2.6 Changingunitstatusinformation..................................................23

3 Managingrestarts........................................................................ 25
3.1 Restartingfunctionalunit..............................................................25
3.2 Restartingsystem........................................................................ 26
3.3 Verifyingsystemstart-up..............................................................27
3.4 Managingcodepreloading...........................................................28
3.4.1 Preloadingthecodefromexternalsourcestounitmemory.........28
3.4.2 Interrogatingsystempreloadinginformation................................ 29
3.4.3 Cancelingcodepreloading...........................................................30

4 Interrogatingrecoveryhistory.......................................................31

5 Recoveryandunitworkingstateadministrationtroubleshooting.....
32
5.1 Unitrestartingfails....................................................................... 32
5.2 Workingstatechangefails........................................................... 33
5.3 Workingstateisincorrect............................................................. 35
5.4 TheactiveOMUcannotstartupproperly.................................... 36

Issue:20 DN02143192 3
AdministeringRecoveryandUnitWorkingStateinIPA-RNC

List of Figures
Figure1 UnitsthathavenoredundancyorSN+redundantunits.......................8
Figure2 Otherredundantunits(2NorN+1units).............................................. 9
Figure3 Structureofrecoveryhistoryrecord....................................................11
Figure4 WO-EXtoSE-NHforSN+ornon-redundantunitwithaBLstate...... 19
Figure5 WO-EXtoSE-NHforSN+ornon-redundantunitwithoutaBLstate.....
19
Figure6 WO-EXtoSE-NHforanactive2NorN+1redundantunit................ 20

4 DN02143192 Issue:20
AdministeringRecoveryandUnitWorkingStateinIPA-RNC

List of Tables
Table1 Theworkingstatesoffunctionalunits.................................................. 7
Table2 Theplug-inunit'sLEDdisplaywithdifferentworkingstates...............22

Issue:20 DN02143192 5

Summaryofchanges AdministeringRecoveryandUnitWorkingStateinIPA-
RNC

Summary of changes
Changesbetweendocumentissuesarecumulative.Therefore,thelatestdocument
issuecontainsallchangesmadetopreviousissues.
Notethattheissuenumberingsystem,safetyinformation,andproductnamingpractice
arechanging.Formoreinformation,seeGuide to WCDMA RAN Operating
Documentation.
Changes between issues 19 (2014-08-08, RU50) and 20 (2015-06-30, WCDMA16)
Thedocumentisrestructured.
Changes between issues 18 (2012-11-30, RU40) and 19 (2014-08-08, RU50)
Overview of recovery and unit working state administration
Thechapterisrestructured.

Changes between issues 17C (2011-06-10, RU30) and 18 (2012-11-30, RU40)


Restarts

Blackboxforminginformationisadded.

Restarting system

Blackboxforminginformationwithitsexampleisadded.

6 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Overviewofrecoveryandunitworkingstate
RNC administration

1 Overview of recovery and unit working state


administration
Recovery is used in error conditions and system start-ups. It prevents the escalation of a
fault by isolating the faulty unit and informing the recovery system.

Arecoveryactioncanbe,forexample:

takingaspareunitintouse
changingthestateofafaultyunit
restartingthefaultyunit

Generally,recovery,includingworkingstatechange,isperformedautomaticallybythe
maintenancesoftware.Thesystemrecoversfromfaultsmorereliablywhenits
configurationiscomplete.Ifanautomaticrecoveryisnotpossible,MMLcommandgroup
US - Working State and Recovery Handlingcanbeused.
IftheMMIuserterminalisnotavailable,youhavetoactivaterecoveryoperationswith
theRCJUUSserviceterminalextension.

1.1 Unit working states


The correct working states are WO-EX, SP-EX, BL-EX and BL-ID. If a unit is
permanently in an incorrect working state, an alarm output is given at regular intervals.

Table 1 Theworkingstatesoffunctionalunits

Main state Full state Name

WO WO-EX Working,executing

WO-RE Working,restarting

BL BL-EX Blocked,executing

BL-ID Blocked,idle

BL-RE Blocked,restarting

SP SP-EX Spare,executing

SP-UP Spare,warmingup

SP-RE Spare,restarting

TE TE-EX Teststate,executing

TE-RE Teststate,restarting

Issue:20 DN02143192 7

Overviewofrecoveryandunitworkingstate AdministeringRecoveryandUnitWorkingStateinIPA-
administration RNC

Table 1 Theworkingstatesoffunctionalunits(Cont.)

Main state Full state Name

SE SE-OU Separated,outofuse

SE-NH Separated,nohardware

TR TR-OU Transition,outofuse

g Note: TR-OUisatemporarytechnicalworkingstatehandledbytheRecoverySystem,
anduserinterferenceisnotneededorallowed.

Incaseofahierarchicalunit,whichconsistsofagroupofnestedfunctionalunits,the
parentunitrestrictsthepossibleworkingstateofitschildunits.Youcaninterrogatethe
hierarchyoftheunitswiththeUSIMMLcommand(marking>meansthatunithaschild
units).Theworkingstatesofthechildunitscanbechangedwithoutaffectingtheother
unitsofthetree.

Unit working state change


The unit working state needs to be changed manually in some situations.The change of
unit working state follows particular principles.

Unitworkingstateneedstobemanuallychangedinthefollowingsituations:
diagnositcsconductedmanuallybyUDU
HWreplacement
HWexpansion
eSWupgrade
othercasesthatrequireamanualunitstatechange

Ingeneral,youcanchangetheworkingstateofaunitonlyfollowingtheprinciples
presentedinFigure1:UnitsthathavenoredundancyorSN+redundantunitsandFigure
2:Otherredundantunits(2NorN+1units).Thesubstateisnotdefinedinthe
modificationcommand.NotalltheunitshaveBLstate.

Figure 1 UnitsthathavenoredundancyorSN+redundantunits

WO BL TE SE

8 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Overviewofrecoveryandunitworkingstate
RNC administration

Figure 2 Otherredundantunits(2NorN+1units)

WO SP TE SE

t Tip: MMLsgivepossibleguidancewithworkingstatetransitions.

Astatetransitionisamultiphaseevent,whichnormallyreservestherecoverysystem
andtheMMLterminalforthedurationoftheexecution(astatetransitioncantakea
maximumoffiveminutes).Ifthestatetransitionlastsalongtime(morethanaminute),
youareaskedwhetheryouwanttowaituntilthecommandexecutioniscomplete.You
canselectthecommandtobeexecutedinthebackgroundwhichreleasestheMML
terminal.Inthiscase,youarealsoinformedofthestatetransitionsalreadyperformedby
thestatetransitionalarmprintouts.

Unit status information


In addition to the unit working states, there is also additional status information that is
either fixed or modifiable, which is associated with the state of a unit and describes the
functional characteristics of the unit in more detail.

Normally,mostoftheunitstatusinformationissetandremovedbythesystem.
Theneedtochangestatusinformationoccursmainlyduringcommissioning,inerror
conditions,andduringtesting.
Thestatusyoucanchangeare:

RBAN Recoveryactionscanonlybecarriedoutbytheuser.

FBAN Recoveryactionscannotbecarriedoutbytheusernorthe
system.

FLTY Thefunctionalunitissuspectedtobefaultyandthe
functionalunit'smainworkingstateisnormal(WO/SP).

TCAL Thefunctionalunitisselectedtoruntestcallsandthe
functionalunit'smainworkingstateisBL.

Therecoverysystemsetsstatusinformationasdirectedbyyouorthesystem.Status
informationcanbeoutputusing,forexample,theMMLcommandUSI-Interrogateunit
stateandstatusinfo.

1.2 Restarts
The restart control manages the start-up of the network element as a whole and the
individual start-up of functional units.

Issue:20 DN02143192 9

Overviewofrecoveryandunitworkingstate AdministeringRecoveryandUnitWorkingStateinIPA-
administration RNC

Normally,restartisdoneautomaticallyasarecoveryactionbythesystem.Ifthe
softwarebuildischangedandyouwanttoloadnewprogramblockstotheunits,for
example,SWoreSWupgrade,youcanrestarttheunitsbyusingMMLcommands.
Restartingafunctionalunitonlyaffectsthatunit.Theunit'sstateisthesamewithitwas
inbeforetherestart.WhentheWOunitofa2Nredundantunitisrestarted,theSPunitis
restartedaswell.SeedetailsintheUSUMMLcommandmanual.
Restartingthesystemrestartsallthecomputerunits,includingtheactiveoperationand
maintenanceunit.SeedetailsintheUSSMMLcommandmanual.
Therestartalwaysinterruptstheoperationofthefunctionalunitcompletely.Thesystem
continuestooperatewithoutdisturbancesiftherestartedunitisinSPstate.Therestart
ofaunitinWO-EXstatealwayscausesdisturbancestotheoperationofthesystem.The
restartofanactiveOMU,SFUorRSMUcausesthesystemtorestart.Therestartofan
activeMXUresultsinarestartoftributaryunits.
TherestartofaunitcausedbyanycommandotherthantheUSMMLcommandgroupor
theserviceterminalextensioncommands,takesplacespontaneouslyfromtherecovery
systempointofview,andthereforeentailsrecoveryoperations.Therecoveryoperations
whicharecarriedoutdependonwhatstatetheobjectunitwasinbeforetherestartand
whetheritisarepeatedstartupornot.Inthecaseof2Nunits,ifthespareunitisinstate
SP-EX,animmediateswitchoverisperformedaftertheunitinWO-EXstatehas
restarted.

1.3 Recovery history


Recovery history provides a finite list of previous recovery events on a network element.
The recovery system can store approximately 15 000 pieces of latest recovery history
records.

Related procedures
Interrogatingrecoveryhistoryonpage31

Structure of recovery history record


Recovery history record consists of four parts: general information, recovery excitation,
recovery analysis, and result.

Inthecaseofrecordsbeinglost,therecoveryhistoryprintoutwillbedifferentfromthose
innormalcases.
SeeExamplesofrecoveryhistoryforreference.

10 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Overviewofrecoveryandunitworkingstate
RNC administration

Figure 3 Structureofrecoveryhistoryrecord
Structureoftherecoveryhistoryrecord

Sequentialnumber: 1
Objectunit: 2 Priority: 3 Recoveryexcitationsender
Entrytime : 4 Unit : 8
Recoverystarttime : 5 Family : 9
Recoveryendtime : 6 Process : 10
Recoveryduration : 7 Focus : 11

RECOVERY EXCITATION

12

ANALYSISIN 13
Action : 14
Status : 15
Primaryunit : 16
Primarystate : 17
Primarystatus : 18
Secondaryunit : 19
Secondary state : 20
Secondary status : 21

RESULT
22
Observerunit : 23
Observerfamily : 24
Accusedunit : 25
Accused family : 26
Errorcode : 27

g Note: Fields23-27aredisplayedonlyinfailedrecoverytasks.

1. Sequentialnumber
Itisaconsecutive32-bitnumber,startingfrom0.Everypieceofrecoveryhistory
recordhasauniquesequentialnumber.Withthisnumber,usercaneasilyseeif
therearepiecesofrecoveryhistoryrecordlost.
2. Objectunittypeandindexname
3. Priority
Itindicatesthepriorityoftherecoveryexcitation.Therangeisfrom0to255.
Thisfieldisonlyprintedinsystem-originatedactions,notinuser-originatedactions.
4. Entrytime
Itisthedateandtimewhenrecoveryexcitationisreceivedbytherecoverysystem.

g Note: Fields4-7refertorecoveryexcitationsender.Iftherecoverytaskisinitiatedby
theuser,therecoveryexcitationsenderisRUUSER.

5. Startdateandtimeoftherecoveryevent
6. Enddateandtimeoftherecoveryevent
7. Durationoftherecoveryevent

Issue:20 DN02143192 11

Overviewofrecoveryandunitworkingstate AdministeringRecoveryandUnitWorkingStateinIPA-
administration RNC

8. Unittypeandindex
Itistheunittypeandindexoftherecoveryexcitationsender.
9. Family
Itisthefamilyoftherecoveryexcitationsender.
10. Process
ItistheprocessIDoftherecoveryexcitationsender.
11. Focus
ItisthefocusIDoftherecoveryexcitationsender.
12. Recoveryexcitation
Thesituationorimpulse,whichtriggerstheexecutionofrecoverytask,iscalled
recoveryexcitation.
Thisblockdescribeswhattherecoveryexcitationis,anditusuallycontainsmore
thanonerow.
13. SYMunitwheretheanalysiswasmade
14. Actionthatwastakenduringtherecovery
15. Status:errorcodeindecimalformat
Iftherecoverytaskiscarriedoutsuccessfully,theprintoutwillbe'0000000000'.
16. Primaryunittypeandindex
Theprimaryunitistheobjectunitonwhichtherecoverytaskwascarriedout.
17. Primaryworkingstatename
Ifworkingstateisnotdefined,itshowsahyphenhere.
18. Primarystatusname
Ifthestatusisnotdefined,itshowsahyphenhere.
19. Secondaryunittypeandindex
Thesecondaryunitistheobjectunitonwhichtherecoverytaskwascarriedout,in
thecaseofaswitchover.
20. Secondaryworkingstatename
Ifworkingstateisnotdefined,itshowsahyphenhere.
21. Secondarystatusname
Ifthestatusisnotdefined,itshowsahyphenhere.
22. Hereitdescribestheresultoftherecoveryevent.
Ifitshows'Recoverytaskcompletedsuccessfully',fields23-27willnotbedisplayed.
23. Observerunittypeandindex
Theobserverunitistheunitwhichobservesthefailureoftherecoverytask.
24. Observerfamily
Itisthefamilywhichobservesthefailureoftherecoverytask.
25. Accusedunittypeandindex
Theaccusedunitistheunitwhichcausedthefailureoftherecoverytask.
26. Accusedfamilyname
Itisthefamilywhichcausedthefailureoftherecoverytask.
27. Errorcodeindecimalformat

Related procedures
Interrogatingrecoveryhistoryonpage31

Related references
Examplesofrecoveryhistoryonpage13

12 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Overviewofrecoveryandunitworkingstate
RNC administration

Examples of recovery history


Examples are presented to illustrate the recovery history printouts both in the normal
case and in the case of records being lost.

Examplefornormalcase
Sequential number: 116615
Object unit: NPS1P-0 Priority: 228 Recovery excitation sender
Entry time : 2005-03-21 15:23:57.53 Unit : OMU-0
Recovery start time: 2005-03-21 15:23:57.53 Family : RXEPRB
Recovery end time : 2005-03-21 15:24:09.36 Process : 00009H
Recovery duration : 00:00:11.83 Focus : 002H

RECOVERY EXCITATION
Activated recovery task failed
Last analysis : Soft disturbance switchover
Original analysis : Soft disturbance switchover
Original sending unit : OMU-0
Original sending family : ALARMP
Original fault class : Disturbance
Original alarm(s) : 00001277 - -

ANALYSIS IN OMU-0
Action : Emergency switchover
Status : 0000000000
Primary unit : NPS1P-0
Primary state : TE
Primary status : -
Secondary unit : NPS1P-1
Secondary state : WO
Secondary status : -

RESULT
Recovery task failed
Observer unit : OMU-0
Observer family : RXEPRB
Accused unit : NPS1P-1
Accused family : RXUPRB
Error code : 0000000279
Exampleforthecaseofrecordsbeinglost
Sequential number : 116612
Record lost in OMU-0
Start time : 2005-03-21 15:23:57.53
End time : 2005-03-21 15:24:09.36
Duration : 00:00:11.83
Lost records : 10

Related procedures
Interrogatingrecoveryhistoryonpage31

Related references

Issue:20 DN02143192 13

Overviewofrecoveryandunitworkingstate AdministeringRecoveryandUnitWorkingStateinIPA-
administration RNC

Structureofrecoveryhistoryrecordonpage10

14 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Managingunitworkingstate
RNC

2 Managing unit working state

2.1 Interrogating unit working state and status


The unit states and status provide a fast overall picture of the network element from the
maintenance point of view. You can interrogate the unit states and status with the USI
MML command.

Summary
ThenormalworkingstatesareWO-EX,SP-EX,andSE-NH.
TheINFOcolumnintheprintoutshowsthestatusoftheunit.Adash(-)intheINFO
columnindicatesthenormalsituation,thatis,aunitwithoutstatus.
The>markingaftertheunitindicatesthattheunitisahierarchicalone.Thechildunits
arenotprintedbydefault.
Incaseofahierarchicalunit,thedash(-)intheINFOcolumnindicatestwothings:

allunitsinthehierarchyhavethesamestateastheunititself,and
theunithasnostatus

Ifthereisanm/nmarkingintheinfofield(wheremandnarenumbers),itindicatesthat
munitsofthetotalofnunitsareavailableforthesystem.Ifmisasmallernumberthan
n,itindicatesadefectinsomeunit(s)intheunithierarchy.

Procedure

1 Check that units are in their normal working state (USI)


ZUSI;

Example:One example of the execution printout for USI MML command


WORKING STATES OF UNITS

UNIT PHYS LOG STATE INFO


OMU-0 0000H 4002H WO-EX -
OMU-1 0001H 4002H SP-EX -
. . .
MXU-0 001BH 4243H WO-EX -
MXU-1 0038H 4243H SP-EX -
. . .
NPS1P-0 > 000FH 4612H WO-EX -
NPS1P-1 > 0031H 4612H SP-EX -
. . .

SFU-0 0004H 45FDH SP-EX -


SFU-1 002FH 45FDH WO-EX -
TBU-0 SP-EX -

Issue:20 DN02143192 15

Managingunitworkingstate AdministeringRecoveryandUnitWorkingStateinIPA-
RNC

TBU-1 WO-EX -
. . .
NUMBER OF UNITS DISPLAYED = 53

COMMAND EXECUTED

2 Check the state and status of all units in the hierarchy (USI)
ZUSI:<unit>,<index>::FULL;

3 Interrogate the state and status during the TRIAL configuration


WhentheTRIALconfigurationisrunningandyougivetheUSIMMLcommand,only
theunitslocatedonthesamesidewheretheMMLcommandisgivenareprinted
out.
Ifyouwanttointerrogatetheunitsontheotherside,giveoneofthefollowing
parameters:ORIG/TRIAL/BOTH.ORIGprintsouttheunitsontheoriginalside,
TRIALprintsoutunitsonthetrialside,andBOTHprintsouttheunitsonbothsides.

Example:Display all units from both sides


ZUSI::::BOTH;
<USI::::BOTH;

LOADING PROGRAM VERSION 5.3-0

WORKING STATES OF UNITS IN ORIGINAL SIDE 2008-04-28 18:57:21

UNIT PHYS LOG STATE INFO


OMU-0 0000H 4002H WO-EX -
MXU-0 0007H 4242H WO-EX -
. . .
NPS1P-0 > 000FH 4612H WO-EX -
NPS1P-1 > 0031H 4612H SP-EX -
. . .
ICSU-0 00BFH 444FH WO-EX -

. . .
SFU-1 0006H 45FDH WO-EX -
. . .
TBU-0 WO-EX -
TBU-1 SP-EX -
. . .

NUMBER OF UNITS DISPLAYED = 96

WORKING STATES OF UNITS IN TRIAL SIDE 2008-04-28 18:57:21

16 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Managingunitworkingstate
RNC

UNIT PHYS LOG STATE INFO


OMU-1 0001H 4002H WO-EX -
MXU-1 000CH 4242H WO-EX -
. . .
ICSU-1 00C3H 444FH WO-EX -

SFU-0 0004H 45FDH WO-EX -

NUMBER OF UNITS DISPLAYED = 12

COMMAND EXECUTED

2.2 Activating switchover


Switchover is possible for 2N and N+1 redundant units. You can perform the controlled
switchover only if the spare unit is in state SP-EX.

Purpose
TheswitchoverofOMUstopstheMMLsessionandyouhavetostartanewsessionina
newWOunit.
InthecaseofanN+1redundantunit,thecontrolledswitchoverisallowedifonespare
unitisinstateSP-EX.
TheworkingstatechangeSP->WOdoesnotactivateswitchoverinN+1redundant
units.
Iftheunithasongoingtasks(forexample,itiscurrentlyloadingsomeinformationto
disk)whichcannotbeinterruptedinacontrolledmanner,youcanonlyperformthe
switchoverwiththeforced control(FCD)parameter.Theswitchoverisalso
performedwiththeFCDparameterin2NorN+1redundantunitsifthespareunitisinthe
SP-UP,SP-RE,TE-EX,orTE-REstate.
FCDshouldnotbeusedimmediatelyafteranunsuccessfulattempttoactivate
switchover,asperformingFCDcanresultinlostdata.Also,theusercanselectwhether
towaitforthecommandexecutionorhavethetaskexecutedinthebackgroundonlyfor
N+1unitsiftheswitchoverlastsalongtime.

w NOTICE:Theuseoftheforced control(FCD)parametercancausedamagein
callcontroloperations,forexample,insystemrestarts.
Youcanonlychangetheworkingstateifitcanbedoneinawaywhichdoesnotaffect
therestofthesystem.Theuseoftheforced control(FCD)parameterisintended
forexceptionalsituationswheneverthestatetransitionisnototherwisepossible.Ifyou
resorttothisaction,youtakefullresponsibility.

Issue:20 DN02143192 17

Managingunitworkingstate AdministeringRecoveryandUnitWorkingStateinIPA-
RNC

Change the working state of the working unit from WO to SP (USC)


ZUSC:<unit>,<unit index>:<state>;

2.3 Configuring revertive switchover


If you want to configure one of the 2N top redundancy unit pair as the default working
unit, you can activate the revertive switchover feature and set the unit to default
active status.

Purpose
Therevertiveswitchoverfeatureonlyappliesto2Ntopredundancyunits,andcannotbe
managedbyZUSF,ZUSA,ZUSPorZUSRintrialconfiguration.
Whenadefaultactiveunitrecoversfromfault,therecoverysystemchangestheunit
statefromSP-EXtoWO-EXautomatically.Inthisway,trafficswitchestotheoriginalunit
androute.Thisfeaturehelpsyoutodistributetrafficinaspecificwaythroughthesite
routers.

g Note: Whentherevertiveswitchoverfeatureisenabledandtheunitisindefaultactive
status,youcannotswitchtheunitfromWO-EXstatetoSP-EXstatewithZUSC
command.

Procedure

1 Enable revertive switchover feature in the system (USF)


ZUSF:ON;

2 Set the unit to default active status (USA)


ZUSA:[<unit type>,<unit index>];

g Note: Iftheotherunitofthe2Ntopredundancyunitpairisalreadyindefaultactive
status,thesystemwillclearitsdefaultactivestatusfirstandthensettheunitspecified
inthecommandtodefaultacivestatus.

3 Display default active unit information (USP)


ZUSP:[<unit>,<unit index>];

4 To clear the default active status in the unit, execute the following command
(USR)
ZUSR:[<unit type>,<unit index>];

18 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Managingunitworkingstate
RNC

5 To disable the revertive switchover feature, use the following command (USF):
ZUSF:OFF;

2.4 Taking a unit out of use


If you want to remove a plug-in unit, change the state of the unit to SE-NH before
physically take the plug-in unit out.

Purpose
Iftheunithasongoingtasks(forexample,loadinginformationtodisk)whichcannotbe
interruptedinacontrolledway,itisrecommendedtowaitawhileandtrytakingtheunit
outofuselater.UsetheFCDparameteronlyifyouhaveto.

Before you start


Youneedtofollowparticularprincipleswhenchangingworkingstatefromnormal
workingstateWO-EXorSP-EXtoSE-NH.
Iftheunitisnotredundantorisredundantbytheload-sharingprinciple,changethestate
oftheunitdirectlytoTE,orfirsttoBLstateandthentoTEstate.

Figure 4 WO-EXtoSE-NHforSN+ornon-redundantunitwithaBLstate

(byuser) (bysystem )
WO-EX BL-EX BL-ID

(byuser)

(byuser ) (byuser )
SE-NH SE-OU TE-EX

g Note: NotallunitshaveBLstate.

Figure 5 WO-EXtoSE-NHforSN+ornon-redundantunitwithoutaBLstate

(byuser ) (byuser ) (byuser )


WO-EX TE-EX SE-OU SE-NH

Iftheunitisa2NorN+1replaceableredundantunit,changethestateoftheunitfirstto
SPandthentoTE.

Issue:20 DN02143192 19

Managingunitworkingstate AdministeringRecoveryandUnitWorkingStateinIPA-
RNC

Figure 6 WO-EXtoSE-NHforanactive2NorN+1redundantunit

(byuser ) (byuser )
WO-EX SP-EX TE-EX

(byuser)

(byuser )
SE-NH SE-OU

Procedure

1 Check the unit working state (USI)


ZUSI:<unit type>;
Whenreplacingaplug-inunit,thefunctionalunitsoftheplug-inunitmustbeinSE-
NHstate.

Expected outcome
Whentheworkingstateofaplug-inunit'sfunctionalunitisSE-NH,theLEDdisplayis
off,andyoucancontinuereplacingtheplug-inunit.

Unexpected outcome
Iftheworkingstateofaplug-inunit'sfunctionalunitisWO-EX,gotostep2.
Iftheworkingstateofaplug-inunit'sfunctionalunitisotherthanWO-EX,gotostep
3.

2 Change the state of the unit to SP or BL-EX (USC)


Iftheunitisa2NorN+1replaceableredundantunit,changethestateoftheunitto
SP.
ZUSC:<unit type>,<unit index>:SP;
Iftheunitisnotredundantorisredundantbytheload-sharingprinciple,changethe
unitstatetoBLstate
ZUSC:<unit type>,<unit index>:BL;
IftheunitdoesnothaveaBLstate,changethestatedirectlytoTE.SeeChangethe
unitworkingstatetoTE(USC).
Iftheunithasalong-runningresource,itwillstayintheBL-EXstate.Inthiscase,
makeforcedchangeoftheworkingunitstatefromBL-EXtoTEaftertheunitfailsto
reachtheBL-IDstate.SeeChangetheunitworkingstatetoTE(USC)

3 Change the unit working state to TE (USC)


ZUSC:<unit type>,<unit index>:TE;
MakeforcedchangeoftheunitworkingstatetoTE(USC)
GivevalueFCDtothestate transition controlparameter.

20 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Managingunitworkingstate
RNC

ZUSC:<unit type>,<unit index>:TE::FCD;


Notethatthiscommandcausesdisturbancesinthenetworktraffic.

4 Change the unit working state to SE-OU (USC)


ZUSC:<unit type>,<unit index>:SE;

5 Change the unit working state to SE-NH (USC)


ZUSC:<unit type>,<unit index>:SE;

g Note: Aplug-inunitcanberemovedonlywhenitsfunctionalunitsareintheSE-NH
state.Also,afunctionalunitcanberemovedfromthesystemonlyafterithasbeenset
totheSE-NHstate(separated,nohardware)usingtheUSCMMLcommand.

Example:Changing the state of the unit directly to TE


ChangethestateofDMCUdirectlyfromWOtoTE,usingtheFCDparameter.
ZUSC:DMCU,0:TE::FCD;
Whenyouchangethestateoftheunit,thefollowingpromptappearsaskingyouto
confirmthecommandexecution:
FORCED STATE TRANSITION REQUESTED
WO-EX -> TE
CONFIRM COMMAND EXECUTION: Y/N? Y

Example:Changing the state of DMCU unit first to BL, then to TE and SE

1. ChangetheworkingstateofDMCUfromWO-EXtoBL-ID.
ZUSC:DMCU,<index>:BL;
Whentheunit'sworkingstateisBL-EX,thesystemchangesittoBL-ID.Whenthe
workingstateisBL-ID,youcanproceedandchangeittoTE.
2. ChangetheworkingstatefromBL-IDtoTE-EX.
ZUSC:DMCU,<index>:TE;
3. ChangetheworkingstatefromTE-EXtoSE-OU.
ZUSC:DMCU,<index>:SE;
4. ChangetheworkingstatefromSE-OUtoSE-NH.
ZUSC:DMCU,<index>:SE;

Example:Changing the state of the unit first to SP and then to TE


ChangetheworkingstateofOMU-1fromWO-EXtoTE-EX.BothOMUsareinnormal
workingstate.

1. ChangetheworkingstatefromWO-EXtoSP-EX.
ZUSC:OMU,1:SP;

Issue:20 DN02143192 21

Managingunitworkingstate AdministeringRecoveryandUnitWorkingStateinIPA-
RNC

ThiscommandactivatesOMU'sswitchover.TheworkingstateofOMU-1changesto
SP-EXandtheworkingstateofOMU-0changestoWO-EX.
2. ChangetheworkingstatefromSP-EXtoTE.
ZUSC:OMU,1:TE;

Example:Changing the state of the unit from TE to SE-OU and then to SE-NH
ChangetheworkingstateofOMU-1fromTE-EXtoSE-NH.

1. ChangetheworkingstatefromTE-EXtoSE-OU.
ZUSC:OMU,1:SE;
2. ChangetheworkingstatefromSE-OUtoSE-NH.
ZUSC:OMU,1:SE;

LED display with different working states


You can tell the different working states of a unit from its LED display.

Table 2 Theplug-inunit'sLEDdisplaywithdifferentworkingstates

Working state LED display

WO-EX Green

WO-RE Greenblink

BL-EX Green

BL-ID Green

BL-RE Greenblink

SP-EX Orange

SP-UP Greenblink

SP-RE Greenblink

TE-EX Redblink

TE-RE Redblink

SE-OU Red

SE-NH None

TR-OU Undefined

22 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Managingunitworkingstate
RNC

2.5 Taking a unit back into use


When install the plug-in unit back to subrack, the unit can be changed in a controlled way
to TE-EX state for diagnostic testing or changed to WO or SP state directly using the
FCD parameter.

Change the unit state from TE or SE to WO or SP. (USC)


ZUSC:<unit type>, <unit index>: <working state>;

Example: Changing the working state of OMU-1 from SE-OU to SP

1. ChangetheunitstatefromSE-OUtoTE-EX.
ZUSC:OMU,1:TE;
2. ChangetheunitstatefromTE-EXtoSP-EX.
ZUSC:OMU,1:SP;

2.6 Changing unit status information


Change the unit status information only if you have the required expertise or knowledge
of the consequences to do so.

Purpose
YoucanbothsetandcleartheRBAN,FBANandTCALstatus,butonlycleartheFLTY
statusbytheserviceterminalRCJUUS.StatusRBANandFBANcanonlybesetforthe
hierarchicallyhighestunit,andthesystemtakescareofextendingthestatusthroughout
theentirehierarchy.

Procedure

1 Take the recovery service terminal (RCJUUS) into use (ZLP)


TotakearemoteserviceterminalsessiontotheworkingOMUunit.
ZDDS;
Totaketherecoveryserviceterminalintouse.
ZLP:J,RCJ;
Formoreinformation,seeActivatingserviceterminalextensionsusingtheLP
commandinDMXunitsinService terminal essentials.

2 Set/clear status info (ZJT)


TosetstatusinfogiveprefixSforstatusinfo.
ZJT:<unit type>,<unit index>,,,S,<status>;

Issue:20 DN02143192 23

Managingunitworkingstate AdministeringRecoveryandUnitWorkingStateinIPA-
RNC

ToclearstatusinfogiveprefixC.
ZJT:<unit type>,<unit index>,,,C,<status>;

24 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Managingrestarts
RNC

3 Managing restarts

3.1 Restarting functional unit


You can select the code loading mode in the functional unit restart. The restart type with
parameters is available in the restart commands.

Purpose
Whenrestartingaunit,youcanalsodefinewhethertherestartiscontrolledorforced
withtheparameterrestart control.Ifyoudonotgiveaparameter,thedefault
value,CONTROLLEDRESTART,isused.TheFCD(forcedcontrol)parametermay
causeseveredisturbancesintheoperationofthesystemifaunitinWOstateis
restartedinaforcedmanner.

w NOTICE:
NOTICE: Restartingcriticalunits,forexampletheactiveOMU,restartsthewhole
system.
RestartingaunitinWOstatealwayscausesdisturbancestothesystem.

g Note: YoucannotrestartafunctionalunitwithOPTloadingmodeifyouhavepreloaded
thecodetotheunitmemorypreviously.

Restart a functional unit (USU)


ZUSU:<unit type>,<unit index>:[<restart type>|WARM
def]:[(<code load mode>)]:[FCD|<controlled
restart>def]:[FULL|<child units in same main state>def];

Example:Restarting the spare Operation and Maintenance Unit (OMU)


MakesurethattheOMU-1isspareandnoticethatOPTloadingcannotbeusedifthe
codehasbeenpreloadedtothesystem.
Usethedefaultvaluesoftheparameters.Therestartiscontrolledandoptimalloading
(OPT)isused.
ZUSU:OMU,1;

Example:Activating forced restart of the spare OMU


MakesurethattheOMU-1isspare.
Optimalloading(OPT)isused.
ZUSU:OMU,1:::FCD;

Issue:20 DN02143192 25

Managingrestarts AdministeringRecoveryandUnitWorkingStateinIPA-
RNC

Thefollowingconfirmationtextappearsonscreenaskingyoutoconfirmthecommand
execution:
FORCED RESTART REQUESTED IN OMU-1
CONFIRM COMMAND EXECUTION: Y/N?

Example:Restarting an SP-EX N+1 redundant computer unit with DSK loading


mode
MakesurethattheICSU-0isspare.
ThisexampleisforICSU-0,butitissimilarinothercaseswhenrestartinganN+1
redundantcomputerunitwithDSKloadingmode.
ZUSU:ICSU,0::DSK;

Example:Restarting the unit with cold restart type


MakesurethatthefunctionalunitICSU-0isconfiguredwithCCP1D-Aplug-inunit.
UseCOLDrestarttype.
ZUSU:ICSU,0:COLD;

3.2 Restarting system


You can select the code loading mode in a system restart.The restart type of the system
is WARM.

Purpose
Whenrestartingthesystem,youcanalsodefinewhethertherestartiscontrolledor
forcedwiththeparameterrestart control.Ifyoudonotgiveaparameter,the
defaultvalueCONTROLLEDRESTARTisused.TheFCD(forcedcontrol)parameter
maycauseseveredisturbancesintheoperationofthesystem.

Before you start


Preloadthecodeofthesoftwarebuildtothesystembeforehandifyouwanttorestartthe
systemwithOTOloadingmode.SeePreloadingthecodefromdisktounitmemoryfor
moreinformation.

Restart the system (USS)


ZUSS:[<restart type>|WARM def]:[<code load
mode>]:[FCD|<controlled restart>def]:[BB|<blackbox not
formed> def];

Example:Restarting the system using default values

26 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Managingrestarts
RNC

Activateatotalsystemrestartandusethedefaultvaluesoftheparameters.Ifyouhave
preloadedthecodetothesystempreviously,thedefaultloadingmodeisTOT.Ifyou
haveneverpreloadedthecodetothesystempreviously,thedefaultloadingmodeis
OPT.
ZUSS;

Example:Restarting the system and define code loading mode


Activateatotalsystemrestartanddefinethatthecodeisloadedfromanexternal
source.WhenthecodeloadingmodeisTOT,theunittypedefinitionsarerecreated
accordingtotheunittypespecificXMLdefinitions.
ZUSS::TOT;

Example:Restarting the system after preloading the code of the running software
build to the system
Preloadtherunnningsoftwarebuildtothesystembeforeyourestartthesystem.Allor
essentialpartofcodeofsoftwarebuildwillbepreloadedtotheunitsmemory.After
preloading,restartthesystemanddefinethecodeloadingmodeasOTO.
ZUSS::OTO;

Example:Restarting the system with blackbox forming


ZUSS::::BB;

3.3 Verifying system start-up


You need to verify that the system starts up correctly in the commissioning process.

Procedure

1 Checking that units are in the WO-EX and SP-EX (USI) states
ZUSI;
Ifaunitisnotinthecorrectstate,changethestatewiththeUSCMMLcommand.

2 Disconnect power from the system

w NOTICE:Beforeyoudoit,makesurethatthiswillnotresultinanyproblems,suchas
hardwaredamageordataloss.

Issue:20 DN02143192 27

Managingrestarts AdministeringRecoveryandUnitWorkingStateinIPA-
RNC

3 Connect power to the system


Expected outcome
Thesystemstartsandloadsfromthedisks.
DuringtheOMUstart-up,therecoverysystemanalysestheconditionofthenetwork
elementandstartsthecontrolofthesystemstart-up.Thefollowingtextisprintedon
thealarmprinter:0689 SYSTEM RESTARTED.

4 Wait until active OMU starts in state WO-EX


Thefollowingnoticeisprintedout:0691 WORKING STATE ACTIVATED BY
SYSTEM.
Thefollowingnoticeisdisplayed:MMI SYSTEM READY.
YoucanstartasessionatanMMLterminal.

5 Output working states of units from the MML terminal and monitor that every
unit starts in the state it had before the power cut (USI)
ZUSI;

g Note: TheOMUorTBUisnotnecessarilyinthestateithadbeforethepowercut,
becausethehardwaredefinestheactivityofOMUandTBU.

6 Monitor the alarm printout


Allthestartingunitsshouldcreateanotice:0691 WORKING STATE ACTIVATED
BY SYSTEM.

7 Check that no unnecessary alarms are on (AAP)


ZAAP;
SeeAlarms and recoveryforfurtherinformation.

3.4 Managing code preloading


3.4.1 Preloading the code from external sources to unit memory
The system downtime is reduced during restart by using OTO mode, compared with TOT
mode. If you want to restart the system with OTO loading mode, you should preload the
code of the software build. You can preload the code of the software build to unit
memory while the unit is in normal state.

28 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Managingrestarts
RNC

Purpose
YoucanonlyrestartthesystemwithOTOloadingmodeorTOTloadingmodeifyou
havepreloadedthecodetothememorypreviously.
YoucanonlyrestartafunctionalunitwithDSKloadingmodeorTOTloadingmodeifyou
havepreloadedthecodetotheunitmemorypreviously.

w NOTICE:CodepreloadingmaycauseCPUoverloadandeffectbandwidthinprogress.

Start system preloading (UWS)


ZUWS:[STAT=<package status>]|[NAME=<package name>];
Thefollowingstatusindicatethepreloadingprocess:
PL_S Codepreloadingisstartedintheunit.

PL_F Codepreloadingisfailedintheunit.

PL_C Codepreloadingiscompletedsucessfullyintheunit.

g Note: Iftimerunsoutincodepreloading,thesystemwillsetthepreloadingstatusto
PL_F.

Example:Preloading NW software build


ZUWS:STAT=NW;

Example:Preloading CB940R software build in FB, BU or NW status


ZUWS:NAME=CB940R;

Example:Preloading the running software build


ZUWS;

3.4.2 Interrogating system preloading information


You can view the system preloading status and progress during preloading with
command UWI.

Displaying system preloading information (UWI)


ZUWI;

Example:Display system preloading information


ZUWI;

Issue:20 DN02143192 29

Managingrestarts AdministeringRecoveryandUnitWorkingStateinIPA-
RNC

PRELOAD HANDLING COMMAND <UW_>


< I;
PRELOAD INFORMATION 2009-04-25 07:36:10
NAME CB940R
STATUS NW
DIRECTORY CB_9_4_R
PACKAGE-ID CB 9.4-0
ENVIRONMENT CB 9.4-0
DELIVERY CNR51840 2.7-1
CDID
DEFAULT PACKAGE YES
PRELOAD START TIME 2009-04-25 07:36:08
NUMBER OF PRELOADING FINISHED UNIT 0
NUMBER OF PRELOADING FAILED UNIT 0
NUMBER OF PRELOADING TOTAL UNIT 16
RATE OF PRELOADING FINISHED UNIT 0%
RATE OF PRELOADING FAILED UNIT 0%

COMMAND EXECUTED

3.4.3 Canceling code preloading


You can cancel code preloading to the unit memory with command UWC.

Purpose
Afterpreloadingiscanceled,youcannotrestartthesystemwithOPTloadingmodeor
OTOloadingmode.YoucanonlyuseTOTloadingmodeinsystemrestartinthiscase.

Canceling system preloading (UWC)


ZUWC;

30 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Interrogatingrecoveryhistory
RNC

4 Interrogating recovery history


You can interrogate the recovery history with command USH. By giving certain
parameters you can set up the filtering rules and get the desired information.

Procedure

1 Interrogate the recovery history of a network element (USH)


ZUSH:[[<unit type>|<all> def],[<unit
index>|<all>def]]:[FULL|<child units not
included>def]:[<analysis action>|<all except NO
OPERATION>def]:[[<start date>|<current date>def],[<start
time>|<00-00-00>def]]:[[<end date>|<current date>def],[<end
time>|<current time>def]];

Example:Interrogating the recovery history records of all units doing a unit


switchover from 15:00:00 PM of 21 March, 2005 to 16:00:00 PM of 22 March, 2005
ZUSH:::SWO:2005-03-21,15-00-00:2005-03-22,16-00-00;
TherecoveryhistoryrecordprintedoutcanbefoundinExamplefornormalcasefrom
Recoveryhistory.
Example:Interrogating the recovery history records of OMU-0's unit restarting
from 00:00:00 AM of current date to current time
ZUSH:OMU,0::URT;

Related descriptions
Recoveryhistoryonpage10

Related references
Structureofrecoveryhistoryrecordonpage10
Examplesofrecoveryhistoryonpage13

Issue:20 DN02143192 31

Recoveryandunitworkingstateadministration AdministeringRecoveryandUnitWorkingStateinIPA-
troubleshooting RNC

5 Recovery and unit working state


administration troubleshooting

5.1 Unit restarting fails


Description
Bycheckingsomeoutputsfromthenetworkelement,youcandetectmostofthereasons
fortherestartfailure.ButifthereasonisthattheOMUdiskdriveisnotinworkingorder,
youcandetectitbycheckingtheconditionoftheOMUdiskdrive.
Thetasksoftherecoverysystemdonottakelongiftheoperationofthesystemis
otherwisenormal.However,iftherecoverysystemisbusyforalongtime,thenthereis
someseriousdisturbanceintheoperationofthesystem.Inthiscase,themaintenance
softwareofthesystemisabletohandlethesituationbestwithitsautomaticprocedures.

g Note: Youshouldundernocircumstancesinterferemanuallywiththetasksifyoudo
nothavetherequiredexpertiseorknowledge.

Symptoms
Unitrestartingfails.

Recovery procedures
Followtheinstructionsbelowtaskbytaskuntiltheerrorhasbeendetected.

Checking why restarting fails

Procedure

1 Check execution error outputs


Theerrortextdescribesthereasonfortherestartfailure.

2 Check the recovery history of the object unit (USH)


ZUSH:[<unit>,<unit index>]:::,<time just before
restarting>;
Formoreinstructionsoncheckingtherecoveryhistory,seeInterrogatingrecovery
history.

3 Check the alarm history of the object unit (AHO)


ZAHO:<unit>,<index>::,<time just before restarting>;

32 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Recoveryandunitworkingstateadministration
RNC troubleshooting

Followtheinstructionsgiveninthealarmdefinition.

4 Connect the service terminal to the unit that is restarting and follow the phase
printouts
Further information
Formoreinformation,seePhasesofChorusunitstart-upandPhasesofDMXunit
start-upinPrint-outsofStartingPhasesinDMXunits.

5 Check the condition of the OMU disk drive


Further information
Formoreinformation,seeDiagnosingI/Odevices.

5.2 Working state change fails


Description
Ifaunitworkingstatechangefails,thereasonmaybeoneofthefollowing:

Thesystemistemporarilybusy
Thereisanalarmsituation
Theunitisnotfunctioningcorrectly
Theunithasalong-runningresource.

Thetasksoftherecoverysystemdonottakelongiftheoperationofthesystemis
otherwisenormal.Iftherecoverysystemisbusyforalongtime,thenthereissome
seriousdisturbanceintheoperationofthesystem.Inthiscase,themaintenance
softwareofthesystemisabletohandlethesituationbestwithitsautomaticprocedures.

g Note: Youshouldundernocircumstancesmanuallyinterferewiththetasksifyoudo
nothavetherequiredexpertiseorknowledge.

WhenyouchangetheunitworkingstatetoBLwiththeUSCMMLcommand,thesystem
changestheunitworkingstateautomaticallyfirsttoBL-EX,thentoBL-IDafterthelast
allocatedresourceoftheunithasbeenreleased.However,theunitmayhavealong-
runningresourceandstayintheBL-EXstate.Inthiscase,youmayneedtoforcethe
unitstatetochangefromBL-EXtoTE.Forinstructions,seeForcingaunitstatechange
fromBL-EXtoTE,whenithasalong-runningresource.

Symptoms
Thestatemanagementcommandyouhavegivendoesnotproducethedesiredoutcome
ortheautomaticstatechangefromBL-EXtoBL-IDdoesnotcometrueandunitstaysin
theBL-EXstate.

Issue:20 DN02143192 33

Recoveryandunitworkingstateadministration AdministeringRecoveryandUnitWorkingStateinIPA-
troubleshooting RNC

Recovery procedures

Checking why changing unit working state with the USC MML
command fails
Purpose
Thisprocedureexplainswhattodowhentheunitworkingstatefailstochangeafteryou
givetheUSCMMLcommand.

Procedure

1 Repeat the state management command after a short time


Ifthechangingofworkingstatefailsagain,gotothenextstep.

2 Check execution error outputs and their meaning


Theerrortextdescribesthereasonforthestatetransitionfailure.

Execution error outputs


Normally,theexecutionerroroutputbeginswiththestandardtext:
/*** STATE TRANSITION NOT POSSIBLE ***/
/*** DX ERROR : <error code> ***/
/*** <error text> ****/

3 Check the recovery history of the object unit (USH)


ZUSH:[<unit>,<unit index>]:::,<time just before
restarting>;
Formoreinstructionsoncheckingtherecoveryhistory,seeInterrogatingrecovery
history.

4 Check the alarm history of the object unit (AHO)


ZAHO:<unit>,<index>::,<time just before restarting>;

5 Check the condition of the unit (UDU)

g Note: TheUDUMMLcommandcanonlybegivenwhentheunitisinTEworkingstate.

ZUDU:<unit type>:<unit index>;

6 Check the diagnostic report (UDH)


ZUDH:<unit>,<index>::,<time just before starting
diagnostics>;

34 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Recoveryandunitworkingstateadministration
RNC troubleshooting

Forcing a unit state change from BL-EX to TE, when it has a long-
running resource
Purpose
WhenyouchangetheunitworkingstatetoBLwiththeUSCMMLcommand,thesystem
changestheunitworkingstateautomaticallyfirsttoBL-EX,thentoBL-ID.Iftheunithas
along-runningresource,itwillstayintheBL-EXstate.Thisprocedureexplainshowto
manuallychangetheworkingstateofaunitfromBL-EXtoTEaftertheunitfailedto
reachBL-IDstate.

Procedure

1 Change the working state to TE (USC)


GivevalueFCDtothestate transition controlparameter.
ZUSC:<unit type>,<unit index>:TE::FCD;

g Note: Thiscommandcausesdisturbancesinthenetworktraffic.

5.3 Working state is incorrect


Description
Asregardsthesystem'sperformanceandredundancy,theunitswhoseworkingstateis
TE-EX,TE-REorSE-OUorunitswhicharepermanentlyinstartingstate(regardlessof
mainstate)areinanincorrectworkingstate.

Symptoms
TheunitisinworkingstateTE-EX,TE-REorSE-OU.
TheunitstateisWO-EXFLTY.Theactiveunitisfaultyandnoredundantunitis
available.
TheunitisinstateWO-RE,SP-RE,TE-RE,orBL-RE,constantlyorrepeatedly.
Thealarm1685UNITRESTARTFAILUREisdisplayed.

Recovery procedures

Issue:20 DN02143192 35

Recoveryandunitworkingstateadministration AdministeringRecoveryandUnitWorkingStateinIPA-
troubleshooting RNC

Checking why unit working state is incorrect

Procedure

1 Check the recovery history of the object unit (USH)


ZUSH:[<unit>,<unit index>]:::,<time just before
restarting>;
Formoreinstructionsoncheckingtherecoveryhistory,seeInterrogatingrecovery
history.

2 Check the alarm history of the object unit (AHO)


ZAHO:<unit>,<index>;

g Note: Checkallthealarmsbackwardsuntilyoufindthedefectthatcausedthealarm.

3 Check the diagnostic report (UDH)


ZUDH:<unit>,<index>;

g Note: Checkallthealarmsbackwardsuntilyoufindthe"UNITOK"report.

4 Find out why the start-up has failed


IftheunitisinstateTEanditsstart-upfailsbeforerestartphase3(RP3),diagnosing
oftheunitwiththeMMLcommandUDUdoesnothelp.Inotherwords,itispossible
thatthestart-upofthediagnosticprogramblockshasfailed.Iftheunitisbeingtested
foralongtime(theUSIMMLcommandshows'intest'asthestatus),thiskindof
failurecouldbesuspected.Inthiscase,findoutwhythestart-uphasfailed.

Unit status
IftheunitisinTEstateanditsstart-upfails,thediagnosticscannotberun,neitherbythe
recoverysystemnorbytheoperator.IfthesystemhaschangedtheunitstatetoTEdue
toadefect,theunitalsohasanMONIstatus.Iftheunit'sstateisSE-OUanditsstatusis
BRKN,itmeansthatthediagnosishasdetectedadefectintheunit.

5.4 The active OMU cannot start up properly


Description
IftheactiveOMUcannotstartupproperlywiththecurrentsoftwarebuild,itpreventsthe
wholesystemfromstartingup.Regardlessofthereasonfortheproblem,itis
recommendedthattheoperatorshouldreplacetheSOMAFIfiletochangethesoftware
buildasthesolution.

36 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Recoveryandunitworkingstateadministration
RNC troubleshooting

Inthiscase,theoriginalfileSOMAFIGX.IMGlocatedinthe/W0-/SCMANA/directory
needtobereplacedwiththefileSOMAFIGX.IMGlocatedinthefallbackbuild's
SCMANAsubdirectory.
InordertochangethefileSOMAFIGX.IMG,theoperatorneedstoestablishaCOM
connectiontotheactiveOMU.

Symptoms
TheactiveOMUcannotstartupproperlywiththecurrentsoftwarebuild.

Recovery procedures

Replace the SOMAFI file

Procedure

1 Make a COM connection to the active OMU which cannot start up properly

2 Force the OMU to start up to minidebugger mode


WhentheOMUstartsup,pressthe'M'keyintheserviceterminalkeyboard.

Further information
WhentheOMUisinminidebuggermode,thefollowingtextisdisplayedinthe
screen:
MINIDEBUGGER, NO LOADING
PROCESSING FAMILIES NAS
READY - PHASSE 99

3 Load the HME service terminal extension


ZLP:1,HME

4 Make sure that the COM connection is established to the active OMU
ThiscanbedonebycheckingthelocationoftheHMSnode.
Z1CN

Expected outcome
Thefollowingtextisdisplayed:
Active HMS master node in own computer.
ThisshowsthattheCOMconnectionisestablishedtotheactiveOMU.

Unexpected outcome

Issue:20 DN02143192 37

Recoveryandunitworkingstateadministration AdministeringRecoveryandUnitWorkingStateinIPA-
troubleshooting RNC

Ifthetextaboveisnotdisplayed,thentheCOMconnectionisnotestablishedtothe
activeOMU.Inthiscase,theoperatorshouldrepeatsteps1-4untiltheCOM
connectionisestablished.

5 Load the POM service terminal extension


ZLP:P,POM
AfterdoingsotheoperatorcanusethePosixcommand.

6 Find out the name of the fallback build directory


ThiscanbedonebytypingthecontentsoftheSOMAFIfile:
ZPS:type,/shadows/SCMANA/SOMAFIGX.IMG

Further information
Tofindoutthefallbackbuildrecord,firstsearchthebuildwiththestatusFB.Afterthe
statusfield,thereisabuildIDfieldwhichshowsanIDnumberwhoselengthis20
bytes.ThenafterthebuildIDfield,youcanseethebuilddirectoryname.

Example:
........EP.)........SOMAFIGX.PAC 3.2-0 99/09/14 PLAENVC3.PAC 2.10-1 CNR58252
2.2-0 2.10-1...................................................................
.........................:.....P.............x.NW..............A5 3.2-0
1A532BU2....DIR.EMIL.....A5 3.2-0 CNR58252 3.2-0 .......................
.....x.BU..............A5 3.4-0 A5_3_4_0...DIR.A5340....A5 3.4-0 C
NR58252 3.2-0 c...'.. ....................x.UT..............U2 9.1-0
110U2910CD11..DIR.CD11.....U2 9.7-0 CIDI0031 2.3-0 ..................U231B
011..x.FB..............A5 3.4-0 0A5340FB....DIR.A5340FB..A5 3.4-0 C
NR58252 3.2-0 T...'.. ....................x.UD...............................
................................................................................
.....x.UT..............A5 1.11-0 A5_1_11_0..DIR.A51110...A5 1.11-0 C
NR58252 3.2-0 ............................x.UD...............................
................................................................................
.....x.UT..............A5 3.2-0 0A5320BU....DIR.INES.....A5 3.2-0 C
NR58252 3.2-0 ...............................................................
................................................................................
.........
[EOF]

Inthisexample,thefallbackbuild's

statusis:FB
buildIDis:A5 3.4-0 0
directorynameis:A5340FB.

38 DN02143192 Issue:20

AdministeringRecoveryandUnitWorkingStateinIPA- Recoveryandunitworkingstateadministration
RNC troubleshooting

7 Copy the SOMAFIGX.IMG file from the fallback build's SCMANA subdirectory
to the SCMANA directory
ZPS:cp,/shadows/<FB build directory>/SCMANA/SOMAFIGX.IMG,
/shadows/SCMANA/SOMAFIGX.IMG

8 Restart the active OMU


Aftertherestart,theOMUwillstartupproperly.

Further information
TheSOMAFIGX.IMGfilecurrentlyinusecontainsonlythefallbackbuild,andthe
statusofthefallbackbuildisFB.Nowinthesystemthereisonlytheonecreated
build.
Ifyouneedtorecreatesomeotherbuilds,dothefollowing:
a) checkallbuildswithcommandZWQO:EX;
b) createtheotherbuildswiththeZWQCcommand.

Issue:20 DN02143192 39