R1

1
a)
R2
NOTE
1 Proceed with these steps only if the DOT version is 8.0 thru 8.0.2P1 or 8.1RC1. Bug 490825 > here
a)
NOTE
STOP
b)
c) Continue with Action Plan on page 1.
README FIRST
README: Remove Failed Disks before the Disk Re-assignment
No "Failed" Disks can exist in the system for the disk assignment to the new controller to properly execute.
Enter: 'partner aggr status -f ' on the partner node that took over. The word (takeover) is in the console prompt. If there
are failed disks, they must be removed or replaced before the disk reassignment process. Continue with README R2.
System is HA and the partner took over: The disk reassignment should be only be executed when the 'giveback' can
also be executed - Confirm with end-user that the 'giveback' can be executed immediately after the disk
reassignment. If the giveback needs to be scheduled, defer the disk reassignment - Continue with next step.
After the initial 'giveback' is completed from the partner node, the workaround requires logging into the " target" (repaired)
node and executing a controller 'takeover'. Then when the takeover is complete, a 'giveback' is executed to finish the
process of updating the disk ownership caches on both nodes. This AP has been updated to include these steps.
Skip this section if the system is running any version of DOT 7.x, 8.0.2P2, 8.0.3, 8.1RC2 or higher, otherwise continue.
README: Process Change in the Disk Re-assignment and Giveback Sections (Bug 490825)
The partner must have taken over the "target" (down) controller. If a takeover did not occur, skip to next page.
If the system configuration is unknown, follow Section III of this AP to assist in determining if controller is HA and if
the partner took over. If the system is not HA or the partner did NOT takeover, skip to next page.
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
I. Appliance/NVRAM Adapter Visual Checks
II. Node Pre-Checks
III. Node State Check and Shutdown Procedure IX. Boot Diagnostics and test the NVRAM Memory
IV. Capture the Current System Configuration X. Capture new System-ID on replacement NVRAM
V. Locate the NVRAM Adapter and the Internal Red LED XI. Disk Reassign
XII. Boot the Operating System - 'cf giveback' if applicable
XIII. Submit logs and Part Return
VII. Replace the NVRAM Adapter
I.
Step
1 Visually verify if you are working on correct model and READ the CAUTION note below.
Fig 1
Fig 2
Fig 3
Click here
for larger view
2 Go to Section II, "Node Pre-Checks" on next page.
SA600 and V-FAS6000 Series: NVRAM Adapter Visual Checks
Action Description
Page 1 of 15
SECTION OUTLINE for the SA600 and V-FAS6000 Series NVRAM Adapter Replacement
This procedure will take 60-90 minutes
Open the System: Remove the Cables, Cable Management
Tray and extract the System Tray
VI.
Close the System: Reinsert the System Tray and reconnect
the cables
VIII.
STOP !!
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
6u
SA600 or V-FAS Model Number
System Serial Number
IOIOI (Console) Port
PS-2 AC
Switch
PS-1 AC
Switch
FC Port Tray ( 0a-0b, 0c-0d, 0e-0f, 0g-0h )
6 Ethernet
Ports
( e0a-e0f )
NVRAM6 Adapter
RLM Port
CAUTION
If the Data in NVRAM red LED is flashing through the faceplate when power is turned off,
uncommitted customer data may be in the NVRAM . Follow this action plan carefully.
Data in NVRAM LED:
Is a Red LED located
behind the Faceplate.
Activity, Status (!), Power LEDs
Activity LED should be OFF!
NVRAM DATA LED:
Located behind the
faceplate.
Fig 4
II.
Step
1
2 Adhere to anti-static precautions. (A paper ESD strap is included inside the RMA box if you don't have your own)
STOP
3 Remove the replacement NVRAM from the anti-static bag and examine the connectors for damage.
III.
Step
To review the J ob Aid on how to connect to console (IOIOI) port and serial emulator options, here >> Console Attach
2
a)
S
T
O
P
3 The target node needs to be halted or " waiting for giveback" for this part replacement. LCD Detail >> LCD Panel
4 Confirm the console response is " LOADER>" or " Waiting for giveback" . If it is " LOADER>" , go to Section IV.
5 If the console response is: " Waiting for giveback..." follow steps 5a-5c.
a) At the "Waiting for giveback " prompt, Press: Ctrl-C
b) At the message: "Do you wish to halt this node rather than wait [y/n]? " Enter: "y"
c) After the system drops to the LOADER> prompt, continue with next section.
6 Go to Section IV, "Capture the Current System Configuration" on next page.
1 Always capture the nodes console output to a text file, ex: NetApp-dispatch-num.txt , even if using the end-user's computer.
SA600 and V-FAS6000 Series: Node Pre-Checks
Action Description
Verify the " Order Reference 8xxxxxxxxx number on the RMA packing slip is the same as the Part Request (PREQ) number
listed in your dispatch notes.
Page2 of 15
This procedure is for a NVRAM6/Cluster interconnect card. If the part # on the RMA packing slip contains " 1926" in it,
the part is a VI-MetroCluster card and this is not the proper action plan. See VI-MetroCluster AP.
WARNING for HA (Active-Active) configurations:
If the failure has caused a controller failover you may have been dispatched on the surviving controller's serial number, not the
failed one. If the LCD panel displays the word " takeover" or " cf takeover" , this node it UP. See Explanation of LCD panel
messages in the link in Step 3.
Action Description
SA600 and V-FAS6000 Series: Node State Check and Shutdown Procedure
Walk up visual observation checks - Identifying the correct node to be serviced, i.e.. the "target" node.
NOTE: A "controller failover" will turn the Status ( ! ), middle LED from GREEN to AMBER on the target and partner nodes.
The " Activity" ( ) LED, left most in figure, on the target node should be OFF. If it is flashing, this node is still up - or you
are looking at the partner node, not the target node. See STOP message.
The (Activity) LED should be off.
No flashing on the target node.
! (Status) LED will turn from
GREEN to AMBER on the target
and partner node if a controller
" failover" has occurred.
Wai t i ng f or gi veback. . . ( Pr ess Ct r l - C t o abor t wai t )
Wai t i ng f or gi veback. . . ( Pr ess Ct r l - C t o abor t wai t )
^C
Thi s node was pr evi ousl y decl ar ed dead.
Pausi ng t o check HA par t ner st at us . . .
par t ner i s oper at i onal and i n t akeover mode.

You must i ni t i at e a gi veback or shut down on t he HA
par t ner i n or der t o br i ng t hi s node onl i ne.


The HA par t ner i s cur r ent l y oper at i onal and i n t akeover mode.
Thi s node cannot cont i nue unl ess you i ni t i at e a gi veback on t he par t ner .
Once t hi s i s done t hi s node wi l l r eboot aut omat i cal l y.

wai t i ng f or gi veback. . .

Do you wi sh t o hal t t hi s node r at her t han wai t [ y/ n] ? y
Hal t i ng. . .
Upt i me: 3m10s
Syst emhal t i ng. . .

LOADER>
Step 5: Hitting Enter
displays this prompt
Step 5a): Enter: CTRL-C
Information on
Partner Status
Step 5b): Enter: " y"
Fig 5
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
IV
Step
1 a)
b) From the LOADER> prompt enter " autoboot" to initiate a prom bootstrap.
c)
d) Enter "5" for "Maintenance mode boot".
e) If asked " Continue with boot?" Answer: "y"
2 Continue with Section IV on next page.
Page 3 of 15
SA600 and V-FAS6000 Series: Capture the Current System Configuration
Action Description
NOTE If the Motherboard fails to boot to the Maintenance menu due to an error, skip to Section V.
Reference the example of console output below and follow these steps. The .. (dots) represent deleted text to highlight
the specific output messages to key on.
When this message appears: " Press CTRL-C for boot menu" , press CTRL-C (^C) to load the " Boot options menu" .
After about 30-40 seconds, the " Maintenance menu" will appear.
LOADER> autoboot
Loadi ng X86_64/ f r eebsd/ i mage1/ ker nel : 0x100000/ 3375736 0x538280/ 3221872
. . . . .
Copyr i ght ( C) 1992- 2010 Net App.
Al l r i ght s r eser ved.
*******************************
* *
* Press Ctrl-C for Boot Menu. *
* *
*******************************
^CBoot Menu will be available.

Pl ease choose one of t he f ol l owi ng:

( 1) Nor mal Boot .
( 2) Boot wi t hout / et c/ r c.
( 3) Change passwor d.
( 4) Cl ean conf i gur at i on and i ni t i al i ze al l di sks.
( 5) Mai nt enance mode boot .
( 6) Updat e f l ash f r ombackup conf i g.
( 7) I nst al l new sof t war e f i r st .
( 8) Reboot node.
Sel ect i on ( 1- 8) ? 5

You have sel ect ed t he mai nt enance boot opt i on:
. . . . .
. . . . .

I n a Hi gh Avai l abi l i t y conf i gur at i on, you MUST ensur e t hat t he par t ner node i s ( and r emai ns) down, or t hat
t akeover i s manual l y di sabl ed on t he par t ner node, because Hi gh Avai l abi l i t y sof t war e i s not st ar t ed or
f ul l y enabl ed i n Mai nt enance mode.

FAI LURE TO DO SO CAN RESULT I N YOUR FI LESYSTEMS BEI NG DESTROYED

NOTE: I t i s okay t o use ' show/ st at us' sub- commands such as ' di sk show or aggr st at us' i n Mai nt enance mode
whi l e t he par t ner i s up

Cont i nue wi t h boot ? yes
. . . . .
. . . . .
. . . . .
*>
Step 1b): Enter " autoboot"
Step 1d): Enter " 5"
Step 1e): If this node has a partner node this
message will be displayed. Answer: "y" .
maintenance mode console prompt
Step 1c): Wait for
this message, then
hit ^C (CTRL-C)
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
IV
Step
3 Next, from the *> prompt enter " disk show -v" to view which SAS and FC Adapter ports are driving disks- See Text Box 3 below.
include switch paths for it's virtual storage. Example of McData and Cisco switch paths here >V-Series Disk Show
4
5 At the >*>prompt enter " halt" (after prom initialization the console will display the LOADER> prompt)
6 Go to Section V, "Locate the NVRAM Adapter and the Internal Red LED" on next page.
Take note of all the "unique" Adapter port numbers displayed. See Text Box STEP 4. In this example: SAS Adapters 5a, 5c and
FC Adapters 0c, 0d are displayed.
Page 4 of 15
SA600 and V-FAS6000 Series: Capture the Current System Configuration (cont.)
Action Description
NOTE
The " disk show -v" sample output below is abbreviated console output. If DOT 8.x, a HOME column is also listed with
OWNER for each disk, which displays the node's systemname (and system-ID). After the controller is replaced it is
necessary to confirm each SAS/FC Adapter port is seeing its storage. If a V-Series (V6000), the 'disk show -v' will
*> disk show -v
Local Syst emI D: 1573753606


DI SK OWNER POOL SERI AL NUMBER HOME
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
5c. 02. 4 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7VRRF f as6040cl 2- ams( 1573753632)
5c. 02. 3 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7WMNQ f as6040cl 2- ams( 1573753632)
5c. 02. 10 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7WF87 f as6040cl 2- ams( 1573753632)
5c. 02. 20 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7WMNE f as6040cl 2- ams( 1573753632)
5c. 02. 19 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7WWR4 f as6040cl 2- ams( 1573753632)
5c. 02. 6 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7W80G f as6040cl 2- ams( 1573753632)
5c. 02. 16 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7WSH9 f as6040cl 2- ams( 1573753632)
5c. 02. 7 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7W3KA f as6040cl 2- ams( 1573753632)
. . . . .
. . . . .
0d. 41 f as6040cl 2- ams( 1573753632) Pool 0 J LVT29GC f as6040cl 2- ams( 1573753632)
0d. 43 f as6040cl 2- ams( 1573753632) Pool 0 J LVT7BUC f as6040cl 2- ams( 1573753632)
0d. 33 f as6040cl 2- ams( 1573753632) Pool 0 J LVS4EHC f as6040cl 2- ams( 1573753632)
. . . . .
. . . . .
0c.21 f as6040cl 1- ams( 1573753606) Pool 0 J LVT0KDC fas6040cl1-ams(1573753606)
0c.18 f as6040cl 1- ams( 1573753606) Pool 0 J LVT2HZC fas6040cl1-ams(1573753606)
0c.28 f as6040cl 1- ams( 1573753606) Pool 0 J LVS585C fas6040cl1-ams(1573753606)
. . . .
5a.01.13 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7W3XZ fas6040cl1-ams(1573753606)
5a.01.21 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7WSX8 fas6040cl1-ams(1573753606)
5a.01.16 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7W3YT fas6040cl1-ams(1573753606)
5a.01.12 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7WS0R fas6040cl1-ams(1573753606)
5a.01.14 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7W8GM fas6040cl1-ams(1573753606)
5a.01.18 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7WX7E fas6040cl1-ams(1573753606)
5a.01.22 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7WY15 fas6040cl1-ams(1573753606)
5a.01.11 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7W8N2 fas6040cl1-ams(1573753606)
. . . . .
. . . . .
*>
*> halt
A typical listing will display many
more disks and FC/SAS adapters
than this partial listing. Step 5: Enter " halt" to exit to the LOADER> prompt
Step 3: The disk show -v command prints out
the System ID of the Local System (1573753606). It also
prints the owner of each disk under the HOME heading
which lists the node's system name. This system name is
(fas6040cl1-am) and owns disks: 0c.21, 0c.18,
0c.28,5a.01.13, 5a.01.21, etc.
NOTE- Partner owned disks are intermixed in the
output. The partner hostname is 'fas6040cl2-ams' and
it's System ID is (1573753632).
Example Only
Step 4: Under the DISK heading, all SAS &
FC Adapters are listed. In this example SAS
adapter 5a and 5c and FC adapter ' 0c and
0d' are seen, but typically there are more.
After the controller is replaced, confirm the
same adapters are listed meaning there is an
active SAS/FC path to the disks.
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
V.
Step
1
STOP
2
An internal view of the NVRAM LED is here >> NVRAM6-LED
3
a)
4
a)
VI.
Step
Note Appliance should have been " halted" and then powered off with power cables removed.
1 Label each cable connector with its port number and then unplug all the cables.
Note If possible keep the cables in the cable clips on the cable tray to keep them in the correct position for reconnection.
2
3 Continue with Section VI on next page.
Page 5 of 15
SA600 and V-FAS6000 Series: Locate the NVRAM Adapter and the Internal Red LED
Action Description
Confirm the console response is " LOADER>" . Next turn off both power supplies (Refer to Fig 2 in Section I.) on the back of the
head unit and disconnect the power cords from the power supplies.
(i) The dispatch should indicate which slot the NVRAM6 is installed; 1 or 2.
(ii ) INVRAM is slot 1: This is a " standard" HA (Active-Active) configuration and will have two cluster interconnect
cables connected to the partner node.
(iii) INVRAM is slot 2: This is a SINGLE Appliance configuration or a Fabric MetroCluster configuration. (If a
MetroCluster, a separate cluster card called a " VI-MetroCluster" would be installed in slot 1 for the cluster
interconnect. The cluster interconnect on the NVRAM6 in slot 2 would not be used.)
Check the NVRAM red LED as seen through the faceplate. If it is not
flashing, skip to Section VI. Refer to Fig to right >>
If the NVRAM card is in PCI slot 1 and the LED is flashing between
the cables, confirm with end-user/NGS the partner took over
successfully. If so, the LED can be safely ignored (Partner node
LCD will toggle "takeover)" and the partner's console prompt after
login, will display the word " (takeover)" .
If a successful takeover did not occur, engage NGS to
determine if it is OK to proceed with the replacement.
If the NVRAM is in PCI slot 2 and the LED is flashing, this node did
not flush the data in the NVRAM. Re-connect the power supplies and
turn "ON" both power supplies.
If the node boots up to the " login" prompt, engage end-user to login and issue a " halt" command to flush the data
in the NVRAM. (Hit <enter> to get the login prompt) If the appliance fails to boot up successfully, engage NGS for
guidance.
SA600 and V-FAS6000 Series: Remove the Cables, Cable Management Tray and extract the System Tray
Action Description
If cable management tray is installed, remove it by gently pushing in the sides of the tray at the arrows and lifting the tray up. Ref
Fig 7a-b.
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
NVRAM (shown in slot
1), arrow shows the
location of LED which
is behind the faceplate.
Cable
Management
Tray
Pinch both sides
" in" to remove.
Fig 7a
Fig 7b
Fig 6
VI.
Step
4
VII.
Step
1 Remove media converters if any, from the NVRAM adapter.
2
3 Pull the NVRAM6 Adapter out of the slot as shown in fig 10 below.
4 Remove the replacement NVRAM adapter from its anti-static bag and examine the edge contact pads for damage.
5 Align the NVRAM adapter with the appropriate expansion slot and firmly push it into the slot.
6 Install the primary and secondary PCI adapter retaining brackets. Ref Fig 10 below.
7 Re-attach the media converters if removed, to the NVRAM adapter.
8 Go to Section VIII, "Reinsert the System Tray and reconnect the cables" on next page.
Page 6 of 15
SA600 and V-FAS6000 Series: Remove the Cables, Cable Management Tray and extract the System Tray (cont.)
Action Description
Loosen the six (blue) thumbscrews that secure the motherboard tray to the chassis as noted below. Open the Motherboard tray by
pulling the cam handle down and pulling on the handle to slide the tray out of the system until the NVRAM Adapter is accessible.
SA600 and V-FAS6000 Series: Replace the NVRAM Adapter
Action Description
Loosen the blue thumbscrew on the primary and secondary PCI retaining brackets and lift the retaining brackets up and off the
chassis frame. Ref Fig 10 below.
STOP !!
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
Blue Thumbscrews need to be
loosened to open up the system tray.
Lower Cam Handle and pull
to extract the tray
6
Fig 9
Fig8
Secondary bracket
PCI card
Fig 10
Loosen both blue thumbscrews to remove brackets
NOTE:
In a STANDARD HA (Active-Active) configuration,
the NVRAM adapter is installed in PCI slot 1.

If this is a SINGLE Appliance or MetroCluster the
NVRAM is installed in PCI slot 2.
Primary bracket
VIII.
Step
1 Reinsert the motherboard tray and raise the cam lever and tighten the 6 thumbscrews. (Reference pictures in Section VI)
2 Re-attach the Cable Management Tray if removed. (Reference pictures in Section VI)
3
IX.
Step
1
2 Reconnect the power cords and turn the power supplies on.
3
4
a. Immediately Press ^C (CTRL-C) to access the "Boot menu".
b. If a 'System ID mismatch' warning message below is displayed, answer : " y" .
c. Next, drop to the "LOADER>" prompt from the Boot Menu by following the linked process > here
d. Continue with Section IX on next page.
Page 7 of 15
SA600 and V-FAS6000 Series: Boot Diagnostics and test the NVRAM Memory
Action Description
Re-attach laptop to the console port and capture the display output even if using the end user's computer.
IMMEDIATELY after the console message " Starting AUTOBOOT press Ctrl-C to abort" is displayed, press Ctrl-C
(^C) key a couple times to abort the autoboot. See Console output example below.
IF you miss the window to abort the autoboot, look for this message: " Press CTRL-C for boot menu" and complete steps
4a-4d, otherwise if at the " LOADER>" prompt, skip to step 5 on next page.
SA600 and V-FAS6000 Series: Reinsert the System Tray and reconnect the cables
Action Description
Cables: Fully insert each cable that was removed to its proper port until it clicks in. Test by pulling on them. Especially the
FC and SAS ports!
Phoeni x Tr ust edCor e( t m) Ser ver
Copyr i ght 1985- 2006 Phoeni x Technol ogi es Lt d.
. . . . . . .
. . . . . . .
Por t i ons Copyr i ght ( C) 2002- 2008 Net App

CPU Type: I nt el ( R) Xeon( R) CPU L5410 @2. 33GHz

Starting AUTOBOOT press Ctrl-C to abort...
Loadi ng x86_64/ f r eebsd/ i mage1/ ker nel : . . . . 0x100000/ 3386728 0x53b000/ 3222096
0x84da50/ 1190096
Aut oboot of PRI MARY i mage abor t ed by user .

LOADER->
STEP 3: Press " CTRL-C"
" ." = Deleted lines to save space
. . . . . . .
. . . . . . .
*******************************
* *
* Pr ess Ct r l - C f or Boot Menu. *
* *
*******************************
^C
Boot Menu wi l l be avai l abl e.
Rest or i ng / var f r om/ cf car d/ x86/ f r eebsd/ var f s. t gz
WARNI NG: Syst emi d mi smat ch. Thi s usual l y occur s when r epl aci ng CF or NVRAM car ds!
Over r i de syst emi d? {y| n} [ n] y
STEP 4b: Enter: " y"
STEP 4a:
Press " CTRL-C"
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
IX.
Step
5 Test the NVRAM with the diagnostics by entering " boot_diags" at the " LOADER>" prompt.
6 In the main Diagnostic Menu enter " nvram" to select the NVRAM diagnostic.
7 If asked " OK to run NVRAM diagnostic (yes/no)?" Answer: " yes"
8 Continue with Section IX on the next page
SA600 and V-FAS6000 Series: Boot Diagnostics and test the NVRAM Memory (cont.)
Action Description
Page 8 of 15
LOADER> boot_diags
Loadi ng x86_64/ di ag/ di ag. kr n: . . . 0x200000/ 12606928 0xe05dd0/ 4664936 0x1278c38/ 8 Ent r y at
0x00202018
St ar t i ng pr ogr amat 0x00202018


Copyr i ght ( c) 1992- 2009 Net App.
i ni t mca f or BSP

Di agnost i c Moni t or
ver si on: 5. 4. 7
bui l t : Fr i Aug 27 16: 06: 30 PDT 2010
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
al l Run al l syst emdi agnost i cs
mb FAS6040 mot her boar d di agnost i c
mem Mai n memor y di agnost i c
nvr am NVRAM di agnost i c
agent agent & r l mdi agnost i c
cf - car d Compact Fl ash cont r ol l er di agnost i c
sas SAS cont r ol l er di agnost i c
st r ess Syst emwi de st r ess di agnost i c

Commands:
Conf i g ( pr i nt a l i st of conf i gur ed PCI devi ces)
Def aul t ( r est or e al l opt i ons t o def aul t set t i ngs)
Exi t ( exi t di agnost i cs)
Hel p ( pr i nt t hi s commands l i st )
Opt i ons ( pr i nt cur r ent opt i on set t i ngs)
Ver si on ( pr i nt t he di agnost i c ver si on)
Run <di ag . . . di ag> ( r un sel ect ed di agnost i cs)

Opt i ons:
Count <number > ( l oop sel ect ed di agnost i c( s) ( number ) of passes)
Loop <yes| no> ( l oop sel ect ed di agnost i c( s) )
St at us <yes| no> ( pr i nt st at us messages)
St op <yes| no> ( st op- on- er r or / keep r unni ng)
Xt nd <yes| no> ( ext ended t est s / r egul ar t est s)
Mchk <aut o| of f | on| hal t > ( machi ne check cont r ol )
Cpu <0| 1> ( def aul t cpu)
Seed <number > ( r andomseed ( 0: use machi ne gener at ed number ) )

Ent er Di ag, Command or Opt i on: nvram
WARNI NG! Do not r un t he NVRAM di agnost i c i mmedi at el y af t er a
syst emcr ash or i f t her e i s a possi bi l i t y t hat l og
dat a i s st or ed. Run onl y on new boar ds, or af t er a
nor mal syst emshut down, or i f t her e i s no chance of
pr eser vi ng cust omer dat a.

OK t o r un NVRAM di agnost i c ( yes/ no) ? yes
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
STEP 6: Enter " nvram" to select the
NVRAM diagnostic
STEP 5: Enter " boot_diags"
The model maybe any FAS6000
or V6000 series or a SA600
STEP 7: Enter " yes"
IX.
Step
9 In the NVRAM Diagnostic menu, enter " 1" to select the Comprehensive NVRAM test.
10
move the battery, click >> here
11 Exit the NVRAM test menu by entering " 99" at the menu prompt.


12 Continue with Section IX on the next page
STOP
If the "NVRAM env" test FAILS, the battery is likely too discharged to operate properly without a period of recharging.
Engage NGS and ask if they want the battery moved from the original card to the replacement. For instructions to
Page 9 of 15
SA600 and V-FAS6000 Series: Boot Diagnostics and test the NVRAM Memory (cont.)
Action Description
Confirm these four comprehensive tests show PASSED:1) " *** Comprehensive NVRAM Memory test" ,
2) " *** Comprehensive NVRAM IB test" , 3) " *** Comprehensive NVRAM env test" , and 4) " *** Comprehensive NVRAM
test" . If not all show as 'PASSED' call NGS and report the failure.
NVRAM Di agnost i c
- - - - - - - - - - - - - - - -
NVRAM car ds pr esent :
Sl ot 1: NVRAM6 512 MB ( 0x0000003100000000 t o 0x000000311e000000)
Fi r mwar e 11, Ser i al 634871, Rev G1, Par t 111- 00138
Bat t er y on, Char ger of f , I B Li nk( s) copper / copper

1: Compr ehensi ve NVRAM t est 73: Di spl ay NVRAM st at us
2: NVRAM memor y menu 74: Di spl ay NVRAM conf i g space
3: NVRAM I B menu 76: Upgr ade NVRAM f i r mwar e [ Xt nd]
5: NVRAM envi r onment al menu 77: Cl ear NVRAM pr oper t i es [ MFG onl y]
6: NVRAM EEPROM t est
7: NVRAM FLASH t est 90: NVRAM car d sel ect i on
8: NVRAM i 2c t est 91: Enabl e/ di sabl e l oopi ng
92: St op/ cont i nue on er r or
70: Set NVRAM pr oper t i es [ MFG onl y] 93: Ext ended/ nor mal t est mode
71: Di spl ay NVRAM pr oper t i es 99: Exi t
72: Di spl ay NVRAM EEPROM

Sel ect t est or f eat ur e by number [ 0] : 1

Test i ng NVRAM car d i n sl ot 1
. . . . .
. . . . .

****** Comprehensive NVRAM memory test .......... PASSED
I nt er nal l oopback RDMAWt est . . . . . . . . . . . . . . . . PASSED
I nt er nal l oopback RDMAR t est . . . . . . . . . . . . . . . . PASSED
I nt er nal l oopback SEND t est . . . . . . . . . . . . . . . . . PASSED
****** Comprehensive NVRAM IB test .............. PASSED
. . . . .
. . . . .
NVRAM env subsyst emt est . . . . . . . . . . . . . . . . . . . . PASSED
NVRAM bat t er y t est . . . . . . . . . . . . . . . . . . . . . . . . . . PASSED
NVRAM char ger t est . . . . . . . . . . . . . . . . . . . . . . . . . . PASSED
****** Comprehensive NVRAM env test ............. PASSED
****** Comprehensive NVRAM test ................. PASSED

1: Compr ehensi ve NVRAM t est 73: Di spl ay NVRAM st at us
2: NVRAM memor y menu 74: Di spl ay NVRAM conf i g space
3: NVRAM I B menu 76: Upgr ade NVRAM f i r mwar e [ Xt nd]
5: NVRAM envi r onment al menu 77: Cl ear NVRAM pr oper t i es [ MFG onl y]
6: NVRAM EEPROM t est
7: NVRAM FLASH t est 90: NVRAM car d sel ect i on
8: NVRAM i 2c t est 91: Enabl e/ di sabl e l oopi ng
92: St op/ cont i nue on er r or
70: Set NVRAM pr oper t i es [ MFG onl y] 93: Ext ended/ nor mal t est mode
71: Di spl ay NVRAM pr oper t i es 99: Exi t
72: Di spl ay NVRAM EEPROM

Sel ect t est or f eat ur e by number [ 1] : 99
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
STEP 10: Confirm these
four comprehensive tests
ALL PASSED.
STEP 11: Enter " 11" to
exit the NVRAM test menu
STEP 9: Enter " 1" for
Comprehensive NVRAM Test
IX.
Step
13
X.
Step
1
2 a) From the LOADER> prompt enter " autoboot" to initiate a prom bootstrap.
b)
c) If a 'System ID mismatch' warning message is displayed due to the new Controller Module , answer : " y" .
d) Enter '5' for "Maintenance mode boot".
e) If asked " Continue with boot?" Answer: "y" to exit to the maintenance mode >*> prompt.
NOTE
Under NO CIRCUMSTANCES bypass the system halt on a NVRAM battery voltage issue.
3 Continue with Section X on next page.
Page 10 of 15
SA600 and V-FAS6000 Series: Capture new System-ID on replacement NVRAM
Action Description
The console prompt should be the LOADER> prompt if Diags were exited properly.
When this message appears: " Press CTRL-C for boot menu" , press CTRL-C (^C) to load the " Special boot options
menu" . After about 30-40 seconds, the " Maintenance menu" will appear.
If the System fails to boot to the Maintenance menu, confirm the NVRAM6 Adapter is properly seated. Engage NGS for
assistance.
STOP
If the system reports the battery voltage is too low or a critical failure, do NOT proceed - Do NOT bypass the system stop. Engage
NGS for assistance.
SA600 and V-FAS6000 Series: Boot Diagnostics and test the NVRAM Memory (cont.)
Action Description
Then enter " exit" to exit the diagnostics.
. . . . . . .
. . . . . . .
*******************************
* *
* Pr ess Ct r l - C f or Boot Menu. *
* *
*******************************
^C
Boot Menu wi l l be avai l abl e.
Rest or i ng / var f r om/ cf car d/ x86/ f r eebsd/ var f s. t gz
WARNI NG: Syst emi d mi smat ch. Thi s usual l y occur s when r epl aci ng CF or NVRAM car ds!
Over r i de syst emi d? {y| n} [ n] y

Pl ease choose one of t he f ol l owi ng:
( 1) Nor mal Boot .
( 2) Boot wi t hout / et c/ r c.
( 3) Change passwor d.
( 4) Cl ean conf i gur at i on and i ni t i al i ze al l di sks.
( 5) Mai nt enance mode boot .
( 6) Updat e f l ash f r ombackup conf i g.
( 7) I nst al l new sof t war e f i r st .
( 8) Reboot node.
Sel ect i on ( 1- 8) ? 5
.....
.....
.....
Cont i nue wi t h boot ? yes




STEP 2c): Enter: " y"
STEP 2b):
Press " CTRL-C"
STEP 2d): Enter " 5"
STEP 2e): If this node has a partner node this
message will be displayed. Answer: "y" .
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers

Ent er Di ag, Command or Opt i on: exit

Phoeni x Tr ust edCor e( t m) Ser ver

Copyr i ght 1985- 2005 Phoeni x Technol ogi es Lt d. Al l Ri ght s Reser ved
. . . . .
. . . . .
CPU Type: AMD Opt er on( t m) Pr ocessor 252
LOADER>
Step 13: Enter: "exit" to exit the Diags.
The prom will initialize displaying many
messages. After about 10-20 seconds, the
it will drop to the LOADER> prompt.
X.
Step
NOTE
4
5
STOP There are new considerations prior to executing the 'disk reassign' and additional steps for 'cf giveback' in the next two Sections.
6 Go to Section XI, "Disk Reassign" on next page.
Page 11 of 15
SA600 and V-FAS6000 Series: Capture new System-ID on replacement NVRAM (cont.)
Action Description
If the disk configuration (disk show -v) was not captured in Section IV, engage NGS to determine the System_ID of the old
NVRAM by examining the Autosupports at NetApp or if HA configuration, by examining the Autosupports on the root vol for this
node which is hosted on the partner.
Enter " disk show -v" at the maintenance mode prompt " *>" to display the new system ID.
Compare the new system ID to the old system ID. Retrieve the old ID by opening up the console log file to where the disk show -
v was originally executed in Section IV, step 3. For DOT 8, always use the HOME column, not OWNER.
*> disk show -v
Local Syst emI D: 1943753293


DI SK OWNER POOL SERI AL NUMBER HOME
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
5c. 02. 4 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7VRRF f as6040cl 2- ams( 1573753632)
5c. 02. 3 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7WMNQ f as6040cl 2- ams( 1573753632)
. . . . .
0d. 41 f as6040cl 2- ams( 1573753632) Pool 0 J LVT29GC f as6040cl 2- ams( 1573753632)
0d. 43 f as6040cl 2- ams( 1573753632) Pool 0 J LVT7BUC f as6040cl 2- ams( 1573753632)
. . . . .
0c.21 f as6040cl 1- ams( 1573753606) Pool 0 J LVT0KDC fas6040cl1-ams(1573753606)
0c.18 f as6040cl 1- ams( 1573753606) Pool 0 J LVT2HZC fas6040cl1-ams(1573753606)
. . . . .
5a.01.13 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7W3XZ fas6040cl1-ams(1573753606)
5a.01.21 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7WSX8 fas6040cl1-ams(1573753606)
5a.01.16 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7W3YT fas6040cl1-ams(1573753606)
5a.01.12 f as6040cl 1- ams( 1573753606) Pool 0 9QJ 7WS0R fas6040cl1-ams(1573753606)
In this example, the local System ID for the new Controller is 1943753293.
The old MB System ID was 1573753606 (disk show -v from Section IV).
The disks need to be reassigned to the local System ID. Example Only
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
XI.
Step
1
NOTE The disk reassignment process takes several seconds and a message is printed for each disk that is reassigned.
A.
A1
Capture the DOT version at the maintenance prompt " *>" by entering: 'version' .
STOP
A2
(i)
(ii)
(iii)
(iv)
(v)
(vi)
A3 Login as " root" to the Partner node. End-user may be required to provide password.
A4 Enter: ' priv set advanced ' at the prompt for the following command to work. Prompt will include " * " .
A5
A6
A7
Do NOT run any other commands from the partner node before performing a 'cf giveback' !
A8 Continue with step 2 on next page.
On the partner node, enter: 'aggr status -f ' to make sure there are no failed disks in the system as they will
need to be replaced before the disk reassignment and giveback. Inform customer to open a support case if
there are "failed" disks.
Page 12 of 15
SA600 and V-FAS6000 Series: Disk Reassign
Action Description
Follow procedure-A if the node was successfully taken over by its partner.
Follow procedure-B on next page if this node is a single controller configuration or the partner did NOT takeover.
Execute the " A" steps on the partner node. Engage-user to assist and for the password.
The question that must be answered by the customer is: Are there any Windows applications running that
would inhibit a cf giveback at this time because the giveback process needs to be executed right after the
disk re-assignment is finished?" IF the customer states the giveback cannot be performed now, the 'disk
reassignment' must be postponed until it can be performed - Engage NGS if questions.
If the giveback can be preformed now, continue with step A3, otherwise continue with step (ii).
Provide the customer and NGS support the new "Local" system-ID that was displayed in the last Section
"Capture the new System-ID". Note, the "old" system-ID was captured in Section IV. Inform customer to
engage NGS when it's time to do the reassignment to make sure it's entered properly.
Confirm the "target" system is left at the maintenance mode prompt "*>" for the disk-reassignment to be
performed later.
Confirm all FC and Ethernet and Controller Interconnect cables are fully inserted into their proper port.
All controller interconnect ports and FC Ports configured as "initiators", should have the "Link" LED ON.
If the OS version is NOT a DOT 8 version, skip to step A3 below. If this system is running DOT 8.0 thru 8.0.2P1 or
8.1RC1, the "disk reassign" and 'cf giveback' steps need to be performed consecutively and then a second
Takeover/Giveback is executed from the "target" node once the "target" node comes online. Follow these steps
carefully.
CAUTION
Skip to Section XIII to complete this dispatch.
NOTE
The partner console prompt must have the word " (takeover)" in it. If not, verify with end-user or NGS that the
takeover did NOT occur. If it did not, use Method B
At the console prompt enter: ' disk reassign -s <old_system_ID> -d <new_system_ID> '.
Cut-n-paste the old and new System IDs from the console Log.
The following message will be displayed if the system is running DOT 8.x. This is a reminder that once the giveback
is complete another Takeover and Giveback has be executed from the "target" (repaired) node. This will be done
later. Enter 'y' to continue.
Enter 'y' to the question " Would you like to continue (y/n)?"
Di sk owner shi p wi l l be updat ed on al l di sks pr evi ousl y bel ongi ng t o Fi l er wi t h
sysi d 1573753606.
Woul d you l i ke t o cont i nue ( y/ n) ? y
di sk r eassi gn: A gi veback must be done i mmedi at el y f ol l owi ng a r eassi gn of par t ner di sks.
Af t er t he par t ner node becomes oper at i onal , do a t akeover and gi veback of
t hi s node t o compl et e t he di sk r eassi gn pr ocess.
Do you want t o cont i nue ( y/ n) ? y
Enter: ' y '
REMINDER of takeover/giveback required
from the " target" (repaired) node.
Enter: ' y '
Command Example Only:
partner-system name(takeover)*> disk reassign -s 1573753606 -d 1943753293
*> version
NetApp Release 8.0.1 7-Mode: Thu J an 27 20: 23: 12 PST 2011
Command
Example Only:
A console message will be displayed for
each disk changing ownership (System ID)
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
XI.
Step Action Description
B.
B1
B2
B3 Continue with step 2.
2
3 Skip to step 4 if this is not a V-series Filer (V6000). If V-series Filer, perform the below Additional Steps.
Additional Steps for V-Series Filer
4 At the maintenance mode prompt: " * >" , enter ' halt ' to exit to " LOADER> ".
5 Go to Section XII, "Boot the Operating System - 'cf giveback' if applicable" on next page.
Page 13 of 15
SA600 and V-FAS6000 Series: Disk Reassign (cont.)
Single Controller configuration or the partner did NOT takeover. Execute the " B" steps from
Maintenance mode on the replacement Controller.
At the maintenance mode " * > " prompt enter: ' disk reassign -s <old_system_ID> -d <new_system_ID> '. Cut-n-
paste the old and new System IDs from the console Log.
Enter 'y' to the question " Would you like to continue (y/n)?"
From the console port on " target" controller on which you replaced the NVRAM Adapter (in maintenance mode):
Enter ' disk show -v ' to display the disks reassigned to the new System ID.
STOP!
Verify the system-id for this node's disks listed under " HOME" if the column exists, use OWNER if not, and the new " Local
System ID" are the same. If not, confirm the correct system-ids were entered on the 'disk reassign' command. If problems,
do NOT proceed, call NGS for assistance.
*> disk show -v
Local Syst emI D: 1943753293



DI SK OWNER POOL SERI AL NUMBER HOME
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
5c. 02. 4 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7VRRF f as6040cl 2- ams( 1573753632)
5c. 02. 3 f as6040cl 2- ams( 1573753632) Pool 0 9QJ 7WMNQ f as6040cl 2- ams( 1573753632)
. . . . .
0d. 41 f as6040cl 2- ams( 1573753632) Pool 0 J LVT29GC f as6040cl 2- ams( 1573753632)
0d. 43 f as6040cl 2- ams( 1573753632) Pool 0 J LVT7BUC f as6040cl 2- ams( 1573753632)
. . . . .
0c.21 -------------------------- Pool 0 J LVT0KDC (1943753293)
0c.18 -------------------------- Pool 0 J LVT2HZC (1943753293)
. . . . .
5a.01.13 -------------------------- Pool 0 9QJ 7W3XZ (1943753293)
5a.01.21 - - - - - - - - - - - - - - - - - - - - - - - - - - Pool 0 9QJ 7WSX8 (1943753293)
5a.01.16 - - - - - - - - - - - - - - - - - - - - - - - - - - Pool 0 9QJ 7W3YT (1943753293)
5a.01.12 - - - - - - - - - - - - - - - - - - - - - - - - - - Pool 0 9QJ 7WS0R (1943753293)
. . . . .
. . . . .
Example Only
The new local System ID for the Controller is 1943753293. The
owner name (fas6040cl1-ams) may or may not be shown. But
those disks should reflect the new local System ID.
Command Example Only:
partner-system name(takeover)*> disk reassign -s 1573753606 -d 1943753293
Di sk owner shi p wi l l be updat ed on al l di sks pr evi ousl y bel ongi ng t o Fi l er wi t h
sysi d 1573753606.
Woul d you l i ke t o cont i nue ( y/ n) ? y
Enter: ' y '
A console message will be displayed for
each disk changing ownership (System ID)
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
The " disk show -v" command displays the connectivity to the third party array and any (optional) NetApp disks.
If the storage listing is missing or incomplete:
(i) Confirm all FC connections are in the correct HA port and firmly seated on the NetApp controller.
Issues? Engage NGS.
S
T
O
P
XII.
Step
1 At the "LOADER>" prompt, enter "autoboot" to boot Data Ontap.
STOP
2 (i)
(ii)
A. Example: Complete boot ( node operating in stand-alone mode or auto-giveback is enabled)
B. Example: Partial boot (node is part of a HA configuration and was taken over by its partner)
3 Continue with Section XII on next page.
NOTE
An automatic giveback might be invoked if the option 'cf.giveback.auto.enable' is enabled. The console will
eventually display " giveback completed" and when finished, will have a "login" prompt when you hit <enter>.
Page 14 of 15
SA600 and V-FAS6000 Series: Boot the Operating System - 'cf giveback' if applicable
Action Description
While the system is booting, visually confirm the "link" (typically green) LEDs are lit on all FCAL & SAS adapters that have a cable
in them to verify the cable (and GBIC) are properly seated. FC Adapters configured as targets and Ethernet link LEDs will turn
"ON" when the system is almost UP.
If the node autoboots to a login prompt (hit <enter>for response) - see NOTE and " A. Example" below - the node is
operating in stand-alone mode or no partner takeover occurred. Skip to Step 3.
If the node displays " Press Ctrl-C for Maintenance menu to release disks" and " Waiting for giveback" after the
<enter> key is hit, this node was taken over by its partner - see "B. Example" below.
Loadi ng X86_64/ f r eebsd/ i mage1/ ker nel : 0x100000/ 3375736 0x538280/ 3221872 0x84abf 0/ 1189648
Ent r y at 0x80148020
Loadi ng X86_64/ f r eebsd/ i mage1/ pl at f or m. ko: 0x96e000/ 504604 0xa85c60/ 476544 0x9e9320/ 16452
0xaf a1e0/ 27720
. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . .
*******************************
* *
* Pr ess Ct r l - C f or Boot Menu. *
* *
*******************************
. . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . .

Dat a ONTAP ( f as3240- ams. ams2k3dom. ngsl abs. net app. com)
login:
These are
typical Boot strap
console messages.
If the node is operating in stand-
alone mode, you should eventually
get a " login" prompt when you
hit <enter>.
.... Many typical system startup
messages removed for clarity
Phoeni x Tr ust edCor e( t m) Ser ver
Copyr i ght 1985- 2006 Phoeni x Technol ogi es Lt d.
Al l Ri ght s Reser ved
. . . . .
. . . . .
Copyr i ght ( C) 2000- 2003 Br oadcomCor por at i on.
Por t i ons Copyr i ght ( C) 2002- 2008 Net App

CPU Type: I nt el ( R) Xeon( R) CPU E5220 @2. 33GHz

St ar t i ng AUTOBOOT pr ess Ct r l - C t o abor t . . .
. . . .
*******************************
* *
* Pr ess Ct r l - C f or Boot Menu. *
* *
*******************************
Chel si o T3 RDMA Dr i ver - ver si on 0. 1
. . . . .
Waiting for giveback...(Press Ctrl-C to abort wait)
STEP 2: Will get this response
after hitting <enter>.
NOTE B.1:
If you see this message, this node is part
of a HA configuration and the partner node
took over for it.

If the " cf giveback" fails due to partner
" not ready" , wait 5 minutes for the
NVRAMs to synchronize. If the giveback
fails due to " open CIFS sessions" , failed
disks or for any other reason, contact
NGS.
"." = Deleted lines
to save space
LOADER>
LOADER> autoboot
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers
XII.
Step
STOP
4 On the "partner node", enter: "cf giveback" - READ text box Note B.1 on previous page.
a) Once the 'target node' (one with the new NVRAM card) comes back on line, login to it and enter: "cf takeover" .
b)
XIII.
Step
1
2 Email the console log with the NetApp Reference Number in the Subject Line to xdl-tpm-console-logs@netapp.com
3 Place the defective part in the antistatic bag and seal the box.
4 Follow the return shipping instructions on the box to ship the part(s) back to NetApps RMA processing center. If the
shipping label is missing see process to obtain a shipping label here > Missing Shipping Label?
5 Verify with customer that the system is OK and if working with NGS ask them if it is OK to be released.
6 Close dispatch per Rules of Engagement.
If the target system is UP, request end-user to send NetApp an Autosupport so the configuration setup can be verified by NGS.
Use the following command:
filer>options autosupport.doit <enter NetApp FSO/case # here>. Without the <>brackets
If the target system is down, issue the autosuppport from the partner system.
(The FSO number is 7 digits and begins with 5xxxxxx. Case numbers are ten digits and begin with 2xxxxxxxxx)
Page 15 of 15
SA600 and V-FAS6000 Series: Boot the Operating System - 'cf giveback' if applicable (cont.)
Action Description
SA600 and V-FAS6000 Series: Submit logs and Part Return
Action Description
Once the 'takeover' of the 'partner node' is complete, (takeover>) is part of the console prompt, enter: 'cf giveback' to
complete the process. This process synchronizes the system-ID between the nodes.
If the partner took over AND the system is running DOT version 8.0 thru 8.0.2P1 or 8.1RC1, the giveback needs to be executed
now. Perform step 4 below. (Reference for bug 490825, README R2). Otherwise skip to Sec XIII.
NVRAM Adapter Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers