SAW Page 1 of 4

HP Integrity Superdome 2 Servers - How to Replace XFabric

Link Cable Online
Issue
The XFabric (also known as Jlink) Cables in a Superdome2 server can be exchanged online with using the OA
(Onboard Administrator) HR (Health Repository) stop link and start link commands.
The procedure might be used when there is an issue on a specific XFabric link.
Example of an issue on XFM 1 port 1 to IOX 9 port 1 XFabric link is:

CAE event:
Event Identification :
Event ID : 2016
Event Time : Wed Jan 26 16:21:51 2011
Indication Identifier : 4201620110126162151
...

Summary:
A system interconnect link (XFabric) failed to initialize at full performance.
Full description:
An XFabric link was brought up to initialize the system or boot a partition and the link could not be trained at full
performance. The FRUs used to implement this link need to be checked for problems.
Probable cause 1:
For cabled links: The cable may be connected to the wrong port. If the link was previously able to train, the cable is likely
connected to the correct port.
For cabled links: The wrong length cable is being used. If the link was previously able to train, the correct length cable is
likely installed.
The enclosure number may be set incorrectly.
The XFabric link is not functioning properly.
Recommended action 1:
Check to make sure the cable is connected to the correct port.
Check to make sure the correct length cable is being used.
Check the enclosure number settings.
Check the XFabric link. The link may be part of a single FRU, or may connect through multiple FRUs. The FRU list is
included as a reference. Check for physical damage on the FRU connection points and ensure proper mating/seating
occurs. If the problem persists, replace only one FRU at a time in the order given below. Test the system between each
FRU replacement.

Replaceable Unit(s) :
Part Manufacturer : Not Available
Spare Part No. : Not Available
Part Serial No. : Not Available
Part Location : 0x0100ff01ff010053
Additional Info : Not Applicable

Part Manufacturer : Not Available

Spare Part No. : Not Available
Part Serial No. : Not Available
Part Location : 0x090001ffff01ff53
Additional Info : Not Available

Part Manufacturer : HP
Spare Part No. : AH341-67001
Part Serial No. : MYJ02705HU
Part Location : 0x0100ff01ff00ff51 enclosure1/xfm1
Additional Info : XFabric Link Id : 14

http://saw.cce.hp.com/km/saw/print.do?docId=emr_na-c02791159-1 12/4/2013
SAW Page 2 of 4

Part Manufacturer : HP
Spare Part No. : AH338-67004
Part Serial No. : DEH03300BW
Part Location : 0x0900ffff01ffff69 enclosure9/midplane0
Additional Info : XFabric Link Id : 0
...

SHOW INDICT:

FRU Type: Xbar Flex Module Link Connector

Location: 0x0100FF01FF010053 Resource path not applicable.
Timestamp: Wed Jan 26 16:18:48 2011
Indictment State: Indicted
Deconfig State: Deconfiguration of this resource not supported

FRU Type: IO Expander Fabric Link Connector

Location: 0x090001FFFF01FF53 Resource path not applicable.
Timestamp: Wed Jan 26 16:18:48 2011
Indictment State: Indicted
Deconfig State: Deconfiguration of this resource not supported

Bay 1 XFM Status:

Health: OK
Power: On
Unit Identification LED: Off
Diagnostic Status:
Internal Data OK
Management Processor OK
Thermal Warning OK
Thermal Danger OK
Firmware Mismatch OK
Indicted OK
Link 1: Degraded <-----!
Link 2: OK
Link 3: OK
Link 4: OK
Link 5: OK
Link 6: OK
Link 7: OK
Link 8: OK

IOX 9 Status:
Status: OK
Power:
Bay 1: On
Bay 2: On
Unit Identification LED: Off
Diagnostic Status:
Internal Data OK
Management Processor OK
Thermal Warning OK
Thermal Danger OK
Cooling OK
Device Failure OK
Firmware Mismatch OK
Indicted OK
...
Xfabric Link Status:
Link 1: Degraded <-----!
Link 2: OK

http://saw.cce.hp.com/km/saw/print.do?docId=emr_na-c02791159-1 12/4/2013
SAW Page 3 of 4

Link 3: OK
Link 4: OK
Link 5: OK
Link 6: OK

HR (Health Repository) Error Logs on OA show always following event:

Keyword: PDH_ICM_REINIT_CCFIFO
Description: Firmware is initializing an I/O host bridge.
Brief Descr: Chassis Code Fifo pointers are reinitialized due to invalid values in ICM
Cause: Forward progress, no action required.

Solution
The following procedure can be used to perform the following:
 Do a dummy replacement of the suspect XFabric link cable. Therefore, just stop and start the link without
touching the cable (skip step 3 below). The link will be initialized again, so one can see if the problem still exists.
 Reseat the suspect XFabric link cable. The cable may just have a seating issue (change step 3 below to just
reseat the suspect cable on both sides). If one faces a stable issue, this should be the attempt to fix the issue
before any replacement is done.
 Exchange the suspect XFabric cable.
Attention: Make sure one follows ESD guidelines and is careful when touching the suspect cable to not harm any other
parts in the SD2.
1. Stop the indicted fabric link.
Note that one needs to stop just one of the two ports (indicted locations) to stop the complete link.
Both link ports and both related FRUs (XFM and IOX) will be indicted.
The LEDs the affected ports will be turned on:

OA1> show hr
OA1 HR> stop link <Indicted Location>

In our example:

OA1 HR> stop link 0x0100FF01FF010053

Link has been stopped.
Request indictments...
Setting IOX-9 LED: ON
Setting Chassis-1 XFM-1 LED: ON
OA1 HR>

2. Wait 3 minutes.
3. Exchange the suspect XFabric cable.
In this example: Exchange the XFM 1 port 1 to IOX 9 port 1 XFabric cable.
4. Start fabric links again (This may take a few minutes.).
Note that one needs to start just one of the two ports (indicted locations) to start the complete link.
The LEDs on both affected ports will be turned off again.
If link training was successful, all four indicted locations (two link ports, XFM and IOX) will be now acquitted:

OA1> show hr
OA1 HR> start link <Indicted Location>

In our example:

OA1 HR> start link 0x0100FF01FF010053

Setting IOX-9 LED: OFF
Setting Chassis-1 XFM-1 LED: OFF
Link training successful. The link started.
OA1 HR>

5. Wait 3 minutes, then check if the locations have been indicted again:

OA1> show hr
OA1 HR> show indict

http://saw.cce.hp.com/km/saw/print.do?docId=emr_na-c02791159-1 12/4/2013
SAW Page 4 of 4

6. If there are no further indictments, check the status of the involved FRUs.
In this example: Check status of XFM 1 and IOX 9:

OA1> show XFM status 1

OA1> show IOX status 9

NOTE: If the link issue is not resolved after the XFabric cable has been replaced, one will need to schedule
some down time. The following FRUs will need to be ordered and have on hand for the repair; an XFM
Module and an IOX Backplane. Continue the troubleshooting during the scheduled downtime.

http://saw.cce.hp.com/km/saw/print.do?docId=emr_na-c02791159-1 12/4/2013