Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Contents
Key Benefits 2
Summary 2
Index of Figures 3
Index of Tables 3
Acronyms and Definitions 4
Background 7
Server Modules 7
PowerEdge M1000e Midplane and
I/O 17
Power 28
Cooling 34
System Management 37
Conclusion 42
Acknowledgments 42
Key Benefits
16servermodules
3x2(Redundant)I/Ofabrics
RedundantPower
RedundantCooling
RedundantChassisManagement
Controllers
AvocentiKVMinterface
LCDControlPanel
Summary
TheDellPowerEdgeM1000eModularServerEnclosure
isabreakthroughinenterpriseserverarchitecture.The
enclosureanditscomponentsspringfroma
revolutionary,groundupdesignincorporatingthelatest
advancesinpower,cooling,I/O,andmanagement
technologies.Thesetechnologiesarepackedintoa
highlyavailablerackdensepackagethatintegratesinto
standardDelland3
rd
party1000mmdepthracks.
Theenclosureis10Uhighandsupports:
Upto16servermodules.
Upto6network&storageI/Ointerconnect
modules.
Ahighspeedpassivemidplaneconnectstheserver
modulesinthefrontandpower,I/O,and
managementinfrastructureintherearofthe
enclosure.
ComprehensiveI/Ooptionssupportduallinksof20
Gigabitspersecondtoday(with4xDDRInfiniBand)
withfuturesupportofevenhigherbandwidthI/O
deviceswhenthosetechnologiesbecomeavailable.
Thisprovideshighspeedservermodule
connectivitytothenetworkandstoragenowand
wellintothefuture.
Thoroughpowermanagementcapabilitiesincluding
deliveringsharedpowertoensurefullcapacityof
thepowersuppliesavailabletoallservermodules.
Broadmanagementabilityincludingprivate
Ethernet,serial,USB,andlowlevelmanagement
connectivitybetweentheChassisManagement
Controller(CMC),Keyboard/Video/Mouseswitch,
andservermodules.
UptotwoChassisManagementControllers(CMC
1isstandard,2
nd
providesoptionalredundancy)and
1optionalintegratedKeyboard/Video/Mouse
(iKVM)switch.
Upto6hotpluggable,redundantPowerSupplies
and9hotpluggable,N+1redundantFanModules.
Page2
SystemFrontControlpanelw/LCDpanelandtwo
USBKeyboard/MouseandoneVideocrashcart
connections.
Index of Figures
Figure1PossibleServerModuleSizes,FrontPanelView..................................................................... 8
Figure2ExampleServerModuleConfigurations .................................................................................. 8
Figure3DualSocketModularServerArchitecture................................................................................ 9
Figure4ModularServerFabricI/OMezzanineCard,FibreChannel4Gbps .................................. 10
Figure5M600DualSocketModularServerwithIntelProcessors .................................................... 13
Figure6M605DualSocketModularServerwithAMDProcessors .................................................. 14
Figure7HalfHeightModularServerFrontPanelView..................................................................... 14
Figure8ModularServerRemoval ......................................................................................................... 15
Figure9PowerEdgeM1000eFrontView............................................................................................... 17
Figure10PowerEdgeM1000eMidplaneFrontView........................................................................... 19
Figure11PowerEdgeM1000eMidplaneRearView............................................................................ 20
Figure12HighSpeedI/OArchitecture.................................................................................................. 21
Figure1310GEPotentialGrowthPath................................................................................................... 23
Figure14PowerEdgeM1000eModularServerSystemBandwidth................................................... 23
Figure15PowerEdgeM1000eRearView.............................................................................................. 25
Figure16DifferencebetweenPassthroughandSwitchModules ...................................................... 26
Figure17I/OModule,DellPowerEdgeGigabitEthernetPassthrough ............................................ 26
Figure18I/OModule,DellPowerEdgeGigabitEthernetSwitch ...................................................... 26
Figure19PowerEdgeM1000eSinglePhasePDUPowerSystem....................................................... 30
Figure20PowerEdgeM1000eThreePhasePDUPowerSystem........................................................ 30
Figure21PowerEdgeM1000e2360WattPowerSupply..................................................................... 31
Figure22PowerRedundancyModes..................................................................................................... 32
Figure23PowerArchitecture.................................................................................................................. 33
Figure24PowerEdgeM1000eFan.......................................................................................................... 34
Figure25ServerModuleCoolingAirProfile........................................................................................ 35
Figure26I/OModuleCoolingAirProfile ............................................................................................. 35
Figure27PowerSupplyCoolingAirProfile......................................................................................... 36
Figure28SystemManagementArchitectureSimplifiedBlockDiagram.......................................... 37
Figure29PowerEdgeM1000eChassisManagementController ........................................................ 38
Figure30ChassisManagementControllerFrontPanel....................................................................... 38
Figure31PowerEdgeM1000eiKVMModule ....................................................................................... 40
Figure32iKVMFrontPanel .................................................................................................................... 41
Figure33PowerEdgeM1000eLCDPanelRecessedPosition ............................................................. 41
Figure34PowerEdgeM1000eLCDPanelDuringUsage.................................................................... 42
Figure35LCDGraphicExamples........................................................................................................... 42
Index of Tables
Table1ModularServerFabricI/OMezzanineFabricBandCOptions............................................ 11
Table2TypicalModularServerFabricConfigurations ....................................................................... 12
Table3HalfHeightModularServerDimensions................................................................................. 13
Table4HalfHeightServerModuleOptions ......................................................................................... 16
Table5PowerEdgeM1000eModularSystemFeaturesandParameters........................................... 18
Page3
Table6RelativeComparisonofIndustryStandardHighSpeedI/O................................................. 22
Page4
Table7EthernetI/OModuleOptions..................................................................................................... 27
Table8FibreChannelI/OModuleOptions........................................................................................... 27
Table9InfiniBandI/OModuleOption................................................................................................... 28
Table10TypicalModularServerSystemRackHeightandCableReduction.................................. 28
Table11PowerEdgeM1000eSinglePhaseandThreePhasePDUOptions...................................... 29
GBGigabyte
GbpsGigabitspersecond
GEGigabitEthernet
Gen1Generation1
Gen2Generation2
GPIOGeneralPurposeI/O
GUIGraphicalUserInterface
HAHighAvailability
HBAHostBusAdapter
HCAHostControllerAdapter
HDDHardDiskDrive
HPCCHighPerformanceComputingCluster
IBInfiniBand
ICHInput/OutputControllerHub
I/OInput/Output
iDRACIntegratedDellRemoteAccessController
iKVMIntegratedKVM
I/OInput/Output
IOHInput/OutputHub
IOMInput/OutputModule
IPCInterprocessCommunications
IPMIIntelligentPlatformManagementInterface
IPv6InternetProtocolversion6
IRIntegratedRAID
iSCSIInternetSCSI
ISOInternationalOrganizationforStandardization
KVMKeyboardVideoMouse
LANLocalAreaNetwork
LCDLiquidCrystalDisplay
LEDLightEmittingDiode
LOMLANonMotherboard
LRLongReach
LVLowVoltage
MbpsMegabitspersecond
NICNetworkInterfaceCard
MBMegabyte
MCHMemoryControllerHub
MHzMegaHertz
MVMidVoltage
N+0N(1,2,3)pluszero,nofaulttolerance
N+1N(1,2,3)plusone,faulttolerancewithonebackupunit
N+NN(1,2,3)plusN(1,2,3)withNbackupunits
NPIVN_PortIDVirtualization
OPSFOpenShortestPathFirst
Page5
OSOperatingSystem
PCIePCIExpress,orPeripheralComponentInterconnectExpress
PDUPowerDistributionUnit
PMBusPowerManagementBus
QDRQuadDataRate
QOSQualityofService
R2Release2
RACADMRemoteAccessControllerandAdministrator
RAIDRedundantArrayofInexpensiveDisks
RIPRoutingInformationProtocol
SANStorageAreaNetwork
SASSerialAttachedSmallComputerSystemInterface,orSerialSCSI
SAS6/IRSASI/OCard6withintegratedRAID
SATASerialAdvancedTechnologyAttachment,orSerialATA
SDDCSingleDeviceDataCorrection
SDRSingleDataRate
SFPSmallFormfactorPluggableTransceiver
SMASHCLPSystemsManagementArchitectureforServerHardwareCommandLineProtocol
SNMPSimpleNetworkManagementProtocol
SOLSerialoverLAN
SRShortReach
SSHSecureShell
SSLSecureSocketsLayer
TCP/IPTransmissionControlProtocol/InternetProtocol
TFTPTrivialFileTransferProtocol
TOETCP/IPOffloadEngine
USBUniversalSerialBus
VGAVideoGraphicsArray
vKVMVirtualKVM
VLANVirtualLAN
vMediaVirtualMedia
VRRPVirtualRouterRedundancyProtocol
W2K3WindowsServer2003OperatingSystem
WOLWakeOnLAN
WSMANWebServicesforManagement
XMLExtensibleMarkupLanguage
XFP10GigabitSmallFormFactorPluggableTransceiver
Page6
Background
ITISALLABOUTEFFICIENCY.ThenewPowerEdgeM1000eisdesignedtohelpyoubemore
efficientwithyourtime,yourpowerandcooling,yourinvestment,andyoursystems
performance.ItisabreakthroughDellengineeredandpatentpendingdesignthatmaximizes
flexibility,powerandthermalefficiency,systemwideavailability,performance,and
manageability.Thechassisintegratesthelatestinmanagement,I/O,powerandcooling
technologiesinamodular,easytousepackage.Designedfromthegrounduptosupportcurrent
andfuturegenerationsofserver,storage,networking,andmanagementtechnologies,the
PowerEdgeM1000eincludestheheadroomnecessarytoscaleyourenvironmentforthefuture.
ThePowerEdgeM1000eModularServerEnclosuresolutionsupportsservermodules,network,
storage,andclusterinterconnectmodules(switchesandpassthroughmodules),ahigh
performanceandhighlyavailablepassivemidplanethatconnectsservermodulestothe
infrastructurecomponents,powersupplies,fans,integratedKVMandChassisManagement
Controllers(CMC).ThePowerEdgeM1000eusesredundantandhotpluggablecomponents
throughouttoprovidemaximumuptime.
Introducingnewlevelsofmodularitytoserverbasedcomputing,thePowerEdgeM1000e
providesidenticalandsymmetricfabricoptionsBandCforeachmodularserver.EthernetI/O
switchessupportI/OsubmodulesthatprovideexternalI/Oflexibilityofstackingports,10GE
copperports,or10GEopticalports.Truemodularityatthesystemandsubsystemlevelprovides
simplicityofextensionandenhancement,nowandinthefuture.
Server Modules
Virtuallyunlimitedinscalability,thePowerEdgeM1000eEnclosureprovidesultimateflexibility
inserverprocessorandchipsetarchitectures.BothIntelandAMDserverarchitecturesare
plannedforintroductionintotheinfrastructure,whilecuttingedgemechanical,electricaland
softwareinterfacedefinitionsenablemultigenerationalserversupportandexpansion.
ThePowerEdgeM1000eenclosuresupportsupto16HalfHeightservermodules,each
occupyingaslotaccessibleinthefrontoftheenclosure.ServerModulesotherthanstandard
HalfHeightformfactorarealsosupportedinthefuture,occupyingfullslotheights,dualslot
widths,orboth.Themechanicalslotsintheenclosurecansupportserversthataretwicethe
heightand/orwidthoftheHalfHeightmodule,asshowninFigure1.
ServerModulescanbefreelylocatedwithineach2x2HalfHeightquadrant.Themechanical
designofthePowerEdgeM1000ehassupportstructuresforHalfHeightServerModulesabove
orbelowDoubleWidthServerModules,andforHalfHeightServerModulessidebysidewith
FullHeightServerModules,asshowninFigure2.
Page7
ThefirsttwomodularserversavailableforthePowerEdgeM1000eModularEnclosureareHalf
Height2CPUsocket,8DIMMcapablesystems,withtwo2.5inchhotpluggableHardDiskslots,
andtwoflexiblefabricmezzaninecardsinadditiontothetwoonboard1GigabitEthernetLAN
onMotherboard(LOMs).AsimplifiedblockdiagramofthePowerEdgeM600modularservers
architecture,basedontheIntelprocessor/chipsetarchitecture,isshowninFigure3.
Figure1PossibleServerModuleSizes,FrontPanelView
Figure2ExampleServerModuleConfigurations
Server Processors
ThefirstgenerationofHalfHeightservermodulessupportafullrangeofdualandquadcore
processorsineitherLowVoltage(4068W),MediumVoltage(8095W)orFullPower(120W)
profiles.Somerestrictionsonmemorysupportmayapplytothe120Wquadcorefullpower
processors,baseduponthesystempowerenvelopeandcoolingcapabilities.BothAMDandIntel
processorarchitecturesaresupported.
Server Memory
MemorysupportinthefirstgenerationHalfHeightservermodulesiscomprisedof8DIMM
slots.IntelbasedarchitecturesuseFullyBufferedDIMMtechnology,whileAMDbased
Page8
architecturesuseDDR2DIMMtechnology.Interleaving(2waylockstep),ECC,memorysparing
Figure3DualSocketModularServerArchitecture
andChipKill/SDDCaresupportedwithbothAMDandIntelbaseddesigns.Memorymirroring
issupportedonthePowerEdgeM600Intelbaseddesign.SupportingarangeofDIMMcapacities
from512MBto8GB,eachservermodulemaycontainamaximumof64GBofmemory.
Server I/O
Eachservermoduleconnectstotraditionalnetworktopologies,whileprovidingsufficient
bandwidthformultigenerationalproductlifecycleupgrades.I/Ofabricintegrationencompasses
networking,storage,andinterprocesscommunications(IPC).TounderstandthePowerEdge
M1000earchitecture,itisnecessarytofirstdefinefourkeyterms:fabric,lane,linkandport.
Page9
Afabricisdefinedasamethodofencoding,transporting,andsynchronizingdatabetween
devices.ExamplesoffabricsareGigabitEthernet(GE),FibreChannel(FC)orInfiniBand(IB).
FabricsarecarriedinsidethePowerEdgeM1000esystem,betweenservermoduleandI/O
Modulesthroughthemidplane.Theyarealsocarriedtotheoutsideworldthroughthephysical
copperoropticalinterfacesontheI/Omodules.
AlaneisdefinedasasinglefabricdatatransportpathbetweenI/Oenddevices.Inmodernhigh
speedserialinterfaceseachlaneiscomprisedofonetransmitandonereceivedifferentialpair.In
reality,asinglelaneisfourwiresinacableortracesofcopperonaprintedcircuitboard,a
transmitpositivesignal,atransmitnegativesignal,areceivepositivesignal,andareceive
negativesignal.Differentialpairsignalingprovidesimprovednoiseimmunityforthesehigh
speedlanes.Variousterminologyisusedbyfabricstandardswhenreferringtolanes.PCIecalls
thisalane,InfiniBandcallsitaphysicallane,andFibreChannelandEthernetcallitalink.
Alinkisdefinedhereasacollectionofmultiplefabriclanesusedtoformasinglecommunication
transportpathbetweenI/Oenddevices.Examplesaretwo,fourandeightlanePCIe,orfourlane
10GBASEKX4.PCIe,InfiniBandandEthernetcallthisalink.Thedifferentiationhasbeenmade
herebetweenlaneandlinktopreventconfusionoverEthernetsuseofthetermlinkforboth
singleandmultiplelanefabrictransports.SomefabricssuchasFibreChanneldonotdefinelinks
astheysimplyrunmultiplelanesasindividualtransportsforincreasedbandwidth.Alinkas
definedhereprovidessynchronizationacrossthemultiplelanes,sotheyeffectivelyacttogether
asasingletransport.
AportisdefinedasthephysicalI/Oendinterfaceofadevicetoalink.Aportcanhavesingleor
multiplelanesoffabricI/Oconnectedtoit.
TherearethreesupportedhighspeedfabricsperPowerEdgeM1000eHalfHeightservermodule,
withtwoflexiblefabricsusingoptionalpluginmezzaninecardsontheserver.Theportsonthe
servermoduleconnectviathemidplanetotheassociatedI/OModules(IOM)intherearofthe
enclosure,whichthenconnecttothecustomersLAN/SAN/IPCnetworks.
Figure4ModularServerFabricI/OMezzanineCard,FibreChannel4Gbps
Page10
Thefirstembeddedhighspeedfabric(FabricA)iscomprisedofdualGigabitEthernetLOMsand
theirassociatedIOMsinthechassis.TheLOMsarebasedontheBroadcom5708NetXtremeII
Ethernetcontroller,supportingTCP/IPOffloadEngine(TOE)andiSCSIbootcapability.In
addition,fulliSCSIHBAcapabilitywithfullprotocoloffloadandabroaderarrayofOSboot
supportisplannedtobeavailableinearly2008.Whileinitiallyserversreleasedinthisseries
haveadualGELOMconfiguration,thesystemmidplaneisdesignedtoallowfuturesupportfor
uptoquadGELOMsintheHalfHeightformfactor.
OptionalcustomerconfiguredfabricsaresupportedbyaddinguptotwodualportI/O
Mezzaninecards.EachHalfHeightservermodulesupportstwoidenticalI/OMezzaninecard
connectorswhichenableconnectivitytoI/OfabricsBandC.TheseoptionalMezzaninecards
offerawidearrayofEthernet(includingiSCSI),FibreChannel,andInfiniBandtechnologies,with
otherspossibleinthefuture.Figure4showsanexampleofadualportFibreChannel4Gigabits
persecond(Gbps)mezzaninecard.Liketheconnector,I/OMezzaninecardformfactorsare
commontobothfabricBandC,forahighlevelofflexibilityinfabricconfiguration.
Theoptionalmezzaninecardsaredesignedtoconnectvia8lanePCIetotheservermodules
chipset.ForPCIeGen1thisprovidesupto16Gbpsofdatabandwidthpermezzaninecard.
MezzaninecardsmayhaveeitheronedualportASICwith4or8lanePCIeinterfacesordual
ASICs,eachwith4lanePCIeinterfaces.BothPCIefabricsandexternalgoingfabricsarerouted
throughhighspeed,10Gigabitpersecondcapableairdielectricconnectorpinsthroughthe
planarandmidplane.Forbestsignalintegritythesignalsisolatetransmitandreceivesignalsfor
minimumcrosstalk.Differentialpairsareisolatedwithgroundpinsandsignalconnector
columnsarestaggeredtominimizesignalcoupling.
WakeonLAN(WOL)issupportedbytheservermodulesLOMs.EthernetWOLforFabricsB
andCisalsosupported.BootfromSANissupportedbyFibreChannelandiSCSIenabledcards.
ThePowerEdgeM1000esystemmanagementhardwareandsoftwareincludesFabric
ConsistencyChecking,preventingtheaccidentalactivationofanymisconfiguredfabricdeviceon
aservermodule.SincemezzaninetoI/OModuleconnectivityishardwiredyetfullyflexible,a
usercouldinadvertentlyhotplugaservermodulewiththewrongmezzanineintothesystem.
Forinstance,ifFibreChannelI/OModulesarelocatedinFabricCI/OSlots,thenallserver
modulesmusthaveeithernomezzanineinfabricCoronlyFibreChannelcardsinfabricC.Ifa
GEmezzaninecardisinaMezzanineCslot,thesystemautomaticallydetectsthis
misconfigurationandalertstheuseroftheerror.Nodamageoccurstothesystem,andtheuser
hastheabilitytoreconfigurethefaultedmodule.
Table1showsoptionsfortheinitialreleaseoffabricmezzaninesinthePowerEdgeM1000e
productline.ThesamemezzaninecardcanbeusedinFabricBorC.Table2showssome
typicalservermodule
FabricType Ports Speedperport Features
GigabitEthernet 2 1.25Gbps TOE,iSCSIboot,JumboFrames,
WOL
FibreChannel4G 2 4.25Gbps BootfromSAN,NPIV
FibreChannel8G* 2 8.50Gbps BootfromSAN,NPIV
4xDoubleDataRate
(DDR)InfiniBand*
2 20.00Gbps
10GigabitEthernet* 2 10.3125Gbps TOE,iSCSIoffload,JumboFrames
*planned
Page11
Table1ModularServerFabricI/OMezzanineFabricBandCOptions
TypicalFabric
Configurations
FabricA FabricB FabricC
LowCostCompute
ClusterNode
2x1GELOM Empty Empty
Ethernet/SANserver
orHAClusterNode
2x1GELOM 2x1GEMezzanine 2xFC4Mezzanine
HighPerformance
ClusterNode
2x1GELOM 2x4DDRInfiniBand Empty
UnifiedFabric 2x1GELOM 2x1GEMezzanine 2x10GEMezzanine
HighPerformance
UnifiedFabric
2x1GELOM 2x10GEMezzanine 2x10GEMezzanine
Table2TypicalModularServerFabricConfigurations
fabricconfigurations,baseduponcurrentandfuturearchitecturalmodels.Speedandlanecount
increasesareanticipatedforfuturemezzaninereleases.
Local Storage
Servermodulelocalstorageiscomprisedofuptotwo2.5inchHardDrives.Bothhotpluggable
SASandnonhotpluggableSATAharddrivesaresupported.Localstoragecontrolleroptions
includelowcostchipsetSATAI/O,SASDaughtercardwithintegratedRAID(SAS6/IR)orSAS
RAIDDaughtercardwithcacheandhighperformanceRAIDprocessor(CERC6/i).
Integrated Server Management
ThePowerEdgeM1000esupportstheintroductionofDellsnewIntegratedDellRemoteAccess
Controller(iDRAC)whichisintegratedoneachM600/605servermodule.iDRACcontains
featuresthataretypicallyaddonoptionstostandardmonolithicrackbasedservers.TheiDRAC
addsvirtualMedia(vMedia)andvirtualKVM(vKVM)functionsaswellasoutofbandGUI
status/inventory.TraditionalIPMIbasedBaseboardManagementController(BMC)featureslike
hardwaremonitoringandpowercontrolaresupported.Additionally,onboardgraphicsand
keyboard/mouseUSBconnectstoanoptionalsystemlevelIntegratedKVM(iKVM)modulefor
localKVMaccess.FullUSBaccessisavailablethroughtheservermodulefrontpanel.
ThenewiDRACisconnectedtotheCMCviadedicated,fullyredundant100MbpsEthernet
connectionswiredthroughthemidplanetoadedicated24portethernetswitchontheCMC,and
exposedtotheoutsideworldthroughtheCMCsexternalManagementEthernetinterface
(10/100/1000M).ThisconnectionisdistinctfromthethreeredundantdataFabricsA,BandC.
UnlikepreviousgenerationsofDellservermodules,theiDRACsconnectivityisindependentof,
andinadditionto,theonboardGELOMsontheservermodule.EachservermodulesiDRAC
hasitsownIPaddressandcanbeaccessed,ifsecuritysettingsallow,directlythrougha
supportedbrowser,telnet,SSH,orIPMIclientonthemanagementstation.
Server Module and Packaging
Page12
Theservermoduleisasheetmetalmodulethathousestheserverplanarandhotplughard
drivesandisdesignedtobetoollessforanyassemblyordisassemblyoperation.Anypart
removalfromtheserverisintuitiveandrequiresnotools.Thisincludesthetopcover,planar
Figure5M600DualSocketModularServerwithIntelProcessors
installation,harddrives,fabriccardsandremovableHardDrivecage.PhotosoftheM600and
M605bladesareshowninFigure5andFigure6respectively,whileoverallbladedimensionsare
showninTable3.
BladeMeasurement Dimension
Length 520mm
Height 193mm
Width 50mm
Table3HalfHeightModularServerDimensions
Server Module Front Panel
TheHalfHeightservermodulesfrontpanelprovidesaccesstotwohotpluggableharddrives,
twobootableUSBconnectors,oneserverpowerbuttonandonestatusindicator.Allserver
modulessupporthotpluggable2.5inchharddiskdrives.Thefrontpanelisvisibleinthephoto
inFigure7.
Server Module Removal and Translating Handle
Theservermoduleismanuallyinstalledbyaligningitwithoneoftheopenfrontslotlocations
andslidingituntiltheconnectorsbegintoengagewiththemidplane.Atranslatinghandle,
whichtravelsinthesamedirectionastheservermodule,isthenusedtofullyengageandlockit
inplace.Toremovetheservermodule,abuttonreleasesthetranslatinghandlelatch.The
handleisthenusedforpullingtheserveroutofthesystem,disengagingtheinternallatchasthe
handleispulledoutwardsandunmatingtheconnectors.
Page13
Figure6M605DualSocketModularServerwithAMDProcessors
Figure7HalfHeightModularServerFrontPanelView
Thereareguidepinsonbothsidesoftheservermodulesignalconnectorforprecisealignmentto
thechassis.Keyingfeaturespreventtheservermodulefrominstallingupsidedown.Adefined
maximumslidingforceinsuresergonomicfriendliness,whileprecisionpointofcontact
positioningandcammingfeaturesensureproperconnectorwipeformaximumsignalquality.
Modularserverpowerisreceivedthroughadedicated2x3powerblock,withwidepowerpins
designatedinsuchawaytopreventpowerrailshortingincaseapinbendsforanyreason.
Page14
Figure8ModularServerRemoval
Server Module Summary
Table4summarizesthefeaturesforthefirsttwoservermodulesshippingforthePowerEdge
M1000e.AdditionalservermodulesareexpectedtoshipfollowingthelaunchofthePowerEdge
M1000e,andthroughoutthelifetimeofthemodularsystem.
Page15
Table4HalfHeightServerModuleOptions
Figure9PowerEdgeM1000eFrontView
TheDellPowerEdgeM1000esupportsuptosixteenHalfHeightservermodules.Thechassis
guideandretentionfeaturesaredesignedsuchthatalternativemoduleformfactorsarepossible.
SeetheServerModulesectionformoredetails.Thechassisarchitectureisflexibleenoughthat
server,storageorothertypesoffrontloadingmodulesarepossible.
Notvisiblyobvious,butimportantnonetheless,arefreshairplenumsatbothtopandbottomof
thechassis.Thebottomfreshairplenumprovidesnonpreheatedairtothepowersupplies,
enablinggroundbreakinglevelsofpowerdensity.Thetopfreshairplenumprovidesnon
preheatedairtotheCMC,iKVMandI/OModules.SeetheCoolingsectionformoredetails.
Page17
Feature Parameter
ChassisSize 10Uhighrackmount
BladesperChassis 16HalfHeight,8FullHeight
TotalBladesina42URack 64HalfHeight,32FullHeight
TotalI/OModuleBays 6(3redundantordualfabrics)
TotalPowerSupplies 6(3+3redundant)
TotalFanModules 9(8+1redundant)
ManagementModulesandInterfaces 2CMCs(1+1redundant),1iKVM,Front
ControlPanel,GraphicalLCDControlPanel
Width,notincludingrackears 447.5mm
Height 440.5mm
Depth,RearofEIAFlangetoRearofChassis 753.6mm
TotalSystemDepth(FrontBezeltoPSLatch): 835.99mm
Table5PowerEdgeM1000eModularSystemFeaturesandParameters
PowerEdge M1000e Midplane
Thoughhiddenfromviewinanactivelyrunningsystem,themidplaneisthefocalpointforall
connectivitywithinthePowerEdgeM1000eModularSystem.Themidplaneisalargeprinted
circuitboardprovidingpowerdistribution,fabricconnectivityandsystemmanagement
infrastructure.Additionallyitallowsairflowpathsforthefronttobackcoolingsystemthrough
ventilationholes.
Asisrequisiteforfaulttolerantsystems,thePowerEdgeM1000emidplaneiscompletelypassive,
withnohiddenstackingmidplanesorinterposerswithactivecomponents.I/OFabricsand
systemmanagementarefullyredundantfromeachhotpluggableitem.Thesystem
managementEthernetfabricisfullyredundantwhentwoCMCsareinstalled,withtwopointto
pointconnectionsfromeachservermodule.
I/Ofabricsareroutedthrough10Gbpscapablehighspeedconnectorsanddielectricmaterial.All
tracesaredifferential100ohmcharacteristicimpedance.Placementofconnectorsandpluggable
componentswasdeterminedbaseduponminimumroutingdistanceforhighspeedcriticalnets.
HighspeedI/Oconnectorholesarebackdrilledinthemidplanetominimizeelectricalstublength
andimprovehighspeedsignalintegrity.MidplanematerialisanimprovedFR408dielectric
material,providingbetterhighfrequencytransmissioncharacteristicsandlowerattenuation.The
I/OChannelshavebeensimulatedto10GBASEKRchannelmodels.Perindustrystandard
requirements,fabricsinternallysupportaBitErrorRateof10
12
orbetter.Atthesystemlevelthis
meansthatahighlevelofinvestmenthasbeenmadetoinsurescalablebandwidth,forcurrent
andfuturegenerationsofserverandinfrastructure.
Page18
Themidplaneservesastransportforapatentpendingtimedivisionmultiplexedserialbusfor
GeneralPurposeI/Oreduction.ThisserialbuscontributesgreatlytothemidplanesI/Olane
countreduction,whichistypicallyburdenedwithasignificantI/Opinandroutingchannelcount
oflargelystaticorlowspeedfunctions.Forinstance,allFibreChannelI/OPassthroughmodule
LEDandSFPstatusinformationiscarriedoverthisbus,whichaloneeliminatesoverone
hundredpointtopointconnectionsthatwouldotherwiseberequired.Thetimedivision
multiplexedserialbusisfullyredundant,withhealthmonitoring,separatelinksperCMCand
errorcheckingacrossalldata.
Figure10PowerEdgeM1000eMidplaneFrontView
Thesystemisdesignedforreceptaclesonallmidplaneconnectorsandpinsonallpluggable
components,soanypotentialforbentpinsislimitedtothepluggablefieldreplaceableunit,not
tothesystem.ThiscontributestothehighreliabilityanduptimeofthePowerEdgeM1000e
modularsystem.
Themidplaneisphysicallyattachedtotheenclosurefrontstructuralelement.Itisalignedby
guidepinsandedgesinall3axes.Thisprovidesclosetolerancealignmentbetweentheserver
modulesandtheirmidplaneconnections.
Themidplanehasbeencarefullydesignedtominimizetheimpacttotheoverallsystemairflow.
Themidplanedesignwasevaluatedatvariouspointsthroughoutthedesigncycletomake
tradeoffsbetweenthermalperformanceandmidplanecosts.Earlyanalysisofthemidplane
showedlargeincreasesinairflowduetoamovefrom15%to20%midplaneopening.The
ventilationholesresultingfromthisexpansionareseeninFigure10andFigure11.These
changesaddedtothedemandingrequirementsofthemidplanedesign,butprovidedthebest
overallsystemsolution.
Page19
Blade
Signal
Connectors
Blade
Power
Connectors
Figure11PowerEdgeM1000eMidplaneRearView
Whileyoumightthinkthatmultiplemidplanesarebetterfortheproduct,therereallyisno
systemadvantageifthemidplaneisproperlydesigned.Allmidplaneroutingisfullyisolated,
supportingallchassispower,fabric,systemmanagementandfaulttolerancerequirements.The
singleintegratedmidplanealsosimplifiesmanufacturingassembly,reducingnetacquisitioncost
tothecustomer.
PowerEdge M1000e I/O
I/OisfullyscalabletocurrentandfuturegenerationsofservermodulesandI/OModules.There
arethreeredundantmultilanefabricsinthesystem,asillustratedinFigure12.
FabricAisdedicatedtoGigabitEthernet.Thoughinitialservermodulereleasesaredesignedas
dualGigabitEthernetLOMcontrollersontheservermoduleplanar,themidplaneisenabledto
supportuptofourGigabitEthernetlinksperservermoduleonFabricA.Potentialdata
bandwidthforFabricAis4GbpsperHalfHeightservermodule.
Page20
Power
Supply
Connectors
Fan
Connectors
I/O Module
Connectors
LCD and
Control
Panel
Connector
CMC
Connectors
iKVM
Connector
OneimportantnewcapabilityinthePowerEdgeM1000eisfull10/100/1000MEthernetsupport
whenusingEthernetpassthroughmodules.Inthepast,customerswerelimitedtoconnecting
onlytoexternalswitcheswith1000Mports.Nowthesystemcanconnecttolegacyinfrastructure
whetherusingEthernetpassthroughorswitchtechnology.Thistechnicaladvanceusesinband
signalingon1000BASEKXtransport,andrequiresnouserinteractionforenablement.
External
Fabric
Connec-
tions
M1000e Midplane Half Height Modular Server (1 of 16)
Fabric B
Mezzanine
Fabric C
Mezzanine
Fabric A
LOM
Fabric A1
Ethernet
I/O Module
Fabric A2
Ethernet
I/O Module
Fabric B1
I/O Module
Fabric B2
I/O Module
Fabric C1
I/O Module
Fabric C2
I/O Module
1-2 lane
1-4 lane
1-4 lane
1-2 lane
MCH/
IOH
4-8 lane
PCIe
8 lane
PCIe
8 lane
PCIe
Half Height Modular Server (16 of 16)
Fabric B
Mezzanine
Fabric C
Mezzanine
Fabric A
LOM
MCH/
IOH
4-8 lane
PCIe
8 lane
PCIe
8 lane
PCIe
1-4 lane
1-4 lane
1-2 lane
1-2 lane
1-4 lane
1-4 lane
1-4 lane
1-4 lane
CPU
CPU
CPU
CPU
Figure12HighSpeedI/OArchitecture
Page21
FabricBandCareidentical,fullycustomizablefabrics,routedastwosetsoffourlanesfrom
mezzaninecardsontheservermodulestotheI/OModulesintherearofthechassis.Supported
bandwidthrangesfrom1to10Gbpsperlanedependingonthefabrictypeused.Table6shows
comparativebandwidthbetweencurrentandforthcomingfabrics.SymbolRateisdefinedasthe
maximumfabricspeedwithalldataencodedasperitsencodingmechanism.DataRateisthe
maximumamountofactualdatabitsthatcanbetransportedonthefabric.Somefabricsare
definedassinglelaneonly,whileothersaredefinedinvariouslaneincrementsforlinktolane
aggregation,inordertoprovideincreasedtotalbandwidth.
Aseachmezzaninecardisconnectedbyan8lanePCIelinktotheserverchipset,thereareno
throttlepointsinthesystemwhereI/Obandwidthisconstricted.Whenmultilane10GBASEKR
becomesareality,DellanticipatesservermoduleswithPCIeGen2orbettersupport,providing
fullendtoendI/ObandwidthfromtheservermodulestotheI/OModules.
Fabric Encoding
SymbolRate
PerLane
(Gbps)
DataRate
PerLane
(Gbps)
DataRate
PerLink
(Gbps)
LanesPerLinkPer
IndustrySpecification
PCIeGen1 8B/10B 2.5 2 8(4lane) 1,2,4,8,12,16,32
PCIeGen2 8B/10B 5 4 16(4lane) 1,2,4,8,12,16,32
PCIeGen3 scrambling 8 8 32(4lane) 1,2,4,8,12,16,32
SATA3G 8B/10B 3 2.4 2.4 1
SATA6G 8B/10B 6 4.8 4.8 1
SAS3G 8B/10B 3 2.4 2.4 1Any
SAS6G 8B/10B 6 4.8 4.8 1Any
FC4G 8B/10B 4.25 3.4 3.4 1
FC8G
8B/10B+
scrambling 8.5 6.8 6.8 1
IBSDR 8B/10B 2.5 2 8(4lane) 4,12
IBDDR 8B/10B 5 4 16(4lane) 4,12
IBQDR 8B/10B 10 8 32(4lane) 4,12
GE:1000BASEKX 8B/10B 1.25 1 1 1
10GE:10GBASEKX4 8B/10B 3.125 2.5 10(4lane) 4
10GE:10GBASEKR 64B/66B 10.3125 10 10 1
Table6RelativeComparisonofIndustryStandardHighSpeedI/O
Ethernet Bandwidth Migration Path
Page22
ThebandwidthgrowthpathforEthernetisfromGEto10GE,supportedthroughthreebackplane
standardsdefinedbytheIEEE802.3apCommittee.1000BASEKXsupportsGEthrough1.25
Gbpsdifferentialbackplanesignaling.10GBASEKX4supports10GEthroughfourlanesof3.125
Gbpsdifferentialbackplanesignaling.Thefourlanesareaggregatedtoprovidethesumtotal10
Gbpsbandwidthrequirement.10GBASEKRisanemergingIEEE802.3apstandardthatsupports
10GEthroughasinglelaneof10.3125Gbpsdifferentialbackplanesignaling.Intermsof
customerimpact,thismeansthattodaysmodularsystemsusing10GBASEKX4arelimitedto
one10GEchannelper4lanesofinternaldifferentialsignaling,whileinthefuturethosesame
lanescancarrymultiplelanesof10GBASEKRandprovidearesultantincreaseintotalsystem
throughput.SeeFigure13foranillustrationofthe10GEgrowthpath.Thisgrowthpathapplies
tobothFabricsBandC.
GEfabricsusesinglelanesperlinkover1000BASEKX.Initially10GEistobesupportedthrough
10GBASEKX4,whichlikeInifiniBandusesfourlanesperlink.Astechnologymaturesand
enableshighlevelintegration,10GBASEKRisexpectedtoreplace10GBASEKX4asthe10GE
backplanetransportofchoice,enabling10Gbpsoversinglelaneswithinthesystemmidplane.
Finally,multiplelanesof10GBASERpermezzaninecanprovideexpandedEthernetbandwidth
upto80Gbpsperfabricmezzanine.
2x10GE
Mezzanine using
4 lane
10GBASE-KX4
10GBASE-KX4
takes 4 lanes to
achieve 10 Gbps
data bandwidth,
2 links per
mezzanine for 20
Gbps total
2x10GE
Mezzanine using
1 lane
10GBASE-KR
10GBASE-KR
takes 1 lane to
achieve 10 Gbps
data bandwidth,
2 links per
mezzanine for 20
Gpbs total
2x4x10GE
Mezzanine using
multilane
10GBASE-KR
Up to 4 lanes of
10GBASE-KR
can achieve 40
Gbps data
bandwidth,
8 links per
mezzanine for 80
Gpbs total
Figure1310GEPotentialGrowthPath
Total System Bandwidth
Inmostcases,onlyasmallpercentageofthisbandwidthisusedinasystem,asillustratedin
Figure14.Atypicalconfigurationisdefinedas4lanesofGEand2lanesofFibreChannelper
servermodule,anduseslessthan10%ofthesystemspotentialbandwidth.Anexampleofa
unifiedEthernetconfiguration,aggregatingallnetwork,storageandInterprocess
CommunicationsonEthernetlinks,isshownas2lanesofGEand4lanesof10GEperserver
module,using25.7%ofsystempotentialbandwidth.AnexampleofaHighPerformance
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
MaximumPotentialBandwidth
HighPerformanceSAN/IPC
UnifiedEthernet
Typical
S
y
s
t
e
m
C
o
n
f
i
g
u
r
a
t
i
o
n
TotalSystemBandwidth
Page23
Figure14PowerEdgeM1000eModularServerSystemBandwidth
SAN/IPCconfigurationisshownas2lanesofGE,2lanesofFC8,and2lanesofQDRInfiniBand,
andstillusesonly35%ofPowerEdgeM1000epotentialbandwidth.Theresultsofthegraph
showthatthePowerEdgeM1000eisfullyscalableforI/Oinfrastructureneedsformany
generationsofserversandswitches.
AssumingafullpopulationofGEand10GbElanesinFabricsA,BandC,thePowerEdgeM1000e
candelivertheoreticaltotalbandwidthof5.44Terabitspersecond.However,thisextreme
bandwidthcomparisonmaynoteverbefullyrealizedinrealworldapplicationusage.Italso
doesnotaddressfactorssuchasflexibilityoffabricmigrationpaths.
ThePowerEdgeM1000eisdesignedforfullsupportofallnear,mediumandlongtermI/O
infrastructureneeds.WhilethePowerEdgeM1000esystemsbandwidthcapabilitiesleadthe
industry,thePowerEdgeM1000eisalsointelligentlydesignedformaximumcost,flexibilityand
performancebenefit.Anexampleofthisistheroutingofdual4lanepathsforFabricsBandC,
anddual2lanepathsinFabricA.Inthenearterm,10GBASEKX4routingsupports10GE
connectivity.10GBASEKX4routingusesall4lanes,eachrunningata2.5Gbpsdatarateto
achievetotallinkdatabandwidthof10Gbps.Todayandintheneartermfuture,10GBASEKX4
isthemostcosteffective,ubiquitoussolutionfor10GEfabricsovermidplanes.Afully
configuredsystemsupports2x2GEand2x210GEimplementationswith10GBASEKX4routing,
supportingtheleadingedgeofunifiednetworktopologies.Suchaconfigurationcould,for
instance,provideredundant10GElinksperbladefortraditionalnetworktraffic,andanotherset
ofredundant10GElinksforiSCSIorFibreChannelOverEthernet(FCoE)networkedstorage.
I/OModulesarefullycompatibleacrossslots.WhileFabricAisdedicatedtotheservermodule
LOMs,requiringEthernetswitchorpassthroughmodulesforI/OslotsA1andA2,FabricsBand
CcanbefreelypopulatedwithEthernet,FibreChannelorInfiniBandsolutions.AspartofDells
efforttosimplifyIT,thereisnoconfusingsupportmatrixtofollowformezzaninesorI/O
modulesandnomultiplicationofmezzanineandI/OModuleformfactorsanddesignstandards,
withresultantcompatibilityorconfigurationconcerns.Dellsupportsonemezzaninedesign
standardandoneI/OModuledesignstandardfortruemodularcomputing.
PowerEdge M1000e Rear View
TherearofthePowerEdgeM1000eEnclosurecontainssystemmanagement,cooling,powerand
I/Ocomponents.AtthetopoftheenclosureareslotsfortwoChassisManagementCardsandone
integratedKeyboard/Video/Mouseswitch.TheenclosureshipsbydefaultwithasingleCMC,
withthecustomerhavingtheoptionofaddingasecondCMCtoprovideafullyredundant,
activestandbyfaulttolerantsolutionformanagementaccessandcontrol.
InterleavedinthecenterofthechassisarefansandI/OModules.Thisarrangementoptimizesthe
balanceofairflowthroughthesystem,allowinglowerpressurebuildupinthesystemand
resultinginlowerairflowrequirementsforthefans.
Page24
I/OModulesareusedaspairs,withtwomodulesservicingeachservermodulefabricandfully
redundant.I/OModulesmaybepassthroughsorswitches.Passthroughmodulesprovidedirect
1:1connectivityfromeachLOM/mezzaninecardportoneachservermoduletotheexternal
network.SwitchesprovideanefficientwaytoconsolidatelinksfromtheLOMorMezzanine
cardsontheservermodulestouplinksintothecustomersnetwork.TheEnclosureaboveis
shownwith10/100/1000MbEthernetandFC4PassthroughModules.Figure16showsthe
differencebetweenthesetwotypesofIOMs.Figure17showstheDellGigabitEthernet
PassthroughModule.
Figure15PowerEdgeM1000eRearView
TheDellandCiscoEthernetswitchesforthePowerEdgeM1000eenclosureuniquelydemonstrate
thelevelofforethoughtthatwentintothedesignoftheI/Omodules.TheseEthernetswitch
modulesintroducesubmoduleI/Oexpansionformaximumflexibilityandextension,acapability
neverbeforeofferedinmodularsystems.Theyprovideasinglehardwaredesignthatallows
scalingfromalowcostallGEsolution,toasolutionthatutilizesswitchstackingtechnologiesto
interconnectmultipleswitcheswithinorbetweenchassis,and/ortoasolutionthatsupportsallof
theseplus10GEuplinkstothecorenetworkwithflexibleinterfacetypes,bothcopperand
optical.Figure18showstheDellPowerConnectGigabitEthernetSwitchModulewithitstwo
submodulebaysavailableforswitchI/Oexpansion.
Page25
iKVM
CMCs
(1 blank)
Fans
I/O Modules
(blank in
middle)
Power Supplies
(3 blanks to left)
Figure16DifferencebetweenPassthroughandSwitchModules
Figure17I/OModule,DellPowerEdgeGigabitEthernetPassthrough
Page26
Figure18I/OModule,DellPowerEdgeGigabitEthernetSwitch
TheinitialoptionsforI/OModulesareoutlinedinTable7,Table8andTable9.Dellplansfuture
I/OModulereleasessupportingalltechnologyupgradepaths,including8GFC,QDRIBand
10GE.
I/OModule
/Features
10/100/1000M
Ethernet
Passthrough
DellPowerConnectM6220
EthernetSwitch
CiscoCatalystBladeSwitch
3032*,3130SG*,
3130SX*
Internal
ports
16 16 16
External
ports
16x
10/100/1000M
4xfixedcopper10/100/1000M
plus2ofthefollowingoption
modules:
1)2x24Gstackingports
2)2x10GOptical(XFPSR/LR)
uplinks
3)2x10GcopperCX4uplinks
4)2x10GBASETcopper
uplinks*
3032:4fixedcopper10/100/1000M+2
optionalbayseachsupporting2x1G
copperoropticalSFPs
3130SG:Above+2x32GStackwise
PlusStackingports
3130SX:
Above+upto2xX2modulesfor10G
CX4orSR/LRuplinks
Speed 10/100/1000M Internalports1G
Externalports
FixedRJ4510/100/1000M
CX4uplink10GE
OpticalXFP10GE
10GBASET10GE
Internalports1G
ExternalportsFixedRJ45
10/100/1000M
X2CX4uplink10GE
X2SR/LRMOptical10GE
Manage
ability
N/A CLI/Web CLI/Web,Ciscomgmttools
Features Supportfor
10/100/1000M
speeds(hardset
andauto
negotiation)
Layer3routing(OSPF,RIP,
VRRP),10GEconnections,
Stacking,IPv6,Layer2/3QOS,
AccessControlLists
L2switching,BaseL3routing(static
routes,RIP),AccessControllists,L2/3
QoS
OptionalupgradestoIPServices(Adv.
L3protocolsupport)andAdvancedIP
Services(IPv6)
*planned
Table7EthernetI/OModuleOptions
I/OModule/
Features
EmulexFC4Passthrough Brocade4424AccessGatewayandFC4Switch
Internalports 16 8/16
Externalports 16x4GSFP 4/8x4GSFP
Speed 1/2/4G 1/2/4G
Manageability N/A CLI/Web,
BrocadeandEMCFabricManagementtools
Features N/A AccessGatewayModeenablesNPIVfunctionalityon
externalports(enhancedinteroperability,simplified
setup,doesntconsumeanFCdomain)
12or24portenablementoptions
Page27
Table8FibreChannelI/OModuleOptions
I/OModule/
Features
CiscoSFSM7000InfiniBandSwitch*
Internalports 16
Externalports 8CopperorOpticalports
Speed 4xDDR
Manageability Fabricbasedmanagement
*planned
Table9InfiniBandI/OModuleOption
Atthebottomofthechassisarethesystempowersupplies.Seethepowersectionformore
detailsonpowersupplyoperationandthepowersysteminfrastructure.
Cable Management
Oneoftheadvantagesofamodularserversystemisthereductionincablemanagementneeds
withinaracksystem.Theinclusionoffabricswitches,integratedKVMandsystemmanagement
aggregationattheCMCsprovidessixfoldorbettercablereduction.Table10showsa
comparisonofatypicalreductionavailablewhenusingthePowerEdgeM1000eModularsystem
withintegratedswitches,comparedtotraditionalrackandstackcomponents.The
configurationinthetableassumesaserverwithfourEthernetportsandtwoFibreChannelports.
InsupportofthePowerEdgeM1000eDellisreleasinganimprovedmodularsystemcable
managementsystemtoeasesysteminstallationinDellorotherindustrystandardracks.
Dellleadstheindustryinpower/performanceefficiencywithitsEnergySmartproducts,andthe
PowerEdgeM1000econtinuesthistradition.Amodularsystemhasmanyadvantagesover
standardrackmountserversintermsofpoweroptimization,andthisaspectwasafocalpoint
throughoutthePowerEdgeM1000esconceptualizationanddevelopment.Thekeyareasof
interestarePowerDeliveryandPowerManagement.
Power Delivery
InsupportofthePowerEdgeM1000e,DellintroducesaSinglePhase60AmpPDUandaThree
Phase30AmpPDU.Becausethreephase30Amppoweristhemostcommondatacenterhigh
powerfeed,thesystemhasbeensizedtofitperfectlywithinthatenvelope,takingonethree
phase30Ampfeedpersetofthreepowersuppliesunderanyloadingcondition.Thethreephase
powergridiscomplementedbythe3+3redundantpowersystemofthePowerEdgeM1000e.
FeaturesforthePDUsareshowninTable11.PDUssupportworldwidestandardhighpower
connectionsforeaseofinstallation,andstandardzeroUverticalmountingwithinDellracks.
TypicalconfigurationsforthePowerEdgeM1000ewithsinglephaseandthreephasePDUsare
showninFigure19andFigure20.
ThePowerEdgeM1000eprovidesindustryleadingpowerefficiencyanddensity,accomplished
throughhighlyefficientcomponents,improveddesigntechniquesandafreshairplenumthat
reducestheairtemperaturetothepowersupplycomponents.Loweroperatingtemperature
equatestohigherpowerdensityforthepowersupply,exceeding21Wattspercubicinch,and
higherpowerefficiency,betterthan86%at20%load,andhigheratheavierloads,approaching
91%efficiencyundernormaloperatingconditions.
Figure19PowerEdgeM1000eSinglePhasePDUPowerSystem
PSU-1
2360W
PSU-6
2360W
PSU-5
2360W
PSU-4
2360W
PSU-3
2360W
PSU-2
2360W
Three
Phase
30 Amp
PDU-1
1
2
Three Phase
Power Cord
(30 Amp
Service)
Three
Phase
30 Amp
PDU-2
Three Phase
Power Cord
(30 Amp
Service)
3
1
2
3
13.8 Amps
Single Phase
24 Amps
Three Phase
Page30
Figure20PowerEdgeM1000eThreePhasePDUPowerSystem
Figure21PowerEdgeM1000e2360WattPowerSupply
PowerefficiencyinthePowerEdgeM1000edoesnotstopwiththepowersupply.Everyaspectof
efficiencyhasbeentweakedandimprovedfrompreviousdesigns,addingmorecoppertoPC
boardpowerplanestoreduceI
2
Rlosses,improvedinductorsandothercomponents,increasing
efficienciesofDCDCconverters,andreplacingsomelinearvoltageregulatorswithmore
efficientswitchingregulators.
PowerredundancyinthePowerEdgeM1000esupportsanynecessaryusagemodel.The
PowerEdgeM1000erequiresthree2360Wattpowersuppliestopowerafullypopulatedsystem
orsixpowersuppliesinafullyredundantsystem.Figure22showstypicalpowerredundancy
models.IntheN+Npowersupplyconfiguration,thesystemprovidesprotectionagainstACgrid
lossorpowersupplyfailures.Ifonepowergridfails,threepowersupplieslosetheirACsource,
andthethreepowersuppliesontheothergridremainpowered,providingsufficientpowerfor
thesystemtocontinuerunning.IntheN+1configurationonlypowersupplyfailuresare
protected,notgridfailures.Thelikelihoodofmultiplepowersuppliesfailingatthesametimeis
remote.IntheN+0configurationthereisnopowerprotectionandanyprotectionmustbe
providedatthenodeorchassislevel.TypicallythiscaseisanHPCCorotherclustered
environmentwhereredundantpowerisnotaconcern,sincetheparallelismoftheprocessing
nodesacrossmultiplesystemchassisprovidesalltheredundancythatisnecessary.
Themidplanecarriesall12VoltDCpowerforthesystem,bothmainpowerandstandbypower.
TheCMCs,LCDandControlPanelarepoweredsolelyby12VoltStandbypower,insuringthat
chassislevelmanagementisoperationalinthechassisstandbystate,wheneverACpoweris
present.Theservermodules,I/OModules,Fans,andiKVMarepoweredsolelyby12VoltMain
power.
Page31
Page32
Figure22PowerRedundancyModes
Figure23PowerArchitecture
Power Management
Powerisnolongerjustaboutpowerdelivery,itisalsoaboutpowermanagement.The
PowerEdgeM1000eSystemaddsanumberofadvancedpowermanagementfeatures.Mostof
thesefeaturesoperatetransparentlytotheuser,whileothersrequireonlyaonetimeselectionof
desiredoperatingmodes.
Sharedpowertakesadvantageofthelargenumberofresourcesinthemodularserver,
distributingpoweracrossthesystemwithouttheexcessmarginrequiredindedicatedrack
mountserversandswitches.ThePowerEdgeM1000eintroducesanadvancedpowerbudgeting
feature,controlledbytheCMCandnegotiatedinconjunctionwiththeiDRAConeveryserver
module.Priortoanyservermodulepoweringup,throughanyofitspowerupmechanismssuch
asACrecovery,WOLorasimplepowerbuttonpress,theservermoduleiDRACperformsa
sophisticatedpowerbudgetinventoryfortheservermodule,baseduponitsconfigurationof
CPUs,memory,I/Oandlocalstorage.Oncethisnumberisgenerated,theiDRACcommunicates
thepowerbudgetinventorytotheCMC,whichconfirmstheavailabilityofpowerfromthe
systemlevel,baseduponatotalchassispowerinventory,includingpowersupplies,iKVM,I/O
Modules,fansandservermodules.SincetheCMCcontrolswheneverymodularsystemelement
powerson,itcannowsetpowerpoliciesonasystemlevel.
Page33
IncoordinationwiththeCMC,iDRAChardwareconstantlymonitorsactualpowerconsumption
ateachservermodule.Thispowermeasurementisusedlocallybytheservermoduletoinsure
thatitsinstantaneouspowerconsumptionneverexceedsthebudgetedamount.Whilethe
systemadministratormaynevernoticethesefeaturesinaction,whattheyenableisamore
aggressiveutilizationofthesharedsystempowerresources.Nolongeristhesystemflying
blindinregardstopowerconsumption,andthereisnodangerofexceedingpowercapacity
availability,whichcouldresultinaspontaneousactivationofpowersupplyovercurrent
protectionwithoutthesefeatures.
Thesystemadministratorcanalsosetprioritiesforeachservermodule.Thepriorityworksin
conjunctionwiththeCMCpowerbudgetingandiDRACpowermonitoringtoinsurethatthe
lowestprioritybladesarethefirsttoenteranypoweroptimizationmode,shouldconditions
warranttheactivationofthisfeature.
ThePowerEdgeM1000eintroducescompliancetoPMBusSpecification1.1,usingthisnewpower
managementstandardforstatus,measurementandcontrol.PowerEdgeM1000epowersupplies
continuouslymonitorACinputcurrent,voltageandpower,enablingexposureofdatatoDell
OpenManageITAssistantortootherenterpriselevelmanagementtools.Realtimepower
consumptionisnowviewablepersystem.
Cooling
ThecoolingstrategyforthePowerEdgeM1000esupportsalowimpedance,highefficiency
designphilosophy.Drivinglowerairflowimpedanceallowsthesystemtodrawairthroughthe
systemataloweroperatingpressurethancompetitivesystems.Insomecases,thiscanbeupto
40%lessbackpressurethansimilarlyconfiguredcompetitiveproducts.Thelowerbackpressure
reducesthesystemfanpowerconsumedtomeettheairflowrequirementsofthesystem.
Thelowimpedancedesigniscoupledwithahighefficiencyairmovingdevicedesigned
explicitlyforthePowerEdgeM1000e.Theefficiencyofanairmovingdeviceisdefinedasthe
workoutputofthefanascomparedtotheelectricalpowerrequiredtorunthefan.Thesystem
fanoperatesatefficienciesupto40%greaterthantypicalaxialfansonthemarket,which
correlatesdirectlyintosavingsinthecustomersrequiredpowertocool.
Figure24PowerEdgeM1000eFan
Page34
Thehighefficiencydesignphilosophyalsoextendsintothelayoutofthesubsystemswithinthe
PowerEdgeM1000e.TheServerModules,I/OModules,andPowerSuppliesareincorporated
intothesystemwithindependentairflowpaths.Thisisolatesthesecomponentsfrompreheated
air,reducingtherequiredairflowconsumptionsofeachmodule.
Figure25ServerModuleCoolingAirProfile
Figure26I/OModuleCoolingAirProfile
Page35
TheServerModulesarecooledwithtraditionalfronttobackcooling.AsshowninFigure25,the
frontofthesystemisdominatedbyinletareafortheindividualservermodules.Theairpasses
throughtheservermodules,throughventingholesinthemidplane,andisthendrawnintothe
fanswhichexhausttheairfromthechassis.Thereareplenumsbothupstreamofthemidplane,
betweenthemidplaneandtheblades,anddownstreamofthemidplane,betweenthemidplane
andthefans,tomoreevenlydistributethecoolingpotentialfromthethreecolumnsoffans
acrosstheservermodules.
TheI/OModulesuseabypassducttodrawambientairfromthefrontofthesystemtotheI/O
Moduleinlet,asseeninFigure26.Thisductislocatedabovetheservermodules.Thiscoolairis
thendrawndownthroughtheI/OModulesinatoptobottomflowpathandintotheplenum
betweenthemidplaneandfans,fromwhereitisexhaustedfromthesystem.
ThePowerSupplies,locatedintherearofthesystem,usebasicfronttobackcooling,butdraw
theirinletairfromaductlocatedbeneaththeservermodules,asseeninFigure27.Thisinsures
thatthepowersuppliesreceiveambienttemperatureair.
Figure27PowerSupplyCoolingAirProfile
Thishardwaredesigniscoupledwithathermalcoolingalgorithmthatincorporatesthe
following:
ServerModulelevelthermalmonitoringbytheiDRAC
I/OModulethermalhealthmonitors
FancontrolandmonitoringbytheCMC
TheiDRAConeachservermodulescalculatestheamountofairflowrequiredonanindividual
servermodulelevelandsendsarequesttotheCMC.Thisrequestisbasedontemperature
conditionsontheservermodule,aswellaspassiverequirementsduetohardwareconfiguration.
Concurrently,eachIOMcansendarequesttotheCMCtoincreaseordecreasecoolingtotheI/O
subsystem.TheCMCinterpretstheserequests,andcancontrolthefansasrequiredtomaintain
ServerandI/OModuleairflowatoptimallevels.
Page36
FansareN+1redundant,meaningthatanysinglefancanfailwithoutimpactingsystemuptime
orreliability.Intheeventofafanfailure,systembehaviorisdependentontheresultant
temperaturesofthesystem,asmonitoredbytheServerModuleiDRACandI/OModules.The
CMCcontinuestointerprettheairflowneedsofeachserverandI/Omoduletocontrolthefan
speedsappropriately.Thesystemdoesnotrampthefanstofullspeedintheeventofafan
failureunlessdeemednecessarybyonboardmonitoring.
ThePowerEdgeM1000eisengineeredforgoodsoundqualityinaccordancewiththeDell
Enterpriseacousticalspecification.Comparedtopreviousgenerationsofproductsthefanshave
morelevelsofcontrol,allowingmuchfinertuningofthefanbehavior.Firmwareisoptimizedto
choosethelowestfanspeedsandthereforethelowestacousticaloutputforanyconditionand
configuration.
System Management
DellsPowerEdgeM1000emodularserverdeliversmajorenhancementsinmanagementfeatures.
Thesefeaturesarebasedondirectcustomerfeedbackandassistcustomersdeployingmodular
serversintheirenvironment.Eachsubsystemhasbeenreviewedandadjustedtooptimize
efficiencies,minimizingtheimpactstoexistingmanagementtoolsandprocesses,andproviding
futuregrowthopportunitiestostandardsbasedmanagement.
Blade-16
CMC 1
L2+Switch
CMC MPU
Chassis
Mngmt
CMC 2
L2+Switch
CMC MPU
Chassis
Mngmt
Blade-1
iDRAC iDRAC
IOM-1
Mgmt
IOM-6
Mgmt
Local KVM
Management Network 1
Management Network 2
Analog
video, USB
Keyboard
and mouse
I2C
Power
Supply 1
Power
Supply 2
Power
Supply 6
Chassis
buttons, FRU,
LEDs
PWM, TACH
Intranet Intranet
OSCAR
FAN 1 FAN 2 FAN 9
I2C
LCD Panel
Figure28SystemManagementArchitectureSimplifiedBlockDiagram
Page37
Figure28illustratesthemanagementconnectionswhichtransferhealthandcontroltraffic
throughoutchassis.Thesystemmanagementfabricisarchitectedfor100BaseTEthernetover
differentialpairsroutedtoeachmodule.Therearetwo100BaseTinterfacesbetweenCMCs,one
switchedandoneunswitched.AllsystemmanagementEthernetisroutedfor100Mbps
signaling.EverymodulehasamanagementnetworklinktoeachCMC,withredundancy
providedatthemodulelevel.Failureofanyindividuallinkcausesfailovertotheredundant
CMC.
Chassis Management Controller (CMC)
ChassismanagementisaccessedandmanagedbytheChassisManagementControllerorCMC.
ThePowerEdgeM1000epossessesatleastoneCMCandsupportsanoptionalredundant
module,eachoccupyingaslotaccessiblethroughtherearofthechassis.Redundancyis
providedinanActiveStandbypairingofthemodulesandfailoveroccurswhentheactive
modulehasfailedordegraded.TheCMCinterfacesthroughdualstacking10/100/1000Ethernet
portsandoneserialport.TheCMCserialportinterfaceprovidescommonmanagementofupto
sixI/Omodulesthroughasingleconnection.
Figure29PowerEdgeM1000eChassisManagementController
Page38
Figure30ChassisManagementControllerFrontPanel
TheCMCprovidessecureremotemanagementaccesstothechassisandinstalledmodules.It
managesorfacilitatesthemanagementofthefollowing:
Status, Inventory and Alerting for server modules, chassis infrastructure and I/O
Modules
CentralizedconfigurationforiDRAC,I/OModulesandCMC
SSL/SSH
Powersequencingofmodulesinconjunctionwiththedefinedchassispowerstates
Powerbudgetmanagementandallocation
Configuration of the embedded management switch, which facilitates external access to
manageablemodules.
Remoteusermanagement
Userinterfaceentrypoint(web,telnet,SSH,serial)
Monitoring and alerting for chassis environmental conditions or component health
thresholds.Thisincludesbutisnotlimitedtothefollowing:
o Realtimepowerconsumption
o Powersupplies
o Fans
o Powerallocation
o Temperature
o CMCredundancy
o I/Ofabricconsistency
SMASHCLPfeaturesprovidingchassisservermodulepowercontrolfunctions
WSMAN
CIMXML
SNMP
RACADM
Ethernettrafficmanagement(firewall)
Ethernetswitchmanagement
PrivateVLAN
PrivateDHCPservice
PrivateTFTPservice
VirtualMedia
VirtualKVMinterfacebetweentheIPnetworkandtheservermoduleiDRACvKVM
FirmwaremanagementofCMC,IOM,iDRAC,iKVM
Integrated Dell Remote Access Controller (iDRAC)
Theservermodulebasemanagementsolutionincludesadditionalfeaturesforefficient
deploymentandmanagementofserversinamodularserverformfactor.Thebasecircuit,which
integratestheBMCfunctionwithhardwaresupportforVirtualKVM(vKVM)andVirtualMedia
(vMedia),iscalledtheintegratedDellRemoteAccessControlleroriDRAC.iDRAChastwo
Ethernetconnections,oneforeachCMC,providingsystemmanagementinterfaceredundancy.
Page39
DellmodularserversnowincludevKVMasastandardfeature,routingtheoperatorskeyboard
output,mouseoutputandvideobetweenthetargetservermoduleandaconsolelocatedonthe
systemmanagementIPnetwork.WithuptotwosimultaneousvKVMsessionsperblade,remote
managementnowsatisfiesvirtuallyanyusagemodel.vMediaisalsonowstandard,providing
emulationofUSBDVDR/W,USBCDR/W,USBFlashDrive,USBISOimageandUSBFloppy
overanIPinterface.ConnectiontovKVMandvMediaisthroughtheCMC,withencryption
availableonaperstreambasis.
HighlightsoftheiDRACsolutionincludethefollowing:
Dedicatedmanagementinterfaceforhighperformancemanagementfunctions
VirtualMedia
VirtualKVM
IPMI2.0OutOfBandmanagement
SerialoverLANredirection
SMASHCLP
Bladestatusandinventory
Activepowermanagement
IntegrationwithActiveDirectory
Security,LocalandActiveDirectory
Integrated Keyboard and Mouse Controller
ThemodularenclosuresupportsoneoptionalIntegratedKVM(iKVM)module.Thismodule
occupiesasingleslotaccessiblethroughtherearofthechassis.TheiKVMredirectslocalserver
modulevideo,keyboard,andmouseelectricalinterfacestoeithertheiKVMlocalportsorthe
PowerEdgeM1000efrontpanelports.TheiKVMallowsasingledonglelessVGAmonitor,USB
keyboard,andUSBmousetobeattachedandthenmovedbetweenservermodulesasneeded
throughanonscreendisplayselectionmenu.TheiKVMalsohasanAnalogConsoleInterface
(ACI)compatibleRJ45portthatallowstheiKVMtotietheinterfacetoaKVMappliance
upstreamoftheiKVMviaCAT5cabling.DesignedwithAvocenttechnology,theACIport
reducescostandcomplexitybygivingaccessforsixteenserversusingonlyoneportonan
externalKVMSwitch.
Figure31PowerEdgeM1000eiKVMModule
Page40
TheiKVMcontainsanewseventeenthbladefeature,connectingtheCMCCommandLine
InterfaceviatheKVMswitchandallowingtextbaseddeploymentwizardsonVGAmonitors.
iKVMfirmwareisupdatedthroughtheCMC.
Figure32iKVMFrontPanel
Control Panel and LCD
Thecontrolpanelcontainsthelocaluserinterface.Functionsincludechassisleveldiagnostic
LEDs,LCDdisplayandpowerbutton.Thisdeviceisnothotpluggableandisalwayspowered,
eveninchassisstandbymode.
Figure33PowerEdgeM1000eLCDPanelRecessedPosition
Page41
Page42
Figure34PowerEdgeM1000eLCDPanelDuringUsage
TheChassisLCDshowschassisinformation,chassisstatusandfaults,majorservermodulefaults
andprovidesaninterfaceforservermoduleandCMCnetworkconfiguration.Includedisan
errormessagewhennoCMCispresentinthechassis.ServiceTag,AssetTagandIPAddressare
noweasilyvisibleoneveryPowerEdgeM1000emodularsystemthroughtheLCDdisplay.The
LCDpanelcanberetractedintothechassisbody,orextendedandangledoncedeployedforfull
visibilitynomatterwherethePowerEdgeM1000eismountedintherack.
*Graphicimagesmaydifferfromfinalproduct
Figure35LCDGraphicExamples
Conclusion
ThePowerEdgeM1000eModularServerEnclosureisabreakthroughinenterpriseclassserver
design.DelloptimizedthenewPowerEdgeM1000eModularServerEnclosureandM600/605
ServerModulesto:
MaximizeflexibilitymodularI/O,power,cooling,andmanagementarchitecture.
Maximizelongevityoptimizedpowerandcoolingdesignsupportscurrentandfuture
generationsofservermodulesandI/O.I/Obandwidthtosupportnotonlytodaysgeneration
of10GbEthernet,20GbpsInfiniBandand4GbpsFibreChannel,butupto40GbpsQDR
InfiniBand,10GbpsSerialEthernet,and8GbpsFibreChannel.
LowerTCOlowercostthanmonolithicserverswithequivalentfeatures.Bestinclasspower
andcoolingefficiency.
Acknowledgments
TheauthorwouldliketothankRichardCrisp,MikeRobertsandK.C.Coxefortheir
contributionstothiswhitepaper.