Dynamic

Interconnection
Networks
Dr. Rajender Nath

Dynamic Interconnection Networks

Shared Path Interconnection Networks

Single Shared Bus

Multiple Shared Bus

Switching Interconnection Networks

Crossbar

Multistage Interconnection Networks

(MINs)

Single Shared Buses

Dr. Rajender Nath

Single shared Buses

Some of the simplest and earliest parallel machines used
buses.
All processors access a common bus for exchanging data.
The distance between any two nodes is O(1) in a bus. The
bus also provides a convenient broadcast media.
However, the bandwidth of the shared bus is a major
bottleneck.
Simple and cost-effective for small-scale multiprocessors
Typical bus based machines are limited to dozens of
nodes. Sun Enterprise servers and Intel Pentium based
shared-bus multiprocessors are examples of such
architectures.
Not scalable (limited bandwidth;
electrical
complications)
4

Dr. Rajender Nath

Three Generations of Buses

Locked Buses

Pended Buses

Split Transaction Buses

Locked Buses
Difference in three generation of
buses lies in the way memories are
accessed.
Write Phase:
Bus Cycle: Address and data are
transferred via the bus to the MC
Memory Cycle: The memory write
operation is completed by MC

Memory Write on Locked Bus

Dr. Rajender Nath

Memory Write on Pended and Split Transaction Buses

Dr. Rajender Nath

Phases of Memory Read Operations

1. Bus Cycle: The adddress is

transferred via the bus to the MC
2. Memory Cycle: The memory read
operation is executed by the MC
3. Bus Cycle: The data is transferred
via the bus to the requesting
processor

Dr. Rajender Nath

Memory Read on Locked Bus

10

Dr. Rajender Nath

Memory Read on Pended Bus

11

Dr. Rajender Nath

Memory Write on Split Transaction Bus

12

Dr. Rajender Nath

Bus Arbiter

The process of selecting the next bus

master is called bus arbitration
The duration of a master's control of the
bus is called bus tenure
Types

Centarlized

With independent request &

grant line

Daisy Chained

Decentarlized
13

Dr. Rajender Nath

Centralized Arbiter with independent Request & Grant

14

Dr. Rajender Nath

Pros and Cons

Pros
It can realize any bus
allocation policy FCFS,
LRU, RR, Fixed priority etc.

Faster

Cons

Large number of bus

exchange lines==>cost

15

Dr. Rajender Nath

Daisy Chained Bus Arbiter

16

Dr. Rajender Nath

Pros and Cons

Pros

Easy to implement
Cost Effective
Only one line, extension is easy

Cons

17

Slow ==> long chain

Lack of fairness in allocation
scheme

Dr. Rajender Nath

Decentralized Rotating Arbiter with independent Requests & Grants

18

Dr. Rajender Nath

Multiple Shared Bus

Types

1-Dimensional

2- or 3-dimensional

Cluster Bus System

Hierarchical Bus System

19

Dr. Rajender Nath

1-Dimensional Multiple Bus Multiprocessor

20

Dr. Rajender Nath

b-of-m arbiter for Multibus system

21

Dr. Rajender Nath

Hierarchical Bus System

22

Dr. Rajender Nath

Crossbars (N x N)
Each

port has link to every other port

Crossbar

+ Low latency and high throughput

P
P

- Cost grows as O(N^2)

so not very scalable.

- Difficult to arbitrate
Examples of machines that employ crossbars include the
Sun Ultra HPC 10000 and the Fujitsu VPP500.

23

Dr. Rajender Nath

Summary of Networks?

Buses have excellent cost scalability, but

poor performance scalability.
Crossbars have excellent performance
scalability but poor cost scalability.
Multistage
interconnects
strike
compromise between these extremes.
24

Dr. Rajender Nath

Multistage Interconnection Networks

25

Dr. Rajender Nath

Types of MINs
Depends upon
Types of switch used e.g. 2X2, 4x4, 8x8
etc.
Interstage connection used e.g. Perfect
shuffle, inverse suffle, crossbar,
butterfly, baseline, cube connection
etc.

26

Dr. Rajender Nath

NxN Omega Network

Type of switch: 2x2

ISC: perfect suffle
No. of stages: log2N
No. of switch per stage: N/2

27

Dr. Rajender Nath

Omega Network: 2x2 switch

Switch states

28

Dr. Rajender Nath

Omega Network: Perfect Shuffle

shuffle(i)

29

Dr. Rajender Nath

8x8 Omega Network

30

Dr. Rajender Nath

8x8 Omega Network

All stages are same, so can use

recirculating network.

Single path from source to destination.

Can add extra stages and pathways to
minimize collisions and increase fault
tolerance.
Can support combining. Used in IBM
RP3.
31

Dr. Rajender Nath

Omega Network: Routing

Let s be the binary representation of the source and d
be that of the destination processor.
The data traverses the link to the first switching node. If
the most significant bits of s and d are the same, then
the data is routed in pass-through mode by the switch
else, it switches to crossover.
This process is repeated for each of the log p switching
stages.
Note that this is blocking switch.

32

Dr. Rajender Nath

Network Topology
Myrinet-2000 Clos Network for 128 Hosts

Backplane of the M3E128 Switch

M3-SW16-8F fiber line
card (8 ports)

33

Dr. Rajender Nath

Thank You

34

Dr. Rajender Nath