PRESENTED BY:-

GROUP 7
PRAVESH TIWARI
RAMAN KUMAR GARG
TANMAY ANAND
 About CareGroup
• CareGroup was formed on October 1,
1996
• The Beth Israel Hospital, Deaconess
Hospital and Mount Auburn Hospital
came together on that day to form
CareGroup
• The Group produced US$ 1.6 billion in
revenue
• It was second largest group of
hospitals in eastern Massachusetts
• It had a highly appreciated IT
Structure
 IT Architecture at
CareGroup
• Most Advanced web
infrastructure, email,
voice/wireless system, data
centre in health care to
CareGroup’s belief
• The datacentre had 3 backup
generators with no power
outage in three years
• IBM mainframe replaced with
unix/linux servers
Network at
CareGroup
• Cisco studied CareGroup’s
network and concluded that
it didn’t require immediate
attention
• No accountability for
network changes
 Network Collapse at CareGroup
• Napster, software designed to locate and copy information across the
network was the root cause of the network collapse
• The software was left running in basic mode without proper testing,
it began to explore surrounding network, seeking and copying data
across computers
• The network was clogged due to huge data transfers through a
centrally located switch
• Network components started calculating alternative paths and
became confused on choosing primary and backup path
• Network went “out of spec” as algorithm for computing data paths
could not operate correctly
 Action to Counter Collapse
• Halamka CIO of CareGroup called in Cisco for urgent assistance. Cisco
initiated CAP process consequentially to counter the network collapse

• Hospital switched to backup procedures and access to network was

blocked to avoid intermittent use of computers

• Establishment of command center as a central point of communication

• Establishment of runners available to retrieve specimens, tests,

equipment, supplies at any time

• Manual procedures for orders and pharmacy, paper based

documentation of all activity came into effect
 Solving the problem
• CareGroup IT and the Cisco SWAT team spent fixing
looping issues in overall network
• Cisco team removed an old model router, which had
problem with upgradation in its firmware
• Post restoration of the computer network, the paper
based information was transferred to systems in 48 hrs.
This was done after monitoring the proper working of
computer systems for 24 hrs
 Lessons Learned
• Do not hesitate to bring the experts to make sure your network is
configured properly – Two Cisco experts will remain on site at Beth Israel
Deconess

• Do not let any one individual in your IT group become the sole point of
failure - As CareGroup relied heavily on only one resource, they should have
used a backup

• Keep your working knowledge current – Don’t allow your knowledge to

stagnate

• Beware of users armed with just enough knowledge to be dangerous – IT

should be vigilant with the users experimenting with their limited
knowledge
 Lessons Learned
• Institute rigorous network change control – Proper timings of changes in
network to be made by the engineering team with proper testing

• There are limits to customer-centric responsiveness – Balance the changes

to be made strictly according to customer’s needs

• Have backup procedures in which you can have confidence – Backup

should be robust and should last long
• Component redundancy is not rough; you need alternative access methods
– Like telephones and modern capabilities of PCs

• Life-cycle-manage your network components – Replace components in a

period
Thank You 