Tech Info

GRID COMPUTING
Sandeep Kumar Poonia

Head Of Dept. CS/IT
B.E., M.Tech., UGC-NET
LM-IAENG, LM-IACSIT,LM-CSTA, LM-AIRCC, LM-SCIEI, AM-UACEE
WHY GRID COMPUTING?
 40% Mainframes are idle

 90% Unix servers are idle

 95% PC servers are idle
 0-15% Mainframes are idle in peak-hour

 70% PC servers are idle in peak-hour
Source: “Grid Computing” Dr Daron G Green

OUTLINE
 Introduction to Grid Computing

 Methods of Grid computing
 Grid Middleware
 Grid Architecture
ELECTRICAL POWER GRID
ANALOGY
Electrical power The Grid
grid  users (or client applications) gain
 users (or electrical access to computing resources

(processors, storage, data,
appliances) get access to
applications, and so on) as needed
electricity through wall with little or no knowledge of where
sockets with no care or those resources are located or what
consideration for where or the underlying technologies,
how the electricity is hardware, operating system, and so
actually generated. on are
 “The power grid” links  "the Grid" links together computing
resources (PCs, workstations, servers,
together power plants of
storage elements) and provides the
many different kinds mechanism needed to access them.
WHY NEED GRID COMPUTING?
 Core networking technology now accelerates at a much

faster rate than advances in microprocessor speeds
 Exploiting under utilized resources
 Parallel CPU capacity
 Virtual resources and virtual organizations for

collaboration
 Access to additional resources

WHO NEEDS GRID COMPUTING?
 Not just computer scientists…

 scientists “hit the wall” when faced with situations:
 The amount of data they need is huge and the data is stored in
different institutions.
 The amount of similar calculations the scientist has to do is
huge.
 Other areas:
 Government
 Business
 Education
 Industrial design
 ……

LIVING IN AN EXPONENTIAL WORLD
(1) COMPUTING & SENSORS
Moore‘s Law: transistor count doubles each 18 months

Magnetohydro-
dynamics
star formation
LIVING IN AN EXPONENTIAL WORLD:
(2) STORAGE
 Storage density doubles every 12 months
 Dramatic growth in online data (1 petabyte =

1000 terabyte = 1,000,000 gigabyte)
 2000 ~0.5 petabyte
 2005 ~10 petabytes
 2010 ~100 petabytes
 2015 ~1000 petabytes?
 Transforming entire disciplines in physical and,
increasingly, biological sciences; humanities
next?
DATA INTENSIVE PHYSICAL SCIENCES
 High energy & nuclear physics

 Including new experiments at CERN
 Gravity wave searches

 LIGO, GEO, VIRGO
 Time-dependent 3-D systems (simulation, data)
 Earth Observation, climate modeling
 Geophysics, earthquake modeling
 Fluids, aerodynamic design
 Pollutant dispersal scenarios
 Astronomy: Digital sky surveys

ONGOING ASTRONOMICAL MEGA-SURVEYS
 Large number of new surveys MACHO
 Multi-TB in size, 100M objects or larger 2MASS
 In databases SDSS

 Individual archives planned and under way DPOSS
GSC-II
 Multi-wavelength view of the sky COBE
 > 13 wavelength coverage within 5 years MAP
NVSS
 Impressive early discoveries FIRST
 Finding exotic objects by unusual colors GALEX
 L,T dwarfs, high redshift quasars ROSAT
 Finding objects by time variability OGLE
 Gravitational micro-lensing ...
COMING FLOODS OF ASTRONOMY DATA
 The planned Large Synoptic Survey Telescope
will produce over 10 petabytes per year by 2008!

 All-sky survey every few days, so will have fine-grain
time series for the first time
DATA INTENSIVE BIOLOGY AND MEDICINE
 Medical data
 X-Ray, mammography data, etc. (many petabytes)
 Digitizing patient records
 X-ray crystallography

 Molecular genomics and related disciplines
 Human Genome, other genome databases
 Proteomics (protein structure, activities, …)
 Protein interactions, drug delivery
 Virtual Population Laboratory (proposed)
 Simulate likely spread of disease outbreaks
 Brain scans (3-D, time dependent)
A BRAIN
IS A LOT
OF DATA!
(MARK ELLISMAN, UCSD)

And comparisons must be
made among many
We need to get to one micron to know location of every cell. We’re just now
starting to get to 10 microns – Grids will help get us there and further
Fastest virtual supercomputers
As of April 2013, Folding@home – 11.4 x86-equivalent

(5.8 "native") PFLOPS.

As of March 2013, BOINC – processing on average 9.2
PFLOPS.
As of April 2010, MilkyWay@Home computes at over
1.6 PFLOPS, with a large amount of this work coming from
GPUs.
As of April 2010, SETI@Home computes data averages
more than 730 TFLOPS.
As of April 2010, Einstein@Home is crunching more than
210 TFLOPS.
As of June 2011, GIMPS is sustaining 61 TFLOPS.
HOW GRID COMPUTING WORKS

Idol time
Idol CPU
Super computer,
Big mainframe…
Idol CPU
Idol time
Source: “The Evolving Computing Model: Grid Computing” Michael Teyssedre

Idol time
Virtual machine Idol CPU
Virtual CPU…
Idol CPU
Idol time

0% idol
Grid 0% idol
Computing
0% idol
0% idol
GRID ARCHITECTURE

Autonomous, globally distributed computers/clusters
WHAT IS A GRID?
 Many definitions exist in the literature
 Early defs: Foster and Kesselman, 1998
―A computational grid is a hardware and software

infrastructure that provides dependable, consistent,
pervasive, and inexpensive access to high-end
computational facilities‖
 Kleinrock 1969:
―We will probably see the spread of ‗computer utilities‘,
which, like present electric and telephone utilities, will
service individual homes and offices across the country.‖
3-POINT CHECKLIST (FOSTER 2002)
1. Coordinates resources not subject to
centralized control

2. Uses standard, open, general purpose protocols
and interfaces
3. Deliver nontrivial qualities of service
• e.g., response time, throughput, availability,
security
DEFINITION
Grid computing is…
 A distributed computing system

 Where a group of computers are connected
 To create and work as one large virtual

computing power, storage, database, application,
and service
DEFINITION
Grid computing…
 Allows a group of computers to share the system

securely and
 Optimizes their collective resources to meet
required workloads
 By using open standards
GRID COMPUTING
Grid computing is a form of distributed computing
whereby a "super and virtual computer" is composed of a
cluster of networked, loosely coupled computers, acting in
concert to perform very large tasks.

Grid computing (Foster and Kesselman, 1999) is a
growing technology that facilitates the executions of
large-scale resource intensive applications on
geographically distributed computing resources.
Facilitates flexible, secure, coordinated large scale

resource sharing among dynamic collections of
individuals, institutions, and resource
Enable communities (―virtual organizations‖) to share

geographically distributed resources as they pursue
common goals
Ian Foster and Carl Kesselman
A COMPARISON
SERIAL PARALLEL GRID
 Fetch/Store  Fetch/Store  Fetch/Store
 Compute  Compute/  Discovery of Resources
communicate
 Interaction with remote

 Cooperative game application
 Authentication /
Authorization
 Security
 Compute/Communicate
 Etc
DISTRIBUTED COMPUTING VS. GRID
 Grid is an evolution of distributed computing

Dynamic


 Geographically independent
 Built around standards
 Internet backbone
 Distributed computing is an ―older term‖

 Typically built around proprietary
software and network
 Tightly couples systems/organization
WEB VS.
GRID
 Web
 Uniform naming access to documents

http://
http://
 Grid - Uniform, high performance access to computational

resources
Software
Catalogs
Sensor nets
Colleges/R&D
Labs
IS THE WORLD WIDE WEB A
GRID ?
 Seamless naming? Yes

 Uniform security and Authentication? No

 Information Service? Yes or No
 Co-Scheduling? No
 Accounting & Authorization ? No
 User Services? No
 Event Services? No
 Is the Browser a Global Shell ? No
WHAT DOES THE WORLD WIDE WEB BRING TO
THE GRID ?
 Uniform Naming
 A seamless, scalable information service

 A powerful new meta-data language: XML
 XML will be standard language for
describing information in the grid
 SOAP – simple object access protocol
 Uses XML for encoding. HTML for protocol
 SOAP may become a standard RPC
mechanism for Grid services
 Uses XML for encoding. HTML for protocol
 Portal Ideas
THE ULTIMATE GOAL

 In future I will not know or care
where my application will be
executed as I will acquire and pay
to use these resources as I need
them
WHY GRIDS?
 Large-scale science and engineering are done
through the interaction of people, heterogeneous
computing resources, information systems, and

instruments, all of which are geographically and
organizationally dispersed.
 The overall motivation for ―Grids‖ is to facilitate

the routine interactions of these resources in order
to support large-scale science and Engineering.
AN EXAMPLE VIRTUAL ORGANIZATION:
CERN‘S LARGE HADRON COLLIDER
1800 Physicists, 150 Institutes, 32 Countries

100 PB of data by 2010; 50,000 CPUs?
GRID COMMUNITIES & APPLICATIONS:
DATA GRIDS FOR HIGH ENERGY PHYSICS
~PBytes/sec
1 TIPS is approximately 25,000
Online System ~100 MBytes/sec SpecInt95 equivalents
Offline Processor Farm

There is a “bunch crossing” every 25 nsecs.
~20 TIPS
There are 100 “triggers” per second
~100 MBytes/sec

Each triggered event is ~1 MByte in size
~622 Mbits/sec
Tier CERN Computer Centre
or Air Freight (deprecated) 0
Tier
France Regional Germany Regional Italy Regional FermiLab ~4 TIPS
1 Centre Centre Centre
~622 Mbits/sec
Tier Caltech
~1 TIPS
Tier2 Centre
Tier2 Centre
Tier2 Centre
Tier2 Centre
~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS
~622 Mbits/sec 2
Institute
Institute Institute Institute
~0.25TIPS Physicists work on analysis “channels”.
Each institute will have ~10 physicists working on one or more
Physics data cache
~1 MBytes/sec channels; data for these channels should be cached by the
institute server
Tier
Physicist workstations 4
www.griphyn.org www.ppdg.net www.eu-datagrid.org

INTELLIGENT INFRASTRUCTURE:
DISTRIBUTED SERVERS AND SERVICES

THE GRID:
A BRIEF HISTORY
 Early 90s
 Gigabit testbeds, metacomputing
 Mid to late 90s

 Early experiments (e.g., I-WAY), academic software projects
(e.g., Globus, Legion), application experiments
 2002
 Dozens of application communities & projects
 Major infrastructure deployments
 Significant technology base (esp. Globus ToolkitTM)
 Growing industrial interest
 Global Grid Forum: ~500 people, 20+ countries
HOW IT EVOLVES

Utility computing
Service grid
Data grid Virtualization

Service-oriented
Open standard
Processing grid
EARLY ADOPTERS
 Academic
 Big science

 Life science
 Nuclear engineering
 Simulation…
MARKET POTENTIAL
 Financial services:
risk management and compliance

 Automotive:
acceleration of product development
 Petroleum:
discovery of oils
Source: “Perspectives on grid: Grid computing - next-generation distributed computing" Matt Haynos, 01/27/04
Criteria for a Grid:
Coordinates resources that are not subject to
centralized control.
Uses standard, open, general-purpose protocols

and interfaces.
Delivers nontrivial qualities of service.
e.g., response time, throughput, availability, security
Benefits
Exploit Underutilized resources
Resource load Balancing
Virtualize resources across an enterprise
Data Grids, Compute Grids
Enable collaboration for virtual organizations
WHY DO WE NEED GRIDS?
 Many large-scale problems cannot be solved by a
single computer

 Globally distributed data and resources
GRID APPLICATIONS
Data and computationally intensive applications:
This technology has been applied to computationally-
intensive scientific, mathematical, and academic problems
like drug discovery, economic forecasting, seismic analysis
back office data processing in support of e-commerce

 A chemist may utilize hundreds of processors to screen
thousands of compounds per hour.
 Teams of engineers worldwide pool resources to analyze
terabytes of structural data.
 Meteorologists seek to visualize and analyze petabytes of
climate data with enormous computational demands.
Resource sharing
 Computers, storage, sensors, networks, …
 Sharing always conditional: issues of trust, policy,
negotiation, payment, …
Coordinated problem solving
 distributed data analysis, computation, collaboration, …
GRID TOPOLOGIES
• Intragrid
– Local grid within an organisation

– Trust based on personal contracts
• Extragrid
– Resources of a consortium of organisations
connected through a (Virtual) Private Network
– Trust based on Business to Business contracts
• Intergrid
– Global sharing of resources through the
internet
– Trust based on certification
COMPUTATIONAL GRID
―A computational grid is a hardware and software infrastructure
that provides dependable, consistent, pervasive, and inexpensive

access to high-end computational capabilities.‖
‖The Grid: Blueprint for a New Computing Infrastructure‖,

Kesselman & Foster
Example : Science Grid (US Department of Energy)

DATA GRID
 A data grid is a grid computing system that deals with
data — the controlled sharing and management of
large amounts of distributed data.

 Data Grid is the storage component of a grid environment.
Scientific and engineering applications require access to
large amounts of data, and often this data is widely
distributed. A data grid provides seamless access to the
local or remote data required to complete compute
intensive calculations.
Example :
Biomedical informatics Research Network (BIRN),
the Southern California earthquake Center (SCEC).
BACKGROUND: RELATED
TECHNOLOGIES
 Cluster computing
 Peer-to-peer computing

 Internet computing
CLUSTER COMPUTING
 Idea: put some PCs together and get them to
communicate

 Cheaper to build than a mainframe
supercomputer
 Different sizes of clusters
 Scalable – can grow a cluster by adding more PCs

CLUSTER ARCHITECTURE

PEER-TO-PEER COMPUTING
 Connect to other computers
 Can access files from any computer on the

network
 Allows data sharing without going through
central server
 Decentralized approach also useful for Grid
PEER TO PEER ARCHITECTURE

METHODS OF GRID COMPUTING
 Distributed Supercomputing
 High-Throughput Computing

 On-Demand Computing
 Data-Intensive Computing
 Collaborative Computing
 Logistical Networking
DISTRIBUTED SUPERCOMPUTING
 Combining multiple high-capacity resources on

a computational grid into a single, virtual
distributed supercomputer.

 Tackle problems that cannot be solved on a
single system.
 Examples: climate modeling, computational
chemistry
 Challenges include:
 Scheduling scarce and expensive resources
 Scalability of protocols and algorithms
 Maintaining high levels of performance across
heterogeneous systems
HIGH-THROUGHPUT COMPUTING
 Uses the grid to schedule large numbers of
loosely coupled or independent tasks, with the
goal of putting unused processor cycles to

work.
 Schedule large numbers of independent tasks
 Goal: exploit unused CPU cycles (e.g., from

idle workstations)
 Unlike distributed computing, tasks loosely
coupled
 Examples: parameter studies, cryptographic
problems
On-Demand Computing
 Uses grid capabilities to meet short-term
requirements for resources that are not
locally accessible.

 Models real-time computing demands.
 Use Grid capabilities to meet short-term
requirements for resources that cannot
conveniently be located locally
 Unlike distributed computing, driven by cost-
performance concerns rather than absolute
performance
 Dispatch expensive or specialized
computations to remote servers
COLLABORATIVE COMPUTING
 Concerned primarily with enabling and

enhancing human-to-human interactions.

 Enable shared use of data archives and
simulations
 Applications are often structured in terms of a
virtual shared space.
 Examples:
 Collaborative exploration of large geophysical data sets
 Challenges:
 Real-time demands of interactive applications
 Rich variety of interactions
Data-Intensive Computing
 The focus is on synthesizing new information
from data that is maintained in geographically
distributed repositories, digital libraries, and

databases.
 Particularly useful for distributed data mining.

 Examples:
•High energy physics generate terabytes of distributed data, need complex
queries to detect “interesting” events
•Distributed analysis of Sloan Digital Sky Survey data
LOGISTICAL NETWORKING
 Logistical networks focus on exposing storage
resources inside networks by optimizing the
global scheduling of data transport, and data

storage.
 Contrasts with traditional networking, which
does not explicitly model storage resources in the
network.
 high-level services for Grid applications
 Called "logistical" because of the analogy it bears
with the systems of warehouses, depots, and
distribution channels.
P2P COMPUTING VS GRID
COMPUTING
Differ in Target Communities


 Grid system deals with more complex, more

powerful, more diverse and highly interconnected
set of resources than
P2P.
A TYPICAL VIEW OF GRID
ENVIRONMENT
Grid Information Service Grid Information Service

system collects the details of Details of Grid resources

the available Grid resources
and passes the information 1
to the resource broker.
2
4
Computational jobs
3
Grid application
Processed jobs
Computation result
User
A User sends computation
Resource Broker
A Resource Broker distribute the
or data intensive application
to Global Grids in order to
jobs in an application to the Grid Grid Resources
resources based on user’s QoS Grid Resources (Cluster, PC,
speed up the execution of requirements and details of available Supercomputer, database,
the application. Grid resources for further executions. instruments, etc.) in the Global
Grid execute the user jobs.
GRID MIDDLEWARE
 Grids are typically managed by grid ware -
a special type of middleware that enable sharing and
manage grid components based on user requirements
and resource attributes (e.g., capacity, performance)

 Software that connects other software components or
applications to provide the following functions:
Run applications on suitable available resources
– Brokering, Scheduling
Provide uniform, high-level access to resources
– Semantic interfaces
– Web Services, Service Oriented Architectures
Address inter-domain issues of security, policy, etc.
– Federated Identities
Provide application-level status
monitoring and control
MIDDLEWARES
 Globus –chicago Univ
 Condor – Wisconsin Univ – High throughput

computing
 Legion – Virginia Univ – virtual workspaces-
collaborative computing
 IBP – Internet back pane – Tennesse Univ –
logistical networking
 NetSolve – solving scientific problems in
heterogeneous env – high throughput & data
intensive
TWO KEY GRID COMPUTING GROUPS
The Globus Alliance (www.globus.org)
 Composed of people from:
Argonne National Labs, University of Chicago, University of
Southern California Information Sciences Institute,
University of Edinburgh and others.

 OGSA/I standards initially proposed by the Globus Group
The Global Grid Forum (www.ggf.org)

 Heavy involvement of Academic Groups and Industry
 (e.g. IBM Grid Computing, HP, United Devices, Oracle,
UK e-Science Programme, US DOE, US NSF, Indiana
University, and many others)
 Process
 Meets three times annually
 Solicits involvement from industry, research groups, and
academics
GRID USERS
 Many levels of users
 Grid developers

 Tool developers
 Application developers
 End users
 System administrators
SOME GRID CHALLENGES
 Data movement
 Data replication

 Resource management
 Job submission
SOME OF THE MAJOR GRID PROJECTS
Name URL/Sponsor Focus
EuroGrid, Grid eurogrid.org Create tech for remote access to super

Interoperability European Union comp resources & simulation codes; in
(GRIP) GRIP, integrate with Globus Toolkit™

Fusion Collaboratory fusiongrid.org Create a national computational
DOE Off. Science collaboratory for fusion research
Globus Project™ globus.org Research on Grid technologies;
DARPA, DOE, development and support of Globus
NSF, NASA, Msoft Toolkit™; application and deployment
GridLab gridlab.org Grid technologies and applications
European Union
GridPP gridpp.ac.uk Create & apply an operational grid within the
U.K. eScience U.K. for particle physics research
Grid Research grids-center.org Integration, deployment, support of the NSF
Integration Dev. & NSF Middleware Infrastructure for research &
Support Center education
Grid in India-GARUDA
•GARUDA is India's Grid Computing
initiative connecting 17 cities across the

country.
•The 45 participating institutes in this
nationwide project include all the IITs and
C-DAC centers and other major institutes
in India.
GLOBUS GRID TOOLKIT
 Open source toolkit for building Grid systems and
applications
 Enabling technology for the Grid
 Share computing power, databases, and other tools securely

online
 Facilities for:
 Resource monitoring
 Resource discovery
 Resource management
 Security
 File management
DATA MANAGEMENT IN GLOBUS
TOOLKIT
 Data movement
 GridFTP

 Reliable File Transfer (RFT)
 Data replication
 Replica Location Service (RLS)
 Data Replication Service (DRS)
GRIDFTP
 High performance, secure, reliable data transfer protocol
 Optimized for wide area networks
 Superset of Internet FTP protocol

 Features:
 Multiple data channels for parallel transfers
 Partial file transfers
 Third party transfers
 Reusable data channels
 Command pipelining
MORE GRIDFTP FEATURES
 Auto tuning of parameters
 Striping

 Transfer data in parallel among multiple senders and
receivers instead of just one
 Extended block mode
 Send data in blocks
 Know block size and offset
 Data can arrive out of order
 Allows multiple streams
STRIPING ARCHITECTURE
 Use ―Striped‖ servers

LIMITATIONS OF GRIDFTP
 Not a web service protocol (does not employ
SOAP, WSDL, etc.)

 Requires client to maintain open socket
connection throughout transfer
 Inconvenient for long transfers
 Cannot recover from client failures
GRIDFTP
RELIABLE FILE TRANSFER (RFT)
 Web service with ―job-scheduler‖ functionality for data
movement

 User provides source and destination URLs
 Service writes job description to a database and moves

files
 Service methods for querying transfer status
RFT
REPLICA LOCATION SERVICE (RLS)
 Registry to keep track of where replicas exist on physical
storage system

 Users or services register files in RLS when files created
 Distributed registry
 May consist of multiple servers at different sites
 Increase scale
 Fault tolerance
REPLICA LOCATION SERVICE (RLS)
 Logical file name – unique identifier for contents of
file
 Physical file name – location of copy of file on
storage system

 User can provide logical name and ask for replicas
 Or query to find logical name associated with
physical file location
DATA REPLICATION SERVICE (DRS)
 Pull-based replication capability
 Implemented as a web service
 Higher-level data management service built on top of RFT

and RLS
 Goal: ensure that a specified set of files exists on a storage
site
 First, query RLS to locate desired files
 Next, creates transfer request using RFT
 Finally, new replicas are registered with RLS

CONDOR
 Original goal: high-throughput computing
 Harvest wasted CPU power from other machines

 Can also be used on a dedicated cluster
 Condor-G – Condor interface to Globus resources

CONDOR
 Provides many features of batch systems:
 job queueing
 scheduling policy
priority scheme


 resource monitoring
 resource management
 Users submit their serial or parallel jobs
 Condor places them into a queue
 Scheduling and monitoring
 Informs the user upon completion

NIMROD-G
 Tool to manage execution of parametric studies across
distributed computers
 Manages experiment
 Distributing files to remote systems

 Performing the remote computation
 Gathering results
 User submits declarative plan file

 Parameters, default values, and commands necessary for
performing the work
 Nimrod-G takes advantage of Globus toolkit features
NIMROD-G ARCHITECTURE

GRID CASE STUDIES
 Earth System Grid
 LIGO

 TeraGrid
EARTH SYSTEM GRID
 Provide climate studies scientists with access to

large datasets
 Data generated by computational models –
requires massive computational power
 Most scientists work with subsets of the data
 Requires access to local copies of data

ESG INFRASTRUCTURE
 Archival storage systems and disk storage systems at

several sites
 Storage resource managers and GridFTP servers to
provide access to storage systems
 Metadata catalog services
 Replica location services
 Web portal user interface

EARTH SYSTEM GRID
EARTH SYSTEM GRID INTERFACE

LASER INTERFEROMETER
GRAVITATIONAL WAVE
OBSERVATORY (LIGO)

 Instruments at two sites to detect gravitational waves
 Each experiment run produces millions of files
 Scientists at other sites want these datasets on local storage
 LIGO deploys RLS servers at each site to register local

mappings and collect info about mappings at other sites
LARGE SCALE DATA REPLICATION
FOR LIGO
 Goal: detection of gravitational waves
 Three interferometers at two sites

 Generate 1 TB of data daily
 Need to replicate this data across 9 sites to make

it available to scientists
 Scientists need to learn where data items are,
and how to access them
LIGO
LIGO SOLUTION
 Lightweight data replicator (LDR)
 Uses parallel data streams, tunable TCP windows, and

tunable write/read buffers
 Tracks where copies of specific files can be found
 Stores descriptive information (metadata) in a

database
 Can select files based on description rather than filename
TERAGRID
 NSF high-performance computing facility
 Nine distributed sites, each with different

capability , e.g., computation power, archiving
facilities, visualization software
 Applications may require more than one site
 Data sizes on the order of gigabytes or terabytes

TERAGRID
TERAGRID
 Solution: Use GridFTP and RFT with front end
command line tool (tgcp)

 Benefits of system:
 Simple user interface
 High performance data transfer capability
 Ability to recover from both client and server software
failures
 Extensible configuration
TGCP DETAILS
 Idea: hide low level GridFTP commands from users
 Copy file smallfile.dat in a working directory to another

system:
tgcp smallfile.dat tg-login.sdsc.teragrid.org:/users/ux454332
 GridFTP command:
globus-url-copy -p 8 -tcp-bs 1198372 \
gsiftp://tg-gridftprr.uc.teragrid.org:2811/home/navarro/smallfile.dat
\
gsiftp://tg-login.sdsc.teragrid.org:2811/users/ux454332/smallfile.dat
GRID ARCHITECTURE
THE HOURGLASS MODEL
 Focus on architecture issues
Applications
 Propose set of core services as
Diverse global services

basic infrastructure
 Used to construct high-level,
domain-specific solutions
(diverse)
Core
 Design principles
services
 Keep participation cost low
 Enable local control
 Support for adaptation
 ―IP hourglass‖ model Local OS
LAYERED GRID ARCHITECTURE
(BY ANALOGY TO INTERNET ARCHITECTURE)
Application

Internet Protocol Architecture
“Coordinating multiple resources”:
ubiquitous infrastructure services, Collective
app-specific distributed services Application
“Sharing single resources”:

negotiating access, controlling use Resource
“Talking to things”: communication

(Internet protocols) & security Connectivity Transport
Internet
“Controlling things locally”: Access
to, & control of, resources Fabric Link
EXAMPLE:
DATA GRID ARCHITECTURE
App Discipline-Specific Data Grid Application
CollectiveCoherency control, replica selection, task management,

(App) virtual data catalog, virtual data code catalog, …

CollectiveReplica catalog, replica management, co-allocation,
(Generic)certificate authorities, metadata catalogs,
Resource Access to data, access to computers, access to network
performance data, …
Connect Communication, service discovery (DNS), authentication,

authorization, delegation
Fabric Storage systems, clusters, networks, network caches, …

SIMULATION TOOLS
 GridSim – job scheduling
 SimGrid – single client multiserver
scheduling

 Bricks – scheduling
 GangSim- Ganglia VO
 OptoSim – Data Grid Simulations
 G3S – Grid Security services Simulator –

security services
SIMULATION TOOL
 GridSim is a Java-based toolkit for modeling,

and simulation of distributed resource

management and scheduling for conventional
Grid environment.
 GridSim is based on SimJava, a general

purpose discrete-event simulation package
implemented in Java.
 All components in GridSim communicate with

each other through message passing operations
defined by SimJava.
SALIENT FEATURES OF THE GRIDSIM
 It allows modeling of heterogeneous types of
resources.
 Resources can be modeled operating under space-

or time-shared mode.
 Resource capability can be defined (in the form of
MIPS (Million Instructions Per Second)
benchmark.
 Resources can be located in any time zone.
 Weekends and holidays can be mapped
depending on resource‘s local time to model non-
Grid (local) workload.
 Resources can be booked for advance reservation.
 Applications with different parallel application
models can be simulated.
SALIENT FEATURES OF THE GRIDSIM
 Application tasks can be heterogeneous and they can
be CPU or I/O intensive.
 There is no limit on the number of application jobs
that can be submitted to a resource.

 Multiple user entities can submit tasks for execution
simultaneously in the same resource, which may be
time-shared or space-shared. This feature helps in
building schedulers that can use different market-
driven economic models for selecting services
competitively.
 Network speed between resources can be specified.
 It supports simulation of both static and dynamic
schedulers.
 Statistics of all or selected operations can be recorded
and they can be analyzed using GridSim statistics
analysis methods.
A MODULAR ARCHITECTURE FOR GRIDSIM
PLATFORM AND COMPONENTS.
Application, User, Grid Scenario’s input and Results
Appn Conf Res Conf User Req Grid Sc … Output
Grid Resource Brokers or Schedulers

GridSim Toolkit
Appn Res entity Info serv Job mgmt Res alloc Statis
modeling
Resource Modeling and Simulation
Single SMPs Clusters Load Netw Reservation
CPU
Basic Discrete Event Simulation Infrastructure
SimJava Distributed SimJava
Virtual Machine
PCs Workstation SMPs Clusters Distributed
Resources

Tech Info

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Tech Info

Caricato da

Copyright:

Formati disponibili

GRID COMPUTING

Sandeep Kumar Poonia

 40% Mainframes are idle

Sandeep Kumar Poonia

 0-15% Mainframes are idle in peak-hour

Source: “Grid Computing” Dr Daron G Green

Sandeep Kumar Poonia

Sandeep Kumar Poonia

 Core networking technology now accelerates at a much

 Parallel CPU capacity

 Virtual resources and virtual organizations for

Sandeep Kumar Poonia

 Not just computer scientists…

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

 High energy & nuclear physics

Sandeep Kumar Poonia

 Astronomy: Digital sky surveys

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

As of April 2013, Folding@home – 11.4 x86-equivalent

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

 To create and work as one large virtual

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Facilitates flexible, secure, coordinated large scale

Enable communities (―virtual organizations‖) to share

Sandeep Kumar Poonia

 Grid is an evolution of distributed computing

Sandeep Kumar Poonia

 Distributed computing is an ―older term‖

Sandeep Kumar Poonia

 Grid - Uniform, high performance access to computational

 Seamless naming? Yes

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

 The overall motivation for ―Grids‖ is to facilitate

Sandeep Kumar Poonia

Offline Processor Farm

Sandeep Kumar Poonia

www.griphyn.org www.ppdg.net www.eu-datagrid.org

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Data grid Virtualization

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia

Sandeep Kumar Poonia