Sei sulla pagina 1di 77

Project No.

FP6-004265

CoreGRID

European Research Network on Foundations, Software Infrastructures and


Applications for large scale distributed, GRID and Peer-to-Peer Technologies

Network of Excellence

GRID-based Systems for solving complex problems

D.STE.07 – Design Methodology of the Generic


Component-based Grid Platform

Due date of deliverable: 30 April 2008


Actual submission date: 6 May, 2008

Start date of project: 1 September 2004 Duration: 48 months

Organisation name of lead contractor for this deliverable:


University of Westminster

Project co-funded by the European Commission within the Sixth Framework Programme
(2002–2006)
Dissemination level
PU Public PU

Keyword list: Design methodologies, component model, tools/system com-


ponents, Problem Solving Environments, portal components
CoreGRID FP6-004265 1

Contents
1 Executive Summary 4

2 Introduction 5

3 Platform Architecture 6
3.1 Middleware Architecture and Design . . . . . . . . . . . . . . . . 6
3.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2.1 Advantages for using the GCM . . . . . . . . . . . . . . . 7
3.3 Collective Communications Support . . . . . . . . . . . . . . . . 9
3.3.1 Brief recall of the GCM collective interfaces . . . . . . . . 9
3.3.2 Use of collective interfaces to build up a Grid-aware mid-
dleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4 Dynamic Service Aggregation Platform . . . . . . . . . . . . . . . 14
3.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.4.2 The Virtual Clusters Platform . . . . . . . . . . . . . . . 14
3.4.3 WSPeer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4.4 Integration and Contribution to Platform Architecture . . 17

4 Interoperability Issues 19
4.1 Interoperability in Component-based Platforms . . . . . . . . . . 19
4.1.1 Introduction to Interoperability in Component-based Plat-
forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.1.2 Component Model and Framework Interoperability . . . . 19
4.1.3 Composition-level interoperability . . . . . . . . . . . . . 21
4.1.4 Interoperability – discussion . . . . . . . . . . . . . . . . . 23
4.2 Workflow Level Interoperability . . . . . . . . . . . . . . . . . . . 23

5 Development and Execution Environment 27


5.1 Component-Based Integrated Toolkit . . . . . . . . . . . . . . . . 27
5.1.1 Integration in a Generic Component Platform . . . . . . . 27
5.1.2 GCM-Based Design . . . . . . . . . . . . . . . . . . . . . 28
5.1.3 Life-Cycle Controller Modification to Prevent Deadlocks . 30
5.2 Grid Integrated Development Environment - GIDE . . . . . . . . 31
5.2.1 User Groups . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.2.2 Composition . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.2.3 Deployment Perspective . . . . . . . . . . . . . . . . . . . 34
5.2.4 Monitoring Perspective . . . . . . . . . . . . . . . . . . . 34
5.2.5 Steering Perspective . . . . . . . . . . . . . . . . . . . . . 35
5.3 Mediator Component Framework . . . . . . . . . . . . . . . . . . 36
5.3.1 Classification of mediator components . . . . . . . . . . . 38
5.3.2 GCM-based design . . . . . . . . . . . . . . . . . . . . . . 39
5.4 gUSE - A Service based Environment for Grid Application De-
velopers and End Users . . . . . . . . . . . . . . . . . . . . . . . 43
5.4.1 gUSE as a Grid application developer environment . . . . 43
5.4.2 gUSE as an end user environment . . . . . . . . . . . . . 44

CoreGRID - Network of Excellence


CoreGRID FP6-004265 2

5.4.3 gUSE as a scalable, fault tolerant layer to ease Grid sys-


tem management . . . . . . . . . . . . . . . . . . . . . . . 44
5.4.4 gUSE system architecture . . . . . . . . . . . . . . . . . . 45

6 Autonomic Support 47
6.1 Autonomic support of non functional features in GCM . . . . . . 47
6.1.1 Autonomic components . . . . . . . . . . . . . . . . . . . 47
6.1.2 Autonomic behaviour support in GCM . . . . . . . . . . . 49
6.1.3 Experimental results . . . . . . . . . . . . . . . . . . . . . 50
6.1.4 Specific contributions to STE institute . . . . . . . . . . . 51
6.2 Automation of Application Deployment on Grids . . . . . . . . . 51
6.2.1 Application Submission . . . . . . . . . . . . . . . . . . . 53
6.2.2 Resource Discovery . . . . . . . . . . . . . . . . . . . . . . 54
6.2.3 Resource Selection . . . . . . . . . . . . . . . . . . . . . . 54
6.2.4 Deployment Planning . . . . . . . . . . . . . . . . . . . . 55
6.2.5 Deployment Enactment . . . . . . . . . . . . . . . . . . . 55
6.2.6 Application Execution . . . . . . . . . . . . . . . . . . . . 56
6.2.7 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

7 Security Issues in Component-Based Grid Systems 57


7.1 Security Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
7.2 Existing security infrastructures . . . . . . . . . . . . . . . . . . . 58
7.2.1 Grid Security Infrastructure . . . . . . . . . . . . . . . . . 58
7.2.2 Shibboleth . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.3 Security Issues – Conclusion . . . . . . . . . . . . . . . . . . . . . 60

8 Use Cases – Overview and Examples 61


8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
8.2 Wrapping legacy software . . . . . . . . . . . . . . . . . . . . . . 61
8.2.1 GENIE Overview . . . . . . . . . . . . . . . . . . . . . . . 61
8.2.2 Component-based version of GENIE . . . . . . . . . . . . 62
8.3 Componentizing existing applications . . . . . . . . . . . . . . . . 63
8.3.1 Jem3D overview . . . . . . . . . . . . . . . . . . . . . . . 63
8.3.2 Componentising Jem3D . . . . . . . . . . . . . . . . . . . 63
8.4 Developing component-based codes . . . . . . . . . . . . . . . . . 64
8.4.1 GriCoL . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
8.4.2 Use Case: Biochemistry Application . . . . . . . . . . . . 65
8.5 Use Cases – Conclusions . . . . . . . . . . . . . . . . . . . . . . . 66

9 Software Licensing Issues1 67


9.1 Some GPL facts . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
9.2 Incompatibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
9.2.1 Apache License . . . . . . . . . . . . . . . . . . . . . . . . 67
9.2.2 Eclipse Public License (EPL) . . . . . . . . . . . . . . . . 67
9.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
1
This section is based on text originally written by Tony Arbona and Gaston Freire from
Grid Systems, S.A. and provided via the EU GridCOMP project.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 3

10 Conclusions and Future Work 70

CoreGRID - Network of Excellence


CoreGRID FP6-004265 4

1 Executive Summary
This deliverable reports on the considerations and methodology for the design
and implementation of a Generic Component-based Grid Platform. It collates
the innovative efforts of a number of partners which together address the es-
sential aspects of an effective Generic Component Platform. We first outline
the platform architecture, giving an overview of the Grid Component Model
(GCM) in Section 3.1, paying particular attention to the middleware and the
extensions to the communication support that it provides in Section 3.2. The
Virtual Clusters project is a further extension which provides support for mo-
bile devices through functional services known as “Aggregators” (Section 3.3).
We address then address interoperability issues in Section 4, by first discussing
the interoperability in component based platforms (Section 4.1), and then work-
flow level interoperability in Section 4.2. To be practical and useful, any Grid
platform must be adequately supported by development and monitoring tools.
Section 5.1 presents a component based integrated toolkit, which is a framework
that enables developers to develop Grid applications that transparently use grid
resources. The modular design of a component-based runtime environment in-
cludes components for performance monitoring, data dependencies and task
scheduling. All these features are brought together for the user with a homoge-
nous interface in the Grid Integrated Development Environment (GIDE). The
GIDE (Section 5.2) is an Eclipse based set of plugins which provides integrated
tools for component composition and programming, deployment, monitoring
and steering of component applications. Extending the concept of homogenous
sets of component level interfaces between software systems, is the Mediator
Component Framework presented in Section 5.3. Its goal is to integrate system-
component capabilities into application code, achieving both application steer-
ing and performance adaptation to achieve the most efficient execution on the
available Grid resources. Section 5.4 introduces an alternative workflow-based
Grid application development and enactor infrastructure - gUSE (Grid User
Support Environment), which enables the creation, testing, execution, moni-
toring and sharing of Grid applications through intuitive graphical interfaces.
Autonomic support is a key aspect of any generic Grid infrastructure, contribut-
ing to scalability, performance and robustness. Section 6 focuses on the sup-
port provided through autonomic components, which automatically take care of
some non-functional properties aimed at achieving some autonomic reconfigura-
tion goal. Section 7 addresses security issues. Here, the specific requirements of
addressing security for component applications through component containers
is discussed. A distributed component application requires communication not
only between the client and components, but it also involves direct connection
between components. This section discusses authentication, authorization, del-
egation, communication, sandboxing, and virtual organization building. Section
8 presents some use cases exploring the approaches investigated in the course
of the project. In particular, the need to address legacy code is discussed and a
biochemistry use case application is presented. In Section 9, we address the need
for an appropriate licensing strategy for the dissemination of results. Finally,
we conclude and summarize in Section 10.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 5

2 Introduction
Grid technology has the potential to enable coordinated sharing and utilisation
of resources in large-scale applications. In Grid infrastructures the platform is
the heart of the system. It is the cohesive bond enabling virtualisation and
control of resources, users, policies and applications regardless of geographic or
administrative boundaries.
In many cases, existing, contemporary Grid platforms are feature-rich, such
that the requirements of end users are a subset of the available functionalities.
This limits the scalability of the platform to be unidirectional – always scal-
ing upwards. Additionally, the feature-rich philosophy introduces considerable
software and administrative overheads, even for relatively simple demands. As
a consequence, the majority of the platforms cannot easily be adopted for a
specific purpose. Instead, they involve substantial consideration for changes.
In our proposed design methodology, we address this problem by developing
our Generic Component-based Grid Platform, which could scale automatically
based on the context of use.
Recently, component-based software technologies have emerged as a modern
approach to software development for distributed and Grid applications. The
wide adoption of component-based software development and in particular the
use of suitable programming models for compatibility and interoperability are
key issues towards building effective future Grids. Examples of component
models applicable to this field include the Common Component Architecture
(CCA) [8], the CORBA Component Model (CCM) [56], and the emerging Grid
Component Model (GCM) [18].
However, the lack of longer-term experience and the complexity of the tar-
get systems demand much more research results in the field. Our generic Grid
platform design methodology relies on this advanced approach based on the
GCM and on composable component frameworks addressing different concerns
of the platform functionalities. The component-oriented design methodology
supports flexible component adaptation statically and dynamically. The compo-
nent frameworks facilitate designing the platform architecture, support extensi-
bility of different aspects of the platform and allow easy to use and consistency-
preserving reconfigurations.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 6

3 Platform Architecture
3.1 Middleware Architecture and Design
3.2 Background
A component is a software entity that gives a standardized description of what
it provides (server interfaces) and requires (client interfaces). By means of these
interfaces, components can be connected (bound) together and communicate,
thus constituting a component assembly. Component models specify the way
in which components are defined and interact.
Fractal [32] is a model that organizes components hierarchically. Each com-
ponent is either composite (composed of other components), or primitive (a
black box encapsulating a basic functional block). The Fractal specification [15]
proposes a generic, typed component model in which components are runtime
entities that communicate exclusively through interfaces. One of the crucial
features of this model is its support for hierarchical composition. Another key
feature is its support for extensible reflective facilities: each component is asso-
ciated with an extensible set of controllers that enable inspecting and modifying
internal features of the component. Controllers provide a means for capturing
extra-functional behaviours such as varying the sub-components of a composite
component dynamically or intercepting incoming and outgoing operation in-
vocations. The GCM proposal [18] is an extension of the Fractal component
model that specifically targets Grid environments.
The Common Component Architecture (CCA) [8] specifies the means for in-
teraction among components. In CCA, components interact using ports, which
are interfaces pointing to method invocations. Components in this model define
provides-ports to provide interfaces and uses-ports to make use of non-local in-
terfaces. The enclosing framework provides support services such as connection
pooling, reference allocation and other relevant services. Dynamic construc-
tion and destruction of component instances is also supported along with local
and non-local binding. Though CCA enables seamless runtime interoperabil-
ity between components, one of the main weaknesses of the CCA is the lack
of support for hierarchical component composition and for control mechanisms
thereof.
The CCM [6] is a language-independent, server-side component model which
defines features and services to enable application developers to build, deploy
and manage components to integrate with other CORBA services. The CCM is
an extension of the CORBA object model defined to overcome its complexities.
The CCM specification introduces the concept of components and the defini-
tion of a comprehensive set of interfaces and techniques for specifying imple-
mentation, packaging, and deployment of components. The CCM provides the
capabilities for composing components (through receptacles) and permits con-
figuration through attributes. However, in contrast to the Fractal component
model, the CCM does not permit hierarchical composition; that is, recursively
composing components to form more complex, composite components.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 7

3.2.1 Advantages for using the GCM


The GCM [18] is the CoreGRID component model for Grid computing, built
on top of the already existing Fractal model. Its main features are sketched in
Figure 1.

Hierarchical GCM users have the possibility of programming GCM compo-


nents as compositions of existing GCM components. The new, composite
components are first class components, in that they can be used in every
context where non-composite, elementary components can be used, and
programmers need not necessarily perceive these components as compos-
ite, unless they explicitly want to consider this feature.
Collective interaction patterns GCM supports collective interaction pat-
terns. In particular, 1-to-N and N-to-1 ports, with different, configurable
distribution and collection policies. As a result, collective broadcast,
scatter, multicast, gather and reduce patterns are provided as primitives.
Non standard ports In addition to standard use/provide (client/server)
ports and to the collective ports, GCM provides stream ports (support-
ing data flow streams between components), data ports (supporting data
sharing among components) and event ports.
Autonomic management of components GCM supports Manager com-
ponents as well as standard controllers in the component membrane.
Autonomic management of notable composite components (i.e. com-
posite components modelling common, reusable, parametric parallelism
exploitation patterns) is also supported, taking care of component per-
formance tuning.
XML based ADL The GCM component program architecture is described
using a proper XML-based Architecture Description Language (ADL)
that decouples functional program development from the underlying tasks
needed to deploy, run and control the components on the component
framework.

Figure 1: Main features of the GCM

Currently, a reference implementation of our Generic GCM-based Grid Plat-


form is being developed on top of ProActive [61] within the GridCOMP STREP
FP6 EU project [38]. Following ProActive design solutions this reference im-
plementation of GCM uses asynchronous method calls with futures for dealing
with possibly high latency. Moreover, GCM components have a request queue,
which contains the received requests, and an activity, which defines the service
policy of the requests.
Concerning the development of a component application, we distinguish two
essential roles: the developer and the assembler. The developer provides the
functional (business) code, while the assembler is in charge of composing the
system, taking into account the requirements of the application.
Dynamic reconfiguration is crucial in a component-based system. It means
reacting to situations unforeseen by the developer. These reactions are imple-

CoreGRID - Network of Excellence


CoreGRID FP6-004265 8

mented by the assembler inside non-functional (NF) pieces of code. This is why
having the ability to specify at will the control part of any GCM component
is important. A detailed presentation of our work and results on autonomic
support of non-functional features in GCM is included in Section 6.1.
As all those non-functional pieces of code may end up to be complex and
interact in a subtle way, GCM also promotes the idea of designing the membrane
of a GCM component as a GCM-based system of components. At this non-
functional level, the two above mentioned roles are relevant. This non-functional
system is then diluted into the functional hosting component and explicitly
defines what are the subtle interactions that can arise between the functional
and non-functional parts as shown in Figure 2.

hns
Labels of Interfaces :
Controller
b1 Component F = functional; NF = non-functional
nns -First letter (component):
Controller object nfs h/n/f/e: host/inner NF/inner F/external F
Controller b2
-Second letter (interface):
Component
b4 n/f/i/m: external NF/ external F/internal F/internal NF
nfc -Third letter (interface):
c/s: client/server
Membrane b3 -Example:
hmc hns = external NF server interface of the host component
ffs b9 fns
b5 ffc
hfs b8 hfc
b6 fnc b10
hic b7 his
Functional Content hms ens
b11
hnc efs efc

Figure 2: New structure for the membrane of Fractal/GCM components

Here we give a more detailed explanation about some possible roles of NF


interfaces exposed by the membrane, making references to Figure 2. For sim-
plicity, we focus on the NF interfaces of the host component, they give a high
level view of the structure suggested for the component management:

• hns: Host component external NF Server interfaces as exist in the Fractal


specification; they allow external entities to access controllers inside the
membrane;

• hnc: Host component external NF Client interfaces; they give to inner


NF components access to external NF interfaces;

• hms: Host component internal NF Server interfaces; they give inner func-
tional components access to interfaces of inner NF components;

• hmc: Host component internal NF Client interfaces; they give to inner


NF components access to interfaces of inner functional components.

All these interfaces give the membrane a better structure and enforce decoupling
between the membrane and its externals. For example, to connect nfc with fns,
GCM adds an additional stage: we have first to perform binding b3, and then

CoreGRID - Network of Excellence


CoreGRID FP6-004265 9

binding b9. This avoids nfc to be strongly coupled with nfs: to connect nfc to
another interface of kind nfs, only binding b9 has to be changed.
In Figure 2, some of the links are represented with dashed arrows. Those
links are not real bindings but “alias” bindings (e.g. b3); the source interface is
the alias and it is “merged” with the destination interface. These bindings are
similar to the export/import bindings existing in Fractal (b6, b10) except that
no interception of the communications on these bindings is allowed in Fractal.
To conclude, the GCM provides possibilities consisting in implementing as
components (part of) the membrane and thus benefiting from the component
structure, which allow membrane to be designed in a component-oriented way,
and to be reconfigurable.
The aim is to show the applicability of GCM to the Integrated Toolkit, one
key element of the Generic component-based Grid Platform. This is detailed
further in Section 5.1.2

3.3 Collective Communications Support


3.3.1 Brief recall of the GCM collective interfaces
The GCM adds the idea of collective interfaces to the Fractal component model
following previous work and results in the area of multiparadigm communica-
tions for Grid computing [36].
Multicast interfaces provide abstractions for one-to-many communications,
by transforming a single invocation onto a list of invocations. When a single
invocation is transformed into a list of invocations, these generated invocations
are forwarded to connected server interfaces. The semantics of the propaga-
tion of the invocation and of the distribution of the invocation parameters are
customizable by means of a parameter distribution policy, and the result of
an invocation on a multicast interface is a list of results, or a reduction of it.
Invocations forwarded to the connected server interfaces may occur in parallel,
which is one of the main reasons for defining this kind of interface: it enables
parallel invocations.
Gathercast interfaces provide abstractions for many-to-one communications,
by transforming a list of invocations into a single invocation. A gathercast in-
terface coordinates incoming invocations before continuing the invocation flow:
it can define synchronization barriers and gather incoming data. Invocation
return values are automatically redistributed to the invoking components. Syn-
chronization barriers and gathering operations are customizable by means of
an aggregation policy, as well as redistribution policies for invocations return
values.
There is also an other kind of interface called gather-multicast (Fig. 3).
This interface, specific to composite components, is built by the concatenation
of a gathercast and a multicast interface:

• the server gather-multicast (Fig. 3.a) exposes its gathercast interface and
is connected to internal components through a multicast interface;

• the client gather-multicast (Fig. 3.b) connects internal components on a

CoreGRID - Network of Excellence


CoreGRID FP6-004265 10

gathercast interface, exposing them externally through a multicast inter-


face.

internal multicast external multicast


a. client interface b. client interface

.
. . . = .
. = . . .
. . .

external gathercast internal gathercast


client interface client interface

Figure 3: GCM gather-multicast interface type

Obviously, the naive solution for performing collective communications among


composites bound by a gather-multicast interface creates bottlenecks both in
the gather and the multicast sides of the interface. Efficient communication
requires some form of direct bindings between the inner components accord-
ing to the redistribution pattern. For this reason, we have introduced new
controllers associated to primitive components and to the gather-multicast in-
terfaces. These controllers are capable of optimizing these collective interfaces
by distributing the ’MxN’ behavior i.e. creating dynamically new client in-
terfaces on the ’M’ client components, and server interfaces in the ’N’ server
components and connect them directly, according to data distribution. (see
Figure 4). The optimization takes three main arguments:

• a reconfiguration function (R): used by a special controller to create the


necessary interfaces (in general, a multicast client interface and a gather-
cast server interface) and the direct binding scheme.

• a dispatch function (D): associated to the generated client interfaces, it is


responsible for the adaptation of the method call signatures. In general,
it transforms the original single call, of the non-optimized version, into a
list of calls. It is also in charge of dispatching method calls;

• a gather function(G): associated to the generated server interfaces, it is re-


sponsible for the adaptation of method call signatures. In general, it trans-
forms the received calls into a single call, expected by the non-optimized
version of the server interface.

The next subsection depicts how the gather-multicast interface and its MxN
optimization have been used to implement a GCM-based runtime to support
MPI-like SPMD applications. Succeeding to host communication intensive ap-
plications while achieving good performances is a strong argument in favor of
using a component-based approach to design runtime and middleware for grid
computing in general, not only for supporting HPC scientific applications.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 11

and Original single interfaces

and Generated collective interfaces 1:0

1:1 ...

0:0
1:2

0:1

...
0:2

0:3

2:0

...
Direct (optimized) bindings 2:1

Original bindings

Figure 4: Example of a GCM gather-multicast usage and its optimisation using


an MxN interface type

3.3.2 Use of collective interfaces to build up a Grid-aware middle-


ware
In the context of the French ANR-funded project DiscoGrid [24], we have de-
signed a GCM-based runtime support for hiererchical MPI-like SPMD applica-
tions.
The DiscoGrid Runtime is a component-based infrastructure that models a
hierarchy of resources forming a computing Grid, and offers support to the con-
cepts and programming primitives of the DiscoGrid API [50]. The DiscoGrid
API intends to provide programmers with a notion of hierarchy in their dis-
tributed applications. This hierarchy is the expression, at the program level,
of the underlying physical resource topology. The originality of the DiscoGrid
communication API resides in using hierarchical identifiers to involve different
sets of processes. The grid resources are organized as a tree and, depending
on hierarchical identifiers of the source and the destination, i.e. a leaf or a
node, programmers can perform either point-to- point or multi-point commu-
nications, respectively. The DiscoGrid communication API defines classical
point-to-point primitives such as Send and Receive as well as collectives ones
such as Barrier, Reduce, AllReduce. Point-to-point primitives have a special
semantic where a Send operation targeting a node in the resource hierarchy
can be seen as a multicast, and a Receive can be seen as a gathercast. The API
also defines the Update primitive, which is an automated way of updating data
shared between processes, using an associated meta-information, such as data
location and neighboring. In the case of domain-decomposition problems, the
Update primitive is useful to synchronize borders of each domain.
The DiscoGrid Runtime is a composition of two sorts of components:

• the primitive wrapper is the most elementary component used to form


the entire component infrastructure. It wraps an MPI process and is

CoreGRID - Network of Excellence


CoreGRID FP6-004265 12

responsible for the MPI to/from Java communication. By default, it


only presents a single server interface and a single client interface that
are bound to a generic clustering component that represents the upper
level of the hierarchy. The primitive wrapper is also responsible for the
encapsulation of MPI messages into objects with information useful to
route messages through the component infrastructure.

• the composite clustering is a generic component capable of clustering


lower-level components (both primitive wrappers and composite cluster-
ing). It presents two interfaces: a gather-multicast server interface, capa-
ble of receiving messages from other clustering components and dispatch-
ing messages to inner components and a gather-multicast client inter-
face capable of gathering messages from inner-components and dispatch
them to other clustering components of the same level. These gather-
multicast interfaces implement the main collective operations (reduce,
scatter, gather, broadcast and barriers) as aggregation and dispatch poli-
cies, selected dynamically according to the requested operation.

By default, the entire component model is enclosed in a generic composite,


that exposes externally the entire DiscoGrid application. The interest of such
encapsulation is that one can envisage to take profit of GCM features to compose
complex simulations by loosely coupling simpler applications or even expose one
whole application as a WebService.

High-level DiscoGrid communication support

• Point-to-point communications: A point-to-point communication between


processes on different clusters happens when two process do not have
the same hierarchical identifier prefix. In practice, a message is properly
tagged in the wrapper component, sent upwards in the hierarchy and
routed through composite interfaces until it reaches the destination.

• Collective communications: As in MPI, standard collective communica-


tions assume that all processes of a given level must participate on the
process. So, a collective communication is said to take place at a given
level. In general, messages are gathered on the most external clustering
component that represents the given level. Depending on the operation,
the aggregation policy also reduces the data. If necessary, the parameters
are properly split by the parameter dispatch policy, that also generate the
method calls and invoke them on the proper bound interfaces.

• Neighborhood-based communication: This kind of communication is rep-


resented by the update primitive, defined in the DiscoGrid API. The non-
optimized implementation of the update operation uses the initial compo-
nent bindings and gather-multicast interfaces. This update communica-
tion can be considered a generalization of the MxN communication, that
instead of just triggering a one shot communication between two sets of
components (of size M and N), trigger a set of such MxN communications

CoreGRID - Network of Excellence


CoreGRID FP6-004265 13

(i.e. a generalized MxN communication). Thanks to gather-multicast in-


terfaces, the runtime can do this generic MxN communication at once:
each process participates with its shared data of the given level, messages
are then gathered in the most external clustering component of the level,
splitted up into pieces and dispatched properly to the correct destinations
using a dispatch policy that looks like a scatter, but taking into account
neighborhood information to split and determine who receives each piece
of data. However, there is some room for optimization, which is done
as follows. The optimization of gather-multicast interfaces is defined by
three functions: R defines the creation of a multicast client interface and
a gathercast server interface (the generated interfaces attached to each
primitive component in the Figure 4) and the bindings according to the
neighborhood information (the arrows crossing the composite component
boundaries in the Figure 4). D splits the parameter of the update call and
dispatches the data directly to the connected neighbors. And G blocks
until messages from neighbors are arrived on the gathercast interface and
order them properly, before passing the message to the wrapped MPI
process.

Some performance benchmarks confirm that the optimized update operation


implementation is worth considering (see Figure 5).

12000 DiscoGrid Non-Optimized


Pure MPI
DiscoGrid Optimized
10000

8000
Time (ms)

6000

4000

2000

0
8 16 32 64 128
Number of Nodes

Figure 5: Poisson-3D update operation duration

Overall, the usage of the GCM collective interface types and the introduc-
tion of numerous optimisations that we have been able to conduct prove that
a middleware for grid computing based upon a component-based approach can
exhibit flexibility but also, good performances in supporting performance de-
manding inter-component interactions. As the Generic Component-based Grid
platform may face communication intensive patterns, this is a good point in
favor of such an approach.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 14

3.4 Dynamic Service Aggregation Platform


3.4.1 Introduction
In this section we briefly present the Virtual Clusters platform, the related
contribution to WP7 stemming from it, and the further opportunities for in-
tegration with WSPeer. We will first provide some overview of the Virtual
Clusters platform and WSPeer, then move further into the most significant as-
pects of the projects in regards to work done in the Institute of Systems, Tools
and Environments, and finally lay the roadmap for future integration activities.

3.4.2 The Virtual Clusters Platform


The Virtual Clusters Platform (from now on “VC”) is a proxy-based middle-
ware that enables the efficient integration of mobile and potentially small-scale
devices (collectively referred to from now on as simply “mobile services” or
“mobile devices”) in Grid systems as service providers -and not just service
consumers, towards what are called “hybrid systems”. ”Efficient“ in this con-
text refers to a number of desired attributes the integrated system should have:
performance efficiency in dealing with many mobile nodes at the same time;
minimal administrative and management overhead due to the large number of
mobile devices integrated in a relatively stable system; introduction of efficient
programming models and techniques on top of this integrated system, that
provide high level, abstract views of the underlying, heterogeneous, and very
dynamic mobile domain.

Overview/Architecture The VC is based around the concept of Aggrega-


tor services (or simply “Aggregators”). Aggregators are proxies to services
available in mobile devices. In order for Aggregators to assume their proxy
responsibilities, mobile devices that offer a specific service are grouped to form
what is called a Virtual Cluster. It is a cluster because it is a collection of
functional services (although it does not bear much similarity to the traditional
computational clusters), and it is Virtual because potential clients do not and
cannot “see” the mobile services directly. Instead, clients only access the avail-
able functionality through the Aggregators. This happens in a standardized
manner through WS technologies (HTTP/SOAP) despite the potential hetero-
geneity and multitude of binding protocols available “behind” the VC abstrac-
tion. The Aggregators reside in proxy nodes, which can be either dedicated
nodes (preferably on the “edge” of the static part of the hybrid system) or
typical functional nodes since Aggregators can be duplicated or migrated and
the middleware does not have a large footprint [40].
Aggregators are not simple proxy services though: the are assisted by a
number of well orchestrated middleware services that provide support for im-
portant non functional aspects of the system such as mobility monitoring, failure
masking, failure tolerance, collective operations and more.

Dynamic Properties Here we will mention only the dynamic properties of


the VC platform that are relevant to the Platform Architecture deliverable

CoreGRID - Network of Excellence


CoreGRID FP6-004265 15

Figure 6: High Level Architecture of the Virtual Clusters Platform

context. For more information the reader is referred to [41].

• Dynamic Service Aggregation


Aggregators are generated only once for each VC: the first time that a
particular service becomes available in the system. From then on, mo-
bile devices offering the same service will only have to register with the
relevant Aggregator -a process that is already semi-automated and in the
near future will require human intervention only in case of security con-
siderations or special policy enforcing. Aggregators -with the help of the
middleware services, are able to detect and automatically activate mobile
services that have registered previously in a seamless manner.
• Mobility Prediction and Monitoring
Amongst the main design goals and the core driving force behind the VC
platform was the inherent support for mobile services. This entails some
sort of ability to forecast or predict changes due to mobility, and a flexible
but powerful mechanism for monitoring and adapting to environmental
changes in general, service mobility in particular. In the VC platform,
there is a mobility prediction mechanism – a Virtual Distance mechanism
[39] – to calculate the probability of a device moving out of coverage. In
brief, this is based on an algorithm that keeps track of a short history
of mobility patterns of the device in question. This mechanism is non
intrusive for the mobile device and it has no requirements on the device
provider, which makes it appealing for such dynamic systems. It is purely
distance-oriented only and does not include issues related to location or
personalisation.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 16

• Abstraction Layer
The implementation of the VC abstraction, enables a unified and consis-
tent interface to multiple and dynamic services/resources, possibly utiliz-
ing different binding protocols. Server-side components available in the
underlying fabric layer can make use of a variety of transport protocols,
such as WS, RMI, or the LSC (see below: “Binding Extensions”). At the
programming level, the VC abstraction will significantly ease software de-
velopment and simplify some high-level programming models, such as the
master-worker and the voting models, and will provide the foundations for
collective (group) operations on the VC. Furthermore, it provides trans-
parency from the underlying binding protocols and dynamic addressing
schemes, and works as a bridging layer that provides unified access to
heterogeneous services. Finally, the VC abstraction also enables the en-
capsulation of internal failures, protecting this way the rest of the system
from the inherent instability of the underlying nodes.
• Failure Management Support
Dealing with the consequences of device failures has a three-fold meaning:
fault tolerance – for high reliability, failure masking – for stabilized avail-
ability, and failure recovery – to support both. To achieve fault tolerance,
the VC takes advantage of the cardinality of the underlying services in
the mobile domain, to automatically mirror the execution of a task to
more than one available services. For failure masking, failures within the
cluster generate notifications that only go up to the proxy layer while
they are dealt with internally with mechanisms such as state migration
or redistribution. The VC abstraction enables failure masking by dealing
with failures without requiring the intervention of the requesting client.
Finally, for failure recovery, and since the nature of failures in the mo-
bile domain is fundamentally different than in traditional static domains,
slightly different techniques are required. First of all, a temporary discon-
nection of the device, due to intermittent wireless connectivity, is regarded
as a failure in a traditional system. However, the device might actually
come back online almost immediately. In such case, there is enough flex-
ibility in the VC platform, to allow the immediate reactivation of the
device, without any consequences in the higher layers, and without the
need to redistribute or restart the execution. If, however, the failure is
permanent or non-recoverable, then there is build-in support for auto-
matic redistribution of the task to available services, as well as support
for migratable services in case of predicted failures.
In [39] an extended discussion about dependability issues in such hybrid
environments is presented, along with the detailed description of the VC
approach to dependability.

3.4.3 WSPeer
WSPeer is an API for hosting and invoking Web services. Unlike most Web
service middleware, WSPeer is not based on a container model in which appli-

CoreGRID - Network of Excellence


CoreGRID FP6-004265 17

cations are placed into the container environment and managed by it. Instead
an event-based architecture is used in which applications add themselves as
listeners to events generated by WSPeer. Events are generated as the result of
deployment, publishing, and the arrival of SOAP messages, leaving the appli-
cation in control of how these are handled. This makes the architecture simple
for integrating applications with it.
WSPeer has evolved out of an interest in combining the strengths of Web
services, specifically the interoperability benefits of WSDL and SOAP and the
decentralized resource sharing and discovery mechanisms of P2P systems. This
allows for the complimenting of standard Web services means for message trans-
fer and service description with P2P mechanisms for discovery as well as Net-
work Address Translation System (NAT) and firewall traversal.
Web service XML standards allow interoperable service definition and mes-
sage exchange while decentralised resource sharing and discovery have proven
themselves to be scalable and resilient to node failure while enabling transient
nodes to interact. WSPeer is architected to allow for multiple protocol and
middleware bindings to facilitate Grid and P2P interoperability. Currently,
WSPeer supports P2P communication using JXTA and P2PS and Grid and
Web services through WS-RF and WS-Notification. In addition, there is also
support for firewall traversal using Styx as well as an in-house lightweight XML
communication protocol for mobile devices bundled within WSKPeer, a J2ME
version of WSPeer. Because multiple bindings can coexist, a node exposed with
WSPeer functionality can bridge static Grid environments with more dynamic
ones.

3.4.4 Integration and Contribution to Platform Architecture


By integrating the Virtual Clusters architecture with WSPeer discovery and
invocation capabilities, the VC environment would benefit on three different
and distinct levels: (i) communication and discovery among the VC layer; (ii)
communication between the VC Aggregators and the underlying mobile ser-
vice nodes; and, (iii) direct P2P interaction between mobile nodes. The first
aspect would allow for the VC layer to form a dynamic network that is able
to self discover and heal in the case that Aggregators are leaving and joining
the network. The WSPeer communication and discovery capabilities, beyond
the base network forming and maintenance, also provides the tools needed for
VCs to discover and communicate with one another when migration events are
triggered. WSPeer can also be beneficial in the second aspect, that of commu-
nication between the VC and the underlying mobile and transient nodes in the
cluster. Lastly, mobile nodes can make use of decentralized P2P infrastructure
to dynamically discover and directly interact with one another using WSPeer’s
unicast and multicast protocols. WSPeer already has bindings for JXTA, P2PS,
and Styx, which would expand the current capabilities of the VC Aggregators
to allow these communication protocols.

Hierarchical Structures Hierarchical structures and topologies are becom-


ing important in distributed systems because of their flexibility and available

CoreGRID - Network of Excellence


CoreGRID FP6-004265 18

options. For example, different protocols and different policies could be en-
forced in different branches of the hierarchy for better fine tuning and more
application-specific support. While the VC platform is inherently ready to
provide the backbone for hierarchical clustering, WSPeer would bring in an-
other dimension in the form of P2P-style bindings and communication. This
will relax a number of constraints of the VC platform and will enable more
straightforward communication between proxies at the same hierarchy level. It
will also enable flexible topologies and mobility at every hierarchical branch to
be supported seamlessly as the VC proxies will be able to dynamically adapt to
environmental changes easily and in a cooperative fashion thanks to WSPeer
P2P bindings.

Binding Extensions This is important in the context of mobile and po-


tentially limited environments because of the inadequacy of mobile devices to
support traditional service middleware (e.g. Globus or Tomcat/Axis). It is
therefore desired to have a flexible system that can understand a variety of
binding protocols so that mobile users can select the one that better suits their
own device. For example, some of the stronger mobile devices can opt for a
full fledged web services container with all the benefits and the problems this
will bring. Another might choose a less “heavy” approach such as RMI or even
RMI-OP for Java micro devices. Yet another mobile provider could go for the
least “expensive” approach of all that comes in the form of the Lightweight
Service Container (LSC) developed in UoW [41].
Integration with WSPeer will extend the range of immediately supported
bindings from the existing HTTP/SOAP, RMI, and LSC, to also include Styx,
P2PS and JXTA, thus providing even more options for mobile service providers.

Better Non-Functional Properties Support The P2P style interactions


of WSPeer will assist the VC proxies in a number of significant non-functional
properties. Direct communication between VC proxies in a P2P manner maps
directly to high level VC proxy chaining, a process that takes place in response
to (and in support of) mobility in order to forward execution results back to the
requesting client. Migration techniques will also take advantage of WSPeer since
cooperation between VC proxies becomes more efficient through notification
mechanisms that are directly supported through WSPeer.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 19

4 Interoperability Issues
4.1 Interoperability in Component-based Platforms
4.1.1 Introduction to Interoperability in Component-based Plat-
forms
Interoperability can be defined as an ability of two or more entities to commu-
nicate and cooperate despite differences in the implementation language, the
execution environment, or the model abstraction [66]. Today, a popular solu-
tion for interoperability between components is Web Services where standard-
ized protocols based on XML provide the common language for applications to
communicate [30]. This has been successfully applied also to high-performance
modules like ASSIST modules, wrapped as GRID.it components [1]. Interop-
erability has been outlined as a requisite for the GCM: a realistic programming
model, in the context of heterogeneous systems, component frameworks and
legacy software, must be able to interoperate with existing component frame-
works and allow applications to be built from a mixture of various kinds of
components. Naturally, the GCM proposes to achieve general component in-
teroperability through standardized web services. Beside it there are alternative
interoperability approaches: our idea is to introduce mediators and adapters to
build up an ad-hoc interoperability layer between selected component frame-
works without superimposing on them another general-purpose interoperability
framework (like a CORBA bus, or a meta-component model implemented on
top of some selected existing component frameworks [59]).
In this section, we describe two levels of interoperability:

• Component model and framework interoperability

• Interoperability at the composition level

The first case is exemplified on our experience with the GCM and CCA com-
ponent models [51] and the specific implementations: ProActive and MOCCA [52].
We demonstrated how these models can be mapped to each other and how the
implementations can interoperate. The second case is demonstrated on the
example of GridSpace programming environment, which proposes a high-level
scripting approach to component application composition and allows integrat-
ing multiple middleware technologies and component models, including CCA
and GCM components as well as Web Services and legacy jobs.

4.1.2 Component Model and Framework Interoperability


We identified two approaches of interoperability between CCA and GCM: the
encapsulation of a single CCA component and of a complete CCA system,
consisting of several CCA components, to be seen as a GCM component.
The first solution enables the creation (instantiation) of a CCA component
as a primitive Fractal component in a single address space. It relies on a wrapper
that encapsulates a CCA component, and exposes cca.Services interface to a
CCA component (see Fig. 7). Before instantiation we should know the type of

CoreGRID - Network of Excellence


CoreGRID FP6-004265 20

Figure 7: a) Integration of a single CCA component into a Fractal one ; b)


Interoperability between CCA and Fractal components

a component in order to define the Fractal type of the component; this might
be obtained from a provided ADL description.
In the second case CCA components are created in their own framework
and they are connected to Fractal components running in their framework.
Complete interoperability between two frameworks requires instantiation of
a whole CCA assembly, and ability to interact with it from a Fractal framework
as if it was a Fractal composite component. In this case, we have a CCA
component or a set of CCA components which are created and connected among
themselves by a CCA framework (e.g. MOCCA). So, we wrap the component
assembly as a Fractal component in such a way that it can be connected to
other Fractal components.
The solution we propose is based on a wrapper which adds a Membrane to a
CCA assembly. The wrapper should interact with the CCA framework only via
BuilderService external interface (obtained in framework dependent manner).
The implementation of a GluePort is framework specific, and translates the
Fractal invocations to CCA invocations and reversely. The GluePorts expose
Fractal interfaces to the outside world, and they can be connected (bound) to
other Fractal components using BindingController and Component interfaces
of the wrapper. The wrapper uses the BuilderService to connect exported
CCA ports to corresponding GluePorts using CCA framework, so the commu-
nication between CCA component assembly and GluePorts is handled by the
CCA framework. In other words, the Wrapper component is both a CCA and
a Fractal component.
In order to verify the proposed above solution a prototype using Java-based
ProActive and MOCCA implementations was developed.
Integration of a single component was realized as outlined above. A wrapper
which encapsulates a CCA component and which exposes Services interface to
a CCA component is created by the Fractal framework. The wrapper instanti-
ates the CCA component as a local object and it invokes setServices (this)

CoreGRID - Network of Excellence


CoreGRID FP6-004265 21

on a CCA component, passing the reference to itself. The CCA component


registers its uses and provides ports, and consequently the wrapper can create
direct (local) bindings to exported CCA ports.
In the real interoperability scenario we assume that there are CCA compo-
nents running in a framework and connected using a mechanism specific to this
framework (e.g. a script, or Java API), forming the existing CCA assembly. It
should be noted that both Client and Server Glue components are conceptually
symmetric and their role is to translate invocations from one framework to the
other.
It was the implementation choice to create a Server Glue as ProActive com-
ponent which includes the MOCCA code, whereas a Client Glue is created as
MOCCA component with an “embedded” ProActive one.

4.1.3 Composition-level interoperability


Our research was focused also on the high-level approach for component compo-
sition, based on a scripting paradigm. There is a need for a high-level program-
ming approach which would enable combining the composition in space and
composition in time in a way which is flexible and convenient for a program-
mer. The approach should not be limited to a single component model, since
many models are available for programming Grid applications. Moreover, being
focused on the Grid environment, it should conceal the complexity of the un-
derlying infrastructure, automating the process of component deployment and
resource selection where possible. It would be also valuable if the solution could
facilitate such aspects as component configuration and passing parameters to
the application. Such a solution would eventually form a powerful application
development, deployment and execution tool for the Grid.
A system designed to meet the above mentioned requirements based on a
scripting approach is called GridSpace [54]. This solution is especially well
suited for rapid application development (RAD), prototyping and experiment-
ing scenarios. A scripting approach provides also the full flexibility of using a
programming language, allowing a rich set of control constructs of component
application (workflow). The high-level notation allows hiding all the details of
underlying Grid infrastructure, so the programmer may focus on the application
logic and automating the process of resource selection and component deploy-
ment. Our approach is in line with the suggestion that any component-based
platform to be successful should support scripting as a mechanism for glueing
application components together [57].
As the goal of our Ruby-based GScript language is to provide constructs for
component deployment, composition and invocation the component methods,
there is a need to provide an execution engine, responsible for providing the
information on components in use and to hide the complexity of the underly-
ing grid infrastructure. The architecture of the proposed system is shown in
Figure 8. Registry is used for storing all technical information about available
components, containers and the state of the Grid resources, updated by Moni-
toring system. The registry is available as a Web service, and is also possible to
use a local registry, which stores the same information in the Ruby script format,

CoreGRID - Network of Excellence


CoreGRID FP6-004265 22

Figure 8: Architecture of GridSpace scripting environment

which is useful for quick development and debugging. The Optimizer module
is responsible for supporting decisions on (automatic) component deployment.
The role of optimizer is similar to the deployment framework as proposed in
the following sections of this document. The Invoker module transparently
performs remote calls on component ports of different frameworks.
The invoker has an extensible architecture which allows plugging in adapters
responsible for interacting with different technologies. As of component tech-
nologies, there are adapters for communicating respectively with MOCCA or
ProActive components. For Web Services the adapter can use a client from
a standard library of a scripting language. There is also an adapter for the
MOCCA-ProActive Glue components; those glue components allow composi-
tion in space (direct connections) between the two frameworks [51]. More details
on the invoker and the architecture of the system can be found in [11].
Composition-level interoperability is supported by the GridSpace environ-
ment in the following way. The scripting API for composition and deployment of
components is neutral with respect to the used component model. Script invo-
cations are translated to underlying Fractal, CCA or GCM APIs. If more than
one component model is supported, it is possible to combine components from
different models into a single application. In the case of a component workflow,
the runtime system is the central point of inter-component communication, so
it acts as an intermediary passing results from one component invocation to
another. It is also possible to integrate modules developed in other technolo-
gies, e.g. Web services or WSRF in such a workflow. In the case of composition
in space, when direct links between components are involved, it may be neces-
sary to introduce glue ports between the heterogeneous components, to enable
translation of invocations between them. As our research on interoperability
between GCM and CCA [51] suggests, it is possible to introduce such a generic
glue which can bridge components from different models and frameworks.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 23

4.1.4 Interoperability – discussion


On the level of inter-component model interoperability we can draw the follow-
ing conclusions. The analysis of CCA and GCM component models, shows that
despite some differences, it is feasible to integrate components from one model
into another framework, as well as to create the glue code which enables inter-
framework interoperability. The prototype functionality has been verified with
a number of examples, including a non-trivial application (simulation of gold
cluster formation[53]) and integrated with the ProActive library. We observed
that if the properties of two different component models can be well under-
stood, then the generation of wrappers and glue code bridging two different
component frameworks can be generic and thus automated.
On the composition-level interoperability, we can also note that the vari-
ous component models can be equivalent to some extent and it is possible to
propose an environment which can conceal the differences between the under-
lying frameworks. Therefore we can achieve interoperability and facilitate the
application composition by hiding the low-level details, while making them still
available for more advanced users who need full control over the application
ecexution.

4.2 Workflow Level Interoperability


Grid interoperation at the level of workflows can be achieved inside one workflow
or within several different workflow systems. Therefore, we can talk about
intra- or inter workflow interoperation, respectively. In both cases, we have to
examine two different aspects of interoperation. One aspect is the execution of
workflow components on computing resources spanning several grids. The other
is the movement of data between heterogeneous data resources and workflow
components. SZTAKI and University of Westminster are collaborating within
the framework of the CoreGrid project to address these interoperability aspects
and were concentrating on intra-workflow interoperability so far.
The generic requirements towards intra-workflow interoperability are shown
in Figure 9 [45]. The input files of the workflow can come from file systems,
like SRM or SRB, or from database management systems and the result can
also be fed into any of these solutions. Both the file systems and the databases
can be located in different production grids, and the jobs of the workflow can
also be mapped to different grids. These grids can be based on different grid
middleware and may require different user certificates for authentication.
The P-GRADE grid portal and application development environment [44]
was utilized to implement the generic interoperability scenario above.
Components of a P-GRADE workflow can be executed in various Grids and
can send input/output data to each other. The multi-grid support in the P-
GRADE portal allows downloading multiple certificate proxies to one portal
session and mapping these to multiple Grids. The job submission mechanisms
of the portal also support submitting jobs to various Grid middleware such
as GT2, GT4 and g-Lite. Figure 10 illustrates a P-GRADE workflow where
the different jobs are mapped to different production Grids (EGEE, UK-NGS,

CoreGRID - Network of Excellence


CoreGRID FP6-004265 24

Figure 9: Generic requirements towards intra-workflow interoperation of grid


data resources

TeraGrid) and are running on multiple grid middleware. The job executable
can be provided by the user or can be selected form a legacy code repository
realized by the GEMLCA [23] solution.
The second aspect of intra-workflow interoperability is data access. Intra-
workflow interoperation of grid data resources allows data to be input from
or output to different file storage systems or database solutions, located in
several different grids. The P-GRADE portal allows data to be fed from and
to GridFTP, SRB and SRM based file systems, and also to utilise databases
exposed by OGSA-DAI. As P-GRADE already supported accessing GridFTP
and SRM catalogues, our current work extended this towards SRB and OGSA-
DAI.
In order to achieve workflow level interoperation of SRB and the already
supported data sources (GirdFTP catalogues and EGEE storage elements),
workflow level integration of P-GRADE and SRB was necessary [46]. As input
and output files in P-GRADE are represented as ports, the natural way to
integrate SRB with P-GRADE was to extend it with a new port type called
SRB. This integration raises three challenges. The first challenge is setting up
and configuring the SRB client environment to enable access to multiple SRB
servers at the same time and by extending the multi-Grid capabilities of P-
GRADE to SRB resources. The second challenge was enabling the creation of
SRB input and output ports in the P-GRADE workflow editor. And finally, to
access the SRB data collection to retrieve the selected files before job execution,
and also to copy SRB output files back to the SRB resource. When moving these

CoreGRID - Network of Excellence


CoreGRID FP6-004265 25

Figure 10: Workflow level interoperability with P-GRADE Grid Portal

files the portal machine can easily become a bottleneck if all SRB file transfers
occur and terminate on this machine. In order to overcome this shortcoming the
implemented solution utilises direct file transfer between the SRB resource and
the executor site whenever it is possible. This workflow level integration now
allows the seamless interoperation of SRB catalogues, GridFTP file systems and
EGEE storage elements (based on SRM) at the level of P-GRADE workflows.
Jobs of one workflow can run in different grids (US OSG, UK NGS, EGEE)
and utilise data resources based on different technologies (SRB, SRM, GridFTP,
local) from these different grids.
In order to achieve the generic interoperation of grid data resources data
coming from relational or other databases has to also be accommodated into
workflows. A generic tool exposing database operations as services is OGSA-
DAI. OGSA-DAI was selected as the middleware level solution to be integrated
with P-GRADE due to its flexible architecture and widespread utilization by
the grid community.
The primary aim when accessing a database in the grid is to move computa-
tion to the data rather than the other way round. This is justified by the huge
size of databases and also by the fact that the service representation requires
the conversion of data to SOAP-XML format. When integrating OGSA-DAI
to P-GRADE we have to keep this very important feature of OGSA-DAI in-
tact and move the data only when it is absolutely necessary. Moving the data
is inevitable when some complex analysis has to be carried out on a data set
delivered as a result of a database operation (e.g. a query), and the computing
resources available on the database server are not adequate to run this analysis.
It is quite common in an e-science scenario that the result of a query produces
a set of data that forms the input of a parameter sweep experiment. P-GRADE

CoreGRID - Network of Excellence


CoreGRID FP6-004265 26

supports the creation of parameter study workflows, as explained in [4]. There-


fore, an important requirement towards the integration is that data coming as
a result of OGSA-DAI queries should be able to serve as input to these param-
eter sweeps. As the parameter study workflows may require hundreds or ever
thousands of resources to run, the only feasible solution is to move the data to
computational resources where the analysis is carried out.
Based on the above, the major principles of OGSA-DAI P-GRADE integra-
tion are the following:

• Allow a set of database operations to be delivered to OGSA-DAI resources


as P-GRADE workflow components (move operations to data).

• Allow delivering the results of OGSA-DAI operations, when it is required,


to form the input of parameter study workflows in P-GRADE.

The integration of OGSA-DAI and P-GRADE requires the definition of a


new job type, called OGSA-DAI job in P-GRADE. The most important ques-
tion here is what an OGSA-DAI job is doing and at what level of abstraction
it should be defined by the user.
An OGSA-DAI job describes a set of database operations. OGSA-DAI
receives these operations in the form of a perform document and returns them
as a response document. Therefore, an OGSA-DAI job in P-GRADE is a job
that receives its input as a perform document. The job itself is responsible for
interpreting this document, connecting to the defined OGSA-DAI service and
returning the results as a response document. It is then the responsibility of the
following jobs in the portal to interpret this response document and use it, for
example as input for parameter sweep workflows. Work is currently undertaken
to fully implement this solution.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 27

5 Development and Execution Environment


5.1 Component-Based Integrated Toolkit
The Integrated Toolkit, previously presented in [20, 64], is a framework that
enables the easy development of Grid-unaware applications, i.e. those that
transparently use the Grid resources. It is mainly formed by an interface and
a runtime.
On the one hand, the Integrated Toolkit interface offers a simple program-
ming model which only requires the user to select the parts of the application
that will be executed on the Grid (the so-called tasks) and, optionally, to use
very few API methods.
On the other hand, the Integrated Toolkit runtime, described in detail
in 5.1.2, has the following features:

• Performance optimization of the application by exploiting its inherent


concurrency.

• Data-dependency checking at task level, automatically deciding which


tasks can be run at every moment. The best suitable applications for the
Integrated Toolkit are those with large granularity tasks.

• Task scheduling and resource selection taking into account task con-
straints and performance aspects.

5.1.1 Integration in a Generic Component Platform


The Integrated Toolkit is a part of our Generic Component-based Platform,
depicted in Figure 11. It gives support to different kinds of Grid applications,
PSEs and portals, and it builds on top of the Mediator Components and the
Service and Resource Abstraction Layer.

Grid−unaware application Grid−aware


application
PSE
integrated toolkit user portal

application tuning application−level steering


...
manager component information cache component

steering
service and resource abstraction layer
interface
security context

application resource application


information monitoring
persistence broker meta−data
services services
service services repository

Figure 11: Generic component platform

CoreGRID - Network of Excellence


CoreGRID FP6-004265 28

Although the main purpose of the Integrated Toolkit is to provide a straight-


forward programming model for Grid-unaware applications, it could also be an
alternative to develop Grid-aware applications: the componentised structure
of the Integrated Toolkit, detailed in Section 5.1.2, makes possible to use it
as a whole or to deploy solely specific subcomponents, depending on which
functionality interests the programmer.
Moreover, in order to provide the Integrated Toolkit with graphical deploy-
ment, monitoring and steering of the application, it could be used together
with the Grid IDE (Section 5.2). The runtime components of the Integrated
Toolkit could offer a set of non-functional services, corresponding to the default
controller interfaces (such as the LifeCycleController [32]) and to a customized
AutonomicController interface [18]. These services would allow the user to
reconfigure and steer the Integrated Toolkit runtime components through the
Grid IDE, as well as to obtain dynamic information about the execution of the
application.
Concerning the lower layers in the component platform, the Mediator Com-
ponents (Section 5.3) offer system-component capabilities to the Integrated
Toolkit, achieving both steering and performance adaptation. This set of com-
ponents allow to integrate Grid resources and services into one overall system
with homogeneous component interfaces, while heterogeneous software architec-
tures and technologies are situated underneath. Furthermore, the Integrated
Toolkit also relies on a Service and Resource Abstraction Layer which represents
an abstraction from the underlying Grid middleware, by means of a uniform
interface for job submission, file transfer, resource management, etc.

5.1.2 GCM-Based Design


The design of the Integrated Toolkit is based on the GCM [17] and, therefore, its
runtime is defined as a set of GCM components, each of them in charge of a given
functionality. The design, inspired on the GRID superscalar framework [9],
comprises the following components:

• Task Analyser : receives incoming tasks and detects their precedence,


building a task dependency graph. It implements the Integrated Toolkit
interface used by the application to submit tasks: when such a request
arrives, it looks for data dependencies between the new task and all previ-
ous ones. When a task has all its dependencies solved, the Task Analyser
sends it to the Task Scheduler.

• Task Scheduler : decides where to execute the dependency-free tasks re-


ceived from the Task Analyser. This decision is made accordingly to a
certain scheduling algorithm and taking into account three information
sources: first, the available Grid resources and their capabilities; second,
a set of user-defined constraints for the task; and third, the location of the
data required by the task. This component could change its scheduling
strategy on demand, thanks to the dynamic and reconfigurable features
of the GCM.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 29

• Job Manager : in charge of job submission and monitoring. It receives


the scheduled tasks from the Task Scheduler and delegates the necessary
file transfers to the File Manager. When the transfers for a task are
completed, it transforms the task into a Grid job in order to submit it
for execution on the Grid, and then controls the proper completion of the
job. It could implement some fault-tolerance mechanisms in response to
a job failure.

• File Manager : takes care of all the operations where files are involved. It is
a composite component which encompasses the File Information Provider
and the File Transfer Manager components. The former gathers all in-
formation related with files: what kind of file accesses have been done,
which versions of each file exist and where they are located. The latter is
the component that actually transfers the files from one host to another;
it also informs the File Information Provider about the new location of
files.

Task Task Job


Analyser Scheduler Manager

File Manager
File File
Information Transfer
Provider Manager

Figure 12: Design of the Integrated Toolkit runtime

The design presented above benefits from the following GCM properties:

• Hierarchical composition.

• Separation between functional interfaces, used to access the implementa-


tion of the functionalities that the component offers, and non-functional
interfaces, provided by default or custom controllers.

• Synchronous and asynchronous communications.

• Collective interactions between components (in particular, multicast com-


munications).

• ADL-based description of the component structure.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 30

5.1.3 Life-Cycle Controller Modification to Prevent Deadlocks


During the execution of the application, the Integrated Toolkit runtime can
experience errors of different kinds: a job submission that has failed, a problem
with a file transfer, an exception in some point of the code, etc. Unfortunately,
managing an error produced inside the Integrated Toolkit while it is working is
not a trivial issue. Since all its subcomponents are interconnected and commu-
nicate constantly, a failure in one of them could impede the overall system to
work properly.
The general response to such a situation should be to stop the components as
quickly as possible; however, the components that form the Integrated Toolkit
cannot be stopped in any arbitrary order because they have data dependencies.
A dependency between two components A and B appears when A invokes a
synchronous method on B and waits for its result. The problem arises if B is
stopped before it can serve the request from A; in that case, A would remain
blocked waiting for the result of the call and it could never serve the stop control
request.
If one invokes the stop method of the Integrated Toolkit life-cycle controller
(stopFc, see [32]) the call is forwarded to all the hierarchy of components in an
a priori unknown order. Nevertheless, the synchronous calls between subcom-
ponents lead to the dependencies shown in Figure 13, and such dependencies
impose a stop order that must be respected; otherwise, we could experience a
deadlock.

TA TS JM

FM
FIP FTM

Figure 13: Data dependencies between Integrated Toolkit subcomponents

In order to address this problem, a solution has been presented in [63]. It


basically redefines the behaviour of the life-cycle controller to ensure that a
structure of components is stopped in an adequate order that, in the case of
the Integrated Toolkit, is concretely the following one: Task Analyser (TA),
Task Scheduler (TS), Job Manager (JM), File Transfer Manager (FTM), File
Information Provider (FIP).
In that sense, two alternatives are proposed: on the one hand, a language
for the assembler (the person in charge of composing the system) to specify the
stopping sequence of the subcomponents of a composite; on the other, two ver-
sions of an algorithm that automatically determines the proper stopping order
for a structure of components. Both of these proposals focus on the ‘stopping’

CoreGRID - Network of Excellence


CoreGRID FP6-004265 31

receiveStop()?
started stopping sendStopToSubcomp()!

[all subcomponents are


start()? in stopped state]
sendStopToParent()!

stopped

Figure 14: Default behaviour of the life-cycle controller

state shown in Figure 14, providing means to orchestrate the internal operation
of this state. Later in this document, Section 5.3 will present a complementary
modification of the GCM component life cycle, this time extending the ‘started’
state.

5.2 Grid Integrated Development Environment - GIDE


Our vision for the GIDE is to provide the developer with a single environment
for the development pipeline. As can be seen from Figure 15 this includes
graphical composition, deployment, monitoring and steering of grid based ap-
plications.

Grid Interactive Development Environment

Obtaining
Application Program / Monitor &
Compose Deploy Solution
(Algorithm) GCM Steer

Metadata Description incl. ADL, etc.

Figure 15: Component-Based Program Development Pipeline

Our philosophy is to restrict the programmer as little as possible, and enable


the developer full access to all levels of the language hierarchy. By language
hierarchy we mean that the developer will be capable of switching between
developing graphically via a component model view, as well as coding directly
in a suitable computer language using some middleware API. Given that the
underlying component model for our platform is GCM we use Java and Eclipse
as the development platform. This enables the maximum integration with the
Integrated Toolkit (Section 5.1) and the ProActive library [61]. Eclipse is also
well known for its extensibility via the development of suitable plugins and

CoreGRID - Network of Excellence


CoreGRID FP6-004265 32

hence provides a seamless path from code to component model in a uniform


development platform. This ensures that the target user groups can benefit
from a richer set of functionalities [33].
In addition to this, deployment, monitoring and steering are also being
developed as plug-ins. Some monitoring capability is already present in the In-
teractive Control and Debugging of Distribution (IC2D) application [13] which
provides graphical monitoring of Active Objects. We are in the process of ex-
tending this functionality in order to enable the deployment and monitoring of
components. The main advantage of relying on this plug-in-based approach is
that specific features can be activated (plugged-in) on demand. Figure 16 gives
a block diagram representation of the GIDE design.

APPLICATIONS (USE CASES)

USES (Requirements) API

Eclipse
Framework Development IDE IC2D Data Centre IDE

Composition Deployment Monitoring Steering

GIDE
Toolset

ADL Code/ Node


ADL Finalization Component Install/
Parser/ ADL Test Debug Resource Start/Stop
Renderer Deployment Monitor Remove
Verifier Generator Monitor

Figure 16: GIDE Block Diagram

In designing the front-end, we followed the model driven approach using


the Graphical Modelling Framework (GMF) supported by the Eclipse platform.
The underlying architecture for the composition view relies on this model for
the functionalities. The model is well supported by a front end based on the
Graphical Editing Framework (GEF) [25] and inherited features from GEF,
such as event handlers.

5.2.1 User Groups


The Grid IDE is aimed at supporting a number of different user groups. We
can classify the user groups as follows:
Application Developers: Application developers require support for developing
Grid applications through graphical composition as well as having to support
source-code based development. This approach aligns with industrial efforts in
building applications through graphical composition [27]. However, providing
support for developing Grid applications poses additional requirements, includ-

CoreGRID - Network of Excellence


CoreGRID FP6-004265 33

ing support for Grid component models and composite components, and the
complexities of deploying these components over distributed systems. Addi-
tional tools are necessary to enable deployment, monitoring of both component
status and resource, and steering of components to maximise resource utilisa-
tion.
Application Users: The GIDE should facilitate the deployment of applications
and subsequently, the monitoring of deployed applications. The monitoring
process provides a set of opportunities for concerned users to monitor their
running application in real-time. This functionality is also shared with the
application developers who need such facilities to test the application during
development.
Data Centre Operators: Data centres have high turnover rates. Hence there is a
need for a design that would facilitate fast handovers and enable other operators
to assist newcomers in coming to terms with the applications quickly. In order
to achieve this the GIDE is also available as a standalone Rich Client Platform
(RCP) application, which provides only the key functionalities that would be
required in a data centre environment. These features are arranged within the
deployment, monitoring, and steering perspectives. Also, personalisation of
views should be limited as far as possible, so that a uniform design is visible to
all operators in order to enhance handover and communication.
The following are the four GIDE perspectives that meet these needs.

5.2.2 Composition
The composition process is enabled via a fully interactive environment. The
underpinning feature which enables such interactivity is the event driven ap-
proach. Eclipse acts as the host platform to our environment. The Graphical
Editing Framework, GMF-Runtime, and Eclipse facilitate handling of different
events within the environment. These events are captured by the host platform
through a message loop processed by the Eclipse, which are then routed to the
dedicated event handlers or providers of the environment. These event handlers
or providers are linked to the underlying model so that the changes are reflected
upon editing.
A prototype has been completed for the Composition perspective (see Fig-
ure 17). The central area focuses the user on the graphical composition view
which provides the developer with a palette of available components that can be
dragged and dropped on the composition canvas. Components can also be im-
ported from existing ADL files and stored in the component palette. ADL files
conform to the GCM-specification for describing compositions such as in [31].
Components can then be resized and moved, modified and stored. Connec-
tions between the interfaces can be drawn directly between the components
using the connection tool. Composition is achieved by drawing a membrane
around a group of components and defining interfaces. The developer is able
to switch between the graphical view, and a view of the fractal description of
the component as an ADL file. The ADL file can then be exported and used
for deployment.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 34

Figure 17: Component Composition Perspective

5.2.3 Deployment Perspective


This perspective will consist of views needed for application deployment. The
main view is of a deployment descriptor editor to map physical hosts to virtual
nodes. Deployment descriptors are used to associate components with virtual
nodes. Virtual nodes are included in ADL files to specify the number of vir-
tual nodes that the component will need. A developer may have a set of these
deployment descriptors to be used for deployment to different hardware con-
figurations. To complement this view, a view of the hosts and their resource
statuses is also provided, giving a developer the ability to associate sets of hosts
with each deployment descriptor. Within the deployment perspective the oper-
ator is able to launch components simply via drag-and-drop operations before
moving on to steering.

5.2.4 Monitoring Perspective


The monitoring perspective will provide the views that data centre operators
need in order to properly monitor the environment in which components oper-
ate. See Figure 18 for an example Monitor perspective consisting of component
and resource monitor views. Three types of monitoring are necessary in order
to enable proper management of applications. Firstly, monitoring of resources
provides the hardware status of hosts. This includes CPU utilization, hard disk

CoreGRID - Network of Excellence


CoreGRID FP6-004265 35

space, and other platform specific status information. Secondly, monitoring


of the GCM components themselves provides status and location information
along with a zoom-in feature for monitoring sub-components. Finally, we allow
monitoring of active objects, which is necessary for developers/composers to
debug and monitor applications during the development phase.

Figure 18: Monitor Perspective

5.2.5 Steering Perspective


More useful for data centre operators, the aim of the steering perspective will be
to provide views to enable the operator to start, stop and relocate components.
Building on the monitoring and host views, it has as its main focus a component
monitoring view. This view graphically shows the components location and
their status. An additional view shows the geography and resource availability
of the hosts, virtual nodes, as well as the components that are running on
them. Based on these views, the operator has the facility to start, stop or move
components from one virtual node to another while monitoring their status to
ensure correct execution.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 36

5.3 Mediator Component Framework


The mediator component toolkit has been introduced in [19]. Its goal is to
integrate system-component capabilities into application code, achieving both
steering of the application and performance adaptation by the application to
achieve the most efficient execution on the available resources offered by the
Grid.
By introducing such a set of components, resources and services in the Grid
get integrated into one overall system with homogeneous component interfaces.
The advantage of such a component system is that it abstracts from the many
software architectures and technologies used underneath. For example, in the
following, we use the term Grid services for referering to services offered within
a Grid platform, independent of the service architecture actually deployed, like
OGSA/OGSI or WSRF, or even both.
The strength of such a component-based approach is that it provides a
homogeneous set of well-defined (component-level) interfaces to and between
all software systems in a Grid platform, ranging from portals and applications,
via mediator components to the underlying system software. The mediator
components are an integral part of the generic component platform, as shown in
Figure 11. The concrete set of mediator components is considered to be flexible
and extensible to future application needs. Canonical example components are
discussed in the following.

Service and resource abstraction layer


This is the lowest layer of an application runtime stack, as shown in
Figure 11. It implements a set of component interfaces to various kinds
of Grid services and resources, like job schedulers, file systems, etc. Doing
so, it provides an abstraction layer between application components and
system components. The interfaces are supposed to be implemented by
a delegation mechanism that forwards invocations to service providers.
Examples of such runtime environments are the GAT [4], SAGA [43], or
the Java-GAT [55].

Steering interface
A dedicated part of the service and resource abstraction layer is the steer-
ing interface. This component-level interface is supposed to make appli-
cations accessible by system and user-interface components (like portals,
PSE’s, or an application manager) like any other component in the sys-
tem. Adding such an interface contributes to the complete integration of
application and system components by enabling communication in both
directions. One obvious additional use of such an interface would be a
debugging interface for Grid applications.

Application-level meta-data repository


This repository is supposed to store meta data about a specific applica-
tion, storing, e.g., timing or resource requirements from previous, related
runs. The collected information is used by other components to support
resource management (location and selection) and to optimize further

CoreGRID - Network of Excellence


CoreGRID FP6-004265 37

runs of the applications automatically. The meta data to be stored in


this repository needs to use application-dependent schemas; the crux of
its design will be to provide efficient mechanisms for storage and retrieval
without built-in knowledge about data structures. The organisation of
such a repository with namespaces for users and applications poses fur-
ther challenges. As this repository encompasses access to stable storage,
user authentication and authorization become important issues.

Application-level information cache


This component is supposed to provide a unified interface to deliver
all kinds of meta-data (e.g., from a GIS, a monitoring system, from
application-level meta data) to the application. Its purpose is twofold.
First, it is supposed to provide a unifying component interface to all data
(independent of its actual storage), including mechanisms for service and
information discovery. Second, this application-level cache is supposed to
deliver the information really fast, cutting down access times of current
implementations like Globus GIS (up to multiple seconds) to the order of
a method invocation. For the latter purpose, this component may have
to prefetch (poll) information from the various sources to provide them to
the application in time. An implementation of this component has been
described in [5].

Application steering and tuning components


Controlling and steering of applications by the user, e.g. via application
managers, user portals, and PSE’s, requires a component-level interface
to give external components access to the application. Besides the steer-
ing interface, also dedicated steering components are necessary, both for
mediating between application and system components, but also for im-
plementing pro-active steering systems, carrying their own threads of ac-
tivity. These components have to interface with Grid monitoring services
and Grid resource brokers. This way, the steering components provides
a framework for performance tuning, which can be used to improve the
execution time of applications automatically as well as for improving ser-
vices and tools which are involved in the environment. Steering and tuning
components are supposed to utilize both the application-level meta data
repository and the application-level information cache components.

Application manager component


The application manager component establishes a pro-active user inter-
face, in charge of tracking an application from submission to successful
completion. Such an application manager is in charge of guaranteeing
such successful completion in spite of temporary error conditions or per-
formance limitations. (In other settings, such an active component might
be referred to as an agent.) For performing its task, the application man-
ager needs to interoperate with most of the other mediator components,
like the application itself (via the steering interface), the application meta
data repository and cache, as well as available steering components.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 38

application steering tuning


app.− appli−
level cation
comp. comp. info. manager
cache

runtime environment

app. app.
persis− persis− compute compute
tence tence
data server server
service

appli− moni−
resource file
cation toring
meta data broker service server
repository

meta data services Grid resources

Figure 19: Mediator component classification

5.3.1 Classification of mediator components


With respect to their functionality, their implementation, and their integration
into an overall Grid component platform, the proposed mediator components
can be categorized as in the following. Fig. 19 shows this categorization of the
components and services from Fig. 11.

1. Application-level components.
This group of components is intended to be integrated directly with the
application code, and the corresponding process environment at runtime.
We distinguish two sub categories:

(a) Passive components.


These components are merely passive code, running within the appli-
cation’s process environment. They provide Grid-related functional-
ity to the application. These are the following:
• runtime environment
providing the general-purpose interface to the Grid, including
steering interface and security context (not shown in the figure)
• application manager
in a passive variant, providing a proxy to the application persis-
tence service (see below)
• application-level information cache
providing up-to-date status information about resources and their
performance

CoreGRID - Network of Excellence


CoreGRID FP6-004265 39

• tuning component
in a passive variant, using status information to tune application
performance
• steering component
providing a steering interface to the application
(b) Active components.
For pro-active functionality, active components are beneficial that
operate as individual processes, in behalf of a given application, but
independent from the application itself. Such components are typi-
cally bundled with an instance of the runtime environment to form
an individual (application) process. Examples are:
• application manager
providing external requests to an application, possibly for paus-
ing or migrating an application
• tuning component
proactively monitoring resource performance and actively deliv-
ering related events to an application
2. Service-level components.
This group of components is closely-related to Grid services that are op-
erating persistently within a Grid platform (virtual organization). These
are the following components:
• service proxy components
providing a component interface wrapper to existing services like
resource brokers or monitoring services
• application persistence service
providing a persistent service that ensures applications can run to
completion in spite of transient error conditions, without further user
interaction. An implementation of such a service has been described
in [47].
3. Meta-data components.
This group of components provides access to meta data, both of applica-
tions and of other mediator components. Likely, a single implementation
will be able to serve all necessary instances of this group.

• application persistence data repository


providing persistent status information to the application persistence
service
• application-level meta data repository
providing persistent status information to individual applications

5.3.2 GCM-based design


In the following, we will present the design by which application components and
mediator components are integrated with each other using GCM, as published

CoreGRID - Network of Excellence


CoreGRID FP6-004265 40

in [28]. For adaptation either by tuning and management components, or by the


user via the steering component, the application components have to be called
by the mediator components. For this purpose, we propose the interface shown
in Figure 20, with specialized controllers that are added to the application
components’ membrane.
Based on experience gathered when investigating the GCM component frame-
work, we propose two extensions. First, in order to effectively modify the struc-
ture of a running application, we propose to implement an explorer component.
Thus, the user could switch between different implementations of his/her algo-
rithms without the need to stop and re-run their application.
Second, the mediator toolkit can greatly benefit from implementing different
control aspects of the application separately, namely by using controllers. We
propose to introduce into the architecture the following controllers, as depicted
in Figure 20:

• steering – for modifying application parameters, which would allow for


computational steering during runtime

• persistence – for handling checkpoints: initiating checkpoints, as well as


starting (from checkpoint or from scratch) and stopping the application

• distribution – for optimal utilization of allocated resources, and for adapt-


ing to changes in environment (releasing and acquiring resources, changes
in quality of network connections)

• component – for investigating the application’s structure (in terms of com-


ponents) and modifying it (e.g. switching to alternative implementation,
replacing subcomponents)

Note that the component controller is already implemented in GCM. How-


ever, the other controllers have to be added according to the necessary func-
tionality. Another important observation is that communication with the ap-
plication is via its controllers only.

Persistence Controller and Life Cycle Controller The Persistence Con-


troller is the manager of an application instance. Not only is it responsible for
checkpointing, but also for starting (from scratch or from a checkpoint) and
stopping an application. For this to be accomplished, we propose bounding the
Persistence Controller with GCM’s LifeCycleController. The latter is a simple
state automaton (with two states: started and stopped). We propose extending
the state-cycle to service checkpointing, shown in Figure 21.
We propose to extend the started state of the component by adding substates
representing different stages of the running application (created, initialized, run-
ning, and finished), and checkpointing.
The GCM LifeCycleController is responsible for starting and stopping the
component. There are certain conditions under which a component can be
stopped. For example, all method invocations on this component should have

CoreGRID - Network of Excellence


CoreGRID FP6-004265 41

application
components
application Grid−unaware with integrated tookit or Grid−aware

steering persistence distribution component


controller controller controller controller

app−level
components

steering application tuning info cache explorer


component manager component component
toolkit

component

security context
components

application
system

PSE resource information monitoring meta−data


user portal broker system system repository

Figure 20: Generic component platforms with application controllers

STARTED

CREATED INITIALIZED
STOPPED

FINISHED RUNNING

CHECKPOINTING

Compound state of GCM LifeCycleController


State of extended LifeCycleController

Figure 21: Extended states of LifeCycleController

finished (a special interceptor keeps a counter of active method invocations).


Similarly, only a component with all mandatory interfaces bound can be started.
Our system also benefits from this approach. Transitions between stopped
and started states are limited to only a few started substates. The component
must not be allowed to stop while checkpointing is in progress. Additionally,
stopping an application in the running state could mean interrupting the ap-
plication (transition to finished) first.

Application controllers The proposed application controllers (steering – sc,


persistence – pc, distribution – dc, and component – cc) are implemented as
GCM-controllers, part of the membrane, shown in Figure 22 (left).

Dealing with GCM-aware and unaware applications The GCM Media-


tor Toolkit is ready to run not only with applications that have been developed

CoreGRID - Network of Excellence


CoreGRID FP6-004265 42

application
component

application
component sc

pc
dc cc

sc pc dc cc

Figure 22: Controllers inside the component membrane (left) or as subcompo-


nents (right).

with GCM in mind, but also with application objects (rather than components),
from “legacy” applications, as shown in Figure 23.

GCM-unaware application GCM-aware application


Default application component
Default Distribution
Controller User application component
Application manager Application manager dc
Default Persistence
Controller
dc
Application
pc
Default Componentpc
Controller
cc
Default Steering
am cc Controller am
sc

sc

Figure 23: Integrating applications with mediator controllers.

GCM-unaware applications The framework is able to cooparate with ap-


plications that do not use the GCM framework. In that case, a set of default
controllers is created. A user application is encapsulated by the default persis-
tence controller as this component is responsible for starting and stopping the
application, and it has direct access to the application object. This controller,
together with the default distribution, component, and steering controllers, are
integrated to a default application component, which is bound to the rest of
the framework via the application manager.
The default implementations of controllers are very simple. Only the persis-
tence controller is able to perform some actions – starting and terminating the
application. All other methods in this and the remaining controllers throw a not
implemented exception, as they cannot be provided without specific knowledge
about the application.

GCM-aware applications These are very easy to connect to the framework.


The only requirement towards the application developer is to deliver a GCM

CoreGRID - Network of Excellence


CoreGRID FP6-004265 43

component (user application component, see Figure 22, right) with exported
interfaces for each of the controllers (dc, pc, cc, and sc). Internally, they are
expected to be bound to the user’s implementation of the controllers.

5.4 gUSE - A Service based Environment for Grid Application


Developers and End Users
gUSE (Grid User Support Environment) is a flexible and scalable Grid appli-
cation development and enactor infrastructure that provides user environment
on top of widely adopted, production level scientific e-infrastructures. gUSE
enables the creation, testing, execution, monitoring and sharing of distributed
applications through intuitive, graphical interfaces. Grid application develop-
ers can apply gUSE to design complex calculations on top of Web service and
Grid service infrastructures, end users can apply gUSE to select pre-defined
calculations and execute them with their own custom parameters. Grid sys-
tem administrators can benefit from the loosely-coupled gUSE architecture to
deliver stable, scalable and flexible services for Grid and Web service users.

5.4.1 gUSE as a Grid application developer environment


Application developers use gUSE to compose complex applications from ele-
mentary Web services and batch jobs. Such calculations are represented as
data-driven workflows, where the nodes of a workflow are able to perform an
elementary (or complex) computation, while the connections between the nodes
represent data transfer between these algorithms.

• Grid Application Developers can connect batch programs with Web ser-
vices into data-driven workflows that perform complex calculations based
on elementary computational steps.
• Workflow components can be Web services, or jobs that are able to run
as batch computation on Globus Toolkit 2, Globus Toolkit 4 or gLite
service hosts. Short computations can be executed locally on a gUSE
server, avoiding communication with remote host and resulting better
performance.
• Any part of a workflow or even the whole workflow can be executed mul-
tiple times in a parameter study fashion in order to perform the same
computation for input different parameters.
• Applications can be nested into each other, realizing hierarchical appli-
cation structures in which any component of a workflow can also be the
same workflow.
• Workflows can be combined with collector components that perform some
collective evaluation of the partial results that are generated by the various
workflow instances.
• Workflows can be combined with generator components that create input
parameters automatically for parameter study simulations.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 44

• A Grid application can be fed with input parameters from files or from
databases that are accessible through JDBC interfaces (Java Database
Connectivity).

• Additional support is to produce reusable Grid Applications: a grid ap-


plication can be abstracted at the level of a graph, at the level of concrete
workflows and at the level of a workflow instance. A graph defines the
graph topology that can be extended to a concrete workflow by specify-
ing services or grid jobs to it. A workflow instance includes all the input
parameters and it is mapped to Grid resources for execution. Reusabil-
ity provides flexible inheritance between applications from the different
abstract levels.

• Grid applications can be executed, tested and monitored to result a well


tested application that can be offered to end users.

5.4.2 gUSE as an end user environment


• Built-on application repositories can be used to load pre-defined, well
tested applications that are ready for execution.

• Any parameter of a pre-defined application can be fixed, or can be left


open. Open parameters can be changed by end users, fixed parameters
cannot.

• The execution of an application can be monitored and visualized by graph-


ical interfaces.

• End user can be notified in email about relevant events related to the
progress of the Grid application.

• The start of a user application can be initiated manually through the


gUSE GUI, or by a remote process, accessing the gUSE application repos-
itory with a Web service call.

5.4.3 gUSE as a scalable, fault tolerant layer to ease Grid system


management
• gUSE is implemented as a set of services, each of them playing a well
defined role in the Grid application lifecycle . e.g. file storage, work-
flow management or job submission. These Web services discover each
other through a central gUSE information system and communicate using
SOAP messages. Any service can be replaced by another implementation,
any service-to-service communication protocol can be re-implemented.

• gUSE services extend the basic but generic functionality of Globus Toolkit
2, Globus Toolkit 4, gLite and Web service networks with functions spe-
cialized for application developers, application end users and their collab-
oration.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 45

• Any member of the gUSE service set can be stopped and restarted without
stopping the whole user environment. Any gUSE service can be dupli-
cated, can be started multiple times to avoid that function becoming a
performance bottleneck.
• WS-PGRADE (Web Services Parallel Grid Runtime and Application De-
veloper Environment) is the graphical services through which the rest of
the gUSE service stack can be utilized. WS-PGRADE is one implemen-
tation and can be replaced by specialized user interfaces if necessary.
• gUSE provides APIs to easily interface the system to other Grid systems
or distributed computing platforms. Such an additional service can be
for example an interface submitter service to utilize a Grid or cluster
job manager, or it can be a higher level service that e.g. load balances
between existing gUSE job submitter services.

5.4.4 gUSE system architecture


gUSE is a set of Web services that bind together in flexible ways to deliver user
services in Grid and/or Web services environments. User interfaces for gUSE
services are provided by the WS-PGRADE Web application (See Figure 24).
WS-PGRADE is a Web portal hosted in Gridsphere framework. WS-PGRADE
uses the client APIs of gUSE services to turn user requests into sequences
of gUSE specific Web service calls. WS-PGRADE hides the communication
protocols and sequences behind JSR168 compliant portlets.

Figure 24: The three tiers of the gUSE architecture

End users can access WS-PGRADE via Web browsers. A graph editor
component can be downloaded from WS-PGRADE via the browser to the user

CoreGRID - Network of Excellence


CoreGRID FP6-004265 46

machine. The editor can be used to define the static skeleton of workflows, while
the HTML pages of WS-PGRADE provide interfaces to add content to graphs,
to generate complete Grid/Web service applications. The middle tier of the
architecture constitutes of gUSE services. gUse services can be hosted on one
ore more server machines in order to optimize resource usage or end-to-end per-
formance. gUSE services provide data services for users (user database, proxy
credential database, application code database, workflow database, application
history and change log database, etc.), and control services in the system (e.g.
job submission, workflow enactment, etc.). These services discover each other
using a special gUSE service, the information system.
The lowest level of the architecture is the services of Web service, Grid
service networks. Grid services typically provide job execution, large scale file
storage and brokering services in a secured way, while Web services make pre-
defined computational algorithms accessible via SOAP messages. gUSE services
connect to this tier to perform computation, data management on behalf of the
users.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 47

6 Autonomic Support
6.1 Autonomic support of non functional features in GCM
The development of efficient component based Grid applications raises some
problems that are specifically related to the dynamic features of the Grid target
architectures. In particular, programmers must deal with a full set of problems
that are not directly related to the functionality of the application but rather
to non-functional aspects, i.e. aspects that affect how the final results are com-
puted rather than what results are actually computed. Such problems include
performance tuning, security and fault tolerance, just to name a few. All these
“general” problem classes, in turn, include other, more specific problems. As
an example, performance problems include proper parallel activity scheduling
and mapping, load balancing, performance tuning and, last but not least, adap-
tation of program configuration to varying features of the target architecture.
In this section we present some of the results related to component autonomic
management of non functional properties. We first introduce autonomic com-
ponents, then we discuss the techniques used to introduce autonomic behaviour
within components and eventually we discuss results achieved with autonomic
GCM components so far.

6.1.1 Autonomic components


Autonomic components are components that autonomously take care of some
non functional properties aimed at achieving some specific goal. Here we con-
sider a particular subset of autonomic components, namely:

• structured, parallel composite components, that is components conform-


ing to the Fractal [32] and GCM [17] specifications that are hierarchically
built from other, simpler components, in such a way these primitive com-
ponents are assembled according to a well known parallel pattern in a
new composite component,

• whose autonomic control takes care of performance optimization, i.e. whose


autonomic control takes care of all the aspects related to efficiency in com-
posite component execution and, in particular, to the aspects related to
dynamic adaptation of execution to the target architecture features.

As described in [3], we call this kind of components behavioural skeletons.


The term borrows the skeleton keyword from the algorithmic skeleton research
track, as the composite components we are taking into account structure the
inner components of the composite according to well known (parallel) algorith-
mic skeletons. The adjective behavioural reflects the fact autonomic control
co-designed with the skeleton structure of the composite adapts behaviour of
the composite to the “local” (in temporal and spatial dimension) properties of
the target architecture.
Figure 25 shows a functional replication behavioural skeleton. W are the
inner component instances, S represents the port collecting tasks from the the
outside and dispatching them to the W s and C represents the port collecting

CoreGRID - Network of Excellence


CoreGRID FP6-004265 48

(partial) results from the W s and delivering them for further processing to other
components.
An example of behavioural skeleton is the one modelling embarrassingly
parallel computations, that is computations where a set of independent input
tasks (data sets) are processed by the same application code to produce a(n
ordered) set output data sets. This kind of parallel pattern is well know and
largely used in grids to perform image processing, parameter sweeping, business
data filtering, etc. In this task farm behavioural skeleton:

• the composite component hosts a number of identical inner components,


each computing the application producing output data from an input task
data set

• the composite component hosts an autonomic manager component, tak-


ing care of the non functional aspect related to the performance and effi-
ciency achieved while computing the embarrassingly parallel computation

• the composite component hosts a passive autonomic controller, providing


methods to access composite component status and performance parame-
ters and measures, as well as methods implementing the actions eventually
invoked by the autonomic manager

• the composite component provides interfaces to submit tasks to be com-


puted and to gather results (functional interface) as well as interfaces to
interact with the autonomic manager and with the autonomic controller
(non functional interfaces).

The autonomic manager of the task farm skeleton implements a classical


autonomic control cycle, that can be summarized as follows:

ac.1 performance parameters are collected exploiting the autonomic controller


interfaces

ac.2 performance figures are analysed. In particular they are compared to


theoretical figures and, in case of mismatch, corrective actions are planned
according to one of pre-defined strategies programmed in the manager and
specific of the parallel computation pattern modelled by the behavioural
skeleton

ac.3 the corrective actions are executed exploiting again the interfaces provided
by the autonomic controller, and the cycle is restarted

Sample autonomic policies implemented in the task farm autonomic man-


ager include the possibility to increase the number of instances of the inner
component when computation requests arrive more frequently than actual re-
sult dispatching as well as substitution of non responding inner component
instances (e.g. due to network faults or overloads).

CoreGRID - Network of Excellence


CoreGRID FP6-004265 49

AC
AM

Functional Functional
server port
S W C client port

W
skeleton
behaviour
(e.g. ORC)

Figure 25: Generic functional replication GCM behavioural skeleton

6.1.2 Autonomic behaviour support in GCM


We discuss how the behavioural skeletons are implemented in GCM by de-
tailing the implementation of the task farm behavioural skeleton. Task farm
behavioural skeleton has been developed as a (prototype) representative of the
full class of behavioural skeletons, actually. Its implementation has been cus-
tomized from classes of a more general functional replication skeleton that can
be specialized to implement data parallel (stateless and stateful) behavioural
skeletons as well. Within our GCM task farm behavioural skeleton template,
the features discussed in Sec. 6.1.1 above are implemented as follows:
• Autonomic controller is a plain Fractal controller, namely the Autonomic
Behavioural Controller (ABC). The ABC provides the composite task
farm component some interfaces:

1. accessing service time and interarrival time of the composite


2. adding an inner component instance (a worker)
3. removing an inner component instance

• Autonomic manager (AM) is implemented currently as an (active) object.


We have plans to provide it as a full fledged Fractal membrane component
in the future. It provides the composite component the interfaces needed
to:

1. start/stop autonomic manager activity


2. provide/substitute autonomic management policies

Autonomic policies are actually currently implemented directly program-


ming Java methods in the manager object. We already have a prototype
implementation of the managers with policies expressed via “business

CoreGRID - Network of Excellence


CoreGRID FP6-004265 50

rules” [42] and the prototype is currently being integrated in the main
GCM autonomic controllers. In the prototype, rule based implementa-
tion of GCM autonomic managers we used JBoss rules. Such rules look
like the following:

rule "AdaptUsageFactor"
when $workerBean: WorkpoolBean(serviceTime > 0.25)
then $workerBean.addWorkerToNode("");
end

AdaptUsageFactor is the name of the rule, the when clause defines the pre-
conditions activating the rule and the then clause defines the consequences
of (or actions related to) rule activation.
$workerBean: WorkpoolBean(serviceTime > 0.25) and
$workerBean.addWorkerToNode("") represent accesses to the ABC con-
troller, the former being an access to the “getter” interface providing
service time measure, and the latter being an access to the “actuator”
ABC interface adding an inner component instance to the task farm.

6.1.3 Experimental results


We used behavioural skeletons (that is GCM autonomic components) to run
several different applications on several distributed target architectures, ex-
ploiting the ProActive based GCM reference implementation currently being
developed within the EU STREP project GridCOMP. The target architectures
considered range from network of laptops and PCs to workstation clusters (in-
terconnected through Fast Ethernet) and the Grid5000 architecture. We run
simple, synthetic applications and more complex application as well, mainly
coming from GridCOMP use cases workpackage. With all the applications we
have been able to demonstrate:

• scalability of the behavioural skeletons. Medium to coarse grain applica-


tions have been demonstrated to scale on all the architectures considered.
Efficiency is close to the ideal one in case the computational grain of the
inner worker components is suitably coarse.

• behavioural skeletons timely react to increases in task submission speed


by augmenting the instances of the inner worker components used. Re-
configuration time (both in case of stateless computations, such as in the
task farm behavioural skeleton and in case of stateful computations, such
as in the stateful data parallel behavioural skeletons) is negligible w.r.t.
improvements achieved in the application service and completion time.
In case submission speed decreases, the manager arranges to release one
or more of the inner component instances in such a way efficiency is kept
close to the ideal one.

• behavioural skeletons timely react to changes in the load of the resources


used to run the composite component. When additional charge is put

CoreGRID - Network of Excellence


CoreGRID FP6-004265 51

on the used resources, the manager detects a mismatch among the pre-
dicted performance and the actually achieved values and suddenly starts
a reconfiguration process. As in the case of interarrival time changes, if
resources previously overloaded become more and more free, inner com-
ponent instances are deallocated to increase efficiency.

Figure 26 plots scalability (upper part) and reconfiguration behaviour (lower


part) of the rule based implementation of the GCM task farm behavioural
skeleton. The lower plot shows the effect of the intervention of the manager
that decided to double the number of instances of the inner components after
computing half of the total tasks submitted. This is quite an “artificial” policy
(JBoss rule) we used to show the overhead of reconfiguration (in this case
lower than 5% w.r.t. to the total execution time. The implementation of the
GCM component in this case is a prototype directly implemented on top of
SCA/Tuscany and the whole GCM behavioural skeleton is provided as a plain
web service accessible simply exploiting his WSDL to submit tasks.

6.1.4 Specific contributions to STE institute


The design of autonomic components is currently going to be integrated in the
design of IDE presented in Sec. 5.2. Specific widgets and tools will be eventu-
ally available in the IDE to allow autonomic component users to customize or
even rewrite from scratch the policies and mechanisms provided by both ABC
controller and the AM manager. Overall, this will provide user friendly pos-
sibilities to exploit autonomic composites (behavioural skeletons). It is worth
pointing out that in principle such notable composite components and their
internal autonomic management of performance features is completely trans-
parent to the final users, that only perceive some additional (possibly ignored)
non functional interfaces w.r.t. equivalent skeleton components not providing
autonomic managers. Overall, this allows to provide limited but effective tool
support within the IDE.
The autonomic component development also defines a design and imple-
mentation methodology that has influenced other research topics in the STE
institute, such as mediator components, for instance.

6.2 Automation of Application Deployment on Grids


A generic component-based grid platform faces the issue of deploying its ele-
ments. Here, the deployment operation involves all the phases implied from a
high level order such as deploy a set of components: translating an appli-
cation specific description to a system level description, finding and acquiring
suitable resources, transferring required files, launching processes if needed,
and configuring what is needed. Hence, a deployment mediator is a solution
to abstract the grid platform from the complex process of actually deploying
components. Such a mediator provides an automatic service of application
deployment because it should not rely on any human intervention [19].
From the point of view of the execution, a component contains a struc-
tured set of binary executables and requirements for their instantiation. The

CoreGRID - Network of Excellence


CoreGRID FP6-004265 52

1400 800
Measured (coarse grain)
Completion time (secs) (coarse grain)

Ideal (coarse grain)

Completion time (secs) (fine grain)


1200 Measured (fine grain) 700
Ideal (fine grain)
600
1000
500
800
400
600
300
400
200

200 100

0 0
2 4 6 8 10 12 14 16
#worker

9
about 170 secs for 1/2 tasks with
8 N=4 inner component instances

6
#worker

3 Reconfiguration
2 takes place here
(workers doubled) about 85 secs for 1/2 tasks with
1 K=2N inner component instances
0
0 50 100 150 200 250 300
Computation time (secs)

Figure 26: Scalability and reconfiguration effect of GCM task farm behavioural
skeleton

CoreGRID - Network of Excellence


CoreGRID FP6-004265 53

Figure 27: Activities involved in the deployment process of an application.

objectives include generating deployment plans a) to deploy components in


a multi-middleware environment, b) to dynamically alter a previous configu-
ration, adding new computational resources to a running application, c) for
re-deployment, when a complete restart from a previous checkpoint is needed
(severe performance degradation or failure of several resources).
A framework for the automatic execution of applications can be composed
of several interacting entities in charge of distinct activities, as depicted in
Figure 27. The logical order of the activities is fixed (Submission, Discovery,
Selection, Planning, Enactment, Execution). Some steps have to be re-executed
when the application configuration is changed at run-time. Moreover, the steps
in the grey box, that interact closely, can be iterated until a suitable set of
resources is found.
In the following we describe the activities involved in the deployment of an
application on a Grid. We also detail the inputs that must be provided by the
user or the deployment framework to perform such activities.

6.2.1 Application Submission


The process takes into input the information necessary to drive the following
phases. This information is provided through a (logical) file containing a de-
scription of the components of the application, of their interactions, and of the
required resource characteristics.

Application Description The description of (the components of) the sub-


mitted application, written in an application specific language, is composed of
various kinds of data. First, the module description deals with the executable
files, I/O data and configuration files which make up each module (e.g. each
process). Second, there is information to guide the stages related to mapping
the application onto resources, like the resource constraints – characteristics
that Grid resources (computational, storage, network) must possess to execute
the application, the execution platform constraints – software (libraries,
middleware systems) that must be installed to satisfy application dependencies,
the placement policies – restrictions or hints for the placement of subsets of
application processes (e.g. co-location, location within a specific network do-
main, or network performance requirements), and the resource ranking – an

CoreGRID - Network of Excellence


CoreGRID FP6-004265 54

objective function provided by the user, stating the optimization goal of appli-
cation mapping. Resource ranking is exploited to select the best resource, or set
of them, among those satisfying the given requirements for a single application
process. Resource constraints can be expressed as unitary requirements, that is
requirements that must be respected by a single module or resource (e.g. CPU
rating), and as aggregate requirements, i.e., requirements that a set of resources
or a module group must respect at the same time (e.g. all the resources on the
same LAN, access to a shared file system); some placement policies are implic-
itly aggregate requirements. Third, the deployment directives determine the
tasks that must be performed to set up the application runtime environment,
and to start the actual execution.
If an application is built from existing components and services, each with
its own description (metadata), then there should be no need for additional
application description provided as input from the application developer or
user unless this is needed for the composition phase prior to deployment.
As discussed in the following subsections, the provided application descrip-
tion information is used throughout the deployment process.

6.2.2 Resource Discovery


This activity is aimed at finding the resources compatible with the execution
of the application. In the application description several requirements can be
specified that available resources must respect to be eligible for execution. The
requirements can specify hardware characteristics (e.g. CPU rating, available
memory, disk space), software ones (e.g. OS, libraries, compilers, runtime en-
vironments), services needed to deploy components (e.g. accessible TCP ports,
specific file transfer protocols), and particular execution services (e.g. to con-
figure the application execution environment).
Resources satisfying unitary requirements can be discovered, interacting
with Grid Information Services (GIS). Then, the information needed to per-
form resource selection (that considers also aggregate requirements), must be
collected, for each suitable resource found.
The GIS can be composed of various software systems, implementing infor-
mation providers that communicate with different protocols (MDS-2, MDS-3,
MDS-4, NWS, iGrid, custom). Some of these systems provide only static in-
formation, while others can report dynamic information about resource state
and performance, including network topology and characteristics. In order to
interact with such different entities, an intermediate translation layer between
the requirements needed by the user and the information provided is necessary.
Information retrieved from different sources is mapped to a standard schema
for resource description that can be exploited in the following activities inde-
pendently from the information source.

6.2.3 Resource Selection


When information about available resources is collected, the proper resources
that will host the execution of the application must be selected, and the dif-

CoreGRID - Network of Excellence


CoreGRID FP6-004265 55

ferent parts of each component have to be mapped on some of the selected


resources. This activity also implies satisfying all the aggregate requirements
within the application. Thus, repeated interaction with the resource discovery
mechanisms may be needed to find the best set of resources, also exploiting
dynamic information.
At this point, the objective function must be evaluated against the char-
acteristics and available services of the resources (expressed in the normalized
resource description schema), establishing a resource ranking where appropriate
in order to find a suitable solution.

6.2.4 Deployment Planning


A component-based application can require different services installed on the
selected resources to host its execution. Moreover, additional services can be
transferred/activated on the resources or configured to set up the hosting envi-
ronment.
Each of these ancillary applications has a well-defined deployment schema,
that describes the workflow of actions needed to set up the hosting environment
before the actual execution can start.
After resource selection, an abstract deployment plan is computed by gath-
ering the deployment schema of all application modules. The abstract plan is
then mapped on the resources, and turned into a concrete plan, identifying all
the services and protocols that will be exploited in the next phase on each re-
source, in order to set up and start the runtime environment of the application.
For example, to transfer files a protocol (e.g. HTTP, GridFTP) must be
selected, the related services and resources must be started or configured, and
finally the transfer can occur. At the end of this phase, the concrete deployment
plan has been generated, specifying every single task to perform to deploy the
application.
This activity can require repeated interactions with the resource discovery
and selection phases because some problems about the transformation from the
deployment schema to the deployment plan can arise, thus the elimination of
one or more eligible resources can force to find new resources, and restart the
whole planning process.

6.2.5 Deployment Enactment


The concrete deployment plan developed in the previous phase is submitted to
the execution framework, which is in charge of the execution of the tasks needed
to deploy the application. This service must ensure a correct execution of the
deployment tasks while respecting the precedences described in the deployment
plan. At the end of this phase, the execution environment of the application
must be ready to start the actual execution.
This activity must deal with different kinds of software and middleware
systems; the selection of the right ones depends on the concrete deployment
plan. The implementation of the services that will perform this activity must

CoreGRID - Network of Excellence


CoreGRID FP6-004265 56

be flexible enough to implement the functionalities to interact with different


services, as well as to add mechanisms to deal with new services.
Changes in the state of the resources can force a new deployment plan for
some tasks. Hence, this phase can require interactions with the previous one.

6.2.6 Application Execution


The deployment process for adaptive Grid applications does not finish when
the application is started. Several activities have to be performed while the
application is active, and actually the deployment system must rely on at least
one permanent process or daemon. The whole application life-cycle must be
managed, in order to support new resource requests for application adaptation,
to schedule a restart if an application failure is detected, and to release resources
when the normal termination is reached. These monitoring and controlling
activities have to be mediated by the deployment support (actual mechanisms
depend on the middleware).

6.2.7 Validation
The deployment process presented in this section has been implemented in two
deployment systems GEA and Adage. In the ASSIST/Grid.it architecture the
Grid Abstract Machine (GAM, [2]) is a software level providing the abstrac-
tions of security mechanisms, resource discovery, resource selection, (secure)
data and code staging and execution. The Grid Execution Agent (GEA, [22])
is the tool to run complex component-based Grid applications, and actually im-
plements part of the GAM. Adage [48] (Automatic Deployment of Applications
in a Grid Environment) is a research project that aims at studying the deploy-
ment issues related to multi-technology applications. One of its originality is
to use a generic application description model (GADe) [49] to handle several
middleware systems.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 57

7 Security Issues in Component-Based Grid Systems


In this section we describe our research towards the analysis of security-related
issues in component-based grid systems. After introducing important concepts
and issues, we give overview of the existing Grid-related security infrastructures
and we discuss their applicability to the component-based systems.

7.1 Security Issues


When analyzing the security issues in component-based grid systems, we focus
on a simplified scenario of such system usage. As shown in Fig. 28, we have
the component containers running on remote and shared resources, upon which
the components are deployed. The distributed component application requires
communication not only between the client and components, but it also involves
direct connection between components. Therefore the following issues have to
be taken into account when analyzing the security model for such applications.

• Authentication – answers the questions how to verify the identity of the


communicating parties. Another requirement is the Single Sign-On (SSO),
to relieve the users from management of multiple logins and passwords to
access multiple resources.

• Authorization – answers the question what is permitted and which


should be denied. In such scenarios as dynamic deployment of component
code on shared containers, it may involve potentially complex policies, as
well as distributed policy management.

• Delegation – answers the question how can a code running in Component


A authenticate itself to Component B on behalf of a client. The client
credentials should be thus delegated, but without the possibility of losing
the control over the delegated rights.

• Communication security – in distributed and often Internet-wide ap-


plications, such issues as message integrity and privacy should be ad-
dressed, using proper cryptography solutions.

• Sandboxing – answers the question how to protect the code running


on shared (not owned!) computational resources from interfering with
the others. This issue is especially important in the case of dynamic
deployment of component into shared containers.

• Virtual organization building – this is related to community-based


management of access policies, possibly dynamically evolving partnerships
and contracts.

.
The above mentioned issues have a lot in common with problems related
to running distributed application on the Grid, such as e.g. MPI or Service-
oriented architectures. Component model, however, due to the possibility of

CoreGRID - Network of Excellence


CoreGRID FP6-004265 58

Figure 28: A simplified view on a component-based application

dynamic deployment of applications onto shared resources, requires special han-


dling and combining the solutions known from Grid-enabled MPI or Web Ser-
vices. Below we discuss how existing technologies known from the Grid and
Web areas can be effectively used for assuring the security in component-based
systems.

7.2 Existing security infrastructures


Having discussed the issues with component-based systems, we give now the
overview of existing solutions and discuss their applicability to the component-
based world. We have analyzed GSI and Shibboleth standards, as well as the
capabilities offered by H2O system which is a base for MOCCA component
framework.

7.2.1 Grid Security Infrastructure


GSI [67] was developed as a part of Globus project. It is based on Public Key
Infrastructure (PKI) using X.509 certificates. GSI implements extensions for
proxy certificates, which constitute short-lived credentials allowing single sign-
on and delegation. GSI supports such features as authentication, authorization,
transport security based on SSL, and WS-Security for Web Service based ap-
plications. MyProxy credential management system can be used for portal
integration, facilitating usage of the Grid for users who prefer not to manage
their certificates. Extensions of GSI include CAS and VOMS systems for vir-
tual organization management, providing groups, roles or attributed for more
scalable policy management. They provide community servers for managing
lists of users and their roles within VO.
The advantage of GSI comes from its wide deployment on production Grid
infrastructures (e.g. EGEE, NGS, TeraGrid, etc.). The proxy certificate del-
egation mechanism has been widely used in parallel-distributed applications,
including MPICH-G2 and it can be directly used to provide delegation of cre-
dentials for inter-component connections. The problems related to GSI come
from the requirement that the users need to manage their personal certificates,
which is the cumbersome and often created a entry barrier for beginners.
As a feasibility study of using GSI for component-based Grid applications,

CoreGRID - Network of Excellence


CoreGRID FP6-004265 59

Figure 29: Using Shibboleth to access components on the Grid

we have build a prototype support for GSI into MOCCA [52] framework. It
extends the H2O container used by MOCCA with a GSI-based authenticator
module. As a result, the access to MOCCA framework can be granted only to
such clients who can provide a valid proxy certificate which is listed in registered
users database.

7.2.2 Shibboleth
Shibboleth [62] is a federated Web Single Sign-On framework based on SAML
(Security Assertion Markup Language). It is an attribute exchange framework,
open source, developed by Internet2 consortium. In contrast to GSI it re-
quires no user certificates, since all login requests are redirected to user’s home
organization. The SSO is achieved by letting users to use their home organi-
zation logins and passwords to access remote resources. Shibboleth features
attribute-based access control and by mutual agreements between participating
institutions it allows decentralized building of virtual organizations. Shibboleth
is used mainly for integrating Web resources of educational institutions, how-
ever it is the subject of interest for Grid research to overcome problems with
user certificates which are required by GSI.
The proposal of integrating Shibboleth with component container is shown
in Fig. 29. The client contacts the component container, acting as Shibboleth
resource provider. Service provider (component container) redirects the client
to IdP (Identity Provider). The IdP returns a handle (an authentication token)
after successful authentication. Then, the SP requests attributes based on the
handle. The authorization is performed based on attributes. In such scenario
all SPs and IdPs must trust each other.
The problem with the above described scenario is that Shibboleth does
not provide credential delegation mechanism, as it is implemented in GSI.
Also, standard IdPs are prepared to interact with clients which are using Web-
browsers. To overcome that limitation, we have developed a standalone IdP

CoreGRID - Network of Excellence


CoreGRID FP6-004265 60

client, which is a library that can be used to obtain Shibboleth credentials from
any application. The prototype of component container which will act as a
Shibboleth service provider is under development within MOCCA framework.

7.3 Security Issues – Conclusion


In this section we have described briefly our experience with the security-related
issues which can be met when discussing component-based Grid systems. Our
analysis of the issues such as authentication, single sign-on, credential delegation
and VO-management applies to all component-based systems, including those
based on GCM, CCA and others, since all distributed component frameworks
for Grid programming face similar challenges from the security point of view.
In addition to such infrastructures as GSI or Shibboleth, we have to men-
tion that Grid component frameworks such as ProActive or MOCCA are based
on Java platform-based security solutions, providing such features as security
managers, code execution sandboxing based on classloader separation, and con-
figurable security policies. Distributed policy management and decision infras-
tructures such as PERMIS and XACML-based solutions are also the subject of
current research.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 61

8 Use Cases – Overview and Examples


8.1 Introduction
It is generally accepted that component-based software development is becom-
ing the most cost-effective approach to application construction for complex
distributed and Grid systems. This, however, makes the search for the most
appropriate programming model and corresponding programming environments
even more important than before. Arguably the most serious obstacle to the
acceptance of modern component technologies is the so-called software crisis.
Application software, in general, is considered the most complex artefact in
distributed computing; since the lifespan of Grid infrastructures has been so
brief, their software environments rarely reach maturity making the software
crisis especially acute. Hence, portability and support for dynamic properties,
in particular, are critical issues in enabling large production applications for
Grid computing systems. The main aim of this section is to present our ex-
perience in applying component-based development using different approaches
depending on the status and the properties of different application codes [35].

8.2 Wrapping legacy software


Giving high level of attention and support to legacy applications by providing
enabling approaches and tools for their seamless integration into state-of-the-
art component-oriented systems is a high priority issue. Depending on the
properties and the development status of different legacy codes two approaches
can be considered automatically generated wrapping [34], and hand-coded
wrapping [65]. In both cases the wrapper development process can significantly
be simplified by adopting the most efficient strategy and using various tools for
reducing the time for development. An important aspect of this strategy is to
guarantee relatively small overhead introduced by the wrapper software layer.

8.2.1 GENIE Overview


GENIE (Grid ENabled Integrated Earth system modelling) [37] is a scalable
modular platform used to produce and simulate a range of Earth system models
in order to understand the long-term changes in Earth’s climate. The framework
includes different variants of earth modules such as the atmosphere, ocean,
sea-ice, marine sediments, land surface, vegetation and soil and ice sheets. A
typical Earth system model is a composition of different variants of these earth
modules (or different instances of earth modules), which are in fact different
configurations suited for different simulation scenarios. Each Earth module is
bound to the natural laws of physics that govern the exchange of energy, fluxes,
water and other physical or biogeochemical tracers. Further, each Earth module
uses their own module-specific computational models and module-specific data
representations for representing boundary surfaces where the exchanges occur.
Figure 30 shows a sample Earth system model (or configuration). The
model uses different instances of each of the Earth system modules (instance
types are shown within parenthesises). Interactions between different (instances

CoreGRID - Network of Excellence


CoreGRID FP6-004265 62

Atmosphere (EMBM)

Surface Flux

Sea−Ice (Goldstein)

Land (EMBM)

Ocean (Goldstein)

Figure 30: A sample configuration representing an Earth system model

of) Earth modules are shown by arrows. In addition to Earth system modules,
a configuration may also use modules to represent exchanged or accumulated
physical or biogeochemical tracers (for example, Surface flux in the Figure).

8.2.2 Component-based version of GENIE


In the original implementation, the GENIE was implemented in pure FOR-
TRAN without any explicit notion of components and services. The frame-
work included a driver module for orchestrating the execution, inter- and intra-
module communications and data exchanges for a given configuration.
At present, these earth-modules have been componentised by wrapping the
legacy code into software components. To facilitate inter-operatability between
different language-specific implementations and to expose these components on
the Web-Services front, these components have been wrapped through Java in-
terfaces [58]. With these wrappers in place, the overall composition of these
components (and thus different earth system models) may include remote com-
ponents and remote services.
In the context of Grid, problem solving environments or problem solving
portals are finding broader range of audience including scientists. Evolving
GENIE as a, or as a part of a portal is an interesting challenge. This includes
creating component repositories, building automated submission, management,
scheduling and execution mechanisms and building/supporting visualisation
platforms. In addressing this challenge, although we could potentially make
use of existing techniques to handle different issues which may arise within,
there are still issues that need to be addressed.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 63

8.3 Componentizing existing applications


This approach consists of a general componentisation process in order to trans-
form an object-based system into a component-based system. The process
assumes that the target component platform allows connecting components
via provided and required interfaces, and that it minimally supports the same
communication styles as the object platform (e.g., remote method invocation,
streams, and events) [60].

8.3.1 Jem3D overview


Jem3D is a numerical solver for the 3D Maxwell’s equations modelling the time
domain propagation of electromagnetic waves [10]. It relies on a finite volume
approximation method operating on unstructured tetrahedral meshes. At each
time step, the method evaluates flux balances as the combination of elementary
fluxes computed through the four facets of a tetrahedron. The complexity of
the calculation can be changed by modifying the number of tetrahedra in the
domain. This is done through setting the mesh size; i.e., the triplet (m1 × m2 ×
m3) that specifies the number of points on the x, y, and z axes used for building
the mesh. Parallelisation relies on dividing the computational domain into a
number of subdomains; the domain division is controlled by another triplet
(d1 × d2 × d3) that determines the number of subdomains on each axis. Since
some facets are located on the boundary between subdomains, neighbouring
subdomains must communicate to compute the values of those border facets.
The original Jem3D builds on the ProActive library.

8.3.2 Componentising Jem3D


Jem3D was componentised using the approach presented in [60]. Most of the
effort was spent on the architecture recovery activity because of the undocu-
mented and degraded structure of the system. The run-time view of the original
architecture was described using UML object diagrams and UML interaction
diagrams. During the component architecture design, the launcher entity (an
executing Java program) was decomposed into a subdomain factory compo-
nent and an activator component; the former is assigned the responsibilities
for creating, initialising, and connecting the subdomains, while the latter the
responsibilities for obtaining the input data, passing them to the factory, and
starting the computation. The reason for the decomposition was to make the
factory reusable beyond Jem3D. A later iteration of the activity grouped the
factory and the subdomains into a composite domain component, exploiting
the hierarchical composition feature of Fractal/ProActive. Implementing the
interface-based version served to increase confidence in the new component ar-
chitecture and drastically simplified the final component-based implementation.
The component-based implementation involved wrapping classes to form Frac-
tal components and replacing a large part of the injector logic with Fractal ADL
descriptions, as seen next.
Figure 31 shows the static structure of the resulting component-based Jem3D
using a UML component diagram (multicast interfaces are represented as stereo-

CoreGRID - Network of Excellence


CoreGRID FP6-004265 64

*
Steering
Agent

MainCollector Activator

Domain

*
:SubDomain
Factory
:SubDomain[*]
*

Multicast interfaces
Client
Server

Figure 31: Component-based Jem3D structure

typed UML interfaces with special notation). The runtime configuration con-
sists of multiple subdomains, logically arranged in a 3D mesh, with each sub-
domain connected to its neighbours via multicast interfaces. The runtime con-
figuration also includes a dynamically varying number of steering agents. The
main collector is connected to the current set of agents via a multicast interface.
A multicast interface is also used to connect each agent to all other agents.
The initial configuration of Jem3D is described using the Fractal ADL, as
seen in Figure 4 (pseudo code is used for brevity). Note that the ADL is
not used to express the configuration of subdomains, which depends on the
dynamically-determined domain division. Since allowable configurations follow
a fixed, canonical structure in the form of a 3D mesh, a parameterised descrip-
tion would be useful for automatically generating subdomain configurations.
However, the Fractal ADL includes currently no variability mechanisms for ex-
pressing such descriptions. The ADL does include a simple parameterisation
mechanism, which is used to configure the factory with the required subdomain
implementation.

8.4 Developing component-based codes


The development process for new application codes can be simplified by follow-
ing the component-oriented development paradigm [12]. This approach nor-
mally involves the use of a component-based development environment such as
GIDE, SEGL, gUSE, etc. For our example in this section we use an integrated
system is composed of GriCoL, a component-based language for the description
of Grid experiments and SEGL, a problem solving environment capable of uti-
lizing the resources of the Grid to execute and manage complex scientific Grid

CoreGRID - Network of Excellence


CoreGRID FP6-004265 65

applications.

8.4.1 GriCoL
GriCoL [21, 14] is a parallel language for programming complex computer- and
data-intensive tasks without being tied to a specific application domain.
GriCoL combines the component model of programming with the method-
ology of parallel programming. The main elements of this language are com-
ponents, which have a defined internal structure and interact with each other
through a defined set of interfaces. The language is based on the principle of
wrapping functionality in components in order to utilize the capacities of super-
computer applications and to enable interaction with other language elements
and structures. Another property of the language is its extensibility through
the use of components. The language enables new functional components to
be added to the language library. Program codes for the parallel machine are
wrapped in the standard language capsule. An additional property of the lan-
guage is that it is multi-tiered. The multi-tiered model of organization enables
the user when describing the experiment to concentrate primarily on the com-
mon logic of the experiment program and subsequently on the description of
the individual fragments of the program.

8.4.2 Use Case: Biochemistry Application


Penicillins and cephalosporins are the most commonly used beta-lactam antibi-
otics. They inhibit the growth of bacteria by inhibiting the enzymes responsible
for the cell wall building process. However, bacteria have acquired resistance
against beta-lactam antibiotics with the help of enzymes called beta-lactamases.
These enzymes efficiently hydrolyze the antibiotic and therefore render them
ineffective. To prevent this resistance one tries to block the action of the beta-
lactamase enzymes with so-called beta-lactamase inhibitors. The common use
of beta-lactamase inhibitors has led to resistances against them through natural
evolution of the beta-lactamase enzyme. The following use case describes how
the Institute for Technical Biochemistry in Stuttgart models the effect of mu-
tations on the inhibitor resistance of beta-lactamases. The project starts from
a testing series of 30 enzyme variants provided by the user. They are combined
with 10 different substrates, which results in 300 different systems that need
to be simulated. For a better statistical evaluation and a better sampling of
the conformational space, each system is simulated with 10 different starting
velocities. This implies enormous time and effort for a user when setting up,
running and analyzing this large number of simulations by hand.
In the first stage, the preparation stage, the input data required for the sim-
ulation is prepared. For this purpose all possible enzyme-inhibitor-variants are
created, resulting in 300 different topologies. In the second stage, the equilibra-
tion stage, each system to be investigated is brought to equilibrium. This pro-
cess usually needs days to weeks, strongly depending on the numbers of CPUs
available and the size of the system. This stage consists of two phases: An en-
ergy minimization phase in order to eliminate energetically unfavorable starting

CoreGRID - Network of Excellence


CoreGRID FP6-004265 66

conditions, such as deformed bond lengths and bond angles. Subsequently, in


the restraint phase, the system is simulated with gradually decreasing restraints
on the atomic coordinates to slowly adapt it to the simulation conditions. After
that, simulation carries on without any restraints until the user defined require-
ments for equilibrium are fulfilled. After the system has reached equilibrium,
it enters the production phase. In this stage data of the system is collected for
later analysis. In the final stage of the experiment, the analysis, the data from
the production phase is analyzed and different properties are calculated from
it. The calculated properties are averaged over the 10 MD simulations with dif-
ferent starting conditions for each system. A global evaluation then compares
the property under investigation between all systems to find significant differ-
ences, which can then be correlated by the user to experimentally determined
biochemical activity.
Because of the high processing capacities required, a practical realization of
such experiments is possible with the help of Grid technology. A corresponding
tool, which supports the modeling of experiment processes and executes them
in a Grid environment, is therefore indispensable for the user. This GriCoL
component-based language offers all elements which are necessary to describe
the application flow of a real experiment [21].
There were obvious benefits from using the Grid management system. Be-
side the time saved for setting up, submitting, and monitoring the thousands of
jobs, the base for common errors like misspelling input files is also minimized.
Beneficial is also the reusability of the computation modules: once defined for
a specific program, it can be reused for other control flows and can be shared
among colleagues requiring that program. The equilibration control helps to
minimize simulation overhead as simulations which have already reached equi-
librium are submitted to the production phase while others that have not are
simulated further. The storage of simulation results in the experiment database
enables the scientist to later retrieve and compare the results easily. It was also
possible to generate an executable program of maximum efficiency.

8.5 Use Cases – Conclusions


This section provides an overview of three different approaches to developing
component-based Grid applications. In order to re-use legacy codes, the wrap-
per software approach can be adopted in its two flavours hand-written or au-
tomatically generated wrapper code. Another approach applicable to existing
object-oriented software is to componentise the code by introducing appropri-
ate modifications. The third approach is component-oriented development from
scratch. We compare and contrast the three approaches and highlight their ad-
vantages and weaknesses. We present our experience in selecting and applying
the most appropriate approach depending on the status and the properties of
different application codes. The three main approaches can be integrated into
a single development framework. This work can also serve as a starting point
for future developments in the area of component-based methodologies for con-
structing Grid applications.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 67

9 Software Licensing Issues2


9.1 Some GPL facts
• The GPL requires the software vendor to provide the source code of the
application (or a reasonable mechanism to get it) when distributing it.
The applications under the GPL are often accompanied by a compressed
file with the source code.
• GPL is “viral”: any application that uses GPL code, in order to be dis-
tributed, must be licensed under the GPL. To be distributed, all the out-
puts from a prototype software (employing previously developed software,
in the sense of invoking its libraries or API) would need to be licensed as
GPL (that means distributing its source code).
• It is worth mentioning, to avoid common misunderstandings, the fact that
licensing an application under the GPL does not make it automatically
free. A GPL application can be sold. In this case, along with the bina-
ries, the source code will be delivered. However, once the application is
acquired, the GPL gives the receptor the freedom to redistribute it, some-
thing unusual for commercial licenses and that can cause the rejection of
the software industry.

9.2 Incompatibilities
The GPL is incompatible with several other open-source licenses, and it is not
legal to distribute applications containing combinations of incompatible licenses
among their base libraries [29].
The full list of licenses whose compatibility with GPL has been studied can
be read here: http://www.gnu.org/philosophy/license-list.html.

9.2.1 Apache License


This license is used by some libraries included in ProActive (commons, derby,
axis, and so on). It is compatible with the GPL v3, but not GPL v2. This
should be taken into account when choosing the version of GPL to be used.
But the incompatibility between Apache and GPLv2 is not consensual. The
Apache Software Foundation itself does not consider them incompatible [6].
Anyway, to avoid conflicts, GPL v3 is the best option.

9.2.2 Eclipse Public License (EPL)


This is the license of Eclipse and the plug-ins included in its distribution. There
are 2 ways of distributing third-party plug-ins:
• As simple Eclipse plug-ins. In this case, only the third-party-developed
plug-ins are packaged and distributed under some license. This will re-
quire the user to have Eclipse previously installed and then, install the
2
This section is based on text originally written by Tony Arbona and Gaston Freire from
Grid Systems, S.A. and provided via the EU GridCOMP project.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 68

new plug-ins. For certain kinds of users (not developers) this represents
a loss of usability.

• As an RCP (Rich Client Platform) application. In this case, both the


third-party-developed plug-ins and other (essential) ones coming from
the Eclipse framework are packaged and distributed. The resulting RCP
application is independent from Eclipse and does not require additional
installation steps for the user.

The EPL license is incompatible with both version 2 and 3 of the GPL. It is
not legal to distribute applications that combine plug-ins licensed under both
licenses (EPL and GPL) [26].
The EPL-GPL incompatibility mainly affects the RCP applications, so this
form of distribution should be avoided (this applies both to GIDE as IC2D).
The usual way to avoid conflict is to distribute Eclipse GPL plug-ins separately.
In http://www.eclipseplugincentral.com, the official repository of third-party
plug-ins for Eclipse, we can find a multitude of GPL plug-ins.
Nevertheless, there are RCP applications licensed under GPL, and then
violating the license. Examples of this are:

• Azureus: one of the most famous Bittorrent clients


http://sourceforge.net/projects/azureus/

• Talend Open Studio: data integration


http://www.talend.com/

• GumTree: physics experiments GUI


http://sourceforge.net/projects/gumtree/

• JCrypTool: a crypto tool


http://sourceforge.net/projects/jcryptool/

• JTAF: small sports events manager


http://www.jtaf.ch/rex/index.php?article id=1&clang=1

• Relations RCP: a personal wiki


http://sourceforge.net/projects/relations-rcp/

A list of open sourced (under different licenses) RCP applications can be found
here:
http://www.eclipse.org/community/rcpos.php
Even so, our advice is to distribute Eclipse plug-ins, not RCP applications.
This will not be a major drawback, taking into account that prototype software
is targeted mainly to developers. Moreover, having a single Eclipse instance
with all development tools within will enhance usability and reduce memory
consumption.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 69

9.3 Conclusions
We recommend GPL v3 (not GPL v2) to avoid illegal (according to the Free
Software Foundation) license combinations with Apache. Software developed
under Eclipse will have to be licensed as GPL as well. This prevents them
from being packaged as RCPs, and should be distributed separately as Eclipse
plug-ins.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 70

10 Conclusions and Future Work


We have presented the results of a collaborative approach to the design of a
Generic component-based Grid platform. The results presented here represent
the culmination of the research efforts of a number of partners with particu-
lar emphasis on the integration of each approach into a unified Generic Grid
Architecture. Many of the results presented here are still in progress and will
require some more time for completion. One of the major results of the research
was to highlight the difficulties a number of topics central to the generic grid
paradigm. These were legacy code wrapping, in order to enable the maximum
adaptation of the grid model, and automation. Automation is vital to address
scalability and robustness and is a key area of investigation.
However, as previously stated, the lack of longer-term experience and the
complexity of the target systems demand more research results in the field.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 71

References
[1] M. Aldinucci, M. Danelutto, A. Paternesi, R. Ravazzolo, and M. Vanneschi.
Building Interoperable Grid-aware ASSIST Applications via WebServices.
In PARCO 2005: Parallel Computing, pp. 145–152, Malaga, Spain, 2005.

[2] M. Aldinucci, M. Coppola, M. Danelutto, M. Vanneschi, and C. Zoccolo.


ASSIST as a research framework for high-performance Grid programming
environments. In J. C. Cunha and O. F. Rana, editors, Grid Computing:
Software environments and Tools. Springer, Jan. 2006.

[3] M. Aldinucci, S. Campa, M. Danelutto, M. Vanneschi, P. Dazzi,


D. Laforenza, N. Tonellotto, and P. Kilpatrick. Behavioural skeletons in
GCM: autonomic management of grid components. In D. E. Baz, J. Bour-
geois, and F. Spies, editors, Proc. of Intl. Euromicro PDP 2008: Parallel
Distributed and network-based Processing, pp. 54–63, IEEE CS Press, 2008.

[4] G. Allen, K. Davis, T. Goodale, A. Hutanu, H. Kaiser, T. Kielmann, A.


Merzky, R.V. van Nieuwpoort, A. Reinefeld, F. Schintke, T. Schtt, E.
Seidel, and B. Ullmer. The Grid Application Toolkit: Towards Generic
and Easy Application Programming Interfaces for the Grid. Proceedings of
the IEEE, vol. 93(3), pp. 534–550, March 2005.

[5] G. Aloisio, Z. Balaton, P. Boon, M. Cafaro, I. Epicoco, G. Gombas, P.


Kacsuk, T. Kielmann, and D. Lezzi. Integrating Resource and Service
Discovery in the CoreGrid Information Cache Mediator Component. In:
CoreGRID Integration Workshop 2005, Pisa, Italy, December 2005.

[6] Apache Software Foundation. http://www.apache.org/licenses/GPL-


compatibility.html.

[7] R. Armstrong, G. Kumfert, L. McInnes, S. Parker, B. Allan, M. Sottile,


T. Epperly, and T. Dahlgren. The CCA component model for high-
performance scientific computing. Concurr. Comput. : Pract. Exper.,
18(2):215–229, 2006.

[8] R. Armstrong, D. Gannon, A. Geist, K. Keahey, S. Kohn, L. McInnes,


S. Parker, B. Smolinski, Toward a Common Component Architecture for
High-Performance Scientific Computing, Proc. of IEEE Int. HPDC Sym-
posium, p. 13, IEEE CS Press, 1999.

[9] R.M. Badia, J. Labarta, R. Sirvent, J.M. Prez, J.M. Cela, and R. Grima.
Programming Grid Applications with GRID superscalar. Journal of GRID
Computing, vol. 1(2), pp. 151–170, June 2003.

[10] L. Baduel, F. Baude, D. Caromel, C. Delbe, S. Kasmi, N. Gama, and S.


Lanteri. A Parallel Object-Oriented Application for 3D Electromagnetism,
Proc. 18th Int. Parallel and Distributed Processing Symposium, IEEE CS
Press, 2004.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 72

[11] T. Bartynski, M. Malawski, T. Gubala, and M. Bubak. Universal Grid


Client: Grid Operation Invoker. In Parallel Processing and Applied
Mathematics, 7th International Conference, PPAM 2007, Revised Selected
Papers, Lecture Notes in Computer Science, vol. 4967, pp. 1068–1077,
Springer, 2008.

[12] A. Basukoski, V. Getov, J. Thiyagalingam, and S. Isaiadis. Component-


oriented Development Environment for Grid: Design and Implementation,
in Making Grids Work, Springer, 2008 (to appear).

[13] F. Baude, A. Bergel, D. Caromel, F. Huet, O. Nano, and J. Vayssiere.


IC2D: Interactive Control and Debugging of Distribution, Proc. Int. Con-
ference on Large-Scale Scientific Computing, Lecture Notes in Computer
Science, Vol. 2179:193–200, 2001.

[14] H. Bouziane, N. Currle-Linde, C. Perez, and M. Resch. Analysis of Com-


ponent Model Extensions to support the GriCoL Language, In; Making
grids Work, Springer, 2008 (to appear).

[15] E. Bruneton, T. Coupaye, and J.B. Stefani, Recursive and Dynamic Soft-
ware Composition with Sharing, Proc. 7th Int. Workshop on Component-
Oriented Programming, 2002.

[16] M. Coppola, M. Danelutto, S. Lacour, C. Prez, T. Priol, N. Tonellotto,


and C. Zoccolo. Towards a common deployment model for Grid systems.
In S. Gorlatch and M. Danelutto editors, CoreGRID Workshop on Inte-
grated research in Grid Computing, pp. 31–40, CoreGRID, IST, Pisa, Italy,
November 2005.

[17] CoreGrid NoE Institute on Programming Model, Proposals for a Grid


Component Model, Deliverable Report D.PM.02, 2006.

[18] CoreGrid NoE Institute on Programming Model, Basic Features of the


Grid Component Model (assessed), Deliverable Report D.PM.04, 2007.

[19] CoreGrid NoE Institute on Grid Systems, Tools, and Environments, Pro-
posal for Mediator Component Toolkit, Deliverable Report D.ETS.02,
2005.

[20] CoreGrid NoE Institute on Grid Systems, Tools, and Environments, Design
of the Integrated Toolkit with Supporting Mediator Components, Deliver-
able Report D.STE.05, 2006.

[21] N. Currle-Linde, F. Bos, M. Resch. GriCoL: A Language for Scientific


Grids, E-SCIENCE ’06: Proceedings of the 2nd IEEE Int. Conference on
e-Science and Grid Computing, Amsterdam, 2006.

[22] M. Danelutto, M. Vanneschi, C. Zoccolo, N. Tonellotto, R. Baraglia,


T. Fagni, D. Laforenza, and A. Paccosi. HPC Application Execution on
Grids. In V. Getov, D. Laforenza, and A. Reinefeld, editors, Future Gen-
eration Grids, Springer, 2006.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 73

[23] T. Delaittre, T. Kiss, A. Goyeneche, G. Terstyanszky, S. Winter, and P.


Kacsuk. GEMLCA: Running Legacy Code Applications as Grid Services,
Journal of Grid Computing, 3, No. 1-2, pp 75–90, Springer, 2005.

[24] The DiscoGrid Project. http://www.inria.fr/caiman/Stephane.


Lanteri/DiscoGrid

[25] Eclipse Graphical Editing Framework. http://www.eclipse.org/gef/

[26] Eclipse Public License. http://en.wikipedia.org/wiki/Eclipse Public License.

[27] Eclipse - An open development platform. http://www.eclipse.org.

[28] M. Ejdys, U. Herman-Izycka, N. Lal, and T. Kielmann. Integrating Ap-


plication and System Components with the Grid Component Model, In:
CoreGRID Symposium, Las Palmas, Spain, 2008.

[29] S. Forge and C. Blackman, Commercial Exploitation of Grid technologies


and Services. Final Report, SCF Associates, Nov. 2006.

[30] I. Foster. Service-oriented science. Science, 308(5723):814–817, 2005.

[31] The Fractal Project. http://fractal.objectweb.org/

[32] Fractal specification, http://fractal.objectweb.org/specification/


index.html

[33] T. Friese, M. Smith, and B. Freisleben. Grid Development Tools for Eclipse.
Proc. Eclipse Technology eXchange Workshop eTX at ECOOP, 2006.

[34] V. Getov. A Mixed-Language Programming Methodology for High Per-


formance Java Computing, in The Architecture of Scientific Software, pp.
333–347, Kluwer Academic Publishers, 2001.

[35] V. Getov. Integrated Framework for Development and Execution of


Component-based Grid Applications, Proc. IPDPS, IEEE CS Press, 2008.

[36] V. Getov, G. von Laszewski, M. Philippsen, I. Foster. Multi-Paradigm


Communications in Java for Grid Computing, Communications of the
ACM, vol. 44(10), pp. 118–125, 2001.

[37] Grid ENabled Integrated Earth system model (GENIE).


http://www.genie.ac.uk

[38] The GridCOMP Project. http://gridcomp.ercim.org

[39] S. Isaiadis and V. Getov. Dependability in Hybrid Grid Systems: a Virtual


Clusters Approach. In Proc. IEEE John Vincent Atanasoff 2006 Interna-
tional Symposium on Modern Computing (JVA’06), pp. 52–57, IEEE CS
Press, November 2006.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 74

[40] S. Isaiadis and V. Getov. Evaluation of Dynamic Clustering Architecture


for Utilising Mobile Res ources, In Proc. IASTED Int. Conference on Par-
allel and Distributed Computing and Networks, pp. 117–125, Acta Press,
February 2008.
[41] S. Isaiadis, V. Getov, I. Kelley, and I. Taylor. Dynamic Service-Based In-
tegration of Mobile Clusters in Grids. Technical Report TR-0094, Institute
on Grid Systems, Tools, and Environments, CoreGRID, February 2008.
[42] Jboss rules home page, 2008. http://www.jboss.com/products/rules.
[43] S. Jha, H. Kaiser, A. Merzky, and O. Weidner. Grid Interoperability at
the Application Level Using SAGA. In: Proc. 3rd IEEE Int. Conference
on e-Science and Grid Computing, pp. 584–591, IEEE CS Press, 2007.
[44] P. Kacsuk and G. Sipos. Multi-Grid, Multi-User Workflows in the P-
GRADE Grid Portal. Journal of Grid Computing, Vol. 3. No. 3-4., 2005,
pp 221-238, Springer, 2005.
[45] T. Kiss, P. Kacsuk, G. Terstyanszky, and S. Winter. Workflow Level In-
teroperation of Grid Data Resources, Proc. 8th IEEE Int. Symposium on
Cluster Computing and the Grid (CCGrid-2008), pp. 194-201, 2008.
[46] T. Kiss, A. Tudose, P. Kacsuk, and G. Terstyanszky. SRB Data Resources
in Computational Grid Workflows. Conf. Proc. of the UK E-Science All
Hands Meeting, pp. 643–650, September 2007.
[47] E. Krepska, T. Kielmann, R. Sirvent, and R.M. Badia. A Service for
Reliable Execution of Grid Applications. In: Achievements in European
Research on Grid Systems, pp. 179-192, Springer, 2008.
[48] S. Lacour, C. Perez, and T. Priol. A Software Architecture for Automatic
Deployment of CORBA Components Using Grid Technologies, In Proc. 1st
Francophone Conference On Software Deployment and (Re)Configuration
(DECOR’2004), pp. 187–192, Grenoble, France, October 2004.
[49] S. Lacour, C. Perez, and T. Priol. Generic Application Description
Model: Toward Automatic Deployment of Applications on Computational
Grids. In the Proc. 6th IEEE/ACM Int. Workshop on Grid Computing
(Grid2005), Springer, November 2005.
[50] R. Lopez and C. Perez, Improving MPI Support for Applications on Hi-
erarchically Distributed Resources In Proc. EuroPVM/MPI Conference,
LNCS, vol. 4757, 2007
[51] M. Malawski, M. Bubak, F. Baude, D. Caromel, L. Henrio, and M. Morel.
Interoperability of Grid Component Models: GCM and CCA Case Study,
In Proc. CoreGRID Symposium, Springer, 2007.
[52] M. Malawski, D. Kurzyniec, and V. Sunderam. MOCCA – Towards a
Distributed CCA Framework for Metacomputing. In Proc. 10th HIPS
Workshop in Conjunction with IPDPS, p. 174a, IEEE CS Press, 2005.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 75

[53] M. Malawski, M. Bubak, M. Placek, D. Kurzyniec, and V. Sunderam. Ex-


periments with Distributed Component Computing Across Grid Bound-
aries. In Proc. HPC-GECO/CompFrame Workshop in Conjunction with
HPDC, pp. 109–116, Paris, France, 2006.

[54] M. Malawski, T. Gubala, M. Kasztelnik, T. Bartynski, M. Bubak,


F. Baude, and L. Henrio, High-level scripting approach for building
component-based applications on the grid, in Making Grids Work,
Springer, 2008 (to appear).

[55] R.V. van Nieuwpoort, T. Kielmann, and H.E. Bal. User-friendly and
Reliable Grid Computing Based on Imperfect Middleware, In: Proc.
ACM/IEEE Conference on Supercomputing (SC’07), 2007.

[56] Object Management Group Inc, The CORBA Component Model, Revision
V4.0, 2006, http://www.corba.org/.

[57] J. K. Ousterhout. Scripting: Higher-level programming for the 21st cen-


tury. Computer, 31(3):23–30, 1998.

[58] S. Panagiotidi, J. Cohen, J. Darlington, Marko Krznarić, and E. Katsiri.


Service-enabling Legacy Applications for the GENIE Project. In All Hands
Meeting, Nottingham, September 2006.

[59] S. Parker, K. Zhang, K. Damevski, and C.R. Johnson. Integrating


component-based scientific computing software. In M. A. Heroux, P.
Raghavan, and H.D. Simon (Eds.), Frontiers of Parallel Processing For
Scientific Computing, Chapter 15, pp. 271–288, SIAM Press, 2006.

[60] N. Parlavantzas, V. Getov, M. Morel, F. Baude, and D. Caromel, De-


sign Support for Componentising and Grid-enabling Scientific Applica-
tions, Proc. ACM CompFrame07 Symposium, pp. 31–38, ACM Press,
2007.

[61] ProActive Java Grid Middleware. http://www-sop.inria.fr/oasis/


proactive/

[62] Shibboleth, http://shibboleth.internet2.edu/

[63] E. Tejedor, R.M. Badia, P. Naoumenko, M. Rivera, and C. Dalmasso. Or-


chestrating a safe functional suspension of GCM components. Proc. Core-
GRID Integration Workshop, Springer, 2008 (to appear).

[64] E. Tejedor, R.M. Badia , T. Kielmann, and V. Getov. A Component-Based


Integrated Toolkit, in Making Grids Work, Springer, 2008 (to appear).

[65] J. Thiyagalingam, V. Getov, S. Panagiotidi, O. Beckmann, and J. Dar-


lington, Domain-Specific Metadata for Model Validation and Performance
Optimisation, In: Achievements in European Research on Grid Systems,
pp. 165–178, Springer, 2007.

CoreGRID - Network of Excellence


CoreGRID FP6-004265 76

[66] A. Vallecillo, J. Hernandez, and J.M. Troya. Component interoperability.


TR ITI-2000-37, Dept. de Lenguajes y Ciencias de la Computacion, Uni-
versity of Malaga, 2000, http://www.lcc.uma.es/~av/Publicaciones/
00/Interoperability.pdf.

[67] V. Welch, I. Foster, C. Kesselman, O. Mulmo, L. Pearlman, S. Tuecke, J.


Gawor, S. Meder, and F. Siebenlist. X.509 Proxy Certificates for Dynamic
Delegation, Proc. 3rd Annual PKI R&D Workshop, 2004.

CoreGRID - Network of Excellence

Potrebbero piacerti anche