Grid Middleware

The Grid creation of middleware is an essential aspect of the software engineering eort to create an evolving layer of infrastructure residing between the network and applications in todays world where computers are a huge part of every business. But in order to work well in business, computers have to be able to communicate with other computers effectively. The computers in a network should run smoothly and almost seamlessly. This can be difficult to do. That is, without the benefit of grid middleware. Grid middleware is what allows certain applications and software components to interact by creating an evolving layer of infrastructure residing between the network and applications. In the computing world, grids are the distributed systems in which the non-interactive workloads that involve large amounts of files can be found. Grids such as these are usually generated through something such as middleware.

Middleware Overview Middleware is software that will connect the components of software with applications, allowing them to communicate. Middleware is made up of service sets that allow many processes which are running at one time on one or more computing devices to interact. It is basically mediator layer that provide a consistent and homogeneous access to resources managed locally with dierent syntax and access methods Grid Middleware Offers Interoperability Interoperability between two or more networked computers is enabled by grid middleware. Therefore, data exchange between assorted programs through sets of different exchange formats and common protocols is available. When middleware is used in grid computing, it supports distributed architectures, and those support the complex distributed applications. Applications The grid middleware allows the users to submit their program requests to execute jobs, or computations, to their grid. This is handy, because in turn it allows the jobs to be executed anywhere on computer network. The software components, like resource brokers, pass the job to the node, or network point, that is the most appropriate. Middleware is also useful for the security functions, since it allows the users to give themselves authorization through standard certificate infrastructure. In the business world, time is money and grid middleware can save companies precious time. It offers businesses interoperability and seamlessly sends the jobs to the appropriate network point. Grid middleware is what makes everything on a computer network to run well and to ensure that a company is working to its fullest capacity.

Layered Grid Architecture and its components

Grid Fabric level consists of distributed resources such as computers, networks, storage devices and scientific instruments. Core Grid middleware offers services such as remote process management, co-allocation of resources, storage access, information registration and discovery, security, and aspects of Quality of Service (QoS) such as resource reservation and trading. User-level Grid middleware utilizes the interfaces provided by the low-level middleware to provide higher level abstractions and services. These include application development environments, programming tools and resource brokers for managing resources and scheduling application tasks for execution on global resources. Grid applications and portals are typically developed using Grid-enabled languages and utilities such as HPC++ or MPI. An example application, such as parameter simulation or a grand-challenge problem, would require computational power, access to remote data sets, and may need to interact with scientific instruments. Grid portals offer Web-enabled application services, where users can submit and collect results for their jobs on remote resources through the Web.

OUR GRID MIDDLEWARE SOLUTIONS

UNICORE is a vertically integrated Grid computing environment that facilitates the following: A seamless, secure and intuitive access to resources in a distributed environment for end users. Solid authentication mechanisms integrated into their administration procedures, reduced training effort and support requirements for Grid sites. Easy relocation of computer jobs to different platforms for both end users and Grid sites. Why UNICORE? 1. Mature, well-proven Grid technology from Europe; widely used in European and national Grid projects, 1. initiatives, and infrastructures 2. Made for SupercomputingHPCC Grids as well as Grids of Clusters and single CCs 3. Fast and competent support directly from the developers 4. Easy to install and configure clients and services 5. Intuitive graphical user interface and powerful command line client for seamless access to resources 6. Application integration mechanisms on the client, services and resource level 7. Mature workflow support tightly integrated but extensible for dierent, domain-specific workflow languages and engines 8. Interoperability through common open standards 9. Source code (all Java)is available and easy to modify 10. Easily extensible with own developments with the option of integration in official releases Features and Functions 1. User driven job creation and submission: A graphical interface assists the user in creating complex and interdependent jobs that can be executed on any UNICORE site without job definition changes. 2. Job management: The Job management system provides user with full control over jobs and data. 3. Data management: During the creation of a job, the user can specify which data sets have to be imported into or exported from the USpace (set of all files that are available to a UNICORE job), and also which datasets have to be transferred to a different USpace. UNICORE performs all data movement at run time, without user intervention. 4. Application support: Since scientists and engineers use specific scientific application, the user interface is built in pluggable manner in order to extend it with plugins that allows to prepare specific application input. 5. Flow control: A user job can be described as a set of one or more directed acyclic graphs. 6. Single sign-on: UNICORE provides a single sign-on through X.509V3 certificates. 7. Support for legacy jobs: UNICORE supports traditional batch processing by allowing users to include their old job scripts as part of a UNICORE job. 8. Resource management: Users select the target system and specify the required resources. The UNICORE client verifies the correctness of jobs and alerts users to correct errors immediately.

GLOBUS
The Globus project is a multi-institutional research eort that seeks to enable the construction of computational grids providing pervasive, dependable, and consistent access to high-performance computational resources, despite geographical distribution of both resources and users. Why GLOBUS? 1. Allows people to share computing power, databases, and other tools securely online across corporate, institutional, and geographic boundaries without sacrificing local autonomy. 2. The toolkit includes software services and libraries for resource monitoring, discovery, and management, plus security and file management. 3. The Globus Toolkit was conceived to remove obstacles that prevent seamless collaboration. Its core services, interfaces and protocols allow users to access remote resources as if they were located within their own machine room while simultaneously preserving local control over who can use resources and when. 4. The New York Times calls it the "the de facto standard" for Grid computing 5. Globus Online enables you to move, sync, and share your data using just a web browser. It takes care of time consuming, error prone IT tasks so you can focus on your research. 6. It Eliminate unnecessary duplication and wasted storage by providing your users an easy and reliable way to sync big datasets 7. Globus Online helps everyone share and access the data they need, no matter where it's stored. No expensive cloud storage required. FEATURES AND FUNCTIONS 1. Resource management a. Globus Resource Allocation Manager (GRAM): GRAM provides remote execution capability and reports status for the course of the execution. b. Globus Access to Secondary Storage (GASS): GASS is a file-access mechanism that allows applications to pre-fetch and open remote files and write them back. GASS is used for staging-in input files and executables for a job and for retrieving output once it is done. It is also used to access the standard output and error streams of the job. 2. Communication (Nexus) This component provides basic communication mechanisms. These mechanisms must permit the efficient implementation of a wide range of communication methods, including message passing, remote procedure call, distributed shared memory, stream-based, and multicast. Mechanisms must be cogniaant of network quality of service parameters such as jitter, reliability, latency, and bandwidth. 3. Security (GSI Authentication) The GSI provides services, protocols and libraries to achieve the following aims for Grid security: a. Single sign-on for using Grid services through user certificates b. Resource authentication through host certificates c. Data encryption d. Authoriaation e. Delegation of authority and trust through proxies and certificate chain of trust for Certificate Authorities

4. Remote data access The Globus data access module addresses the problem of achieving high performance when accessing parallel file systems and network-enabled IHO devices such as the Pigh Cerformance Storage System (PCSS). 5. Data Management The data management package provides utilities and libraries for transmitting, storing and managing massive data sets that are part and parcel of many scientific computing applications . The elements of this package are GridFTC and Replica Location and Management

6. Executable management. GEM provides mechanisms for matching the characteristics of a computer in a computational grid with the runtime requirements of an executable or library. These mechanisms can be used in conjunction with other Globus services to implement a variety of distributed code management strategies.

LEGION
Legion is middleware; it connects networks, workstations, supercomputers, and other computer resources together into a system that can encompass dierent architectures, operating systems, and physical locations. There is no central "big brother" that oversees and controls each Legion resource: instead, each resource is an independent element. Legion provides a coherent framework in which these elements can be combined into a metasystem. Why Legion? 1. It is scalable and is designed to handle trillions of objects over enormous distances and in a widely variegated computing environment via a scalable software architecture 2. Legion keeps security decisions in the hands of the resource owners as much as possible 3. Legion is customiaable .It has extensible, replaceable components that will permit it to evolve over time to accommodate new and perhaps unexpected requirements of future users. 4. Legion is designed to support a wide variety of applications and programs, and we expect that developers will add new possibilities. Legion also allows allows users to run remotely executed programs from the Legion command line, taking advantage of Legion's distributed resources. 5. Sequential programs run on Legion too. Carameter space studies are the most likely to benefit, since Legion allows you to run multiple copies of a program in parallel on multiple hosts 6. Top-down Approach: Legion started out with the "top-down" premise that a solid architecture is necessary to build an infrastructure of this scope. Consequently, much of the initial design time spent in Legion was in determining the underlying infrastructure and the set of core services over which grid services could be built. FEATURES AND FUNCTIONS 1. Object Model : Legion is an object-based system comprised of independent, logically address space-disjoint, active objects that communicate via remote procedure calls (RCCs). Objects represent coarse-grained resources and entities such as users, hosts, schedulers, files, and directories. Each Legion object belongs to a class, which is itself a Legion object. Much of what is usually considered system-level responsibility is delegated to user-level class objects 2. Security: Legions distributed, extensible nature and user-level implementation prevent it from relying on a trusted code base or kernel. Furthermore, there is no concept of a superuser in Legion. Individual objects are responsible for legislating and enforcing their own security policies. The public key embedded in an objects name enables secure communication with other objects 3. Naming :User-defined text strings called context names identify Legion objects. Context names are mapped by a directory service called context space to unique, location-independent binary names called Legion object identifiers(LOIDs). 4. Scalability:LegionFS distributes files and contexts across the available resources in the system. This allows applications to access files without encountering centraliaed servers and ensures that they can enjoy a larger percentage of the available network bandwidth without contending with other application accesses. 5. Extensibility : LegionFS dierentiates between objects according to their exported interfaces, not their implementations. 6. Adaptability: A wide-area file system must be adaptable to a diverse set of network, load, and system-wide conditions. LegionFS facilitates adaptation by maintaining system-wide metadata. Each object has an associated, arbitrary set of <key,value> pairs. Typical attributes for a host object include load averages, architecture, and operating system

GRIDBUS
The Gridbus Croject is an open-source, multi-institutional project led by the GRIDS Lab. It is engaged in the design and development of service-oriented cluster and grid middleware technologies to support eScience and eBusiness applications. It extensively leverages related software technologies and provides an abstraction layer to hide idiosyncrasies of heterogeneous resources and low-level middleware technologies from application developers. In addition, it extensively focuses on realiaation of utility computing model scaling from clusters to grids and to peer-to-peer computing systems. It uses economic models in efficient management of shared resources and promotes commoditiaation of their services. Thus, it enhances the tradability of grid services and manages efficiently the supply and demand for resources. Why Gridbus? 1. Supports commoditiaation of grid services at Raw resource and Application .level as well as Aggregated Services 2. Gridbus emphasiaes the end-to- end quality of services driven by computational economy at various levels - clusters, peer-to-peer (C2C) networks, and the Grid - for the management of distributed computational, data, and application services. 3. It provides an easy to use interface for the end-user to monitor and control jobs running within the Grid environment 4. It enables service providers to publish the services which they provide along with the costs associated with those services 5. It allows flexible application composition by supporting an object-oriented grid application programming model in addition to a grid job model 6. It has the capability to locate and retrieve the required data from multiple data sources and to redirect the output to storage where it can be retrieved by processes downstream. FEATURES AND FUNCTIONS 1. Gridbus Broker :An economy-capable Data Grid service broker for scheduling distributed data oriented applications across Windows and Unix-variant Grid resources with XML input interface. 2. Alchemi : A .NET-based desktop Grid framework with interface for user-level scheduling. 3. GridSim A toolkit for modeling and simulation of global Grids with network extensions and trace capabilities. 4. Grid Workflow Management System: Workflow Management System and Monitor for Grid systems 5. Grid Market Directory: A directory for publication of Grid Service provides and their services with web services based query interface. 6. Grid Bank: A grid accounting, authentication, and payment management infrastructure with web services interface. 7. Gridscape A tool for the creation of interactive and dynamic Grid test bed web portals implemented as Web application within Tomcat. 8. G-Monitor: A web portal to manage execution of applications on Grids using remote brokers. 9. ExcelGrid : Net based GUI client application for remote submission and monitoring of jobs to Grid middleware and brokering systems such as Gridbus broker and Alchemi. 10. Visual Parametric Modeller A graphical environment for application parameterisation with visual parameterisation of data files. 11. Libra :An economy based scheduler for clusters implemented as CBS plug-in.

A COMPARISON OF OUR PARTNER TECHNOLOGIES

Middleware UNICORE Croperty Focus Pigh level Crogramming models Category Mainly uniform job submission and monitoring Architecture Vertical multi tiered system Globus Low level services Generic computational Layered and modular toolkit Pourglass model at system level C and Java Unix Replacement libraries for Unix & C libraries. Special MCI library (MCICP G), CoG (Commodity Grid) kits in Java, Cython, JSC, Cerl and Web It provides Services various features like discovery of resources, heterogeneity and security issues. Legion Pigh level Crogramming models Generic computational Gridbus Abstractions and market models Generic computational Layered component and utility model Pourglass model at user level C, Java, C# and Cerl Unix and Windows with .NET (Alchemi) Broker Java ACI XMLbased parametersweep language Grid Thread model via Alchemi.

Vertically integrated system Object-oriented metasystem C++ Unix Legion Application Crogramming Interfaces (ACI). Command line utilities

Implementation Abstract Job Object Model Implementation Java Technologies Runtime Clatform Crogramming Environment Unix Workflow environment

Advantages

It provides the interface and java based environment for preparing the jobs and to accessing the supercomputer resources in secure manner Interactive processes are not allowed here.

It provides common model or architecture that will support the new functionality added by designer

It offers various services for computational and data grid.

Disadvantages

It does not provide any common model or architecture to add any new functionality. Open source

It does not address the issues like Management of data load and replica

Distribution Model

Open source

Not open source. Commercial version available NCACI Testbed, Nimrod -L , and NCBioGrid.Additionally , it has been used in the study of axially symmetric steady flow and protein folding applications.

Some Users and EuroGrid, Grid Applications Interoperability Croject (GRIC), OpenMolGrid, and Japanese NAREGI

AppLeS , Ninf , Nimrod-G , NASA ICG Condor-G , Gridbus Broker , UK eScience Croject , GriChyN , and EU Data Grid.

For the other complicated applications like stock exchange creation there is need Open for source development of other technologies at here. eChysics Cortal , Belle Analysis Data Grid, NeuroGrid , Natural Language Engineering, PydroGrid and Amsterdam Crivate Grid.