The Enterprise Service Bus, Re-Examined

The Enterprise Service Bus, re-examined
Updating concepts and terminology for an evolved technology

Skill Level: Intermediate Greg Flurry Distinguished Engineer IBM Software Group, Business Process Optimization Kim J. Clark (kim.clark@uk.ibm.com) Consulting IT Specialist IBM Software Services for WebSphere
18 May 2011 This article clarifies the architectural role of the enterprise service bus (ESB) in a service-oriented architecture (SOA), and refines terminology associated with the ESB and integration in SOA.
Introduction
A series of IBM developerWorks articles titled Exploring the Enterprise Service Bus was introduced in 2007, back when service-oriented architecture and the associated concepts and terms were still maturing. For example, Part 1 of that article series fell into the (still-common) trap of using the term "service" in an ambiguous way; it positioned the ESB as an "integration layer" and suggested the ESB cannot include "business logic." Things have changed since that time. This article defines a very crisp, coherent set of concepts and terms to disambiguate discussions about services, and also clarifies (and corrects) the relationship between the ESB and integration in SOA, and between the ESB and business logic. In addition, the reality that a service-oriented architecture must be implemented is also addressed, offering guidance on implementation approaches with a focus on maintaining the all-important factor of separation of concerns.
The Enterprise Service Bus, re-examined Copyright IBM Corporation 2011. All rights reserved.
Page 1 of 26
developerWorks
ibm.com/developerWorks
The "service" in SOA

SOA leverages services to simplify, formalize and bring agility to the creation of business solutions. The idealized SOA is shown in Figure 1, where a requester interacts with a provider to achieve some business-relevant task. Figure 1. An abstract view of SOA: Direct service interaction
Where is the "service" in Figure 1? Consider these statements: "We identified the Process Order service as one of the services needed to run our business." "A service is a repeatable business task." These comments suggest "service" is not a physical thing, but instead an abstract, logical, or conceptual thing, fundamentally the business task or the business outcome resulting from interaction. Consistent with that interpretation is The Open Group SOA Work Group formal definition of "service" as a "logical representation of a repeatable activity that has a specified outcome." You cannot publish or interact with this "service", yet these actions are commonly associated with a "service". Using this definition, you must conclude that the "service" is not shown, but only implied in Figure 1. You might further conclude the "service" is performed by the provider. Now, consider these statements: "We will publish the Process Order service in the registry." "The registry is used to discover services when designing new solutions." Such statements suggest "service" is more concrete, that is, a specification of something, but still not something with which a requester can interact. Using this definition, you must conclude that the "service" is not shown at all in Figure 1. You might further conclude that any interaction with the provider is specified by the "service". Finally, consider these statements:
The Enterprise Service Bus, re-examined Page 2 of 26
Copyright IBM Corporation 2011. All rights reserved.
developerWorks
"In this use case we invoke the Process Order service." "My SOA has reusable services that can be called by any application." These statements suggest "service" is a physical thing with which a requester can interact. Using this definition, the interpretation of Figure 1 is that the provider is the "service". So, we have at least three very different interpretations for the term "service:" In more formal contexts, "service" refers to the abstract or logical representation of a business task. In more informal contexts, "service" can refer to a concrete description or specification representation of that business task, but more often refers to the physical implementation or realization of that business task. Indeed, "service" in some contexts could refer to all interpretations, perhaps in different phases of the service lifecycle. The above discussion shows that use of the term "service" without context and qualification can be confusing and even counterproductive. It is sometimes possible to infer the meaning from the context, but not always. Rather than use the often ambiguous term "service," the following specific terms will be used throughout the remainder of this article in an attempt to be more precise: Governed service: Refers to a business task made available by the SOA, without being particularly precise as to whether you are talking about the business task, a specification, or an implementation or some combination. For example, "We have a suite of governed services for administering customers' accounts." Service specification: A formalized set of characteristics related to a service interaction that includes: the business task achieved by an interaction, the technical aspects of how to interact, and various qualities of service associated with interaction. The service specification is a formal representation of a "service" that is both a concrete and physical representation. The specification is the precise term in the context of defining reusable services for the enterprise (logical) or publishing services in the registry (physical). Service realization: Refers to a physical implementation of a service specification. As it is a physical entity, the realization can be invoked. This is an informal yet the most common meaning of "service." The realization is the precise term in the context interacting with a service or calling a service. This article will build a picture of the set of things that
Page 3 of 26
developerWorks
might be required for a service realization, including clarification on terms such as mediation, integration, enterprise service bus, and, indeed, provider. Widespread use of these more precise terms would go a long way toward reducing ambiguity in communication around SOA, although it's not expected that they will immediately start to be used in general conversation. While it might be more precise to say "Invoke the Process Order service realization" or "Publish the Process Order service specification," conversational simplifications will often trump precision. This article is intended to refine terminology to reduce confusion, and will try to avoid any ambiguity by using the above terms wherever possible. In addition, you will also find other common term pairings: service operation, service exposure, service governance, and service model. The normally unambiguous meanings of these terms will be made clear in the surrounding text.
Service versus service operation

Another common point of confusion regarding use of the term "service" is whether it refers to a suite of related operations or a single operation. Formal architectural definitions of the term allow multiple operations, but do not mandate it. Common mechanisms for defining partial service specifications, such as Web Services Definition Language (WSDL), support this view. For example, assume you need business tasks that enable add, delete, find, and update of a "customer" business entity. You could create a single governed service defined by a single service specification with four operations, or you could create four governed services defined by four service specifications, each with one operation. It is extremely common, however, that in general conversation the word "service" is used as shorthand for service operation, regardless of whether referring to a service specification or service realization. This too can cause confusion. Remember the Open Group SOA Work Group definition of "service" as "a representation of a repeatable activity that has a specified outcome." This definition itself is another example of loose usage of the term. If something performs a single activity, then it really refers to a service operation. The distinction between service and service operation is important when discussing characteristics described in the service specification. There can be characteristics that are common to the service as a whole; for example, communication transport. There can be characteristics that are specific to a single service operation; for example, response time. Thankfully, many of the concepts, observations and suggestions presented here are
developerWorks
not impacted one way or another by service vs. service operation, but the precise term "service operation" is used where it makes a difference to the discussion. You are encouraged to do the same where precision matters.
Service governance in SOA

You should think of "services" within an SOA as governed services, which recognizes that they are defined by service specifications and conform to an organization's governance policies. This helps to differentiate more clearly between functions that just happen to be exposed in a slightly more usable form (for example, an application that provides an HTTP/SOAP-based API) from things that can truly be seen as business-relevant governed services. Let's take a deeper look at what is meant by "governed" in this context. What is the big difference between SOA and earlier approaches to enterprise architecture? SOA is about exposing core business-aligned functions as reusable services. Let's deconstruct that into two critically important aspects: Governing business alignment of services concerns what business functions are made reusable. Put simply, the governed services must help to achieve the goals of the business. Governance includes up-front identification of the core business functions. But an SOA is not static, just as a business is not static; it slowly matures and evolves as the business grows and changes. So, governance must ensure the continued business alignment of the governed services (in aggregate, the service model). The overall SOA, not the ESB, is responsible for governance of the definition and evolution of the service model, but the ESB does help enforce such governance, which leads to the second aspect. Governing service exposure focuses on how those business functions are made reusable. For governed services to be reusable (or perhaps more accurately, sharable), the way they are exposed must meet a number of key criteria that are also governed by the SOA. Proper exposure includes protocols and transports, appropriate quality of service objectives, an organization's needs around security, monitoring, logging and auditing, a versioning strategy, and proper ownership. These aspects of governance all relate to how a governed service is exposed, and are things that the ESB must enforce. Notice that these are all non-semantic characteristics; that is, they have no effect on the meaning of the business function exposed. Architecturally, it would make sense to separate these from any semantic content, and we'll discuss how we retain that separation later. How do you achieve service governance? As a way of describing the important
Page 5 of 26
developerWorks
characteristics related to a service interaction, the formal service specification plays a key role. A logical service specification can, in practice, be codified to become physical to support automated governance processes that ensure compliance with the business goals of the organization (usually via a service registry and repository). It is clear that for "services" to be of value to an enterprise they must be governed in the ways described above. If they are not, they are not part of the service model and must be considered only a part of the implementation of the SOA, created for a limited, specific usage. An SOA therefore, consists of "governed services" and use of this term is encouraged to remind yourself of the difference between these, and traditional interfaces. Two final points on service governance: We must accept that in some organizations there will be differing domains of governance (for example, departmental, internal to enterprise, external to enterprise, and so on) with differing approaches to governance. What this means is that just because something is a governed service in one domain doesn't mean it complies with the governance policies of other domains, and thus might need further work to be used in another domain. This is as much detail on this broader topic that will be discussed here, but it will no doubt become more prevalent as SOAs evolve and governed environments interact more frequently. Service governance is just one part of an enterprise's overall IT governance. Service governance, which primarily relates to the way that services are identified, exposed, and managed, should not be confused with other forms of IT governance covering subjects such as coding standards or implementation technologies. These are critically important parts of the broader IT governance, but do not relate specifically to service governance.
The service specification

Given the importance of the service specification discussed earlier, let's examine it more closely. Figure 2 shows the service specification's place within a more holistic and mature (but still idealized) SOA, emphasizing and facilitating separation of concerns between the logical service specification and the physical service realization (in this case, the provider). Figure 2. SOA and service specification
developerWorks
Figure 3 shows the various perspectives of a service specification. What is the primary reason for service interaction? The service requester interacts with the service realization to achieve a business task. In section (a) of the figure, you see the business task is described by a set of semantic characteristics that include a description of the business task performed (for example, create a purchase order from a shopping cart) and a definition of the business interface in terms of task-relevant business entities (for example, customer, account, shopping cart, purchase order). You can think of these semantic characteristics as things that are accomplished in business-relevant terms through interaction with a service realization. Figure 3. Aspects of a service specification
Page 7 of 26
developerWorks
Section (b) of the figure shows the syntactic characteristics that refer to the mechanisms needed to actually interact with a service realization; for example, communication protocol, message formats, and security. You can think of these characteristics as the syntax or binding required for successful interaction.
"Service specification" versus other terms Unfortunately, we often see and hear different terms used for the concept we call "service specification." The most common term we find is "service contract." This term has a significant drawback in general use in that a contract implies an agreement between two or more parties. There is really only one party to a specification: the service realization. Architecturally, a true service contract would reference a service realization as one party, and define detailed promises made to the requester party, likely related to qualities of service. Another common term is "service description." The drawback here is that its use in WSDL includes formal support for only a subset of the necessary characteristics.
Section (c) of the figure refers to operational aspects such as response time,
developerWorks
throughput, and availability. These characteristics recognize that service realization ultimately runs in one or more physical containers with limited resources such as CPU cycles, storage, memory, network bandwidth, and so on. You can think of the operational characteristics as qualities of service that are dependent on the appropriate sizing and configuration of the provider container, hence its representation in Figure 3. Separation of concerns mandates that all characteristics in the service specification are external and are those exposed by a service realization. Thus, these characteristics are independent of the service realization implementation, such that the implementation can be altered or improved or upgraded without affecting the service requestors, provided those characteristics are not changed. This also means the service specification is applicable to all users of a realization. Given the definition of service specification, in direct service interaction shown in Figures 1 and 2, you can say that since the service provider is the service realization: The syntactic characteristics offered by the provider must satisfy those in the service specification, and The operational (quality of service) characteristics offered by the provider must satisfy those in the service specification. So, in Figures 1 and 2 the service provider implements the semantics of the service specification (it performs the business task), and it also exposes the business related semantics by implementing the syntactic and operational characteristics. In some of the discussions that follow, it will be convenient to only distinguish characteristics relevant to the business task from those that are not relevant. In such discussions, the syntactic and operational characteristics are grouped together as non-semantic characteristics, and therefore refer only to semantic and non-semantic characteristics.
Use of mediation for service exposure in SOA

The simplistic scenario in Figure 2, where the provider already offers an appropriately governed service, is almost never the case in real life. Let's look at a situation where the non-semantic characteristics of the provider do not match the governance policies, including the service specification. With direct service interaction, as discussed in relation to Figures 1 and 2, the provider must satisfy all characteristics described in the service specification. Further, the provider must adhere to all the service governance policies for an organization. This can be difficult or impossible, depending on the nature of the
Page 9 of 26
developerWorks
governance within that domain. The fact that most providers are not capable of adhering to well-governed service specifications while maintaining separation of concerns is the reason we need mediated service interaction in SOA, shown in Figure 4. Figure 4. An abstract view of SOA: Mediated service interaction
Transformation versus translation This article specifically uses the word translation to mean simply changing the representation of a business entity from one form to another without changing the core meaning of that object. Thus, translation is meant to deal with representation (syntax) rather than meaning (semantics). An example would be mapping <surname>Flurry</surname> to <secondName>Flurry</secondName>, or <DOB>1960-11-03</DOB> to <DateOfBirth>3rd November 1960</DateOfBirth>. Notice that even the data itself might change in format, but its meaning is the same. Translation is typically taking place when there is only one source of data. It would be difficult to change a single source of data into something with different meaning. As an analogy, a good language translator could say something in English and French and have it carry the same meaning, even though the words, phrases, and even sentence structure might be different. The translator is purely mediating between the languages, not adding new meaning. Indeed, adding new meaning is beyond a language translator's role and could, at best, cause confusion or, at worst, offense. Transformation, on the other hand, is used here to denote a change in meaning (semantic). This typically occurs when you transform (or, merge) data from multiple sources, which enables you to create a new entity with a different meaning. Therefore, translation is what you will expect to see in service
developerWorks
exposure, where you are not doing semantic work, and transformation is likely to happen in provider creation (described later), where new entities are being created.
Notice the significant differences between Figure 2 and Figure 4. The first difference is that a mediation acts as an intermediary between the provider and the requester. A mediation is introduced to enable a requestor to interact with a provider that has different characteristics -- but different in what way? Part 1 of the earlier article series explained that "services implement the semantics associated with achieving business goals" while mediation "only modifies syntax associated with achieving the necessary interconnectivity." We should adjust and clarify these statements, based on the discussion in the previous sections. That article, if interpreted correctly, makes it clear that mediation cannot change the semantic characteristics of the interaction. Based on the understanding of the service specification, it is better to say that mediation can manipulate the non-semantic characteristics of an interaction. These include both the syntactic and the operational (quality of service) characteristics in a service specification. Mediations perform a set of actions on the messages that pass through them to mediate the non-semantic characteristics; the actions include protocol switching, translation, encryption, and routing. The next difference between Figures 2 and 4 is that the provider alone is no longer a service realization. This reflects that the provider does not comply with a well-governed service specification. The provider does supply some set of semantic, syntactic, and operational characteristics; in the majority of cases, however, these are not formally defined by or governed in relation to the service model. To distinguish that ungoverned set of characteristics from a service specification, we use the term provider description. The final difference between Figures 2 and 4 is that the service realization is, in effect, a combination of the mediation and the provider. You can say that the mediation augments the provider by exposing it according to the service specification; that is, it assists the provider in satisfying the characteristics in the service specification. So in this context, the mediation is performing service exposure, which is the act of bringing an ungoverned capability in line with a domain of governance by using a mediation to reconcile the non-semantic characteristics. You can now consider that mediations are used within SOA to expose services. It is really more precise to say that a mediation is used to expose a provider according to a service specification; the net result is a service realization. Let's now examine the nature of the service specification in mediated service interaction. Due to separation of concerns, the mediation delegates all responsibility
Page 11 of 26
developerWorks
for semantic characteristics in the service specification to the provider. Thus, the semantic characteristics in the provider description (formal or informal) are reflected in the semantic characteristics of the service specification, as shown by the solid arrow in Figure 4.
Protocol switch versus conversion During service exposure there is often a need to enable one communication protocol between the requester and the mediation and another between the mediation and the provider. A common term used is to switch protocols, but sometimes the term conversion is also used. We believe switch is the most accurate term. In reality, one protocol is used in the interaction with the requester and a second one in the interaction with the provider; there is no conversion.
The mediation supplies all the syntactic characteristics of the service specification, since it is the interaction point for the requester and is wrapping any syntactic characteristics of the provider, as implied by the lack of any relationship to the provider description. The operational characteristics in the service specification are derived from a combination of the operational characteristics of the provider (in the formal or informal provider description) and those of the mediation itself. For example, characteristics such as response time and throughput are clearly impacted by the operational containers for both the mediation and the provider. This is implied by the dotted arrow in the figure. Let's once again restate our observations on service interaction, this time for mediated interaction. Given the strong emphasis on separation of concerns in SOA, the roles of provider and mediation in performing service exposure are architecturally distinct and different: The semantic characteristics offered by the provider must satisfy those in the service specification. The syntactic characteristics offered by the provider need not satisfy those in the service specification, since the mediation acts as an intermediary. The operational quality of service characteristics offered by the provider must exceed those in the service specification, since the mediation can typically only degrade such characteristics. There are some subtle exceptions to this, such as a mediation that actually improves a provider's availability by routing to an alternate provider under appropriate circumstances, but that is beyond the scope of this article. This distinction between mediation and provider is critical to separation of concerns. If the mediation contributes to the semantic characteristics in a service specification, it would become a provider. This would be clear violation of separation of concerns that can lead to a various issues such as confusion as to where to "find" the
developerWorks
implementation of the business task when you need to change it, and indeed who "owns" it from a change control point of view.
Service exposure: Exposing a provider as a governed service The term service exposure will be used regularly from this point on in the article, so it is important to understand that it means to expose a provider as a governed service. We often hear the phrase "expose a service" when talking about service exposure. Unfortunately, this can be misleading as it suggests that a governed service was already there, and you just had to uncover it. You should interpret "service exposure" as making a provider available as a governed service, or put even more explicitly: "expose an existing ungoverned provider as a governed service according to a service specification."
Based on the above, you can state some principles about mediation used for service exposure from an architectural perspective: Semantic transparency: A mediation adds no semantic capabilities. It exposes the provider by satisfying the remaining characteristics in the service specification. Reactive versus proactive: A mediation only interacts with a provider as a result of an interaction with that mediation initiated by a requester. One-in-one-out: Because the provider supplies all the semantic characteristics of the service specification, for each request for a service operation, there will logically be only one invocation of a provider to satisfy the semantic characteristics of that service operation. Stateless: The mediation persists no state relating to a request after the request is satisfied; for example, after a response is returned to the requester. These principles can be very important for identifying the sometimes subtle distinction between the role of a mediation for service exposure and the role of a provider.
The ESB in SOA

Notice there is no ESB shown in Figure 4 and no mention of the ESB in the related discussion. Why? Part 1 of the ESB article series identified the ESB as an architectural pattern in SOA; that is true, but not precise. That article also suggested that an ESB supports mediation flows (mediations); that is both true and precise. While it is common now (it was not when that article was written) to say that the ESB exposes services, it is more precise to say it is the individual mediations that expose providers according to service specifications.
Page 13 of 26
developerWorks
This means that from a pure architectural perspective, service exposure via mediation, supporting the principles in the previous section, is the important architectural pattern in SOA. Thus, it is more precise to say that the ESB architectural pattern is support for a collection of individual mediations performing service exposure. In the earlier article, the term "mediation" was intended to always refer to service exposure. However, we frequently find the term used to refer to any use of capabilities (such as translate and switch) that one finds in the broader topic of integration. This is typically a result of confusion over the role of service exposure, or use of a product that supports mediation for service exposure and other roles. We will discuss such confusion in more detail in the next sections. An ESB can thus be considered an architectural container for mediations performing the role of service exposure, as shown in Figure 5. The figure also shows that the ESB itself is hosted in one or more operational containers that host mediations used to expose providers as governed services for an organization, and enables the convenient construction, configuration, and administration of mediations. These containers are often in the form of products. Figure 5. SOA topology
Service bus vs. enterprise service bus Notice that Figure 5 doesn't actually use the term ESB, but instead uses service bus for the architectural pattern. This represents additional terminology evolution to reflect the realities of SOA. It has become very common to expose providers as
developerWorks
governed services within a particular domain (for example, a department, application, or line of business); to assume that all governed services are exposed at the "enterprise" level is usually incorrect. In such situations, a domain uses a domain service bus to expose governed services, most of which are used only within the domain, but some of which are used across the enterprise. The "enterprise" service bus exists only logically and service federation (see part 4) supports the sharing of governed services across the enterprise. All that said, ESB is an established term, so it will be in common use for some time to come. However, you should recognize that not all "ESBs" span the enterprise. Doing so will also diminish the chances of terms such as "Enterprise ESB" from becoming common. Service registry and repository While we're considering operational containers, where would service specifications be contained? As they represent governed entities, service specifications ideally belong in a service registry and repository that supports fully governed service lifecycle management, including the service model, the resulting service specifications and associated metadata. In some ways, then, the service registry and repository becomes a container for service specifications (Figure 5). It is important to understand that the existence of a service bus does not define the service model or mandate service specifications or the existence of the service registry and repository; governance provides the motivation.
Integration in the context of SOA

This section examines the nature of an integration layer in the context of SOA and relates it to the role of mediation and the service bus. The earlier article positioned the ESB as an "integration layer." This assertion needs significant clarification to reduce confusion and to align with the prescriptive concepts and terms in this paper. "Integration layer" and "integration" are both very ambiguous terms, and their definitions are open to interpretation depending on the context. In nearly all contexts (even SOA), the definition of integration derives from enterprise application integration (EAI), and the definition is "whatever it takes to allow one application to interact with another application." In EAI, integration certainly deals with the non-semantic mismatches between applications, but in EAI it commonly also deals with the semantic mismatches between applications. Semantic mismatches arise where the expected task or outcome of the calling application is not matched by any single interaction with another application. The key point is that no single interaction achieves the meaning of the required service operation. A clear cut example might be for a bookTripItineray() operation which might need to do the following; takePayment(), reserveAccomodation(), bookFlight(),
Page 15 of 26
developerWorks
updateItineraryStatus(), and so on. No individual back end operation performs the required semantics of booking a "trip." So something somewhere has to understand the meaning of those different interactions and tie them together. Thus in EAI, "integration" includes both semantic and non-semantic actions, and an "integration layer" can perform both. Do we need integration of this sort in SOA? Absolutely! You saw above that existing providers sometimes offer the semantic characteristics required by a service specification but do not comply with the non-semantics characteristics. Mediation was introduced to deal with such non-semantic mismatches to expose governed services. In SOA, as well as EAI, you must recognize that existing providers cannot always offer even the semantic characteristics required by the service specifications. Indeed, early in an SOA's maturity cycle, organizations spend most of their time defining and implementing "integrations" in order to build out the service model from existing provider assets. So, in fact, integration is just as important in SOA as in EAI. How do you achieve integration in SOA? In reality, much the same way as in EAI, but with some significant differences in emphasis; EAI is about letting individual applications interact one-on-one with other applications, whereas SOA is about an organization-wide, shared, reusable service model used by all applications. In SOA, as in mature EAI, separation of concerns is key, and thus an "integration layer" in SOA might perform both semantic and non-semantic actions, but must keep them separate architecturally. We clarified above that a mediation used for service exposure can only manipulate the non-semantic characteristics of an interaction. Thus, the service bus can provide only a non-semantic "sub-layer" in an integration layer in SOA. Other parts of the integration layer must manipulate the semantics if the existing providers are unable to satisfy the demands of the semantic characteristics of the service specifications. Thus, it should be very clear that a service bus is not an entire integration layer, at least not an integration layer as defined by most. It should also be clear, however, that a service bus, since it performs non-semantic service exposure included in integration, is part of an integration layer, and that mediation for service exposure is a form of integration. For completeness, we should note that in the rare case where existing providers match exactly the semantics of the service model, the service bus can be considered a limited integration layer. Let's use Figure 6 to elaborate. The top service realization in the figure is identical to that described in Figure 4. A mediation, when used for service exposure, and thereby part of the service bus, handles the non-semantic mismatches between the service specification and an existing provider that matches the semantics implied by the service specification. Thus, the service bus provides a service exposure layer encapsulated by a more functional integration layer. Figure 6. Integration layer, mediation and creation
developerWorks
Now, look at the bottom service realization in Figure 6. In this case, there is no existing provider that matches the semantics implied by the service specification. Typically, this arises where the granularity of the business task is larger than any one provider interaction offers. Creating a larger-grained business task that matches the semantics requires a sequence of interactions, often with intervening "logic" (including the creation of business entities and complex error handling). This is a very different set of capabilities than you expect from mediation used for service exposure. You need an architecturally separate layer to do this semantic work. In a general sense, this additional layer creates larger-grained business tasks from smaller-grained business tasks. In some contexts, this is called service creation, roughly the act of creating a business task that was not present before in other words, implementing new semantic characteristics. A more precise description is that this additional layer performs provider creation as the new semantics must mean an existing provider is not present and even the new provider need not (or even cannot) offer the non-semantic characteristics in the service specification. Figure 6 shows the relationship of the encompassing integration layer to the service exposure layer, the provider creation layer, existing providers, service specifications, and service realizations. The figure emphasizes that mediation (for service exposure) and provider creation contribute to service realization, but they are architecturally distinct due to separation of concerns. It is important to note that, architecturally, the service bus supplies only the service exposure layer.
Page 17 of 26
developerWorks
Let's briefly examine two scenarios that demonstrate the differences between service exposure and provider creation. In the first scenario, shown in Figure 7, assume an existing provider that does exactly what is required by a service specification for the service operation (business task, business and technical interface, protocol/format, and so on), with the exception of the need for every request to the provider to be secured via HTTPS. Figure 7. Service exposure in an integration layer
The addition of encryption is a perfect example of a non-semantic contribution to the service specification. The provider implements the business task and, in this case, the majority (but not all) of the non-semantic characteristics. The mediation enables HTTPS without any change to the provider. It is very clear that the mediation adds absolutely nothing to the semantics in this scenario. As above, mediation for service exposure exists only in the service bus. In the second scenario, shown in Figure 8, there is no existing provider that even remotely delivers the semantic characteristics in the service specification for the service operation. It becomes necessary to interact with several providers, usually with additional intervening logic, to satisfy the semantic characteristics of the specification. Mediation capability is often leveraged simply to support interaction
developerWorks
with the existing providers, not for service exposure. In effect, we are turning a collection of finer-grained business tasks into a single coarser-grained business task consistent with the service specification for the service operation. A concrete example might be retrieving a customer account, whereby you need account information from one system and customer information from another. Neither provider holds the complete semantic characteristics, so it is quite clear that semantic activity must be present to meaningfully merge these different entities; this is well beyond service exposure and into the realm of provider creation. Separation of concerns ensures that the mediation performing the non-semantic service exposure is implemented independently of the composition. Figure 8. Provider creation and mediation in an integration layer
In summary, you see that an integration layer in SOA architecturally contains a purely non-semantic service exposure sub-layer that is fulfilled by the service bus pattern and richer semantic functionality in a provider creation sub-layer. The former is always present and the latter is almost always present. Thus, a service bus (also known as an ESB) itself is generally not sufficient to be an integration layer in SOA.
Business logic versus integration logic

Next, let's investigate the terms "integration logic" and "business logic" used in one
Page 19 of 26
developerWorks
of the earlier articles in the context of the integration layer and the service bus, as described in this article. We show in this section that both terms are ambiguous and not helpful in describing the nature of the service bus or the integration layer. We also suggest more meaningful and helpful alternatives. The earlier article stated that "business logic [deals with] the semantics associated with achieving business goals" while "integration logic only modifies syntax associated with achieving the necessary interconnectivity." Based on the above descriptions of the integration layer and the service bus, you recognize conflicts; for example, business logic is a form of integration logic. The problem is over-simplification and lack of precise terminology. The comparison should not be business logic versus integration logic, but instead about logic categorized by impact and ownership. Logic impact relates to the characteristics in a service specification: Semantic impact implies contribution only to the semantic characteristics. Non-semantic impact implies contribution only to the non-semantic characteristics. Logic ownership relates to the role or group that defines and changes the logic. In the integration layer, there are two distinct ownership roles that matter: Business defines all business goals (the "why"), whether semantic or non-semantic. Business can define and change any logic, with semantic or non-semantic impact, to achieve business goals. IT implements the infrastructure to help achieve business goals (the "how"). IT can define and change only logic with non-semantic impact to achieve business goals. Figure 9 shows the three valid logic categories that result: business-owned semantic logic, business-owned non-semantic logic, and IT-owned non-semantic logic. IT-owned semantic logic is by definition invalid; semantics are about the meaning of the business task. Figure 9. Logic in the integration layer
developerWorks
It is obvious that business-owned semantic logic exists in the provider creation sub-layer. Indeed, that is the purpose of the sub-layer, to introduce new semantics. It is also obvious that IT-owned non-semantic logic exists in the mediation/service exposure sub-layer. This would include IT-owned "policies" or "rules" that could also impact routing, based on operational considerations like provider health. We assert IT-owned non-semantic logic exists in the provider creation sub-layer as well. Why? One example is when a new provider needs to interact with an existing provider using a different message format; IT should own the non-semantic logic needed to translate messages to enable the interaction. This maintains separation of concerns. A less obvious type is business-owned, non-semantic logic, which exists in the provider creation sub-layer and in the service exposure layer. Such logic would not change fundamental semantics, but, for example, only change parameter values involved in the fundamental semantics. For the service exposure layer, an example would be routing between providers; the routing decision could be based on business-owned "policies" or "rules" to give better qualities of service to requests from premium customers. These are business-owned policies, but they have nothing to do with satisfying the semantic meaning of the request. You can now be more precise in statements about logic in an integration layer in SOA. Mediation in the service exposure layer can contain only non-semantic logic. That logic is usually IT-owned, but can be business-owned. Thus, it is fair to say that mediation (and thus the service bus) can contain business logic, but only if it is non-semantic. The provider creation layer can contain business-owned semantic logic and both types of non-semantic logic. You care about the distinction between the different types of logic because it is important to implement them independently; they will need to be updated by different roles, in differing change cycles, often using different tools, and different governance - in effect, maintaining separation of concerns. This is reasonably obvious between IT-owned and business-owned logic, but what we are also introducing here is that
Page 21 of 26
developerWorks
there should be a clear separation between the two types of business-owned logic, since they will likely be owned and changed by different business owners.
Architecture versus implementation

The discussion here, as throughout the earlier article series, is about architecture, architectural layers, architectural roles, and architectural principles, such as separation of concerns. Architecture alone, however, cannot provide business value; the architecture must be implemented using appropriate technologies or products. Next, we'll discuss the pragmatics of architecting and implementing an SOA, the integration layer in particular. Separation of concerns is critical in SOA, and you need to define the necessary architectural layers to achieve a clean separation of concerns. That is the reason for the integration layer itself and for the mediation/service exposure and provider creation sub-layers in the integration layer. A common cause of failed SOA implementations is failure to define a layered architecture that promotes separation of concerns, leading to an implementation that is inflexible and difficult to maintain. For example, some enterprises fail to appreciate the difference between an integration layer and a service bus, thus mixing service exposure and provider creation, in effect inappropriately mixing the responsibilities of business and IT. This does not mean, however, that an idealized layered architecture must be implemented with no compromises. Indeed, compromises almost always have to be made for reasons of performance, cost, technology limitations, and so on. For example, it is possible to use clean logical layering rather than physical layering to improve performance. The question is not whether to compromise, but what and where are the fewest possible compromises that retain the architectural intent and thus maximum separation of concerns. Obviously, to implement the architecture, you must choose an implementation technology for each of the architectural layers. In SOA, there are often products that target a particular layer. Such a product will be very good for development, maintenance, and run time aspects of that layer, but not ideal for other layers. The technology choice for a layer might, however, be based on criteria in addition to the targeted architectural layers; the criteria can include skills, existing product inventory, development cost (tooling significantly impacts such costs), qualities of service, total cost of ownership, or a combination of these. It is perfectly valid to leverage a technology for a layer it does not target if other criteria are more important. The important consideration is not so much matching products to layers, but that the separation of concerns is preserved as much as possible by ensuring the separate architectural layers defined are still easily identified in the final solution. Figure 10 shows a very common situation with integration products, including products that target the service bus (ESB) pattern. ESB products focus on
developerWorks
capabilities that target non-semantic capabilities, like adaptation, translation, routing, and so on, plus the ability to sequence those primitive capabilities. Of course, these capabilities are leveraged to produce the mediations in the service exposure layer. Figure 10. Architecture and integration products
Those same capabilities can be used in the provider creation layer as well. However, service bus mediation flow capability is not designed for the complex decision logic, exception handling, and state management required by the interaction scenarios you find in provider creation. On the other hand, other business process oriented products (for example, a BPEL engine) focus on complex flow capabilities intended for dealing with compositional and exception logic, an ability to hold, and represent in-process state for longer running processes. The provider creation layer often requires such capabilities. As discussed above, the service exposure layer might use sophisticated capabilities that contribute to greater agility (such as rules) when there is no impact on semantics. An example might help. Consider the provider creation scenario in Figure 8. Assume an environment that contains IBM WebSphere Process Server targeting business processes. WebSphere Process Server includes IBM WebSphere ESB, which is a product targeting mediation. The most optimal solution, leveraging targeted development and run time capabilities, would implement mediation using mediation technology in WebSphere ESB. WebSphere Process Server technology would be
Page 23 of 26
developerWorks
the best choice for new provider creation as it provides sophisticated tooling and run time capabilities suited to new provider creation. However, depending on the complexity of the new provider and the skills of the development team, it is possible to use WebSphere ESB technology for provider creation as well. Again, the important goal is retention of separation of concerns. To summarize, for a successful SOA, you must architect layers with clean separation of concerns, then implement the layers retaining separation of concerns as much as possible. You can use any technology to implement a layer, whether the technology targets that layer or not, as long as compromises are understood and any impact on separation of concerns is minimized. Good governance during the model and assemble lifecycle phases goes a long way towards ensuring appropriate separation of concerns.
Conclusion
This article attempted to bring precision and clarity to a broad range of topics and terminology around the SOA architectural pattern enterprise service bus. In fact, we showed that the term "enterprise service bus" is often inappropriate and that a better term is simply "service bus" without more knowledge of the organization it serves; that said, we also acknowledged the term ESB is unlikely to be deprecated. We showed the term "service" without context and qualification can be confusing and even counterproductive, and strongly suggest the purely logical governed service as a more precise, but still slightly ambiguous alternative. Even more precision comes from using "service specification" to refer to a formal representation of the external semantic, syntactic, and operational characteristics of a governed service and "service realization" to refer to a physical implementation of a service specification. We showed that the term "service bus" in reality is logically a representation of (and physically a container for) a collection of mediations that perform semantically transparent "service exposure;" that is, exposing providers according to service specifications. We showed that the service registry and repository provides the needed governance of (and can be considered a container for) service specifications. We showed that service bus is not a complete integration layer, at least not an EAI-style integration layer that performs both semantic and non-semantic actions. We showed that mediation is a form of integration, and that the service bus provides a non-semantic "service exposure" sub-layer that is part of an integration layer in SOA that includes a "provider creation" sub-layer responsible for providing large-grained providers from one or more small-grained providers. We showed how to be more precise in statements about "logic" in an integration
developerWorks
layer in SOA. We showed that business logic and integration logic are terms that can cause confusion, and identified business-owned semantic logic, business-owned non-semantic logic, and IT-owned non-semantic logic as more precise terms. We further showed the service exposure layer can contain only non-semantic logic, but that includes business-owned non-semantic logic, thus allowing business logic in the service bus (ESB). We showed that for a successful SOA, you should first architect the required layers that guarantee separation of concerns and then choose an implementation technology for the each of those layers, comprising only as much as necessary, thus retaining separation of concerns. We also showed that a product targeting one architectural layer can be used for other layers as well, assuming preservation of separation of concerns.
Acknowledgements
The authors would like to thank the following for their contributions to and reviews of this article: Andy Garratt, Guy Hochstetler, Andy Humphreys, Brian Petrini, Rachel Reinitz, Marc-Thomas Schmidt, Andre Tost.
Page 25 of 26
developerWorks
Resources
Exploring the Enterprise Service Bus, Part 1: Discover how an ESB can help you meet the requirements for your SOA solution The Open Group SOA Work Group The Open Group SOA Ontology The Open Group SOA Governance Exploring the Enterprise Service Bus: Part 4: Federated connectivity in the enterprise Designing an ESB Gateway Patterns: Implementing an SOA using an Enterprise Service Bus Understand Enterprise Service Bus scenarios and solutions in Service-Oriented Architecture, Part 1 IBM developerWorks WebSphere
About the authors

Greg Flurry Greg Flurry is a Distinguished Engineer in IBM's Business Process and Service Optimization group. His responsibilities include working with clients on service oriented solutions and advancing IBM's service oriented product portfolio.
Kim J. Clark Kim Clark is an IT Specialist from the United Kingdom working in IBM Software Services for WebSphere (ISSW). Alongside providing guidance to customers he writes and presents regularly on SOA design. He has been working in the IT industry since 1993 spanning object oriented programming, enterprise application integration (EAI), and SOA. He pioneered many of the early projects using SOA Foundation Suite products. Kim holds a degree in Physics from the University of London, England.

The Enterprise Service Bus, Re-Examined - 1105 - Flurry PDF

Caricato da

Informazioni sul documento

Descrizione originale:

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

The Enterprise Service Bus, Re-Examined - 1105 - Flurry PDF

Caricato da

Copyright:

Formati disponibili

Updating concepts and terminology for an evolved technology

The "service" in SOA

Copyright IBM Corporation 2011. All rights reserved.

Service versus service operation

Copyright IBM Corporation 2011. All rights reserved.

Service governance in SOA

The service specification