Event-driven SOA

Event-driven SOA is a form of service-oriented architecture (SOA), combining the intelligence and proactiveness of event-driven architecture with the organizational capabilities found in service offerings. Before event-driven SOA, the typical SOA platform orchestrated services centrally, through pre-defined business processes, assuming that what should have already been triggered is defined in a business process. This older approach (sometimes called SOA 1.0) does not account for events that occur across, or outside of, specific business processes. Thus complex events, in which a pattern of activities—both non-scheduled and scheduled—should trigger a set of services is not accounted for in traditional SOA 1.0 architecture.

SOA 2.0

SOA 2.0 architecture, ("event-driven SOA"), lets business users monitor, analyze, and enrich events to make the connections among disparate events that do not at first appear to be intuitively obvious. This makes these enriched events visible to others, especially business analysts or marketing directors, and also allows the SOA 2.0 system to possibly automate actions to take to address some unique pattern.^[1]

SOA 2.0 is the ability to create high-level business events from numerous low-level system events. Events are created by filtering real-time data (from middleware, applications, databases, and Web services, for example) and infusing it with defining detail such as dependencies or causal relationships discovered by correlating other events.

If it's clear, through the enriched events that are produced by an SOA 2.0 environment, that customer shopping cart abandonment rate has escalated in the last few days, a notification to the marketing department could initiate research into what competitors have done to cause customers to buy products elsewhere. Was there a common product in most shopping carts? If so, what are the prices that are being offered by the competition?

In practice, this relationship of streamed events is processed through a causal vector engine, which performs a lookup based on recently viewed events and assigns a causal vector to an event if a relationship is discovered. If A causes B, the causal vector engine checks if B’s causal vector rule index contains a reference to A. The engine may handle events for different transactions simultaneously, perhaps in a different order than they occurred.

Unlike sequential or procedural systems (in which clients must poll for change requests), event-driven SOA allows systems and components to respond dynamically, in real time, as events occur. SOA 2.0 complements and extends SOA 1.0 by introducing long-running processing capabilities.

Long running processing capability enables the architecture to collect various asynchronous events over a long period of time and correlate these events into causal relationships. SOA 2.0 event patterns can be designed and implemented to look for event relationships that span days, weeks, or months; and when certain criteria are met, trigger a business process to address the event pattern.

SOA 2.0 event-driven programming is structured around the concept of decoupled relationships between event producers and event consumers: an event consumer doesn't care where or why an event occurs; rather, it's concerned that it will be invoked when the event has occurred. Systems and applications that separate event producers from event consumers typically rely on an event dispatcher, or channel. This channel contains an event queue that acts as an intermediary between event producers and event handlers.

Prototypical SOA 2.0 paradigm

The prototypical SOA 2.0 paradigm contains four essential elements:

1. multiple low-level system events that, separately, do not appear to have any relationship, but through pattern detection by comparing these many events some unusual or less obvious correlation becomes clear;
2. some amount of data enrichment by infusion of related information to each event to more clearly illustrate how the many events are related;
3. a trigger condition which when not met, the business-level event is not created, but when the trigger condition is met, the higher-level business event is created;
4. some human or automated process that is invoked when the trigger event is reached.

SOA 2.0 Web Services can be composed in two ways: orchestration and choreography. In orchestration, a central process takes control over the involved web services and coordinates the execution of different operations on the web services involved in the operation. The involved SOA 2.0 services do not know (and do not need to know) that they are part of a composition or a higher business process. Only the central coordinator of the orchestration knows this, so the orchestration is centralized with explicit definitions of operations and the order of invocation of SOA 2.0 services.

Choreography on the other hand does not rely on a central coordinator. Rather, each SOA 2.0 service involved in the choreography knows exactly when to execute its operations (based on defined trigger criteria) and whom to interact with. Choreography is a collaborative effort focused on exchange of messages. All participants of the choreography need to be aware of the business process, operations to execute, messages to exchange, and the timing of message exchanges.

BPEL follows the orchestration paradigm. Choreography is covered by other standards, such as WSCI (Web Services Choreography Interface) and (Web Services Choreography Description Language).

Multiple low-level system events

Causal relationships are inherent in the world around us and are intrinsic to our decision making. The human intelligence processes and gathers these relationships faster than current artificial computational capability can. One of the fundamental obstacles in artificial intelligence is the absence of an automated ability to relate events together as when a human uses human intuition.

Using a Causal Vector Engine, the perception of causality can be enhanced under appropriate spatiotemporal conditions based on structural and temporal rules written into the engine. Perception of complex causal semantics, such as additive, mediated, and bidirectional causalities need to be coded so that the engine can distinguish between events that are related and those that only appear to be related but, in fact, are not.

The engine uses preponderant causal vector rate-of-change propagation to code the relationship among the events and establishes a partial order in which it validates the causality perceived between multiple occurrences. The engine plays and replays the event sequence in different temporal order to infer what could be related topological connections and compares these replays to rules preprogrammed by an analyst.

Multiple low-level system events are processed by the Causal Vector Engine and compared against these rules to trigger higher-level Business Events. It does this through a Causality Vector Engine (CVE) console application which displays events in real-time to business analysts. Where streams of events can be observed as they occur, much like a stock ticker, the CVE console app has several windows that list the same events in different contexts, so the business analysts can see what the CVE is doing with the relationships between them.

The Sequential window shows events in date-timestamp order, one or more other windows in various orders as the CVE works through the list of rules and creates implied relationships between the events. Various buttons and controls exist in the console application that enable the business analysts to create relationships between events on-the-fly and define rules that respond to these relationships.

Business analysts can infuse additional defining detail through an SQL query statement attached to a rule or event context. The CVE app works much like a modern day stock trading application that mutual funds managers use to manage risk. An example of a CVE application and engine can be seen in SILK. ^[2]

Data enrichment

Most ESB implementations contain a facility called "mediation". For example, mediation flows are part of the WebSphere enterprise service bus intercept. Mule also supports mediation flows. Mediation flows modify messages that are passed between existing services and clients that use those services. A mediation flow mediates or intervenes to provide functions, such as message logging, data transformation, and routing, typically the functions can be implemented using the Interception Design Pattern.^[3]

As messages pass through the ESB, the ESB enriches the messages destined for a channel that is monitoring for a high-level business event. That is, for each message, the ESB may query a database to obtain additional information about some data entity within the message. For example, based on Customer ID, the ESB mediation flow could get the zip code that the customer resides in. Or, based on IP address of the originating request by the end-user, the ESB mediation flow could lookup what country, state or county that IP address is in.

These examples represent data enrichment, the concept of adding additional value to existing data, based on the intent of the high-level business event to eventually be triggered.

Mediation flows

An ESB mediation flow is one of the component types in a Service Component Architecture (SCA). Like any SCA component, the program accesses a mediation flow through exports that it provides, and the mediation flow forwards messages to other external services via imports. Special kinds of imports and exports for JMS, called JMS bindings, enable developers to specify the binding configuration and write data handling code. The mediation flow consists of a series of mediation primitives that manipulate messages as they flow through the bus.

Once the developers have coded the custom binding for both export and import, they can start to focus on the mediation flow component. In the WebSphere Integration Developer assembly editor, this is done by the JMS Custom Binding Mediation Component where each operation on the flow component's interface is represented by a request and a response.

Service Data Objects (SDO) framework provides a unified framework for data application development. With SDO, developers do not need to be familiar with any specific API in order to access and utilize data. Through SDO, developers simply work with data from multiple data sources, such as relational databases, entity EJB components, XML pages, Web services, the Service Component Architecture, and JavaServer Pages pages.

Mediation flows are entirely independent from the bindings that are used in the imports and exports. In fact, the purpose of having a conversion into an SDO DataObject instance outside of the flow implementation is because mediation flows can then be built without knowledge of the protocol and format with which messages are sent to and from the mediation module.

Business-level trigger condition

A business-level trigger condition enables the SOA 2.0 architecture to establish real-time customer intelligence, marketing automation and customer loyalty solutions, among other features. Business objects model real-world entities in the architecture such as customers, accounts, loans, and travel itineraries. When the state of one of these objects changes, and a monitoring agent notices this change is significant (when compared to the list of criteria to monitor), an event is created and passed to other monitoring agents.

For example, the detection of an actual business problem or opportunity could lead to increased revenue. If a customer cancels an order, extra manufacturing capacity could reduce the profitability of the production run. A SOA 2.0 event could notify marketing department to create a special sales campaign that would resell the excess capacity, thereby recapturing the original profitable cost-per-unit.

Automatic monitoring of events in operational business process activities as processes execute to see if any immediate action needs to be taken either inside or outside the enterprise. These monitoring agents continually test for specific business conditions and changes in business operations. If necessary, the agents alert people, make recommendations, send messages to other applications or invoke whole business processes when such conditions or changes occur.

Resulting business process

A triggered business process should directly support revenue growth with cost containment, responsiveness to business conditions, or ability to pursue new market opportunities. Resulting business processes could also measure operational progress toward achieving goals, control operational costs by communicating just what is needed to just who needs to know, or report performance status of key processes to key decision makers.

SOA 2.0 Conceptual Examples

Abandoned Shopping Cart

For example, you could construct a CRM event from an "abandoned shopping cart" message (parsing the transaction, customer ID, and time), using other filters to extract the value of goods in the cart and tapping the correlation capabilities of the system to add causal indicators such as whether the commerce site was suffering performance problems. Your CRM event might also include customer value or rank from the customer database.

Engineering Defect

For another example, based on the types of independent service calls received, the SOA 2.0 platform could identify a product defect by detecting the underlying pattern of the separate complaints, then triggering an alert to engineering or production of the possible defect.

Real-time Electricity Market

Example 3: A potential use of event-driven SOA could be a virtual electricity market^[4] where home clothes dryers can bid on the price of the electricity they use in a real-time market pricing system.^[5] The real-time market price and control system could turn home electricity customers into active participants in managing the power grid and their monthly utility bills.^[6] Customers can set limits on how much they would pay for electricity to run a clothes dryer, for example, and electricity providers willing to transmit power at that price would be alerted over the grid and could sell the electricity to the dryer.^[7]

On one side, consumer devices can bid for power based on how much the owner of the device were willing to pay, set ahead of time by the consumer.^[8] On the other side, suppliers can enter bids automatically from their electricity generators, based on how much it would cost to start up and run the generators. Further, the electricity suppliers could perform real-time market analysis to determine return-on-investment for optimizing profitability or reducing end-user cost of goods.

Event-driven SOA software could allow homeowners to customize many different types of electricity devices found within their home to a desired level of comfort or economy. The event-driven software could also automatically respond to changing electricity prices, in as little as five-minute intervals. For example, to reduce the home owner's electricity usage in peak periods (when electricity is most expensive), the software could automatically lower the target temperature of the thermostat on the central heating system (in winter) or raise the target temperature of the thermostat on the central cooling system (in summer).

The event-driven SOA software could shut off the heating element of water heaters to the pre-set response limits established by individual homeowners. For example, if the market price of electricity for a given hour exceeded the home owner's limit, the home owner could plan to go without recharging the water's hot temperature for that hour, when prices were high, and opt to delay the hot water temperature recharge to the next hour when electricity market prices might be lower.

All these criteria would be managed through the home owner's personal computer with internet connection, programming the various devices around the home to consume electricity only when the management software approves of the consumption. The savings represented by this technique, and enabled by event-driven SOA, is like improving the gas mileage in your vehicle. It makes your home energy use more efficient by enabling the consumption of electricity when the real-time prices are lower and inhibiting the consumption of electricity when real-time prices are higher.

SOA 2.0 Implementations

One mechanism that can be used from most SOA 1.0 Enterprise Service Bus implementations is the publish/subscribe facility. By implementing ESB functionality as Pub/Sub messages, no advanced knowledge of system events is needed to create SOA 2.0 message patterns. After an enterprise has implemented many Publish functions, SOA middleware analysts can set about the task of strategizing which of the available Publish messages could be assembled into a unique pattern to detect an SOA 2.0-enriched trigger.

Causal Vector Engine (CVE) mechanics are implemented simply, with an expandable view of SQL constructs written in stored procedures.^[9] If A causes B, and causality must occur within N number of transactions, then SQL ORDER BY timestamp clause creates a result set that increments a counter of all transactions that occurred within a timeframe, N number of matching B to occurrence A transactions. The creation of additional stored procedures is accomplished through the CVE console application or by using any standard database developer's toolkit.^[10]

Medical Applications

Domain algorithms, such as fever/flu/infection domain logic in the cited reference, are used to derive SQL code that applies the selected business rules to the use case. Using CVEs in SOA environments improve business agility because the application of SOA 2.0 principles identify business opportunities that would have otherwise been missed or identified much later.^[11]

Functional magnetic resonance imaging (fMRI) using Granger causality analysis (GCA) detects causal effects among brain regions. The results of one sample test demonstrated positive causal effect between rFIC and the dorsal anterior cingulate cortex (dACC). ^[12]

Oracle Business Intelligence

The Oracle CVE Analytical Engine uses a set of theoretical models, each of which evaluates some or all of the data. When a business analyst configures causal factors, he/she specifies criteria indicating which models should consider which causal factor. ^[13]

See also

• Service-oriented architecture
• Enterprise service bus
• Middleware analyst
• EMML
• Preponderant causal vector rate-of-change propagation

References