IBM WebSphere MQ

IBM MQ is a family of network software products that IBM launched in March 1992. It was originally called MQSeries, and was renamed WebSphere MQ in 2002 to join the suite of WebSphere products. In April 2014, it was renamed IBM MQ.

IBM MQ, (often referred to as "MQ"), is IBM's Messaging solution for Enterprise and IBM's Message Oriented Middleware offering. It allows independent and potentially non-concurrent applications on a distributed system to securely communicate with each other. MQ is available on a large number of platforms (both IBM and non-IBM), including z/OS (mainframe), OS/400 (IBM System i or AS/400), Transaction Processing Facility, UNIX (AIX, HP-UX, Solaris), HP NonStop, OpenVMS, Linux, OS 2200, and Microsoft Windows.

Message-oriented middleware

IBM MQ was the most popular^[1] system for messaging across multiple platforms in 2000, including Windows, Linux, OS/2, IBM mainframe and midrange, and Unix.

There are two parts to Message queue (MQ):

• Message: Messages are collections of binary or character (for instance ASCII or EBCDIC) data that have some meaning to a participating program. As in other communications protocols, storage, routing, and delivery information is added to the message before transmission and stripped from the message prior to delivery to the receiving application.
• Queue: Message queues are objects that store messages in an application.

A Queue Manager, although not strictly required for message-oriented middleware, is an IBM MQ prerequisite and system service that provides a logical container for the message queue, and is responsible for transferring data to other queue managers via message channels.

Programs integrated with IBM MQ use a consistent application program interface (API) across all platforms.

Support of service-oriented architecture

IBM MQ is a key component in IBM's service-oriented architecture (SOA) strategy, providing the universal messaging backbone across 80 different platforms. The growing importance of SOA and the growth of Web Services and other connectivity mechanisms are clearly important developments. Because of the loosely coupled nature of the message queuing model, a large number of existing MQ customers feel that they are already adopting SOA principles. The M.Q Service definition support pack MA93^[2] allows MQ applications to be catalogued as software assets that can be reused and composed as Web Services.

MQ also supports Enterprise Service Bus implementations, both proprietary and open source. For example, the Mule WMQ transport is available with Mule Enterprise Edition version 1.6 or later and, as of 2.2, contains many critical performance and reliability improvements as well as native support for WebSphere MQ-specific features.

APIs

There are many ways to access IBM MQ facilities.

Some of the APIs directly supported by IBM are:

• IBM Message Queue Interface (MQI) for C, COBOL, PL/I, Java, Rexx,^[3] RPG, and C++
• Java Message Service (JMS)
• XMS for C/C++ and .NET^[4]
• .NET
• Representational State Transfer
• SOAP

Additional APIs (not officially supported) are also available via third parties, for example:

• Perl interface (developed and contributed by Hildo Biersma), available from CPAN.^[5]
• Python (programming language) interface PyMQI (originally developed by Les Smithson), available from PyPI^[6]
• Windows PowerShell^[7]

Features

IBM MQ provides assured one-time delivery of messages across a wide variety of platforms. The product emphasizes reliability and robustness of message traffic, and ensures that a message should never be lost if MQ is appropriately configured.

It needs to be remembered that a message in the context of MQ has no implication other than a gathering of data. MQ is very generalized and can be used as a robust substitute for many forms of intercommunication. For example, it can be used to implement reliable delivery of large files as a substitute for FTP.

MQ provides application designers with a mechanism to achieve non-time-dependent architecture. Messages can be sent from one application to another, regardless of whether the applications are running at the same time. If a message receiver application is not running when a sender sends it a message, the queue manager will hold the message until the receiver asks for it. Ordering of all messages is preserved, by default this is in FIFO order of receipt at the local queue within priority of the message.

It provides a means for transforming data between different architectures and protocols, such as Big Endian to Little Endian, or EBCDIC to ASCII. This is accomplished through the use of message data exits. Exits are compiled applications that run on the queue manager host, and are executed by the IBM MQ software at the time data transformation is needed.

IBM MQ allows receipt of messages to "trigger" other applications to run, and thus provides the framework for a message driven architecture.

It implements the Java Message Service (JMS) standard API, and also has its own proprietary API, known as the Message Queuing Interface (MQI), which preceded the JMS several years in existence.

Unlike email, MQ itself is responsible for determining the destination of messages by the definition of queues, so processing of sent messages can be moved to a different application at a different destination. MQ provides a robust routing architecture, allowing messages to be routed via alternative paths around a network of MQ managers. MQ can be implemented as a cluster, where multiple MQ implementations share the processing of messages to allow higher performance and load balancing.

Communication

The primary component of an IBM MQ installation is the Queue Manager. The queue manager handles storage, timing issues, triggering, and all other functions not directly related to actual movement of data.

Queue managers communicate with the outside world either via a direct software connection, referred to by IBM as a "bindings" connection, or via a network or "client" connection. The bindings connection is limited to programs running on the same physical host as the queue manager, whereas applications using a client connection can connect to a queue manager on any other host in the network.

Bindings connections are generally faster, but client connections allow for a more robust, easily changeable application design. For instance, with a client connection, the physical location of the queue manager is irrelevant, as long as it is reachable over the network.

Communication between queue managers relies on a channel. Each queue manager uses one or more channels to send and receive data to other queue managers. A channel is uni-directional; a second channel is required to return data. In a TCP/IP based network, a channel will send or receive data on a specific port. A sending channel has a defined destination and is associated with a specific transmission queue, the mechanism by which messages are queued awaiting transmission on the channel; a receiving channel will receive data from any other queue manager with a sending channel of the same name. When a receiving channel receives a message, it is examined to see which queue manager and queue it is destined for. In the event of a communications failure, MQ can automatically re-establish a connection when the problem is resolved.

The "listener" has the function of detecting connections from incoming channels and managing the connection of the sending to the receiving channels. It is the application's network interface to the queue manager. In a TCP/IP network, the listener will "listen" for connections on a specific port.

Local queues represent the location where data is stored awaiting processing.

Remote queues represent a queue on another queue manager. They define the destination queue, which is one element of the routing mechanism for messages.

To transmit data to a queue on another queue manager, a message is placed on a remote queue. A remote queue is sent via the temporary storage transmission queue associated with a channel. On placing a message on a remote queue, the message will be transmitted across the remote channel. If the transmission is successful the message is removed from the transmit queue. On receiving a message, the receiving queue manager will examine the message to determine whether the message is for itself or is required to forward on to another queue manager. If it is the destination, the required queue will be checked, and if it exists, the message will be placed on this queue, if not, placed on the dead letter queue. MQ has features to manage efficient transmission of data across a variety of communication media, so for example messages can be batched together until a queue reaches a particular depth.

Although the queue is FIFO, it is ordered based on the receipt in the local queue, not the committing of the message from the sender. Messages can be prioritized, and by default, the queue is prioritized in order of arrival. Queues will only be in sequence of addition if the message is added locally. Message grouping can be used to ensure a set of messages are in a specific order, aside from that, if sequence is critical, it is the application's responsibility to place sequence data in the message or implement a handshaking mechanism via a return queue. In reality, ordering will be maintained in straightforward configurations.

The other element of a queue manager is the log, the mechanism used to create the robustness. As a message is placed on a queue or a configuration change is made, the data is also logged. In the event of a failure, the log is used to recreate damaged objects and recreate messages. Only "persistent" messages will be recreated when a failure occurs—"non-persistent" messages are lost. Non-persistent messages can be sent across a channel set to a fast mode, in which delivery is not assured in the event of a channel failure.

MQ supports a wide variety of approaches to application development. Information can be retrieved from queues either by polling the queue to check for available data at suitable intervals, or alternatively MQ can trigger an event, allowing a client application to respond to the delivery of a message.

High availability

As the messaging infrastructure becomes more critical to your business, the need for a high availability configuration becomes even more important. There are three software-based solutions available with IBM MQ; Queue Manager Clusters, Shared Queues, and Multi-Instance Queue Managers.

Queue Manager Clusters: Groups of two or more queue managers on one or more computers are defined to a cluster, providing automatic interconnection, and allow queues to be shared among them for load balancing and redundancy.

Shared Queues (z/OS only): In a Shared Queue environment, an application can connect to any of the queue managers within the queue-sharing group. Because all the queue managers in the queue-sharing group can access the same set of shared queues, the application does not depend on the availability of a particular queue manager. This gives greater availability if a queue manager stops because all the other queue managers in the queue-sharing group can continue processing the queue.

Multi-Instance Queue Managers (available in v7.0.1): Instances of the same queue manager are configured on two or more computers with their queues and meta data residing on shared storage. By starting multiple instances, one instance becomes the active instance and the other instances become standbys. If the active instance fails, a standby instance running on a different computer automatically takes over.

History

Version release dates

Background architectural reference

Developed in Scotland, Telecommunications Access Method (TCAM) came along in 1971 with the birth of TSO. It supported asynchronous messaging, as with MQ. TCAM 3.0 added in reusable disk message queues for recovery soon thereafter, as with MQ. A high-level PL/I program could be used to access TRANSIENT datasets (dynamic message queues). Reading a message from a transient dataset resulted in that message being removed from the queue, as with a non-browse READ with MQ. With the advent of computers, IBM saw an opportunity to apply new technology to the need for message switching. In the early 1960s, IBM marketed computer-like devices, such as the 7740 and 7750 message switching systems.

The IBM System/360 was announced in April 1964 and with it came communication access methods such as BTAM and QTAM (Basic and Queued Telecommunications Access Methods). In the late 1960s, TCAM, the Telecommunications Access Method, offered its users a more advanced form of message switching or message routing. TCAM was widely accepted, especially in the financial and brokerage industries.

In the late 1970s, transaction management systems came into being, each trying to achieve a leadership position in the industry. Within IBM, CICS and IMS were chosen as strategic products to address the need for transaction management. Within both CICS and IMS, each had its version of message switching, IMS being a front-end queued system and CICS having its Transient Data facility as the possible basis for message switching.[citation??]

CICS established itself as a popular transaction management system in the 1968-1971 timeframe. Those users who had adopted TCAM for its message handling capabilities, now wanted a combined use of TCAM with CICS. In December 1971, IBM announced CICS support of TCAM as part of the CICS/OS-Standard product, to be delivered in December 1972. For interested customers, this enabled them to use TCAM for its message handling strengths and also have TCAM-connected terminals or computers interface with CICS online applications.[citation??]

In January 1973, TCAM continued to be supported by CICS/OS-Standard Version 2.3. However, TCAM support was omitted from the initial release of CICS/VS, announced in February 1973 and delivered in June 1974. Needless to say, many CICS-TCAM customers were not happy with that product direction.

With considerable pressure from CICS-TCAM customers, the CICS support of TCAM was reinstated in the CICS/VS 1.1 product, as of September 1974. In addition to the previous DCB support, with this reinstatement of TCAM support, CICS began to support TCAM access via VTAM, also known as the ACB support. CICS TCAM ACB support was discontinued as of the CICS/ESA Version 3 product in 1990.

In 1992, IBM announced a new product called MQSeries. This brand name was later renamed to "WebSphere MQ" (sometimes shortened to WMQ) in 2002 to support the WebSphere family name and the product. In 2014, it was renamed "IBM MQ". MQ was to be the extension of TCAM functionality from IBM-only systems to all other platforms. MQ has an architecture that enables heterogeneous systems to communicate with each other (e.g. IBM, HP, Sun, Tandem, etc.). MQ can be used with CICS systems to send and receive data to/from any other MQ-eligible system. MQ can be used to initiate work in a CICS system or a CICS transaction can initiate work in another CICS or non-CICS system.

IBM MQ now supports 80 different environments and has become the leading message assured delivery switching/routing product in the industry.^[9]

MQ and web services

IBM MQ can be used as a foundation for creating service-oriented architectures. Several additional product options exist to help convert legacy programs into functioning web services through the use of MQ. Larger, heterogeneous enterprises often appear as a federation of somewhat autonomous domains based on lines of business, functional or governance areas. In such environments, some services may be shared or reused only within a single domain, while others may be shared or reused throughout the enterprise. IBM MQ provides the means by which communication exists between lines-of-business or otherwise separate business domains.

A related product in the IBM MQ product family, called IBM Integration Bus (formerly WebSphere Message Broker), enables a diverse and robust set of extensions to queue-based architectures. Using IBM Integration Bus, users can implement a WebServices front-end, complete with WSDL file support that can interact with any queue-based application.

See also

• Advanced Message Queuing Protocol
• Amazon Simple Queue Service
• IBM WebSphere Message Broker
• IronMQ, example of Message Queuing as a Service
• Java Message Service
• Message queue
• Message Queuing as a Service
• Microsoft Message Queuing
• 0MQ

References

External links

Listen to this article (info/dl)

This audio file was created from a revision of the "IBM WebSphere MQ" article dated 2011-10-29, and does not reflect subsequent edits to the article. (Audio help)

More spoken articles

• IBM's Product Overview
• Product page
• WebSphere MQ Official Twitter page
• Introductory Whitepaper
• Trial download
• IBM Redbook: WebSphere MQ V6 Fundamentals
• WebSphere MQ newsgroups
• MQSeries Installation, Configuring inetd, xinet, Using JMSAdmin (In French)
• MQConnect for Unisys MCP
• MQ commands
• IBM Developerworks starting point
• WebSphere MQ Security Trends Report
• 3rd party WebSphere MQ Client
• Lighthouse for IBM MQ. Monitoring, Browsing and Deployment tool