The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for message-oriented middleware. The defining features of AMQP are message orientation, queuing, routing (including point-to-point and publish-and-subscribe), reliability and security.[1]
AMQP mandates the behaviour of the messaging provider and client to the extent that implementations from different vendors are truly interoperable, in the same way as SMTP, HTTP, FTP, etc. have created interoperable systems. Previous attempts to standardize middleware have happened at the API level (e.g. JMS) and this did not create interoperability.[2] Unlike JMS, which merely defines an API, AMQP is a wire-level protocol. A wire-level protocol is a description of the format of the data that is sent across the network as a stream of octets. Consequently any tool that can create and interpret messages that conform to this data format can interoperate with any other compliant tool irrespective of implementation language.
Contents |
AMQP was originally designed to provide a vendor-neutral (i.e. interoperable across multiple vendors) protocol for managing the flow of messages across an enterprise's business systems.
AMQP is middleware to provide a point of rendezvous between back-end systems (data stores and services) and front end systems such as end user applications. The first applications were for trading desks in the financial industry, where real time order and market data are transmitted. Though originally used inside of enterprises, AMQP can easily be used to move messages between organizations.
AMQP lets system architects build common messaging patterns out of a simpler underlying model. Typical messaging patterns are: request-response, in which messages are sent to or from specific recipients, publish-and-subscribe, in which information is distributed to a set of recipients according to various subscription criteria, and round-robin, in which tasks are distributed fairly among a set of recipients. Realistic applications combine these, e.g. round-robin for distributing work plus request-response for sending back responses.
The protocol specification defines a binary wire protocol used between a client and server (also known as a broker). In addition the specification outlines a message queuing model and services that an implementation must provide.
The queuing model of AMQP provides for a wide range of messaging use-cases and further refines the functions of the clients and brokers. The function of brokers can be usefully broken into two kinds: exchanges and message queues. Message queues store messages, and various implementations can achieve various quality of service. For example a slow but tornado-proof message queue would keep redundant copies in multiple geographic regions while a fast but fragile message queue might keep everything in a single process's RAM. To help improve interoperability some of these aspects of the message queues are specified in the protocol, e.g. you can state what you need when asking a message queue broker implementation to create a new queue.
The standard AMQP exchanges have no semantics for storing messages. They route them to queues, which store them on behalf of recipients. Exchanges implement a range of message routing techniques: one-to-one message passing (like email to one recipient), one-to-N (like an email list), one-to-one-of-N (like a queue for the next open checkout), and so on. Since all exchanges accept messages from N senders, AMQP allows all one-to-any routing to be N-to-any. The rules that configure an exchange, known as bindings, can range from very simple (pass everything into this message queue) to procedural inspections of message content. AMQP allows arbitrary exchange semantics through custom exchanges (which can queue, generate, consume, and route messages in any way desired by the implementation).
Messages consist of an envelope of properties used in routing and by applications and a content, of any size. AMQP message contents are opaque binary blobs. Messages are passed between brokers and clients using the protocol commands Basic.Publish and Basic.Deliver. These commands are asynchronous so that conditions that arise from a command's evaluation are signalled by sending additional commands back on the channel that carried the command originally. AMQP also provides a synchronous message delivery command, Basic.Get/Get-Ok.
Examples of error conditions include signalling by an exchange that it could not route a message because no route was found, or signalling that a message queue declined to accept a message (say because it was full). Message brokers may be configured to handle exceptions in different ways. For example, routing the associated message to a dead letter queue or even bringing the broker to a hard stop.
AMQP was developed from mid-2004 to mid-2006 by JPMorgan Chase & Co. and iMatix Corporation who also developed implementations in C/C++ and Java. JPMorgan Chase & Co. and iMatix documented the protocol as an interoperable specification and assigned to a working group that included Cisco Systems, TWIST, IONA Technologies, and iMatix. The working group consists of Bank of America, Barclays, Cisco Systems, Credit Suisse, Deutsche Börse Systems, Envoy Technologies, Inc., Goldman Sachs, HCL Technologies Ltd, Progress Software, iMatix Corporation, INETCO Systems, Informatica Corporation (incl. 29 West), JPMorgan Chase Bank Inc. N.A, Microsoft Corporation, my-Channels, Novell, Red Hat, Inc., Software AG, Solace Systems, StormMQ, Tervela Inc., TWIST Process Innovations ltd, VMware (incl. Rabbit Technologies) and WSO2.
A notable design goal of AMQP was to enable the creation of open standard protocol stacks for business messaging both within and between firms by combining AMQP with one of the many open standards describing business transactions, such as FpML or more generically as a reliable transport for SOAP.
Whilst AMQP originated in the financial services industry, it has general applicability to a broad range of middleware problems.
AMQP defines a number of entities. From a connection perspective the relevant ones are:
The entities used for the actual sending and receiving of messages are all declared on a channel. A declaration assures the issuing client that the entity exists (or was previously declared by another client). Some of the entities are named. The naming is unique within the scope of the entity and its broker. An attempt to declare an entity with the same name results in an error; to change entity properties, one must delete it first (and then redeclare).
Clients have no way to retrieve the list of all available named entities; no such operations are defined in the AMQP specification. Only if the name is known the client can perform operations on the entity.
Names are encoded in UTF-8, must be between 1 and 255 characters in length and must start with a digit, a letter or an underscore character.
Exchanges are the entities to which messages are sent. They are named and have a type as well as properties such as:
Queues are the entities that store received messages. They are named and have properties but not a type. Clients can subscribe to queues to the effect that the message broker delivers (pushes) the contents of the queue to the client. Alternatively clients can pull messages from the queue as they see fit.
Messages are guaranteed to be delivered in the first-in-first-out order, unless certain kinds of rerouting operations (e.g. due to failures) occur or due to the priorities in the message.
The properties of queues are:
Note that queues are scheduled to replace exchanges in AMQP/1.0.
Messages are unnamed and are published to an exchange. A messsage consists of a header and a content body. While the body is opaque data the header contains a number of optional properties:
A binding is a relationship between one queue and one exchange that specifies how messages flow from the exchange to the queue. The binding properties match the routing algorithm used in exchanges. Bindings (and exchange algorithms) can be placed on a curve of increasing complexity:
Not all these are implemented as standard, or by all implementations.
These four entities form the basic model of the AMQP. The key to understand how a message is passed to a queue lies in the relationship between the type of an exchange and the resulting interpretation of message and binding routing keys.
An exchange will deliver up to one copy of a message to a queue if the routing key in the message matches the routing key in a binding (multiple semantically identical bindings will not result in duplication). What constitutes a match depends on the type of exchange:
Other types of exchanges, e.g. vendor-specific, are explicitly permitted in the specification.
The concept of binding named queues to named exchanges has powerful properties (with binding making those two entities independent of each other). It is, for instance, possible to bind a single queue with multiple bindings to the same or to different exchanges. Or multiple consumers can share the name of a queue and bind to it with the same parameters and will therefore get only message that the other consumers did not consume. Or multiple consumers can declare independent queues but share the bindings and get all the message every other consumer would get on the bound exchange with these bindings.
The following specifications of the AMQ protocol have been published, in chronological order:
The 1.0 specification changes the AMQP model illustrated above by removing the concepts of exchanges and bindings, and replacing these with queues and links. This change aims to remedy two problems with the previous approach:
According to John O'Hara [1] however, JPMorganChase and RedHat introduced links into AMQP/1.0 simply to solve an operational problem of slow consumers causing memory build up in brokers.
Other changes include the introduction of a queue addressing schema similar to E-mail and XMPP. This raises addresses to first-class entities, and allows for the publication of service location records using the DNS.
The process of bringing the 1.0 Specification to a Standard involves a requirement elicitation phase, then the release of a "public review" spec (PR) which should be reviewed and asked for comments, optionally resulting in further modifications. When there are no substantive changes to the PR, it is voted to be the 1.0 Recommendation.[3] At least two implementations have passed the test coverage. The Recommendation has been voted to be 1.0 Standard.[4] As of 31 August 2011[update], a the final spec has been produced[5] and was accepted by the AMQP working group.
These are the known publicly available AMQP implementations:
There are many clients, including:
These are the known open specifications that cover the same or similar space as AMQP:
There are also vendor specific, proprietary specifications includes those by the Amazon Simple Queue Service, IBM WebSphere MQ, Microsoft Message Queuing, JMS and the OpenWire as used by ActiveMQ.
There has not as yet been a formal comparison of these and other protocols in the same space, although an informal comparison of XMPP and AMQP may be found here. JMS, the Java Messaging service, is often compared to AMQP. However, JMS is an API specification (part of the Java EE specification) that defines how message producers and consumers are implemented. JMS does not guarantee interoperability between implementations, and the JMS-compliant messaging system in use may need to be deployed on both client and server. On the other hand, AMQP is a wire-level protocol specification. In theory AMQP provides interoperability as different AMQP-compliant software can be deployed on the client and server sides. Note that, like HTTP and XMPP, AMQP does not have a standard API.