Message queue - AbsoluteAstronomy.com

Computer science

Computer science or computing science is the study of the theoretical foundations of information and computation and of practical techniques for their implementation and application in computer systems...

, message queues and mailboxes are software-engineering

Software engineering

Software Engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software...

components used for interprocess communication, or for inter-thread

Thread (computer science)

In computer science, a thread of execution is the smallest unit of processing that can be scheduled by an operating system. The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is contained inside a process...

communication within the same process. They use a queue for messaging – the passing of control or of content. Group communication system

Group communication system

The term Group Communication System refers to a software platform that implements some form of group communication. Examples of group communication systems include IS-IS, JGroups, Spread Toolkit, Appia framework, QuickSilver, and the group services component of IBM's RSCT...

s provide similar kinds of functionality.

Overview

Message queues provide an asynchronous

Asynchronous communication

In telecommunications, asynchronous communication is transmission of data without the use of an external clock signal, where data can be transmitted intermittently rather than in a steady stream. Any timing required to recover data from the communication symbols is encoded within the symbols...

communications protocol

Communications protocol

A communications protocol is a system of digital message formats and rules for exchanging those messages in or between computing systems and in telecommunications...

, meaning that the sender and receiver of the message do not need to interact with the message queue at the same time. Messages placed onto the queue are stored until the recipient retrieves them. Message queues have implicit or explicit limits on the size of data that may be transmitted in a single message and the number of messages that may remain outstanding on the queue.

Many implementations of message queues function internally: within an operating system

Operating system

An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...

or within an application

Application software

Application software, also known as an application or an "app", is computer software designed to help the user to perform specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with...

. Such queues exist for the purposes of that system

System

System is a set of interacting or interdependent components forming an integrated whole....

only.

Other implementations allow the passing of messages between different computer systems, potentially connecting multiple applications and multiple operating systems. These message queueing systems typically provide enhanced resilience

Resilience (network)

In computer networking: “Resilience is the ability to provide and maintain an acceptable level of service in the face of faults and challenges to normal operation.”These services include:* supporting distributed processing* supporting networked storage...

functionality to ensure that messages do not get "lost" in the event of a system failure. Examples of commercial implementations of this kind of message queueing software (also known as message-oriented middleware

Message-oriented middleware

Message-oriented middleware is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple...

) include IBM

IBM

International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...

's WebSphere MQ

WebSphere MQ

IBM WebSphere MQ' is a family of network software products launched by IBM in March 1992. It was previously known as MQSeries, a trademark that IBM rebranded in 2002 to join the suite of WebSphere products. WebSphere MQ, which is often referred to simply as "MQ" by users, is IBM's Message...

(formerly MQ Series) and Oracle Advanced Queuing

Oracle Advanced Queuing

In computing, Oracle Advanced Queuing is a sort of Message Oriented Middleware developed by Oracle Corporation and integrated into its Oracle database....

(AQ). There is a Java

Java (programming language)

Java is a programming language originally developed by James Gosling at Sun Microsystems and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++ but has a simpler object model and fewer low-level facilities...

standard called Java Message Service

Java Message Service

The Java Message Service API is a Java Message Oriented Middleware API for sending messages between two or more clients. JMS is a part of the Java Platform, Enterprise Edition, and is defined by a specification developed under the Java Community Process as JSR 914...

, which has several proprietary

Proprietary software

Proprietary software is computer software licensed under exclusive legal right of the copyright holder. The licensee is given the right to use the software under certain conditions, while restricted from other uses, such as modification, further distribution, or reverse engineering.Complementary...

and free software

Free software

Free software, software libre or libre software is software that can be used, studied, and modified without restriction, and which can be copied and redistributed in modified or unmodified form either without restriction, or with restrictions that only ensure that further recipients can also do...

implementations.

There are a number of open source choices of messaging middleware systems, including JBoss Messaging

JBoss Messaging

JBoss Messaging is the JBoss enterprise asynchronous messaging system. It supersedes JBoss MQ as the default Java Message Service provider in JBoss Application Server 5....

, JORAM

JORAM

is an open-source message broker which is certified to fully implement the Java Message Service API 1.1 . JORAM is available for since May 2000....

, Apache ActiveMQ

Apache ActiveMQ

Apache ActiveMQ is an open source message broker which fully implements the Java Message Service 1.1 . It provides "Enterprise Features" like clustering, multiple message stores, and ability to use any database as a JMS persistence provider besides VM, cache, and journal persistency.Apart from...

, Sun Open Message Queue

Open Message Queue

Open Message Queue is an open source message-oriented middleware project by Sun Microsystems that implements the Java Message Service 1.1 API . In addition to support for the JMS API, OpenMQ provides additional enterprise features including clustering for scalability and high availability, a C...

, Apache Qpid

Apache Qpid

Apache Qpid is an open source messaging system which implements the Advanced Message Queuing Protocol. It provides transaction management, queuing, distribution, security, management, clustering, federation and heterogeneous multi-platform support.-See also:*Apache ActiveMQ*Message-oriented...

, RabbitMQ

RabbitMQ

RabbitMQ is an open source message broker software , using the standard Advanced Message Queuing Protocol . The RabbitMQ server is written in Erlang and is built on the Open Telecom Platform framework for clustering and failover. Rabbit Technologies Ltd., acquired in April 2010 by VMware, develops...

, and HTTPSQS

In addition to open source systems, hardware-based messaging middleware is a growing trend with vendors like Solace Systems

Solace systems

Solace Systems is a content networking company based in Kanata, Ontario, Canada that manufactures and sells middleware appliances. Middleware appliances are rack-mountable network devices that implement message-oriented middleware and content-based routing in purpose-built hardware...

, Sonoa / Apigee and Tervela offering queuing through silicon or silicon/software datapaths.

Most RTOSes, such as VxWorks

VxWorks

VxWorks is a real-time operating system developed as proprietary software by Wind River Systems of Alameda, California, USA. First released in 1987, VxWorks is designed for use in embedded systems.- History :...

and QNX

QNX

QNX is a commercial Unix-like real-time operating system, aimed primarily at the embedded systems market. The product was originally developed by Canadian company, QNX Software Systems, which was later acquired by Canadian BlackBerry-producer Research In Motion.-Description:As a microkernel-based...

operating systems encourage the use of message queueing as the primary IPC or inter-thread

Thread (computer science)

communication mechanism. The resulting tight integration between message passing and CPU scheduling is attributed as a main reason for the usability of RTOSes for real time applications. Early examples of commercial RTOSes that encouraged a message-queue basis to inter-thread communication also include VRTX

Versatile Real-Time Executive

Versatile Real-Time Executive is a real-time operating system nowadays developed and marketed by the company Mentor Graphics. VRTX is suitable for both traditional board-based embedded systems and SoC architectures....

and pSOS

PSOS

pSOS is a real time operating system , created in about 1982 by Alfred Chao, and developed/marketed for the first part of its life by his company Software Components Group...

+, both of which date to the early 1980s.

Usage

In a typical message-queueing implementation, a system administrator

System administrator

A system administrator, IT systems administrator, systems administrator, or sysadmin is a person employed to maintain and operate a computer system and/or network...

installs and configures off-the-shelf message-queueing software (a queue manager

Queue manager

A queue manager is a software program that provides queuing services to applications. It divides work among networked machines....

), and defines a named message queue.

An application then registers a software routine that "listens" for messages placed onto the queue.

Second and subsequent applications may connect to the queue and transfer a message onto it.

The queue-manager software stores the messages until a receiving application connects and then calls the registered software routine. The receiving application then processes the message in an appropriate manner.

There are often numerous options as to the exact semantics of message passing, including:

Durability (e.g. - whether or not queued data can be merely kept in memory, or if it mustn't be lost, and thus must be stored on disk, or, more expensive still, it must be committed more reliably to a DBMS)
Security policies - which applications should have access to these messages?
Message purging policies - queues or messages may have a Time to live
Time to live
Time to live is a mechanism that limits the lifespan of data in a computer or network. TTL may be implemented as a counter or timestamp attached to or embedded in the data. Once the prescribed event count or timespan has elapsed, data is discarded. In computer networking, TTL prevents a data...
Some systems support filtering data so that a subscriber may only see messages matching some pre-specified criteria of interest
Delivery policies - do we need to guarantee that a message is delivered at least once, or no more than once?
Routing policies - in a system with many queue servers, what servers should receive a message or a queue's messages?
Batching policies - should messages be delivered immediately? Or should the system wait a bit and try to deliver many messages at once?
When should a message be considered "enqueued"? When one queue has it? Or when it has been forwarded to at least one remote queue? Or to all queues?
A publisher may need to know when some or all subscribers have received a message.

These are all considerations that can have substantial effects on transaction semantics, system reliability, and system efficiency.

Synchronous vs. asynchronous

Many of the more widely-known communications protocol

Communications protocol

A communications protocol is a system of digital message formats and rules for exchanging those messages in or between computing systems and in telecommunications...

s in use operate synchronously

Synchronization (computer science)

In computer science, synchronization refers to one of two distinct but related concepts: synchronization of processes, and synchronization of data. Process synchronization refers to the idea that multiple processes are to join up or handshake at a certain point, so as to reach an agreement or...

. The HTTP protocol – used in the World Wide Web

World Wide Web

The World Wide Web is a system of interlinked hypertext documents accessed via the Internet...

and in web services – offers an obvious example where a user sends a request for a web page and then waits for a reply.

However, scenarios exist in which synchronous behaviour is not appropriate. For example, AJAX

Ajax

- Mythology :* Ajax , son of Telamon, ruler of Salamis and a hero in the Trojan War, also known as "Ajax the Great"* Ajax the Lesser, son of Oileus, ruler of Locris and the leader of the Locrian contingent during the Trojan War.- People :...

(Asynchronous Javascript

JavaScript

JavaScript is a prototype-based scripting language that is dynamic, weakly typed and has first-class functions. It is a multi-paradigm language, supporting object-oriented, imperative, and functional programming styles....

and XML

XML

Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....

) can be used to asynchronously send text or XML

XML

messages to update part of a web page with more relevant information. Google

Google

Google Inc. is an American multinational public corporation invested in Internet search, cloud computing, and advertising technologies. Google hosts and develops a number of Internet-based services and products, and generates profit primarily from advertising through its AdWords program...

uses this approach for their Google Suggest, a search feature which sends the user's partially typed queries to Google's servers and returns a list of possible full queries the user might be in the process of typing. This list is asynchronously updated as the user types.

Other asynchronous examples exist in event notification systems and publish/subscribe systems.

An application may need to notify another that an event has occurred, but does not need to wait for a response.
In publish/subscribe
Publish/subscribe
Publish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers. Published messages are characterized into classes, without knowledge of what, if any, subscribers there may be...

systems, an application "publishes" information for any number of clients to read.

In both of the above examples it would not make sense for the sender of the information to have to wait if, for example, one of the recipients had crashed.

Applications need not be exclusively synchronous

Synchronization (computer science)

or asynchronous. An interactive application may need to respond to certain parts of a request immediately (such as telling a customer that a sales request has been accepted, and handling the promise to draw on inventory), but may queue other parts (such as completing calculation of billing, forwarding data to the central accounting system, and calling on all sorts of other services) to be done some time later.

In all these sorts of situations, having a subsystem which performs message-queuing (or alternatively, a broadcast messaging system) can help improve the behaviour of the overall system.