Message Queues and more

Overview

  • Messaging
    • Message Oriented Middleware
    • Enterprise Application Integration
    • Messaging models
  • Messaging Standards
    • Java Message Service.
    • AMQP.
    • Other wired protocols.
  • Provider Examples
    • ActiveMQ
    • RabbitMQ
    • Kafka

Messaging

Messaging

  • Messaging – exchange of messages between client applications.

 

  • Message – formatted data describing events, requests and replies.

Messaging allows frequent immediate reliable and asynchronous message transfer between applications with custom message formats.

Message Oriented Middleware

  • Middleware - software components used to enable elements of a distributed system to communicate with each other.

 

  • Message oriented middleware (MOM) – asynchronously working middleware that provides message-based communication between software components in a distributed system.

 

  • MOM allows routing, transformation of messages to a different format and is asynchron, as opposed to other middleware paradigm, ORB and RPC. Components that communicate using MOM are loosely coupled.

 

  • MOM requires at least one additional component in the system (so called message broker).

Enterprise Application Integration

  • Combination of software and architectural principles intended to integrate a number of enterprise applications.
  • Can be considered a middleware framework.

Purpose:

  • Data Integration – information kept consistent throughout the system.

 

  • Vendor independence – making it possible to change one application for an other without having to change the system‘s logic.

 

  • Facilitating development, deployment and maintaining.

 

  • Common frontend – possibility to use one facade for multiple applications.

 

Messaging Models

  • Message Queue
    • Relies on the idea of point to point communication.
    • Message will be deleted after it‘s read by one of the consumers.
    • Load balancing possible. According logic can be implemented in the message broker.

 

  • Publish / Subscribe
    • Producers publish messages to different categories.
    • Subscribers consume messages from categories they are interested in.
    • Retention policy depends on implementation. (wait for every subscriber, fanout ignoring inactive subscribers or use time span for deleting messages).
    • Producers and consumers do not need to know anything about each other.

Hybrid models are also possible.

Messaging Standards

Java Message Service

Java Message Service (JMS) – is a message oriented middleware API that provides features and messaging concepts in order for software components to create, send, receive and read messages.

 

  • Minimizes set of messaging concepts, yet supports creating sophisticated messaging applications.
  • Each JMS compliant application is (1) Asynchronous and (2) Reliable
    • (1) – clients do not have to request messages in order to receive them
    • (2) – JMS API is able to ensure a messages is delivered exactly once. However, lower levels of reliability are also available.
  • JMS supports both Message Queue and Publish / Subscribe paradigms.

JMS Elements

  • Provider – a Java JMS implementation or an adapter to a MOM written in an other programming language.

 

  • Producer – application or process that creates and sends messages

 

  • Consumer – application or process that receives messages.

 

  • Queue – storage that contains messages that have been sent, but not yet consumed. FIFO principle is not applicable here.

 

  • Topic – storage or class of messages that are to be distributed to more than one consumer.

JMS Interfaces

  • ConnectionFactory - provides functionality for creating connections for clients.
  • Connection - channel between a client and the messaging service. It is thread safe and can usually be accessedby multiple clients
  • Session - a serial order for producing and consuming messages. Should only be accessed by one thread at a time. Can be made transacted.
  • Destination - represents the area (normally a remote one) for placing and acquiring messages. Either Queue or Topic. 
  • MessageProducer - Represents a message's producer
  • MessageConsumer - Represents a message's consumer
  • Message - Root interface for JMS Message, that are further divided into types:
ByteMessage TextMessage StreamMessage MapMessage ObjectMessage

Message

Consider changes in JMS 2.0 (May 2013).

Advanced message queuing protocol

  • Efficient flow control schemes.
  • Different reliability levels.
  • Flexible - can be used for different types of connection
Feature AMQP 0.9.1 AMQP 1.0
Uses Exchanges Yes No
Easy to apply to existing brokers Not always Yes
Defines sources and targets Yes No, uses abstract adresses instead.
Specifications for message broker Yes No

Other protocols

  • Simple Text-Oriented Messaging Protocol (STOMP)
    • HTTP Based, language-agnostic protocol.
    • Easy to use, yet supports many messaging features, like authentication, transactions, headers and properties and so on.

 

  • Extensible Messaging and Presence Protocol (XMPP)
    • Xml based real-time extensible messaging protocol.
    • Used primarly for instant messaging and contact list maintenance. Newly also for IoT applications, gaming and other purposes.

 

  • Message Queuing Telemetry Transport (MQTT)
    • Lightweight M2M / IoT publish/subscribe messaging protocol.
    • It is fast, reliable (TCP), simple in terms of putting and extracting payload.
    • Examples of usage: home automation, distributing data from smaller sensors to other devices.

Provider Examples

Example: ActiveMQ

  • Supports many languages and protocols.

 

  • Maintains the order of messages in a queue and dispatches them to consumer in order

 

  • Fully JMS compliant

 

  • Additional Features like Message Groups and Virtual Destinations.

Example: RabbitMQ

  • User friendly admin interface, including useful monitoring tools.

 

  • Extended plugin system.

 

  • Flexible routing using different exchange types from AMQP 0.9.1.

 

  • Clustering and Federation of messaging Servers.

Example: Kafka

  • Positions itself as a distributed streaming platform.

 

  • Very high throughput, even using moderate hardware.

 

  • Supports partitioning of messages over messaging servers. A topic with N partitions will withstand N-1 server failures.

 

  • Messages are called records, each having a key (header), value (payload) and a timestamp.

 

  • Records are retained for a configurable time period. Allows for consumer to rewind and read same messages again. 

 

  • Record is sent to consumer groups, while only one consumer inside a group gets it - enables load balancing and broadcasting. 

Thank you for your attention!

Message Queues and more

By stanislau

Message Queues and more

  • 43