Using F5 BIG-IP and Solace Open Data Movement technology for MQTT message routing and delivery

The "Internet of Things" (IoT) has the potential to not just impact the way we live and work, but to disrupt entire industries. Some of the major industries that are driving innovation in IoT include manufacturing, transportation, and utilities. In this post I’ll talk about an example of how digital manufacturing is helping automobiles be more intelligent.

There are millions of connected automobiles that have software running on them: from programs that keep GPS maps up to date to sensors sending vehicle speeds or temperature information. This information typically is being sent to a centralized management system that enables intelligent decisions, and those back-end systems also need to send data, instructions, and updates to the cars.

We are actively working with a leading multinational automotive company. In this deployment the management system uses MQTT to send notifications to cars that a new software update is available for download. It is critical that these messages are received by the connected cars, but message delivery can be delayed if the car can’t be reached because it’s out of range, or driving through a tunnel, or even if cellular networks are congested.

Using the Solace message broker

To ensure eventual delivery in such situations, you can have the management system send messages to a message broker that in turn distributes them to intended recipients. This decouples your architecture so the many elements of the management system don’t need to worry about the specific destinations to which each message must be delivered, nor track delivery status. They simply send a message to the broker which A) takes care of routing it to requisite destinations, and B) stores a copy of it until every vehicle that needs it has confirmed receipt.

Basic IIoT architecture for connected automobiles

Before diving deeper, let’s look at the architecture that is used in the above use case. The idea is the same, but to support the numbers of connected vehicles and volume of data involved, it’s necessary to implement multiple message brokers. This architecture is provided by Solace which uses a novel architecture that enables efficient publish/subscribe distribution of data and can scale to millions of connected devices. (see Fig 1.).

• Back-end applications connect to a single broker (core message broker) to send a message to any vehicle, and can use messaging techniques preferred by IT applications such as JMS, AMQP and Node.js.
• The core message broker is communicating to several edge message brokers through internal messaging.
• The data for a particular vehicle is routed through one edge message broker and delivered to the vehicle in the matter conducive for it, in this case MQTT.

In the diagram, Vehicle 1 connects to Message Broker 1. That means that messages sent from the management system to the core message broker that are intended for Vehicle 1 only need to go to Message Broker 1 

Fig. 1 – Basic IIoT architecture for connected automobiles

This two-tier approach provides a simple and powerful scaling architecture for north-south communication, since more connected cars can be added by adding more edge message routers without affecting the existing system.

BIG-IP and persistence

For this architecture, we’re aiming to achieve two things:

First, we want to relieve the management system of the burden of keeping track of message delivery to potentially millions of vehicles. The Solace message broker uses MQTT QoS Level 1 which guarantees that a message will be delivered at least once to the receiver. The message is persisted in the edge message broker which allows the backend IoT application to publish messages with confidence that they will eventually reach their destination, but without needing to keep track of message delivery status.

Second, we need to make sure that the vehicle connects to the right message broker so it can receive its messages. This can be achieved by managing a 1:1 mapping between the edge message broker and the vehicle.  

Fig. 2 – F5 BIG-IP in the IoT architecture

BIG-IP version 13+ helps to achieve the 1:1 mapping with support for MQTT message parsing. It sits in front of the message broker and parses the MQTT messages – using iRules – exchanged between the vehicle to the message broker. Data groups on the BIG-IP are used to map a vehicle using a unique Identifier to a pool of message brokers. Using iRules every MQTT message is parsed, then the data group lookup routes the data to the correct pool of message brokers. This helps to achieve a “sticky” load balancing solution so that when a vehicle comes back online it will always get routed and connected to the correct message brokers where the connection state for that vehicle is known to be maintained. We say a “pool of message brokers” in order for the brokers to provide a highly-available service.

Here is some sample code of an iRule that helps with achieving persistence or sticky load balancing:

when MQTT_CLIENT_INGRESS {
log local0. "CLIENT [MQTT::type]"
    if {[MQTT::type] eq "CONNECT" } {
        # Select pool based on vehicle ID
        if { [MQTT::username] eq "D1111111" } {
            log local0. "Load balancing to pool MQTT-BrokerPool1"
            pool MQTT-BrokerPool1
        }
        if { [MQTT::username] eq "D1111112" } {
            log local0. "Load balancing to pool MQTT-BrokerPool2"
            pool MQTT-BrokerPool2
        }
    }       
}

IoT use cases which benefit from having a guaranteed delivery of messages can take advantage of F5’s persistence capabilities. The architecture described above means that specific edge broker instances will maintain 'state' for specific connected vehicles. This requires the MQTT load-balancing to be sticky across sessions. BIG-IP version 13+ supports MQTT message parsing using iRules and thereby provides a key functionality in achieving the end-to-end goal of the solution for this type of architecture.

The industries that are embracing “Internet of Things” applications are still in the early stages of innovation, but market leaders are architecting now for scale and resilience so they can automate more and more processes with a single infrastructure. The automaker in this example has chosen F5’s BIG-IP solution for 1:1 mapping of cars to message brokers, and Solace’s Open Data Movement technology as the MQTT infrastructure for message routing and delivery. The combination provides an elegant solution that can scale to millions of connections to support an increasing number of message exchanges between the vehicles and back end systems as their telematics and in-car services get more sophisticated.