On-Demand MQTT

To send device data to the cloud and receive commands in return, a network connection is required between the device and the cloud. This connection typically utilizes the MQTT protocol.

MQTT is a lightweight, publish-subscribe protocol designed for machine-to-machine communication. It is particularly suited for devices with limited resources or constrained network bandwidth.

The Ioto service provides such an MQTT service that is automatically provisioned when your devices register with the cloud. When a device registers with the Ioto cloud and is claimed for management, it is then provisioned with MQTT X509 certificates for secure communication. A per-device MQTT connection is provided for sending data to the cloud and for receiving device operational commands.

Use Cases

MQTT is used for a variety of essential use cases, including to:

• Centrally gather per-device configuration centrally in the cloud
• Accumulate device performance data and metrics for centralized management
• Receive device operational commands from users and service via the cloud
• Synchronize device database with the cloud-side database
• Archive device logs to the cloud

MQTT Connection Cost

MQTT enables the transmission of device configuration data, performance metrics, and other operational details, as well as sending commands from the cloud to the device.

However, each MQTT connection consumes cloud resources, even when idle. This leads to ongoing costs for cloud providers or customers. Maintaining constant MQTT connections for a small number of devices isn’t problematic, but at scale, the costs can add up.

To reduce these expenses, Ioto supports on-demand MQTT connections, allowing the connection to be established when needed, or on a schedule and then be disconnected when idle.

Ioto On-Demand MQTT

Ioto establishes MQTT connections to the cloud transparently when needed. Whenever an API transmits data to the cloud, Ioto automatically initiates an MQTT connection to the configured device cloud.

On-demand connections are controlled via the mqtt.schedule property in the ioto.json5 configuration. This “cron”-style setting specifies the time window during which MQTT connections are allowed. It also defines a schedule for establishing a connection regardless of demand, enabling the device to receive queued cloud commands.

If the connection remains idle, Ioto will close it after the timeout defined by the mqtt.timeout parameter. Once closed, no new connection will be initiated for at least the mqtt.delay period. These settings provide precise control over when and how MQTT connections are established and managed.

Pooled Subscriptions

To reduce the overhead of registering multiple MQTT subscriptions for incoming messages, Ioto supports a pooled master subscription. For MQTT topics that share the same leading topic prefix, Ioto creates a single subscription and routes incoming messages within the agent to the appropriate subscribing code. This approach greatly minimizes the cost of MQTT connections and simplifies cloud operations.

Database Update Aggregation

The Ioto database replicates changes to a centralized cloud database, with replication configurable on a per-table basis. Some tables remain local to the device, while others are synced with the cloud, either sending updates or receiving them.

To optimize network traffic, the Ioto device database supports buffering of changes, allowing multiple updates to be grouped into a single MQTT message that is sent when the device next connects to the cloud. For example, if your device has a power supply table where voltage levels are updated every second, you can configure it to aggregate these updates and send them to the cloud only once per minute, reducing network traffic by 60x.

Buffering is managed using the delay option in the database schema or on a per-API basis with the delay parameter in the database API.

Design Tradeoffs

There is a balance between lowering network costs with on-demand MQTT connections and maintaining the ability to respond immediately to cloud-initiated commands. When a device is not connected to the cloud, it can’t respond to commands until it reconnects.

This tradeoff can be mitigated by using scheduling and device-side connection strategies. If instant response to cloud commands is critical, a permanent MQTT connection is ideal. However, for more cost-effective cloud management of large-scale installations, on-demand connections offer a suitable solution.