Native MQTT Cloud Architecture and Features

Native MQTT (Message Queuing Telemetry Transport) is Ayla's direct MQTT-based integration model for connecting devices, mobile applications, and Ayla cloud services. This document describes the current customer-facing architecture, supported functionality, security model, and feature set for Native MQTT.

Table of Contents

• Overview
• Architecture at a Glance
• Core Components
• Supported Scope
• Developer-Facing Feature Summary
• Security Model
  • Device Authentication
  • Application Authentication
  • Authorization and ACL Enforcement
• Architectural Flow
• Functional Capabilities
  • Device Onboarding
  • OEM Authentication and Template Association
  • Datapoints
  • Device Commands
  • Schedules
  • OTA Support
  • Presence and Offline Detection
• Deployment Model
• Operational Characteristics
• Current Limitations
• Integration Guidance for Developers
• FAQ
• References

Overview

Native MQTT uses MQTT as the primary transport for real-time, two-way messaging between devices and the Ayla platform. Under this model:

• Devices connect to a dedicated MQTT (Message Queuing Telemetry Transport) broker cluster.
• Cloud services exchange events through broker-connected services and, where needed, Kafka-backed processing. Kafka is a distributed event-streaming platform used here for asynchronous message processing.
• Mobile applications authenticate through a JWT (JSON Web Token)-based broker access flow.

Design goals:

• Improve real-time communication between devices and the cloud.
• Simplify cloud-to-device and device-to-cloud messaging.
• Support broker portability (the ability to swap or evaluate different MQTT broker implementations).
• Maintain a clear separation between transport concerns and business logic services.

  📘

  Optional

  Historically, a device and a mobile app never talk to each other directly, every state change, property update, or command is relayed through a cloud service (REST APIs, polling, or the older CMQTT/MDSS bridge). That indirection adds latency and makes near-real-time features (live property sync, fast online/offline detection) hard to deliver. Under Native MQTT, devices and mobile apps become peers on the same MQTT broker: they publish and subscribe directly to each other's topics for day-to-day data exchange, while the cloud continues to observe the same topics (to persist datapoints, enforce rules, and manage account/device administration) without being a mandatory hop in the data path.

Architecture at a Glance

Native MQTT runs on a dedicated MQTT broker cluster that is separate from the legacy MQTT deployment. Devices and mobile applications connect to this broker using authenticated sessions. Backend services interact with the broker through purpose-built APIs (Application Programming Interfaces), Kafka integrations, and broker extensions.

Key outcome: Native MQTT enables direct, low-latency device messaging while preserving Ayla cloud controls for authentication, authorization, commands, schedules, datapoints, and device lifecycle operations.

Core Components

Supported Scope

Phase 1 scope: Native MQTT currently supports Wi-Fi devices only.

Developer-Facing Feature Summary

Security Model

Device Authentication

Device authentication happens when the MQTT connection is established. A broker extension intercepts the incoming connection and calls the device authentication service. Device information can be served from a cache when available, which reduces load on backend services.

Caching note: Caching reduces backend lookup volume, but the cache lifetime should align with the device revocation policy so revoked devices are blocked promptly.

Implementation note: A successful device authentication produces an authorization context. This context is used to permit the MQTT session and to enforce later topic access decisions.

Application Authentication

Mobile and other application clients obtain a JWT from the AuthN-MQTT service before connecting to the broker.

Token details:

• Signing algorithm: ES384 (Elliptic Curve Digital Signature Algorithm using the P-384 curve)
• Default token lifetime: 900 seconds (15 minutes)
• Claims included: client UUID (Universally Unique Identifier), user UUID, user role, OEM (Original Equipment Manufacturer) ID, audience, issuer, issued-at timestamp, and expiration timestamp
• Key distribution: Public signing keys are exposed through a JWKS (JSON Web Key Set) endpoint so downstream systems can validate tokens independently.

Typical application authentication flow:

1. The application authenticates with Ayla APIs.
2. The application requests a JWT for a client-generated MQTT client identifier.
3. The AuthN service returns the JWT and the broker host address.
4. The application connects to the MQTT broker using the issued JWT.

Authorization and ACL Enforcement

Topic access is controlled by an ACL (Access Control List) engine designed for strong tenant isolation and device-level separation. The ACL model uses compiled rules and user-device mappings to determine whether a client may publish or subscribe to a given topic.

• Tenant scoping is based on oem_id.
• Principal types include device, app, and service.
• Rules support exact topics, prefixes, single-level wildcards, and global matching patterns.
• Policy evaluation considers publish vs. subscribe intent, topic variables, QoS (Quality of Service), retained message policy, and shared subscription constraints.
• The default operational principle is restrictive access unless explicitly allowed.

Isolation guarantee: Native MQTT topic design and ACL enforcement ensure that one device cannot access another device's topic space.

Architectural Flow

The following simplified flow describes how the platform operates end to end:

1. A device or app authenticates and connects to the broker.
2. The broker validates identity using device authentication or JWT verification.
3. The broker builds an authorization context and applies ACL rules for topic access.
4. Devices publish datapoints and acknowledgements upstream through MQTT topics.
5. Cloud services publish commands, schedules, and other management messages downstream.
6. Backend services process broker-originated events using service integrations and Kafka-based consumers where applicable.

Functional Capabilities

Device Onboarding

BLE (Bluetooth Low Energy) onboarding is supported in the current scope. During onboarding, the device publishes an onboarding request, and the Native MQTT service validates the request against Ayla backend APIs.

OEM Authentication and Template Association

After onboarding, the device completes OEM (Original Equipment Manufacturer) mapping and template association. These operations are designed to be robust and idempotent (safe to repeat without unintended side effects).

• OEM authentication supports success, unauthorized, forbidden, and conflict outcomes.
• Template association supports normal creation and idempotent success when the device is already associated.
• Server failures are retried with backoff.

Datapoints

Native MQTT supports both device-to-cloud and cloud-to-device datapoint flows.

• Device to cloud: Devices publish datapoints to MQTT topics; backend subscribers process them for cloud persistence and downstream workflows.
• Cloud to device: Ayla cloud services publish datapoint updates to device topics through the broker.

Device Commands

Commands are delivered in sequence, one at a time per device, ensuring deterministic execution and simpler acknowledgement handling.

• Commands are queued if the device is offline.
• Only one in-flight command per DSN (Device Serial Number) is sent at a time.
• Acknowledgements drive progression to the next command.
• Duplicate acknowledgements are handled idempotently.
• Retry, timeout, and dead-letter handling are part of the orchestration model.

Representative command categories: OTA, timezone, reboot, factory reset, setup mode, service changes, logging, LAN enablement, registration, and Wi-Fi scan requests.

Schedules

Schedules are created in cloud services and propagated to devices over MQTT. Devices acknowledge receipt and store schedules locally.

• Schedules are pushed when templates are associated during onboarding.
• Schedule updates are published when changed in the OEM dashboard.
• Devices store schedules locally for local execution.
• If a device is offline, schedules are delivered on reconnect.
• Schedule state is managed as active or inactive rather than through physical deletion on the device.

OTA Support

OTA (Over-the-Air) updates are supported through the same command-delivery framework used for other commands. OTA commands are queued, delivered over MQTT, and acknowledged by the device. Delivery ordering and command gating ensure OTA flows do not conflict with other in-flight commands.

Presence and Offline Detection

Native MQTT supports connection-state-aware cloud behavior. Devices are treated as online or offline based on MQTT session state, and workflows such as command delivery and schedule propagation check online status before attempting dispatch.

Offline detection relies on MQTT connection semantics and LWT (Last Will and Testament) behavior, letting cloud services react when a device disconnects unexpectedly.

Practical benefit: LWT-based offline detection gives the cloud faster awareness of unexpected disconnects and supports deferred delivery when devices reconnect.

Deployment Model

Native MQTT runs on a dedicated MQTT broker cluster, preferably HiveMQ, with separate supporting services. The existing (legacy) MQTT solution remains supported on its current broker environment.

• Dedicated Native MQTT broker cluster
• Separate service pathing for Native MQTT-capable backend services
• Routing through DNC (Dynamic Network Controller) and edge components using host- and path-based patterns where needed
• New Java-based and refactored APIs for Native MQTT use cases
• Additional service consumers for datapoints, status, and acknowledgements

Operational Characteristics

Current Limitations

• Wi-Fi device support only in the current scope.
• RVD, Node, and Gateway support are outside the current phase.
• BLE onboarding only in the current scope.
• Same-LAN registration is not currently supported.
• File upload and download are not part of phase 1.
• Some areas remain under active design, including message persistence, groups, scenes, and selected migration scenarios.

Integration Guidance for Developers

• Use the Device Agent as the canonical integration path for supported Wi-Fi devices.
• Design device firmware to reliably handle command acknowledgements, schedule storage, and reconnect behavior.
• For mobile applications, obtain broker JWTs through the AuthN service and refresh them before expiry.
• Assume strict topic-level isolation; do not design for wildcard access beyond explicitly granted permissions.
• Use idempotent handling on the client side for retries and duplicate deliveries.
• Plan for online/offline transitions and deferred cloud delivery behavior.

FAQ

Does Native MQTT require a different broker than the existing MQTT solution?
Yes. The current architecture proposes a separate broker cluster for Native MQTT, while the existing MQTT solution continues on its current cluster. This separation isolates architecture changes and enables Native MQTT-specific capabilities.

Can mobile applications connect directly to the broker?
Yes. Mobile applications can connect using a JWT issued by the AuthN-MQTT service. The token is signed with ES384, includes broker audience information, and has a short lifetime for security.

How does the platform detect device offline state?
The platform uses MQTT connection-state semantics and LWT-based patterns to identify unexpected disconnects and update online-status-dependent workflows.

Are commands delivered in parallel?
No. Commands are delivered one at a time, in creation order, for each device. The next command is sent only after the previous command is acknowledged.

References

• Native MQTT: Cloud
• AuthN-MQTT Service API Documentation
• HiveMQ – ACL Management
• Native MQTT – Topic Design
• Design Specification: Device Commands
• Design Spec: Device Schedules
• Native MQTT: Infrastructure Requirements

Glossary