Passthrough Properties

1. Overview

Passthrough Properties is a Native MQTT (Message Queuing Telemetry Transport) capability configured at the device template level. When a property is marked as passthrough, its datapoints are delivered directly through the MQTT broker to subscribed clients and are not stored in DPS (Datapoint Service), Ayla's backend for durable datapoint storage. This helps reduce storage overhead and provides lower-latency data streaming compared to regular properties that go through the DPS storage pipeline.

2. Concept and Design

Ordinary Native MQTT datapoints follow this path

Every hop after the broker exists to make sure the datapoint is durably stored and queryable later.

Passthrough properties are intended for high-frequency and real-time data where low latency is more important than long-term storage. These datapoints are delivered immediately to applications, such as mobile apps displaying live sensor readings or real-time device status updates, but are not stored because historical data is not required.

Storing every update for these types of data can increase storage usage and processing overhead. Passthrough properties use the same MQTT transport path as regular properties. After the datapoint reaches the cloud, the system checks the property's passthrough configuration:

• Passthrough property: The datapoint is delivered to subscribed clients through the MQTT broker but is not stored in the cloud database.
• Non-passthrough property: The datapoint continues through the normal ingestion pipeline and is stored for future access.

This allows real-time data delivery while avoiding unnecessary storage operations for data that does not require persistence.

3. Template Configuration and Classification

3.1 Marking a Property as Passthrough

An admin marks specific properties as passthrough when creating the device template. This setting is applied to individual properties, so the same template can contain both passthrough and regular properties.

3.2 Validation Rules

The following combinations are rejected at template-save time:

3.3 Template Classification

The classification field identifies the type of device template being created. In the Ayla Developer Center, when an administrator creates or updates a template, they can select the Native MQTT classification to indicate that the template is intended for Native MQTT devices.

A key rule is that once the classification is set, it cannot be changed. This means a template created as a Native MQTT template will always remain a Native MQTT template, and a template created as a non-Native MQTT template can never be converted into a Native MQTT template later.

This restriction is important because passthrough properties are supported only for Native MQTT templates. The system validates this rule whenever a property is created or updated.

See Section 9, UI Changes for details on how to configure this in the dashboard.

4. Propagation Flow: Template → Device → Cache

When a property is marked as passthrough in a device template, the change is asynchronously propagated to the Redis (cloud cache). Once the cache is updated, the ingestion pipeline uses the new setting to determine whether to store or skip incoming data for that property.

4.1 Flow Steps

4.2 Sequence Diagram

4.3 Architectural Pattern

The passthrough configuration flow uses an event-driven architecture, where services communicate through events instead of making direct synchronous calls to each other.

This approach provides:

• Service decoupling: Template Service, ADS, and mqtt-native-hub operate independently and do not directly depend on each other for configuration updates.
• Eventual consistency: Changes made to a template are propagated asynchronously, so the cache may not update immediately but will become consistent after the event processing completes.
• Consistent cache updates: Kafka acts as the central event channel to propagate passthrough configuration changes and keep the cache synchronized across services.

4.4 Failure Handling

4.5 Metrics to Monitor

Metrics to monitor are runtime measurements that track the performance, reliability, and health of the passthrough feature. They help identify issues such as delays, retries, processing failures, cache update delays, and duplicate messages.

5. Template Association

When a template is associated with or dissociated from a Native MQTT device, ADS (Ayla Device Service) publishes a message to the ads-template-assoc-dissoc-topic Kafka topic, identified by the device's DSN (Device Serial Number):

{
  "template_id": "<template_id>",
  "dsn": "<dsn>",
  "op": "<event>"
}

op is either template_association or template_dissociation. This is the same event stream that drives the passthrough-property cache update described in Section 4.

6. Datapoint Ingestion Pipeline with Passthrough Filtering

The following section details the runtime ingestion pipeline for datapoints received from connected devices. This flow is triggered during normal device operation and is distinct from the asynchronous template-to-device-to-cache propagation process described in Section 4.

6.1 Flow

6.2 Passthrough Check Logic

After a datapoint reaches ADS through the ingestion pipeline, ADS checks whether the associated property is configured as passthrough.

• If the property is passthrough: ADS does not forward the datapoint to DPS (Datapoint Service) for storage. The datapoint has already been delivered through the MQTT broker to any subscribed clients, so no further processing is required. ADS simply ignores the datapoint and does not store it. No retry is performed.

• If a direct write is attempted for a passthrough property: ADS rejects the write with a 4xx error response indicating the property does not accept direct writes; the datapoint is not delivered and not stored. This gives integrators a clear, actionable signal that distinguishes normal passthrough behavior (data delivered via MQTT but not stored) from an invalid write attempt (rejected outright).

• If the property is not passthrough: ADS forwards the datapoint to DPS for permanent storage. DPS manages any storage failures using retry, backoff, and dead-letter queue (DLQ) mechanisms.

  

6.3 Failure Handling

ADS REST failure behavior:

6.4 Metrics to Monitor

These metrics help monitor the health and performance of the datapoint ingestion pipeline.

7. The mqtt-native-hub Service

The mqtt-native-hub service acts as a filter. It reads datapoints from Kafka, checks the passthrough configuration stored in Redis (cloud cache), and decides whether each datapoint should be discarded or forwarded to DPS (Datapoint Service) for storage. It is called the enforcement point because this is where the passthrough rule is actually applied. For more details refer to Section 6.

8. Infrastructure Components

9. UI Changes

This section explains creating a device template in the Ayla Developer Center and adding one or more passthrough properties to it.

Passthrough properties are only available on templates classified as Native MQTT. This is set on the template's Details tab and cannot be changed after the fact for existing properties.

1. Login to Ayla Developer Center.

2. From the top navigation bar, go to Templates.

3. Open an existing template, or create a new one, and go to its Details tab.

   

4. Locate Is this template for a native-MQTT device? and select:

   • Yes -if the device connects using native MQTT. This unlocks passthrough support for the template's properties.

     

   • No - if the device does not use native MQTT. Passthrough will be unavailable for every property on this template.

     

5. To create properties with passthrough, navigate to Templates page, and select the Properties tab.

   

6. Click ADD to open the New Property dialog.

   

7. In the New Property dialog, the first field is Property Type.

   • TemplateProperty - a standard property. Passthrough is selectable here.

     

   • TemplateFeedProperty - a feed property. Passthrough is Not Applicable for feed properties.

     

8. Enter a Name. Optionally enter a Display name and Comment.

9. Select the Base Type.

   • If the Base Type is set to File or Message, the Pass-through option is Not Applicable.

     

10. Set Direction and Scope from the drop-down list.

11. Select Pass-through checkbox.

12. Once Pass-through is enabled, the system automatically sets the following attributes to Not Applicable and makes them non-editable:

    • Host SW Version

    • Track Only Changes

    • Time Series

    • Ack Enabled

    • Alexa Intents

      

13. Review all fields and Click OK to save the property to the template.

Ayla Customer Dashboard Changes

1. Login to Ayla Customer Dashboard.

2. Navigate to Templates Page.

3. On the Properties details page, you can view the Pass-Through value.

   

10. Glossary

11.Reference

• Confluence: Passthrough Properties Design Specification
• Confluence: Native MQTT: Infrastructure Requirements
• Confluence: MQTT-NATIVE-HUB