DataStream Service v2

Suggest Edits

DataStream Service v2 (DSSv2) is a robust event-based data publishing system that guarantees delivery of IoT data and events with the least amount of latency to the cloud endpoint/destination of choice. This feature reads the data from the kafka topics and delivers it to the respective destination channels. The supported destination channels include AWS Kinesis and Azure Event Hubs.

Using this feature, you can analyze and process specific IoT data in real time while supporting your multi-cloud initiatives. Even during periods of endpoint downtime, the streaming feature will save and restart this traffic once your endpoint is back up again.

DSSv2 supports the following datastream types:

  • activation
  • connection
  • datapoint
  • datapointack
  • location
  • registration

Workflow

  • Rule service posts a message to the stream topic (it could be event hub or kinesis).
  • Respective DSSv2 reads the message from kafka topics.
  • Data is delivered to the respective destination by using the client object. Here client object refers to the connection object which is used to establish the connection to the destination and to send the data.

Use Case

Let us take an example of the dimmer switch product. Dimmer switches are needed for adjusting brightness to have efficient smart home or office lighting requirements. A product developer (OEM) wants to know what is the datapoint status of a particular property (for example “Power” property) of all the dimmer switches.

Start with creating a datastream using Ayla Customer Dashboard or Ayla API. While creating a stream, select required OEM Model, event type, and property. Ayla provides you with an option to select all the properties or specific properties. Please note the result is a bulk of data, so using datastream is an appropriate solution for this use case.

DataStream User Interface

Ayla Customer Dashboard provides user interface to view, create, edit, and delete data streams. The following screenshot shows the DataStream page that you can access from the left navigation pane:

2880

DataStream Homepage

Creating DataStreams for AWS Kinesis

Follow these steps to create datastream for AWS Kinesis:

  1. Make sure that you have created and logged in to your AWS account when using the AWS Kinesis as your transport stream.
  2. In the Ayla Customer Dashboard, navigate to the DataStream homepage, and then click the CREATE STREAM button to open the Create Stream dialog box.
  3. In the Transport field, select Amazon Kinesis. The following dialog box is displayed:
1794

Create Stream - Amazon Kinesis

  1. Enter the information required for each field as follows:
    1. For Name, enter a name to identify the datastream.
    2. For Description, enter key details to describe the datastream. For example, you may want to enter the product type (i.e. dryers, boilers, etc.) associated with this datastream.
    3. For Kinesis Stream ARN, enter the Amazon Resource Name (ARN), which uniquely identifies an AWS resource across all of AWS. Click here to know the general formats of ARNs and other related information.
    4. For AWS Secret Access Key, enter the Access Key ID provided with your account on the AWS Management Console. Click here to log in to your account to obtain this key.
    5. For Kinesis Server-side Encryption, select Enable or Disable from the drop-down list, depending on if you wish to have Kinesis Stream encrypt your incoming data.
    6. For AWS AMS KEY ARN, enter the ARN key. When the encryption option is enabled, all users (consumers) will need to provide this ARN key when trying to read data from this stream.
    7. For OEM Model, select the OEM model to associate with this datastream.
    8. For Stream Type, select the attribute that represents the specific type of data for this datastream:
      1. activation - streams activation data for the devices that have the same OEM model associated with this datastream irrespective of whether these devices are registered or not. Activation event data is generated when the end users activate the device without registering it first.
      2. connection - streams all connection data for the devices that have the OEM model associated with this datastream.
      3. datapoint - streams data on the datapoints configured for the devices that have the OEM model associated with this datastream. If you select this option, the Property Selection field becomes enabled. Select appropriate option in the Property Selection field.
      4. datapointack - streams data on acknowledgments that confirm whether property values were updated.
      5. location - streams geolocation data (i.e. latitude or longitude) for the devices that have the OEM model associated with this datastream.
      6. registration - streams registration-related data for the devices that have the OEM model associated with this datastream.
  2. Click CREATE. This adds the new datastream and starts the stream by default. Notice that the option you chose for Property Selection is shown under the Stream Type column.

Creating DataStreams for Azure Event Hubs

Follow these steps to create datastream for Azure Event Hubs:

  1. Make sure that you have created a Microsoft Azure account and signed in to the Azure Portal.
  2. In the Ayla Customer Dashboard, navigate to the DataStream homepage, and then click the CREATE STREAM button to open the Create Stream dialog box.
  3. In the Transport field, select Azure Event Hubs. The following dialog box is displayed:
1782

Create Stream - Azure Event Hubs

  1. Enter the information required for each field as follows:
    1. For Name, enter a name to identify the datastream.
    2. For Description, enter key details to describe the datastream. For example, you may want to enter the product type (i.e. dryers, boilers, etc.) associated with this datastream.
    3. For Event Hub Namespace, enter the name of the namespace. This Event Hub Namespace provides a unique scoping container referenced by its fully qualified domain name, in which you create one or more event hubs. Click here for more details on how to create this in the portal.
    4. For Event Hub Instance Name, enter the name of the event hub in this field.
    5. For Shared Access Key Name, enter the Shared Access Policy Name in this field. Click here for more information on how to create this name in the portal.
    6. For Shared Access Key, enter the shared access key in this field.
    7. For OEM Model, select the OEM model to associate with this datastream.
    8. For Stream Type, select the attribute that represents the specific type of data for this datastream:
      1. activation - streams activation data for the devices that have the same OEM model associated with this datastream irrespective of whether these devices are registered or not.
      2. connection - streams all connection data for the devices that have the OEM model associated with this datastream.
      3. datapoint - streams data on the datapoints configured for the devices that have the OEM model associated with this datastream. If you select this option, the Property Selection field becomes enabled. Select appropriate option in the Property Selection field.
      4. datapointack - streams data on acknowledgments that confirm whether property values were updated.
      5. location - streams geolocation data (i.e. latitude or longitude) for the devices that have the OEM model associated with this datastream.
      6. registration - streams registration-related data for the devices that have the OEM model associated with this datastream.
  2. Click CREATE. This adds the new datastream and starts the stream by default. Notice that the option you chose for Property Selection is shown under the Stream Type column.

DataStream APIs

DSSv2 enables OEMs to use REST APIs to manage DataStreams (get, create, update, delete). For more information on the APIs, please refer to API Reference.

Updated almost 2 years ago