Integrated Agent: ESP32 v1.12

The Ayla ESP32 solution includes example host application source code, an Ayla Wi-Fi Integrated Agent, and Ayla extensions to the Espressif IoT Development Framework (IDF). These pieces link together to provide an Ayla edge solution for an ESP32 or ESP32-C3 module. Learn more about Ayla Integrated Agents.

First steps

  1. Obtain an Espressif dev kit. Any of the following would work: ESP32-DevKitC,
    ESP32-C3-DevKitM-1,
    ESP32-C3-DevKitC-02.

  2. Create an Ayla user account.
    If you do not have an account already, please contact Ayla Support team to get user account with admin or developer access rights.

  3. Reserve a DSN.
    To reserve a DSN, follow these steps:
    1. Sign in to the Ayla Customer Dashboard.
    2. On the left navigation pane, click Factory Actions.
    3. Click the Reserve DSN button.
    4. For Model, select AY008ESP1.
    5. Click Submit and then download the associated XML file. You will need the DSN value and key from the downloaded file.
    NOTE: If you do not have access to the dashboard, request a DSN from your Solution Engineer or from Ayla Support team.

  4. Sign in to the Ayla Developer Portal and create a template with the details provided in the following table. Note that the template version is different from the ADA version.
    The OEM model describes the application and platform, it could be demo-esp32 for the ESP32-WROOM-32E module, and demo-esp32c3 for the ESP32-C3 modules.

    Visibility:private
    Name:demo-esp32-msgs 1.10
    Description:Demo ESP32 msgs 1.10
    Registration:AP-Mode
    Model:demo-esp32 or demo-esp32c3
    Version:demo_esp_msg 1.10
    Type:Wifi
    To learn how to create templates, please refer to Ayla Developer Portal User Guide.
  5. Select the new template and click the Properties tab.

  6. Copy and paste the following properties into a text file (e.g. properties.csv), and import the file using the Import button.

    base_type,direction,name,scope,mime_type,ack_enabled,host_sw_version,track_only_changes
    message,input,binary_in,user,application/octet-stream,TRUE,FALSE,FALSE
    message,output,binary_out,user,application/octet-stream,FALSE,FALSE,FALSE
    boolean,output,Blue_button,user,text/plain,FALSE,FALSE,FALSE
    boolean,input,Blue_LED,user,text/plain,FALSE,FALSE,FALSE
    string,input,cmd,user,text/plain,FALSE,FALSE,FALSE
    decimal,input,decimal_in,user,text/plain,FALSE,FALSE,FALSE
    decimal,output,decimal_out,user,text/plain,FALSE,FALSE,FALSE
    boolean,input,Green_LED,user,text/plain,FALSE,FALSE,FALSE
    integer,input,input,user,null,TRUE,FALSE,FALSE
    message,input,json_in,user,application/json,TRUE,FALSE,FALSE
    message,output,json_out,user,application/json,FALSE,FALSE,FALSE
    string,output,log,user,text/plain,FALSE,FALSE,FALSE
    string,input,message_start,user,null,FALSE,FALSE,FALSE
    boolean,input,node_batch_hold,user,null,FALSE,FALSE,FALSE
    integer,output,output,user,text/plain,FALSE,FALSE,FALSE
    string,output,schedule_out,user,text/plain,FALSE,FALSE,FALSE
    file,input,stream_down,user,text/plain,FALSE,FALSE,FALSE
    integer,output,stream_down_len,user,text/plain,FALSE,FALSE,FALSE
    integer,output,stream_down_match_len,user,text/plain,FALSE,FALSE,FALSE
    file,output,stream_up,user,text/plain,FALSE,FALSE,FALSE
    integer,input,stream_up_len,user,text/plain,FALSE,FALSE,FALSE
    message,input,string_in,user,text/plain,TRUE,FALSE,FALSE
    message,output,string_out,user,text/plain,FALSE,FALSE,FALSE
    string,output,version,user,null,FALSE,TRUE,TRUE
    
  7. Add schedules to the template. The names should be "sched1" through "sched5", and the
    direction should be to-device.

  8. Select the SUNRISE/SUNSET SUPPORTED option for the schedules.

  9. Under Available Connects, make sure that the LAN Connect and Auto-sync options are selected.
    Do not select BLE Connect unless you will be using the BLE Local Control feature, as it requires additional memory.

Build and Run

GCC/Ubuntu/Docker

This section provides directions for building an Ayla ESP32 Solution v1.12 using a GCC/Ubuntu/Docker development environment. The recommendation is to use Ubuntu version 20.04 tls. Running Ubuntu natively or in a VM is supported.

  1. Download Ayla source code (ada-esp-idf-src-1.12.tgz).
    ada stands for Ayla Device Agent.
    NOTE: If you do not have access to Ayla source code, please request for the access to Ayla Connection from Ayla Support team.
  2. Determine the computer-to-ESP32 serial port:
    1. List the serial ports on your computer (e.g. ls /dev/tty*).
    2. Connect an ESP32 board to your computer using a serial cable.
    3. List the serial ports again. The new serial port on the list is the one to use (e.g. /dev/ttyUSB0).
  3. Install a Docker Engine on your Windows, Mac, and/or Linux host OS.
  4. Create a Docker 64-bit Ubuntu container.
$ docker run --net=host --name=ada17 -w /root -it --device=/dev/ttyUSB0 ubuntu bash

You are user root, and the current working directory is /root. To exit your Docker container, type exit. To restart and re-attach to your Docker container, type the following:
$ docker ps -a
$ docker start ada17
$ docker attach ada17

  1. Prepare your development environment:

    1. Update the package list in your Ubuntu Docker instance:
      # apt update
      
    2. Install miscellaneous packages:
      # apt install nano screen iputils-ping
      
    3. Install prerequisites for Espressif Standard Setup of Toolchain v4.3.2 for Linux.
      # apt install gcc git wget make libncurses-dev flex bison gperf python python-pip python-setuptools python-serial python-cryptography python-future python-pyparsing
      
      If using Ubuntu 20.04, install using the following command:
      #  apt install gcc git wget make libncurses-dev flex bison gperf python3 python3-pip python3-setuptools python3-serial python3-cryptography python3-future python3-pyparsing
      
      If using Ubuntu 20.04, link using:
      # ln -s /usr/bin/python3 /usr/bin/python
      
    4. Create an esp directory, and change directory:
      # mkdir esp
      # cd esp
      
    5. Clone the Espressif IoT Development Framework:
      # git clone -b v4.3.2 --recursive https://github.com/espressif/esp-idf.git esp-idf-v4.3.2
      
    6. Create the IDF_PATH environment variable, and persist the change in .bashrc:
      # export IDF_PATH=$HOME/esp/esp-idf-v4.3.2
      # echo "export IDF_PATH=\$HOME/esp/esp-idf-v4.3.2" >> ~/.bashrc
      
    7. Set up the Espressif tools using the following instructions:
      1. cd $IDF_PATH
      2. export IDF_GITHUB_ASSETS="dl.espressif.com/github_assets"
      3. ./install.sh 'esp32;esp32c3'
      4. . ./export.sh
      5. The above adds to the PATH environment variable. Persist the change in .bashrc:
      # echo "export PATH=$PATH" >> ~/.bashrc
      
  2. Test the development environment with the hello_world example:

    1. Change directory:
      # cd $IDF_PATH/examples/get-started/hello_world
      
    2. Customize configuration. (Defaults should be fine.)
      Set target to either esp32 or esp32c3, depending on your devkit.
      # idf.py set-target esp32 # idf.py menuconfig
    3. Build, flash, and monitor:
      # idf.py build
      # idf.py flash
      # idf.py monitor
      
      Output appears similar to this:
      Toolchain path: /root/esp/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
      Toolchain version: crosstool-ng-1.22.0-80-g6c4433a
      Compiler version: 5.2.0
      ...
      ...
      Hello world!
      This is ESP32 chip with 2 CPU cores, WiFi/BT/BLE, silicon revision 1, 4MB external flash
      Restarting in 10 seconds...
      Restarting in 9 seconds...
      ...
      ...
      
      Press Ctl + ] to exit the monitor.
  3. Install Ayla source code:

    1. In a host terminal, copy ada-esp-idf-src-1.12.tgz to your Docker container:
      $ docker cp /home/matt/Downloads/ada-esp-idf-src-1.12.tgz ada16:/root/esp
      
    2. In your Docker terminal, extract the archive file:
      # cd /root/esp
      # tar zxvf ada-esp-idf-src-1.12.tgz
      
    3. Copy the ada-esp-idf-src-1.12 directory on top of the esp-idf-v4.3.2 directory:
      # cp -R ada-esp-idf-src-1.12/* ./esp-idf-v4.3.2
      
  4. Configure and build the Ayla agent and host application:

    1. Change directory:

      # cd $IDF_PATH/examples/ayla_demo
      
    2. Build, flash, and monitor:

      # idf.py set-target esp32 # or esp32c3 as appropriate
      # idf.py build
      # idf.py erase_flash
      # idf.py flash
      # idf.py monitor
      
    3. Provision the device DSN and OEM credentials. Configure the same OEM model used in the template.

        # nvs-set "ada.f.id/dev_id" AC000W000000001
        # nvs-set "ada.f.id/key" MIIB... (long number)
        # nvs-set ada.f.client/server/default 1
        # esp-reboot
        # oem 000000000 (replace with your Ayla OEM ID)
        # oem model demo-esp32    (replace with OEM model from template)
        # oem key 0123456789abcdef0123456789abcdef (replace with your OEM secret)
        # wifi profile ap ssid Ayla-DevKit
        # setup_mode disable
        # save
    

NOTE: Setting "ada.f.client/server/default" to 1 will force your devices to use the development servers. For production, please set this value to 0: "ada.f.client/server/default 0".

Register your device

The Aura Mobile App running on iOS or Android communicates with your ESP32 device (via Bluetooth) to add the device to your local network, and then register the device to your Ayla user account. For instructions, see Using BLE to set up Wi-Fi for ESP32.

Example Application

Please refer to Example Application page for detailed information about the demo application.

Library Code

The example application links with the following Ayla libraries residing in components/ayla:

LibraryNotes
libadaAyla Integrated agent, see below.
libadbAyla Bluetooth library
libaylaSee Integrated Agent.
libadwSee Ayla Device Wi-Fi for Embedded Systems
libjsmnJSMN (pronounced Jasmine) is a small JSON parser. ADA uses a different version of JSMN than the one in ESP32. The ADA version is in components/ayla/jsmn.
libnetThis library contains a thin layer of macros and functions that interface with LwIP and ESP32.

Library ada

Below are summaries of the files in this library to help you with ESP32 development. For a full description of this library, see Integrated Agent.

al/esp32/ada_lock.c

This file implements an interface to the thread mutexes.

al/esp32/client_task.c

This file contains code for the client thread and its initialization, timers, locking, and work queue.

The function ada_init() does the following:

  • Initializes the logging subsystem, including setting severity levels to be logged for each subsystem.
  • Initializes the timers and the mutex, which protects most of the client data structures.
  • Calls init functions for various subsystems.
  • Calls ada_conf_load() to read configuration settings.
  • Creates the client_task_queue and the client thread.

The client thread runs in the function client_idle(). This function initializes the TLS library, the client_redir_client_html() handler (described below) and the 'client_task_watchdog' function. Then it takes the client lock and enters the idle loop.

The idle loop handles any pending timers that need to be handled in the client thread while still holding the client lock. Then, the idle loop waits for callbacks on the client_task_queue. This client_task_queue waits until the next scheduled timer or until an event occurs. If a callback is found, its handler is called and the loop repeats.

The al/esp32/client_task.c file contains code in the function client_redir_client_html() that redirects HTTP requests for “/client” to “/client.html”. This is necessary for same-LAN web-based device registration.

This file also contains the client_task_watchdog timer, which periodically checks that the client is healthy, and if so, calls the ESP-IDF function esp_task_wdt_reset().

The remainder of the code in the al/esp32/client_task.c file implements the client timers and synchronization needed with the server thread:

  • The functions client_lock_int() and client_unlock_int() implement the locking around the client structures. These functions keep some debugging information that can be useful if there is trouble in this area.
  • The global string pointer client_mutex_func holds the name of the function that last successfully locked the client_lock.
  • The global integer client_mutex_line gives the line number in that function.
  • The global pointer client_mutex_owner is a pointer to the OS task that holds the mutex and is used to detect recursive attempts on the lock.

al/esp32/http_client.c

The al/esp32/http_client.c file implements the interface between the HTTP client of ADA and the httpc functionality of the ESP32.

al/esp32/log.c

The al/esp32/log.c file loads the configuration of the logging system in ADA.

al/esp32/notify_task.c

The al/esp32/notify_task.c implements timers for the notification subsystem, which interacts with ANS.

al/esp32/stubs.c

The al/esp32/stubs.c file contains miscellaneous required routines - some for features that are not functional in this architecture, and some that are simple interfaces between ADA and ESP32.

al/net/server.c

The al/net/server.c file implements an HTTP server that binds to TCP port 80. This is used for Wi-Fi setup and LAN mode. The httpc functionality in ESP32 is not usable while this server is running.

Host Application configuration

The host application is responsible for configuring the ESP-IDF build to include the proper subsystems with the correct settings. An example configuration is provided with the demo host application.

For proper operation of the client health check code (see al/esp32/client_task.c), the task watchdog reset feature should be enabled. It should be configured to panic if not reset within 25 seconds. This can be tested with the "crash hang" command found in the demo code.