Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Google+ Youtube

Github Stackoverflow Pinterest

The value of IoT is the wisdom from analytics and notifications

Overview

Here is a hands-on step-by-step introduction to Microsoft’s IoT Hub.

IoT Hub runs in Microsoft’s Azure cloud, one of several IoT clouds I’ve looked at.

https://linuxacademy.com/azure/training/course/name/azure-iot-enterprise?platform=hootsuite


About the money

  1. Have an Azure account at
    https://azure.microsoft.com/en-us/free/free-account-faq

    The minimum “Pay As You Go” Support Plans is $29/month for a Developer plan. The Standard plan is $300/month.

    1-800-867-1389

  2. Open an Azure Subscription:

    After a one-month free trial period, the “F1 Free Tier” provides each month 8,000 free messages (of .5 KB each) a day.

    The first paid tier of 400,000 messages per month / $50 = 0.000125 per message (about 1 cent).

  3. Visit http://azure.com/iotdev - the “Azure IoT Developer Center”

    • https://internetofyourthings.com
    • aka.ms/iotbestpractices
    • https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-security-deployment
    • https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-security-ground-up
    • https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-security-best-practices
    • https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-security-architecture
    • https://www.enisa.europa.eu/publications/etl2015 - mindmap of threats
  4. Get an IoT device: Intel Edison or Raspberry Pi

    https://catalog.azureiotsuite.com/kits = https://aka.ms/edx-dev225x-az1

    In the Adafruit kit:

    • Assembled Adafruit BME280 Temperature, Pressure & Humidity sensor
    • Photo cell (CdS photoresistor)
    • 2x 560 ohm 5% 1/4W Resistor (striped green, blue, brown)
    • Ceramic Capacitor (0.1uF)
    • Diffused 10mm Red LED
    • Diffused 10mm Green LED
    • Male/Male Jumper Wires - 20 x 6” (150mm)
    • Female/Male ‘Extension’ Jumper Wires - 20x6” (150mm)
    • Full Size Breadboard

    Alternately:

  5. Install a Simulator in C# HubExplorer

    https://azure-samples.github.io/raspberry-pi-web-simulator

    https://azure-samples.github.io/raspberry-pi-web-simulator/build/index.html

    https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-raspberry-pi-web-simulator-get-started Connect Raspberry Pi online simulator to Azure IoT Hub (Node.js)

    OPTION A: GUI Resource Group

  6. Enter the Azure Portal GUI at http://ms.portal.azure.com

  7. Click the green “+ New” at the upper left corner.
  8. Type Resource on top of “Search the marketplace”
  9. Subscription:
  10. Resource group location
  11. Click Pin to dashboard
  12. Click Create

    IoT Hub

  13. Click the green “+ New” at the upper left corner.
  14. Type IoT on top of “Search the marketplace” to select IoT Hub.
  15. Click on the IoT Hub icon.

    iot-azure-iot-hub com

  16. Click the blue Create button.
  17. Click “Sign-up for an Azure subscription” if that comes up.

    PROTIP: For each email, you get one month free to use the $200.

  18. Add a hub Name beginning with “IoT”…
  19. Select the F1 free tier or
  20. Begin with 1 partition for Device-to-cloud.
  21. Select a Location closest to you (US-West or US-East-1).
  22. Check Pin to dashboard.
  23. Click Create.

    CAUTION: Each hub consumes credits (money) regardless of usage.

    https://docs.microsoft.com/en-us/azure/iot-hub/
    Documentation

  24. Name is unique across all accounts in Azure. A red ! appears on the right of the field if it’s not.
  25. Pricing and scale tier: F1 for free to highlight it as yellow.
  26. Click Select.
  27. IoT Hub units: 1.
  28. Device-to-cloud partitions: 2.
  29. Subscription:
  30. For Resource group, use existing and select it.
  31. Check Pin to dashboard.
  32. Click Create.
  33. Look at the running bar under the Navigation bell at the upper-right corner until “Deployment succeeded” appears.
  34. You may need to select Refresh to see the recently created resource group.

    Activation (optional)

  35. PROTIP: Make inactive.

  36. Re-activate the hub for steps below.

    Resource Group

  37. Click Resource Group on the left menu.
  38. Click on the hub defined.
  39. Click on Shared access policies
  40. Select iothubowner
  41. Click on the icon to copy the Connection string – primary key credentials. “Copied” appears.

    Permissions on the secondary key can be changed.

  42. Open a Notepad
  43. Paste this into the Notepad and note that this is the credential to the IoT Hub as full administrator

    Device Explorer

  44. Click Device Explorer in the IoT Hub menu.
  45. Click on the “+ Add” to add new devices and credentials.
  46. Device ID: (myFirstNodeDevice) Enter this name exactly as myFirstNodeDevice
  47. All other fields are left to default
  48. Click Save.

    Create Device Entries for Web001

Use the Device Explorer option to view, create, update and delete devices on the IoT Hub.

  1. Click on Device Explorer.
  2. Click on the “+ Add” to add new devices and credentials.
  3. Device ID: (Web001) Enter this name exactly as Web001
  4. All other fields are left to default.
  5. Click Save.

    Gather Device Credentials

    OPTION A: Azure Portal GUI

    http://ms.portal.azure.com

    OPTION B: Azure CLI command code

    Based on https://docs.microsoft.com/en-us/cli/azure/install-az-cli2
    or https://docs.microsoft.com/en-us/cli/azure/install-azure-cli

  6. On a Mac, at any folder:

    curl -L https://aka.ms/InstallAzureCli | bash

    ===> In what directory would you like to place the install? (leave blank to use ‘/Users/mac/lib/azure-cli’):

    ===> In what directory would you like to place the ‘az’ executable? (leave blank to use ‘/Users/mac/bin’):

    ===> Modify profile to update your $PATH and enable shell/tab completion now? (Y/n):

    ===> Enter a path to an rc file to update (leave blank to use ‘/Users/mac/.bash_profile’):

  7. Reset the Terminal:

    source ~/.bash_profile
    

    Alternately, restart command shell:

    exec -l $SHELL
    
  8. Login using command:

    az login
    
  9. Authenticate: use a web browser to open page https://aka.ms/devicelogin to enter code E946P2YFU,

    Click Continue at “Microsoft Azure Cross-platform Command Line Interface”.

    Login to see on the Terminal:

     "cloudName": "AzureCloud",
     "id": "4effab90-7516-4bdb-902a-18001facad69",
     "isDefault": true,
     "name": "Developer Program Benefit",
     "state": "Disabled",
     "tenantId": "12345678-5f96-4d36-a89b-5ea0f7614e72",
     "user": {
       "name": "your@hotmail.com",
       "type": "user"
    
  10. Query using the az commands

    az 
    

    PROTIP: Drag the right edge of your Terminal window wider to avoid line wrap.

    TODO: A Bash/PowerShell script that does all the below.

    IoT Hub Explorer

    The iothub-explorer is a CLI tool for managing device identities in your IoT hub registry, send and receive messages and files from your devices, and monitor your IoT hub operations.

    The iothub-explorer tool also lets you simulate a device connected to your IoT hub.

    To get started with iothub-explorer using the Visual Studio Code command line:

  11. Install Node.js
  12. Install the latest version from any folder:

    npm install iothub-explorer -g
    
  13. Verify by getting a menu:

    iothub-explorer --help
    
  14. PROTIP: Edit on your Mac’s boot-up file:

    source ~/.bash_profile
    
  15. Add this line to create a custom alias, then save the file:

    alias aih='iothub-explorer'

    NOTE: You can select another alias short name than this example.

  16. Close your Terminal and open it again or, alternately:

    source ~/.bash_profile
    
  17. Now you can verify using an alias:

    aih --version
    

    Device Explorer app for IoT Hub devices

    Use the Device Explorer tool (on Windows 10 only) to manage devices connecting to your IoT hub. For example, you can use this tool to register a device with your IoT hub, monitor messages from your devices, and send messages to your devices. This is not the same as the Device Explorer blade that you can open from your IoT Hub service on the Azure portal.

    The Device Explorer app runs on your local machine and connects to your IoT hub in Azure. It communicates with the following IoT Hub endpoints:

    • Device identity management to provision and manage devices registered with your IoT hub.

    • Receive device-to-cloud to enable you to monitor messages sent from your device to your IoT hub.

    • Send cloud-to-device to enable you to send messages to your devices from your IoT hub.

    Create a device

    az iot hub create –resource-group your-RG –name yourname –location yourlocation

    NOTE: Location is also called region.

    Delete a device

    az iot hub delete --name yourname --resource-group yourRG 
    

    List devices

    az iot device -h
    

    Create an IoT Hub service

  18. Clone an Azure IoT service SDK

    WARNING: These repos contain sub-modules, so add to git clone commands --recursive.

    PROTIP: Since we won’t be uploading changes, we don’t fork and we also add to the command --depth=1.

    • https://github.com/azure/azure-iot-sdk-python (2.7 and 3.5) - 195 M - Video

    • https://github.com/azure/azure-iot-sdk-node - 45 M - Video

    • https://github.com/azure/azure-iot-sdk-java (1.7+) - 49 M -

    • https://github.com/azure/azure-iot-sdk-csharp (C#) for .NET and UWP (Universal Windows Platform) - 57 M -

    • https://github.com/azure/azure-iot-sdk-c for microcontrollers such as RTOS - 60M - Video

    PROTIP: The smallest is Node.

  19. NOTE: The rest of this tutorial uses Python, so:

    cd azure-iot-sdk-python
    
  20. PROTIP: See how much disk space is used by the new folder. On a Mac:

    du -sh
    

    For Python I got 155 M (Megabytes) using --depth=1 versus 195 M with all history.

  21. PROTIP: In the future, to obtain changes to the repo by Microsoft:

    git pull
    

    Create an IoT device and register it with your IoT Hub

    https://youtu.be/wvRE5nvX8GQ by linkedin.com/in/arjmand-samuel-7919934

  22. WARNING: Device names must be unique across all Azure customers. So a GUID.

    The Free tier limits you to one IoT Hub (1 unit) per subscription.

  23. Set Device-to-cloud partitions at 2.

  24. Resource group (unique to your own account):

    https://go.microsoft.com/fwlink/?linkid=848172

    https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-raspberry-pi-kit-node-get-started

    Install Linux Simulator

  25. On Windows 10, install Linux inside Windows by installing the “Windows Subsystem for Linux” (beta as of May 2017) at

    https://go.microsoft.com/fwlink/?linkid=848175

    https://msdn.microsoft.com/en-us/commandline/wsl/install_guide

    Create a user

  26. In the Linux environment, run:

    chmod 555 *
    ./deploy.sh -l
    

    https://github.com/Azure/azure-cli

    Adafruit breadboard

    See video

    Config IoT Device

    On the IoT device (Raspberry Pi):

  27. Open a Terminal window and paste the below:

    git clone https://github.com/codefoster/simple-iot-hub
    cd simple-iot-hub
    

    Alternately, Node.js files from Ken Stroker (stkent@microsoft.com)

    https://github.com/kentstroker/IoT-Live-2017

  28. View file device/index.js. With Node, the entirety is in that file.

  29. To have the Pi’s LED blink and send up a picture to IoT Hub:

    git clone https://github.com/codefoster/iot-workshop
    cd iot-workshop
    

    and simple-iot-hub

    Uses GPIO pins 26 and 20.

    Tutorial

  30. Download folder Mod1Lab4. It contains:

    • deploy.sh contains deployment credentials and the connection string to the IoT Hub

    • CMakeLists.txt defines how the code should be built, and also specifies the dependencies of the C code (including the Azure IoT C SDK libraries that are used to facilitate communication with the IoT Hub).

    • main.c

    • bme280.c and bme280.h is used to take measurements from the temperature/pressure/humidity sensor

    • wiring.h is a C header file that makes it easier for you to run the same app on either a Raspberry Pi or a PC

    Video

  31. Edit deploy.sh to customize the device string used to communicate with a hub. For example:

    HostName=edxiothub-chrishowd-2.azure-devices.net;
    DeviceId=testDevice;
    SharedAccessKey=GB1QcIEu49BuoyDSSbSav+uiXKV1ewgetj7sIy2Q7xI="
    
    HostName=iotzzz.azure-devices.net;
    SharedAccessKeyName=service;
    SharedAccessKey=3k32dkba35EGffd...2viib2gg=
    
  32. Edit the IP address.

    QUESTION: What about IPV6 address?

    Send telemetry data from your device to the IoT Hub

    Azure IoT Hub Service

    The Azure IoT Hub service provides the device-to-cloud and cloud-to-device messaging capabilities and acts as the gateway to the cloud and the other key IoT Suite services. The service enables you to receive messages from your devices at scale, and send commands to your devices.

    The service also enables you to manage your devices. For example, you can configure, reboot, or perform a factory reset on one or more devices connected to the hub.

    IoT Suuite

    The Hub service is core to the Azure IoT Suite.

    See https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-messaging

Azure IoT Gateways

The simple version [2]:
iot-azure-arch-650x348-75kb

Raspberry Pi’s can communicate using these directly supported protocols:

  • HTTP (synchronous waits)
  • AMQP (asychrononous but heavy feature set)
  • MQTT (lower overhead and simpler, but less feature rich, less secure) - 256 bytes or it’s a file upload/download.

A more complex version [1]:

IoT Gateway

Different protocols

iot-azure-gateways-610x301
(Click for full screen picture pop-up)
  1. TODO: Get it:

    git clone https://github.com/azure/azure-iot-gateway-sdk --depth=1
    cd azure-iot-gateway-sdk
    

    Video by Chipalo Street (cstreet@micosoft.com)

    • Connect new and legacy devices
    • Run edge analytics
    • Enable time-sensitive decisions
    • Reduce bandwidth costs - fall back to least cost
    • Operate more reliably
    • Maximize Security

    Filter

    Listen for BLE (Bluetooth Low Energy) messages

  2. Get the repo described in :

    git clone https://github.com/17minds/gateway --depth=1
    cd gateway
    

    Video

    Stream Analytics

    Azure Stream Analytics provides “in-motion” data analysis. IoT Suite uses this service to process incoming telemetry, perform aggregation, and detect events.

    Preconfigured solutions use stream analytics to process informational messages that contain data such as metadata or command responses from devices.

    • Remote monitoring (RMsolution) [2:50]

    https://youtu.be/Tg_RZMXHYj8 by (corywink@microsoft)

    • Predictive Maintenance

    Solutions use Stream Analytics to process the messages from your devices and deliver those messages to other services.

    Create a Stream Analytics job that monitors data received by the IoT Hub.

    Hands-On with Azure Stream Analytics by Jeff Prosise (@jprosise) blogger/co-founder at Wintellect.

    1 Streaming Unit (SU) ~= 1 MB/sec.

    Route Output to Event Hub

    Video

    messageType = ‘a’

    Routing to functions

    Azure Functions are small chunks of code invoked based on a trigger that’s paid for only when it’s running vs. Stream Analytics

    C#, F#, Node.js, Python, PHP, batch, bash, or any executable.

    It connects to many services, such as send an email when a sensor reading exceeds a configured threshold value.

    Video to Create your first function

    Azure Functions Developer Guide has templates:

    • BlobTrigger - Process Azure Storage blobs when they are added to containers. You might use this function for image resizing.
    • EventHubTrigger - Respond to events delivered to an Azure Event Hub. Particularly useful in application instrumentation, user experience or workflow processing, and Internet of Things (IoT) scenarios.
    • Generic webhook - Process webhook HTTP requests from any service that supports webhooks.
    • GitHub webhook - Respond to events that occur in your GitHub repositories.
    • HTTPTrigger - Trigger the execution of your code by using an HTTP request.
    • QueueTrigger - Respond to messages as they arrive in an Azure Storage queue.
    • ServiceBusQueueTrigger - Connect your code to other Azure services or on-premises services by listening to message queues.
    • ServiceBusTopicTrigger - Connect your code to other Azure services or on-premises services by subscribing to topics.
    • TimerTrigger - Execute cleanup or other batch tasks on a predefined schedule.

    Message Routing

    iot-azure-messagerouting-610x256

    Video

    https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-messaging

    Remote Management of Devices

    • Implement a Direct Method on your IoT device and then trigger the method remotely

    • Implement a Firmware Update using a Direct Method and monitor progress of the operation using a Device Twin

    iot-azure-device-twin-650x347

    Queries:

    Among device management plans:
    iot-azure-device-mgmt-650x298-65kb.png

    Create a Cosmos DB storage account

    • Use an Azure Stream Analytics (SA) query job to store telemetry data then make it available for analysis.

    Azure blog Storage and Azure Cosmos DB provide a pre-configured solution to store telemetry

    Azure Cosmos DB stores device metadata and enables device management.

    Video: Device Management lifecycle:

    1. Plan
    2. Provision
    3. Configure
    4. Monitor
    5. Retire

    Job

  3. In Azure, create a new job.

    Query:

    SELECT * INTO blob FROM iotmva
  4. Add Output.

    Storage options:

    • Azure SQL Database can be used as an output for data that is relational in nature or for applications that depend on content being hosted in a relational database. Stream Analytics jobs will write to an existing table in an Azure SQL Database. Note that the table schema must exactly match the fields and their types being output from your job. An Azure SQL Data Warehouse can also be specified as an output via the SQL Database output option as well (this is a preview feature).

    • Blog storage offers a cost-effective and scalable solution for storing large amounts of unstructured data in the cloud.

    • Table storage offers highly available, massively scalable storage, so that an application can automatically scale to meet user demand. Table storage is Microsoft’s NoSQL key/attribute store which one can leverage for structured data with less constraints on the schema. Azure Table storage can be used to store data for persistence and efficient retrieval.

    • DocumentDB (shown in the EDX course)

    • Data Lake Store stores data of any size, type and ingestion speed for operational and exploratory analytics. Stream Analytics needs to be authorized to access the Data Lake Store.

    • Event hub s a highly scalable publish-subscribe event ingestor. It can collect millions of events per second. One use of an Event Hub as output is when the output of a Stream Analytics job inputs into another streaming job.

    • Service bus Topics provide a one-to-many form of communication. While Service Bus Queues.

    • Service Bus Queues provide a one to one communication method from sender to receiver with a First In, First Out (FIFO) message delivery. Typically, messages are expected to be received and processed by the receivers in the temporal order in which they were added to the queue, and each message is received and processed by only one message consumer.

    • Azure Cosmos DB is a fully-managed NoSQL document database service that offers query and transactions over schema-free data, predictable and reliable performance, and rapid development.

    Power BI Visualization

    Power BI can be used as an output for a Stream Analytics job to provide for a rich visualization experience of analysis results. This capability can be used for operational dashboards, report generation, and metric driven reporting.

    Azure Web Apps and Microsoft Power BI provide data visualization capabilities.

    The flexibility of Power BI enables you to quickly build your own interactive dashboards that use IoT Suite data.

    Video

    • Use the Power BI service to view live data coming from your IoT device

References

This summarizes and adds to these blogs and classes:

[1] Developer’s introduction to Azure IoT by Olivier Bloch Senior Program Manager, Azure IoT on March 23, 2016

[2] Microsoft Virtual Academy’s Introduction to Azure IoT published 4 May 2017 is by Jeremy Foster, who has written on Pi extensively at codefoster.com/pi

  1. Introduction to Azure IoT
  2. Introduction to IoT Hub
  3. Creating a Hub, Pricing, and Scale
  4. Device Registry
  5. Protocols and Security - aka.ms/iotarch,

  6. D2C and C2D Messaging
  7. Devices and Kits
  8. Accessing the GPIO
  9. The Gateway SDK
  10. Device Management
  11. Stream Analytics and Routing
  12. Storage and Virtualization

It shows use of the C# SDK.

[3] A more in-depth (4-month) course on EdX shows use of the C SDK.
This is free unless you want verification for $99. Instructors are Geoffrey Morgan ( @geoffmmorgan) of Crank211.com and Chris Howd, Engineer and Software Developer, Microsoft

[4] Using Microsoft Azure* IoT Suite with Intel® IoT Technology Intel blog 1 June 2016.



More on IoT

This is one of a series on IoT:

  1. IoT Acronymns and Abbreviations

  2. IoT Apprentice school curriculum
  3. IoT use cases
  4. IoT reminders prevent dead mobile battery
  5. IoT ceiling dumper

  6. IoT text to speech synthesis
  7. IoT AWS button
  8. Intel IoT
  9. IoT Raspberry hardware
  10. IoT Raspberry installation

  11. IoT Clouds
  12. Samsung IoT Cloud

  13. Predix basics
  14. Predix installation
  15. Predix services
  16. Predix programming