End-to-End Serverless Real-Time IoT with Python
Sunday, October 13, 2019
Python runs on IoT devices. It runs on Azure Functions. It runs pretty much anywhere!
In this article, we'll look at how to build an end-to-end IoT solution using Python. We'll send telemetry from a device to Azure IoT Hub. Then we'll use Azure Functions and Azure SignalR Service to send messages from IoT Hub to a Python app in real-time.
Send telemetry to Azure IoT Hub using MQTT
Azure IoT Hub is a managed service that facilitates bi-directional communication between your IoT application and the devices it manages. It has a lot of capabilities; we'll use it to ingest telemetry from IoT devices.
IoT Hub supports many protocols, including HTTPS, AMQP, and MQTT. We'll use MQTT to send messages from our IoT device using Python.
It only takes a few lines of code to send messages to IoT Hub over MQTT using the Eclipse Paho library.
import paho.mqtt.client as mqtt
client = mqtt.Client(config.device_id, mqtt.MQTTv311)
client.username_pw_set(iothub_user, sas_token)
client.tls_set()
client.connect(config.hub_address, 8883)
client.loop_start()
while True:
reading = sensor.read()
msg = json.dumps(reading)
client.publish(iot.hub_topic_publish, msg)
time.sleep(sample_rate_in_seconds)
The full source code in a device simulator can be found on GitHub.
A huge thanks to Dave Glover for helping get Python working with MQTT and IoT Hub.
Relay messages from IoT Hub to Azure SignalR Service with Azure Functions
Now that device(s) are sending messages to IoT Hub, we need a way of receiving them. One of the easiest ways to do this is using Azure Functions. With the Event Hubs trigger (which can be used with IoT Hub's built-in Event Hubs compatible endpoint), Azure Functions does all the work for us and will execute our Python code whenever a message appears in IoT Hub.
When a message is received from IoT Hub, we want to broadcast it to clients in real-time using WebSockets. Azure SignalR is a managed real-time messaging service in Azure that does this really well. We can integrate it with Azure Functions using an output binding.
The Azure Function is configured using the following function.json file. The function is triggered by messages in IoT Hub, and it has a single output binding for sending messages over SignalR Service:
{
"scriptFile": "__init__.py",
"bindings": [
{
"type": "eventHubTrigger",
"name": "event",
"direction": "in",
"eventHubName": "my-hubname",
"connection": "EventHubConnectionString",
"cardinality": "one",
"consumerGroup": "$Default"
},
{
"type": "signalR",
"direction": "out",
"name": "$return",
"hubName": "devicemessages"
}
]
}
The actual Python function in __init__.py is just a few lines of code:
def main(event: func.EventHubEvent):
event_json = event.get_body().decode('utf-8')
event_item = json.loads(event_json)
return json.dumps({
'target': 'newDeviceMessage',
'arguments': [ event_item ]
})
The IoT Hub message is passed to the function in the event argument. To send a message over SignalR Service, the function simply returns a SignalR message object and the output binding does the rest.
Receive messages over WebSockets from SignalR Service
Azure SignalR Service works with clients in many languages. Python is currently supported by an open source community library.
from signalrcore.hub_connection_builder import HubConnectionBuilder
connection = HubConnectionBuilder() \
.with_url('https://function-app-url') \
.build()
connection.on('newDeviceMessage', process_device_message)
connection.start()
Each time we receive a newDeviceMessage, we pass it to process_device_message() to display on the screen.
Our app uses the terminalplot library to plot our real-time data in the terminal window.
We can also use JavaScript in a browser to plot our data. In fact, we can have many different clients written in different languages that simultaneously connect to SignalR Service and receive real-time messages.
Resources
Python runs on IoT devices. It runs on Azure Functions. It runs pretty much anywhere!
In this article, we'll look at how to build an end-to-end IoT solution using Python. We'll send telemetry from a device to Azure IoT Hub. Then we'll use Azure Functions and Azure SignalR Service to send messages from IoT Hub to a Python app in real-time.
Send telemetry to Azure IoT Hub using MQTT
Azure IoT Hub is a managed service that facilitates bi-directional communication between your IoT application and the devices it manages. It has a lot of capabilities; we'll use it to ingest telemetry from IoT devices.
IoT Hub supports many protocols, including HTTPS, AMQP, and MQTT. We'll use MQTT to send messages from our IoT device using Python.
It only takes a few lines of code to send messages to IoT Hub over MQTT using the Eclipse Paho library.
import paho.mqtt.client as mqtt
client = mqtt.Client(config.device_id, mqtt.MQTTv311)
client.username_pw_set(iothub_user, sas_token)
client.tls_set()
client.connect(config.hub_address, 8883)
client.loop_start()
while True:
reading = sensor.read()
msg = json.dumps(reading)
client.publish(iot.hub_topic_publish, msg)
time.sleep(sample_rate_in_seconds)
The full source code in a device simulator can be found on GitHub.
A huge thanks to Dave Glover for helping get Python working with MQTT and IoT Hub.
Relay messages from IoT Hub to Azure SignalR Service with Azure Functions
Now that device(s) are sending messages to IoT Hub, we need a way of receiving them. One of the easiest ways to do this is using Azure Functions. With the Event Hubs trigger (which can be used with IoT Hub's built-in Event Hubs compatible endpoint), Azure Functions does all the work for us and will execute our Python code whenever a message appears in IoT Hub.
When a message is received from IoT Hub, we want to broadcast it to clients in real-time using WebSockets. Azure SignalR is a managed real-time messaging service in Azure that does this really well. We can integrate it with Azure Functions using an output binding.
The Azure Function is configured using the following function.json file. The function is triggered by messages in IoT Hub, and it has a single output binding for sending messages over SignalR Service:
{
"scriptFile": "__init__.py",
"bindings": [
{
"type": "eventHubTrigger",
"name": "event",
"direction": "in",
"eventHubName": "my-hubname",
"connection": "EventHubConnectionString",
"cardinality": "one",
"consumerGroup": "$Default"
},
{
"type": "signalR",
"direction": "out",
"name": "$return",
"hubName": "devicemessages"
}
]
}
The actual Python function in __init__.py is just a few lines of code:
def main(event: func.EventHubEvent):
event_json = event.get_body().decode('utf-8')
event_item = json.loads(event_json)
return json.dumps({
'target': 'newDeviceMessage',
'arguments': [ event_item ]
})
The IoT Hub message is passed to the function in the event argument. To send a message over SignalR Service, the function simply returns a SignalR message object and the output binding does the rest.
Receive messages over WebSockets from SignalR Service
Azure SignalR Service works with clients in many languages. Python is currently supported by an open source community library.
from signalrcore.hub_connection_builder import HubConnectionBuilder
connection = HubConnectionBuilder() \
.with_url('https://function-app-url') \
.build()
connection.on('newDeviceMessage', process_device_message)
connection.start()
Each time we receive a newDeviceMessage, we pass it to process_device_message() to display on the screen.
Our app uses the terminalplot library to plot our real-time data in the terminal window.
We can also use JavaScript in a browser to plot our data. In fact, we can have many different clients written in different languages that simultaneously connect to SignalR Service and receive real-time messages.