Emotion recognition on an ESP32 device using Scailable AI deployment & sending the results to the cloud via LoRa using the KPNThings network (image by author).

How AI on the Edge (provided by Scailable) and LoRa (provided by KPN Things) jointly enable novel (I)IoT applications.

This tutorial describes a demo we recently presented during the 19th KPN Startup Afternoon; we demonstrated how we can use the Scailable Edge AI deployment platform to deploy a fairly complex AI model (a deep neural network recognizing human emotions) to a small ESP32 device. Subsequently, we used an Arduino MKR WAN 1310 and KPN Things to send the resulting emotions to the cloud via LoRa (find a short video here).

We aim to provide sufficient detail to replicate the full demonstration yourself. Although we think the demonstration is interesting in its own right, we have chosen this setup predominantly…


Visualizing a simple image processing pipeline (image by author).

ONNX is an amazingly useful format for storing Data Science / AI artifacts for version control and deployment. We are happy to share sclblonnx, a python package that enables easy editing and augmenting of ONNX graphs.

Over the last year at Scailable we have heavily been using ONNX as a tool for storing Data Science / AI artifacts: an ONNX graph effectively specifies all the necessary operations to be carried out on input data to generate a desired output. As such, ONNX can be used to not only store complex AL/ML models (as popular ML frameworks like TensorFlow and PyTorch will allow you to do out-of-the-box), but also all the pre- and post-processing that is neccesary for the deployment of the trained model in some specific context. Thus, we use ONNX as a tool for version…


Simple image processing using ONNX (and build from scratch!). Image by author.

ONNX is being used more and more to store complex DNNs; however, its use far extends the simple storing of fitted models. This tutorial shows how to build an image processing pipeline in ONNX — which can subsequently be deployed across devices — with only a few lines of Python code.

ONNX has been around for a while, and it is becoming a successful intermediate format to move, often heavy, trained neural networks from one training tool to another (e.g., move between pyTorch and Tensorflow), or to deploy models in the cloud using the ONNX runtime. However, ONNX can be put to a much more versatile use: ONNX can easily be used to manually specify AI/ML processing pipelines, including all the pre- and post-processing that is often necessary for real-world deployments. …


The ONNX pipeline we will build in this tutorial; image by author.

Hands-on Tutorials

ONNX provides an extremely flexible format to store AI/ML models and pipelines. To learn how, it's instructive to build an ONNX graph by hand.

ONNX has been around for a while, and it is becoming a successful intermediate format to move, often heavy, trained neural networks from one training tool to another (e.g., move between pyTorch and Tensorflow), or to deploy models in the cloud using the ONNX runtime. In these cases users often simply save a model to ONNX format, without worrying about the resulting ONNX graph.

However, ONNX can be put to a much more versatile use: ONNX can easily be used to manually specify AI/ML processing pipelines, including all the pre- and post-processing that is often necessary for real-world deployments. Additionally…


Deploying trained AI models on edge devices might seem challenging. However, using minimal WebAssembly runtimes, and automatic conversion from ONNX to WebAssembly, modular AI/ML model deployment Over the Air (OtA) to pretty much any edge device is possible.

We received a number of responses after posting the video above on LinkedIn (find the original on Youtube). The video shows how an edge device equipped with a camera can be used to recognize hand-written digits and characters. The crux of the video is the demonstration of modular AI deployment. In the first ~20 seconds of the video we demonstrate that the deployed model is able to recognize digits, but fails on characters. Next, the video demonstrates an Over the Air (OtA) update of the deployed model such that the device is also able to recognize characters. This is…


A prototype of the maintenance drone running in the lab at the Centre de Recherche and Infromatique, Lille. (Image by author).

How TensorFlow, ONNX, WebAssembly, and the Scailable platform team up to automatically detect and restore cracks in concrete surfaces.

Imagine having to maintain hundreds of concrete, metal, or plastic silos located in your nearby harbor. As time progresses, the surfaces of these silos, although often cumbersome to inspect due to their size, will require maintenance: as cracks emerge due to wear-and-tear, early detection and repair will significantly drive down maintenance costs and prolong the silo’s lifetime.

The scenario above is exactly the scenario we are facing as part of a joint European INTERREG project called SPEED [1]. Our solution is conceptually easy, but challenging to build: we are endowing a fleet of drones with the ability to autonomously fly…


A part of the Bootstrap Thompson Sampling algorithm (image by author).

An uncommon combination allows efficient sequential learning on the edge

The multi-armed bandit (MAB) problem is a relatively simple (to describe that is, not to solve) formalization of a sequential learning problem that has many applications. In its canonical form, a gambler faces a slot machine (colloquially called a one-armed-bandit) with k >1 arms (hence the term multi-armed bandit). Given a (finite) amount of coins, the gambler sequentially chooses which arm to play — subsequently she inserts a coin and pulls the arm — after each play observing some reward (e.g., a win or a loss — typically not winning coins that can be re-used). As the gambler one-by-one squanders…


Image by Author.

With orthogonal persistence we can implement sequential learning on Edge devices

In previous post(s) we have shown how one can exploit the differences between model training (i.e., requiring all the data and often many passes through the dataset) and prediction (i.e., based on a single function with pre-trained parameters) to efficiently deploy Machine Learning (ML) and Artificial Intelligence (AI) models. By transpiling trained models to WebAssembly it is possible to deploy supervised and unsupervised models efficiently to the cloud and to Edge devices. We have been able to deploy models on devices that have less than 64Kb in available memory. Pretty cool. …


Reducing the memory footprint and improving the speed and portability of deployed models.

The code accompanying this post can be found here.

In recent months we have helped many companies deploy their AI / ML models in various environments. We have contributed to model deployment in Healthcare, and, in the last few months, we have helped several companies move trained models to different types of IoT devices. Particularly in the latter case, requirements are often strict: both the number of compute cycles and the available memory are often limited.

In this post I clarify how we make sure that models trained using standard ML libraries such as PyTorch, Scikit-learn, and Tensorflow can be…


Deploying complex AI models to the browser (and beyond) is surprisingly easy.

Using AI (or ML) models in practice has been a challenge for years. Yes, many nice examples do exist, but in our experience, most companies first face the challenge of bringing their data together, then the challenge of building and validating useful models, and finally the challenge of deploying these models wherever they are most needed (be it in-house, in the cloud, or on the edge).

However, each of these challenges has become easier and easier over the last few years. None of them are exactly trivial yet, but tools abound for each step of the process. …

Maurits Kaptein

I am professor at Tilburg University working on bandits, causality, and Bayesian inference with various applications. I also care about AI & ML deployment.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store