When talking about Internet of Things systems development, the discussion often ends up being about sensors and embedded development. But that’s not what the Internet of Things (IoT) is really about, and is a gross oversimplification of what is involved when developing an Internet of Things system.
An IoT system is never simple, or predictable. Developing for such a system is very difficult. In fact, it is much harder than developing for many other, more well-known or “traditional” software systems. This is caused by, among other things:
- The number of entities/devices;
- The fact that these entities are hard (and sometimes impossible) to access, making it difficult to apply bugfixes or updates;
- The potential for an exponential number of communication lines;
- The complexity that emerges with communication;
- The challenges of the physical world.
IoT development, however, is often approached very similar to something the developer knows, be it embedded programming (which reasons from the devices and doesn’t take into account the overarching concepts) or web development (which reasons from the cloud, develops more centralized systems and assumes anything can change quickly). This is because we don’t have IoT developers yet that are properly trained to develop these systems and because we lack the tools to support us in our work.
While developing IoT systems, I’ve found two aspects to be of paramount importance to be able to develop IoT systems. They have helped me receive fast and accurate feedback, with better production results. These are simulation and visualization.
Simulation
Whenever you’re developing for and deploying a sufficiently complex IoT system, you will quickly find that it is highly impractical (and, more often than not, impossible) to create a realistic environment for testing. This is because of physical and financial constraints: You have to purchase a realistic number of devices, and set them up in a realistic environment.
If you happen to develop an IoT system that consists of less than 10 devices and if an office is a realistic environment for your system, then you’ll have no problems. If, however, your IoT system will be deployed throughout a city, with thousands of devices being operational out in the field simultaneously, you’re in trouble.
The typical approach in this scenario is to purchase around 5 or 10 devices, set them up in a test environment (i.e. your office) and extrapolate all findings from there. If things look good and your system is far enough along, you run a pilot with a willing customer, register and process all findings, and pilot again until you are fairly confident that things will work in an arbitrary setting.
This approach has a number of downsides, but most of them can be summarized in the fact that feedback for your development takes very (and sometimes, very very) long. As your office testing environment is not realistic, you won’t get decent feedback until you are ready for a pilot, which can take months of developing. You might have made fundamental mistakes at that point, which can be extremely time- and money consuming to correct.
I’ve found that the application of simulations in developing IoT systems is fundamental to their success. Using Docker it is now possible to relatively easily start a large number of software entities per workstation or server, isolated from each other and in a completely controlled environment. Couple this with a cloud provider and you have the capacity to start thousands of entities instantly.
In the context of IoT development, this gives us the ability to model real-life situations and scenarios with modest and realistic resources. You can model a realistic number of entities as well as their relations, as long as you take the time to create an accurate deployment scenario. To achieve this, you have to:
- Program your devices as Docker-supported entities;
- Model real-life connection constraints (such as latency and timeouts) based on the connection technology;
- Implement tooling to easily draw and generate networks of entities. We’ve used a graph editor that exported to GraphML, which is easily parsed; every node represents a software entity, and every edge represents a communication path;
- (Optionally) Implement tooling to codify real-life scenarios to be executed against your system.
While this requires additional work, the incredible benefit is that it allows each and every developer in the team to easily run simulations on extremely complex scenarios whenever he wants, often even on his own workstation. This can provide the team with immediate, realistic feedback.
Furthermore, setting up these simulations has an added benefit that they can be sped up to generate data over weeks in moments. This generates instant feedback on scenarios that might have required a pilot of months, allowing developers to work much more efficiently.
Visualization
When developing for a single device, you can easily observe its behavior. You program some code, load it on the device, and see if the LEDs blink or whether the device sends the right message by checking its logs.
If you develop a web service, you can observe its behavior even easier. You change the code and can often immediately see the changes in your browser.
While these approaches might work for individual components, they won’t work for visualizing the execution of an entire IoT system. In fact, more often than not it is not possible to see the execution of an IoT system. Sure, there might be actuators whose effects you can see, but the decision-making process of why the actuator was triggered is not something that is typically observed.
Furthermore, with sufficiently complex systems, logs are extremely hard to parse. Due to the number of active components the logs more often than not are a mess, and humans are not well suited to quickly process a large number of lines of text. We like images and colors. So let’s use images and colors.
Given the simulated environment as described above, it is not difficult to extend this with a visualization environment. An IoT system can most easily be visualized as a graph, so it is usually most natural to implement a graph visualization for your system. To achieve this, the following has to be done:
- Design what you want to visualize, and how the entities could visualize this in a graph: a blinking LED could be a visualized by a blinking node in a graph, a message that was transmitted can be shown by a blinking edge or little “packages” floating from one node to another, etc.;
- Select a graph visualization tool or library;
- Implement an API that, given some visualization commands, draws and updates a graph in real-time (if you’re lucky, your graph visualization tool already has this; an example of this is Ubigraph, although it, unfortunately, isn’t supported anymore);
- Modify your simulated entities to send visualization commands to your visualization tool.
It might require some creativity to map behavior to a graph visualization, but the result is a way to easily see how your IoT system behaves. This makes it much easier to debug complex situations that might take hours of reading logs or to find bugs in the decision-making process that wouldn’t ever have been noticed until it was too late.
Conclusion
I’ve used both simulation as well as visualization in the development of Internet of Things systems, and I feel that they are essential to being able to develop complex systems like this. You need quick feedback and an easy way to see what is happening.
We can actually see that many other fields of software development have seen an evolution in their tooling on specifically these aspects. For web development we have the popularity of interpreted languages, we use TDD or BDD and CI to quickly detect issues and get feedback, and in game development, the popularity of editors such as Unity stems in part from its excellent quick visualizations that give you instant feedback on what is happening.
IoT development is a young field, and we need tooling to support the development of these complex systems. IoT IDE’s will become more and more necessary in the coming years, and simulation, as well as visualization, has to become a part of this. I’m very curious if others see the need for these tools as well because if so, I will consider building such a development environment. So contact me if you’re interested!
Originally published on Medium.
