Inter-connecting IoT devices with APIs

The numbers of IoT devices is on the rise, both in projected figures and devices really being deployed. As of now, most constrained devices are likely to send their data to backend cloud services, sometimes using gateways which are connected to the internet, sometimes via wide area networks - LPWANs - such as LoRa or Sigfox. Examples of cloud backends are Google Cloud IoT Core, Amazon AWS IoT and many more. The primary transmission direction is an uplink: from device to cloud. The reverse direction, or downlink is also used to transmit commands to devices, or even send firmware updates.

The form of communication is likely to be bound to the transmission technology and its capabilities. So a device does have an interface, let’s call it API, but it’s often hidden behind the vendor’s implementation framework, sometimes even buried in binary firmware. This is in stark contrast to the world of Open APIs from the internet industry, say from Twitter or Github. There, developers are used to have open APIs with lots of documentation and bindings for their favourite languages, which in turn boosts developer acceptance.

So what choices do have developers today to have their constrained IoT devices expose an API and be reachable by other devices or gateways? Let’s look at some of the options:

Pre-built everything (or most of the things)

There’s plenty of products out there, so one can easily buy a device with pre-built and pre-installed closed-source firmware, and use API capabilities offered by the vendor. Typically these existing solutions on the market are closer to the end user’s use-case, because they try to solve a problem end-to-end. Solutions can then consist of devices, a gateway and the cloud service, all ready to be used. Which is great in terms of customer experience, because controlling all parts from end-to-end can guarantee a high likelihood of stuff working out well.

However, typically the firmware is closed-source, but offers documented interface endpoints to implement against. Developers can use offered functionality and integrate their own code or applications parts, at least to parts of the solution that have been opened up.

Examples of these are smart home devices, which might be based on a closed procotol from device to gateway, but „open up“ at the gateway side and present HTTP- or MQTT-interfaces.

Pro’s:

  • Most of them are complete working solutions, end-to-end
  • Security is likely to be solved, as part of the existing product warranties

Contra’s:


  • APIs and other mechanisms may be subject to change
  • typically bound to transmission technology (i.e. in Smart Home Solutions in Europe, BidCos is popular, which is a closed-source solution)
  • often, it’s not possible to implement changes on the devices itself, because firmware cannot be changed.

Pre-build, but Open Source

This option is about using an open source solution with pre-exisiting firmware code, and extend it according to own needs. A good example of this is Blynk. After putting the blynk library on a microcontroller (with support for many popular controllers and frameworks, read: Arduino), one can easily create and connect mobile dashboards to a device and exchange data. The gateway-ish middle tier software is Open Source as well, so at least there is an open path to more professional development, where one is able to forward data to backend services other than blynk’s.

Another example i would like to bring up is about Particle and their new Particle Mesh series products. Between devices it’s a IEEE 802.15.4 based mesh network, but their open on the gateway side (because one’s able to code firmware for it), and they’re open on the cloud side, with REST APIs. So their product is open, and one can use pretty much out-of-the box functionality.

Pro’s:

  • Also often working solutions, maybe not completely end-to-end, but hey, we’re devs, so..
  • Open source, possibly with a community behind.
  • Developers can implement their own firmware, using existing firmware code as a framework/library
  • Security likely to be already solved

Contra’s:

  • Still bound to underlying transmission technologies, i.e. blynk requires a TCP/IP network

Custom firmware implementation

This option is about developing your own firmware, using existing libraries and frameworks for protocols and data exchange. It’s the vast field of maker hardware and maker projects that revolve around boards from Arduino or ESP (or many others), supported by the communities behind the products and frameworks. For developers this is can be great fun (well at least if you agree that coding in C/C++ is regarded great fun :), and the supply of libraries and code examples is really good. In this option, developers have the most degrees of freedom when it comes to building an IoT solution.

Pro’s:

  • Freedom of choice for language, frameworks, libraries.
  • Freedom of choice for transmission technology, protocols etc. You like CoAP and EXI, or MQTT-SN + MessagePack, LoRa or HTTP+JSON? go ahead :)
  • Great support and input from maker space communities

Contra’s:

  • Green-field approach takes longer to implement: There are quite some pitfalls in embedded programming, in using (and combining) libraries and parts from different vendors etc.
  • Finding libraries and frameworks for security on constrained devices is still hard and time-consuming.

Pre-generate parts of the solution

This last options is a bit out-of-the-box, because it does not play a role anymore in web development: It’s about generating code from models, (sometimes called Model-driven Architecture or Model-driven Development). The idea is to separate code, architecture and domain knowledge in distinct places, and regenerate code from a domain model whenever the domain or the underlying (code) architecture eeds a change.

An example of this is (in part) Temboo, where integration code for public APIs (such als Flickr, GitHub and many more) is hosted and made available via a model-based interface. Even for embedded devices. So they’re bringing together the world of web APIs with the world of embedded devices. Another example is the overall works of the World Wide Web Consortium, W3C, regarding a Web Of Things. Core belief is that devices should be able to communicate with other devices, gateways and backends over open APIs, which can at best describe themselves using standardized protocols and data formats. That’s what i try to capture within the THNG:STRUCTION generators.

Pro’s:

  • One pro is the idea behind it, to not implement code over and over again but place it in models, which can be reused in multiple (and potentially different) contexts
  • Typically not tied to vendor libraries, because this method delivers code (at best open source)

Contra’s:

  • Finding libraries and frameworks for security on constrained devices is still hard and time-consuming.
  • Offerings for this type of service are limited.

Conclusion

So there are quite some choices here. To sum it up:

  • Companies invested time and money into establishing secure end-to-end solutions. It’s natural for them to reap the benefits, so many system are only partially open to integrations, or open source code. However this is not true for all solutions, and there are good examples around for open ecosystems.

  • IoT transmission technologies, protocols and data formats are highly heterogeneous. When building a solution you have to make a decision at some point in time, which limits your degrees of freedom. If you -for example - build the first prototype for a product using WiFi, chances are low to make it work for LPWANs due to high differences in underlying technology.

  • The kind of openness we’re used to from the internet industry has not yet arrived in the IoT space, which limits the ability of devices to communicate with each other. Approaches such as the Web Of Things are highly interesting but not very wide spread yet.

What can be done? I’m always a fan of collecting requirements for a solution as early in the realization as possible. Two major aspects of requirements regarding IoT projects are transmission technology and power consumption: Where are things going to be deployed, how accessible are they, what do they need to communicate, how much and how often? From there on explore the potential protocols, frameworks and solutions that are available. And though it’s still a concept for future days to come, invest some time in new concepts such as the Thing Description from the Web of Things.