HTTPS on the ESP32 - Part 1, as a server

Many embedded maker projects involve HTTP or MQTT communication, and more often the question arises if one can secure that communication in an easy way. The answer can be tricky and highly depends on the hardware and the OS or embedded framework being used. In this series we'll take a look at the ESP32 using the Arduino framework, and its capabilities regarding TLS.

Why not just prepend https:// and be done?

In regular languages or frameworks for web or desktop development, we're used to just make HTTP calls using https://, or include small code snippets for HTTPS listeners with a server key and certificate and that's (almost) it - securing an API on the transport level isn't that hard, sometime putting an HTTPS-offloading proxy in front also solves the problem. On embedded devices, however, things look much different:

  • Sometimes MCUs do not offer encryption in silicon. The MCU itself of course can calculate encryption and signatures, but is typically (too) slow at it.
  • Some networking/transceiver chipsets include encryption (e.g. all WiFi chip sets have to, otherwise they could not connect to a WiFi..), and they may make this functionality accessible to the outside
  • Some boards feature a secure element - that is a separte security chip for crypto primitives, encryption and storage of keys etc.

Whatever the situation for a given board/framework/OS is, the communication libraries above have to make use of specific features, and the majority of them keep being specific to the board/framework/OS.
But what is needed for HTTPS? This is determined by the cipher specification, and involves a bunch of cryptographic functions:

  • A cryptographic hash function, typically SHA256
  • A symmetric cipher: AES, Camellia
  • An asymmetric cipher/signature algorithm, e.g. RSA, DSA, ECDSA, ...
  • on top of that, key exchange: DH, ECDH, at best in ephemeral variants.

Crypto on the ESP32

Luckily, the ESP32 includes some on-chip crypto functionality. Beside what's necessary for all WiFi-related crypto, it has builtin AES, RSA, SHA-2, ECC and a random number generator. The Arduino Core of ESP32 includes a port of Arm Mbed TLS (see in tools/sdk/include/mbedtls) and also OpenSSL. These functions can be addressed directly, DFRobot has an example for AES-128-ECB. So, the foundation is there.

HTTPS

As a first step, i'd like my ESP32 to be a web server for a REST API, but using HTTPS. Now most of HTTP server code for Arduino works with EthernetServer or WiFiServer, but there's no TLS or link to the mbedtls port beneath. A sample HTTPS implementation can be found at github.com/fhessel/esp32_https_server. It is marked as work-in-progress, but i gave this a try. It uses OpenSSL to handle TLS and offers a HTTP server API to create endpoints and resources etc. I'm using PlatformIO with a NodeMCU ESP32s to set up the example code:

$ mkdir pio-https
$ cd pio-https
$ pio init -b nodemcu-32s --ide vscode

Clone the code from fhessel's repo, arrange it under /src. We going to need data/, https/, tools/ and the example .cpp/.h under src/.

$ mkdir suppl
$ git -C suppl clone https://github.com/fhessel/esp32_https_server.git
$ cp -rp suppl/esp32_https_server/{data,https,tools,https_server.*} src/

The repo's README has a decription of the library, its capabilities and how to set it up. In essence, what needs to be done is:

  • Add WiFi credentials in /data/wifi
  • Create a X509 key and certificate, reformat it as C code so it can be included/compiled into our firmware
$ cd src/data/wifi/
$ mv wifi.example.h wifi.h

Edit wifi.h and put in some valid WiFi WPA2 credentials:

#define WIFI_SSID "<your ssid goes here>"
#define WIFI_PSK  "<your pre-shared key goes here>"
$ cd ../../tools/cert

Edit create_cert.sh and update the X.509 location parts that will make up the DN of the certificate. Run create_cert.sh:

$ . create_cert.sh
Generating RSA private key, 1024 bit long modulus
..........................++++++
...........................................................++++++
e is 65537 (0x10001)
Generating RSA private key, 1024 bit long modulus
..................................................++++++
..................++++++
e is 65537 (0x10001)
Signature ok
subject=/C=DE/ST=NRW/CN=esp32.local
Getting CA Private Key
example.crt: OK
writing RSA key

It creates a CA, then a server key and a request for a certificate, to be signed by the CA. In the end, xxd is used to format the DER-encoded certificate/key into C source code, which gets written to data/cert/cert.h and data/cert/private_key.h. These files, in turn are included by https_server.cpp and used.

Let's compile this:

$ cd ../../..
$ pio run
(...)
Linking .pioenvs/nodemcu-32s/firmware.elf
Building .pioenvs/nodemcu-32s/firmware.bin
Retrieving maximum program size .pioenvs/nodemcu-32s/firmware.elf
Checking size .pioenvs/nodemcu-32s/firmware.elf
Memory Usage -> http://bit.ly/pio-memory-usage
DATA:    [=         ]  13.5% (used 39880 bytes from 294912 bytes)
PROGRAM: [======    ]  57.9% (used 758310 bytes from 1310720 bytes)

Works out well. I'd like to add one more thing which is printing out the IP address on the serial port, to see how to reach the device. Edit src/https_server.cpp, go to line #197 and add:

        Serial.println(WiFi.localIP());

Compile, flash and watch the serial port:

$ pio run -t upload && pio device monitor -b 115200
(...)
............................... connected.
192.168.0.101
Creating server task...
Beginning to loop()...
Configuring Server...
Starting Server...
Server Socket fid=0x1000
Server started.

Looks good. We could use a web browser, point to the IP and inspect the certificate. The browser will complain about the certificate being not valid, because obviously it's self-signed and thus not trusted. But nevertheless it works! On the command line, either curl or openssl are nice alternatives to find out more about the encryption:

$ curl -vvv --insecure https://192.168.0.101/
(...)
*   Trying 192.168.0.101...
* TCP_NODELAY set
* Connected to 192.168.0.101 (192.168.0.101) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=DE; ST=HE; L=Darmstadt; O=MyCompany; CN=esp32.local
*  start date: Jul  4 11:53:12 2018 GMT
*  expire date: Jul  1 11:53:12 2028 GMT
*  issuer: C=DE; ST=HE; L=Darmstadt; O=MyCompany; CN=myca.local
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> GET / HTTP/1.1

curl with a -vvv prints out TLS handshake messages as well as the server certificate. Openssl does the same, with additional levels of details using -debug and -msg.

$ openssl s_client -connect 192.168.0.101:443 -showcerts
(...)
---
Certificate chain
 0 s:/C=DE/ST=HE/L=Darmstadt/O=MyCompany/CN=esp32.local
   i:/C=DE/ST=HE/L=Darmstadt/O=MyCompany/CN=myca.local
-----BEGIN CERTIFICATE-----
MIICHjCCAYcCAQIwDQYJKoZIhvcNAQEFBQAwVzELMAkGA1UEBhMCREUxCzAJBgNV
BAgMAkhFMRIwEAYDVQQHDAlEYXJtc3RhZHQxEjAQBgNVBAoMCU15Q29tcGFueTET
MBEGA1UEAwwKbXljYS5sb2NhbDAeFw0xODA3MDQxMTUzMTJaFw0yODA3MDExMTUz
MTJaMFgxCzAJBgNVBAYTAkRFMQswCQYDVQQIDAJIRTESMBAGA1UEBwwJRGFybXN0
YWR0MRIwEAYDVQQKDAlNeUNvbXBhbnkxFDASBgNVBAMMC2VzcDMyLmxvY2FsMIGf
MA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3AaSEXqVR+GIPTBA92hkoHViO3J0q
84/ysIBauA3f0b/YZdvl+rDa4CbwDw9UiHvGWZ5uMmiXXIWeF+B3A5iI7u/DOQSh
E1zE5SlTFjgID3gAOcBauY0XatbdYPKDcjq5r3TxPWyWtCr9Y5C0YslFIiB5LMRa
Cf/WHxti3rCJJQIDAQABMA0GCSqGSIb3DQEBBQUAA4GBAKKxgpT/7gX605TqTFWW
nO6BuCUvwgmrXqegMfZlUqRYXLS7cWzKMl8wlqGl5UJBNwJtPXcbjzqs+cnD/RH3
9UgTqpCeFH06XiQgtVQi83XisHxgMytVHTaoLoIKJLRMgNbDFzWpexq6reYJU7Jh
dYgEVq6aVFPd9boeGEe23myQ
-----END CERTIFICATE-----
---
Server certificate
subject=/C=DE/ST=HE/L=Darmstadt/O=MyCompany/CN=esp32.local
issuer=/C=DE/ST=HE/L=Darmstadt/O=MyCompany/CN=myca.local
---
No client certificate CA names sent
---
SSL handshake has read 991 bytes and written 512 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: B852C6DB374B8682DBC8761176128C78F2934A449B7F9F0EB0D4D5FAEAF913CD
    Session-ID-ctx:
    Master-Key: 4D09B700856D8A6019FFA9D38B8E24B01CB531EEBFAF582444F605D460C64435D06FDB189D309E70DE47B848B61F8769
    Start Time: 1530785865
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)
---
closed

Good to see: TLS 1.2 is used, the Cipher is up-to-date. On my ESP32, it crashed sometimes with a panic. I did not investigate this further, but as the README says: it's work in progress. What i wanted to now is the amount of time necessary to set up the TLS connection. A small patch to https/HTTPSServer.cpp does the job:

line #182

  // Start to accept data on the socket
  long d1 = millis();
  int socketIdentifier = _connections[freeConnectionIdx]->initialize(_socket, _sslctx, &_defaultHeaders);
  long d2 = millis();
  HTTPS_DLOG((d2-d1));

This measures the time nedded to run HTTPSConnection::initialize, and it prints out:

HTTPSServer->debug: [-->] New connection. Socket fid is:  0x1001
HTTPSServer->debug: 1483

So it's ~1.5 seconds to set up the TLS connection, and that can roughly be measured on the client side as well:

$ time $(echo '' | openssl s_client -connect 192.168.0.101:443 )
(...)
real	0m1.516s
user	0m0.034s
sys	0m0.007s

That's quite some work for the ESP. For single calls to e.g. an API this can be acceptable, for subsequent calls to access a web site on the ESP, it might be too much.

Wrapping up

  • If you really want to work with HTTPS on ESP as a server, make sure to check out github.com/fhessel/esp32_https_server
  • Given the time, a deep dive into openssl and Mbed tls could be worth it to further understand the inner workings
  • Without optimization, TLS is probably too slow to make an ESP a secure web server

In the next post, we're going to look at the ESP32 being a HTTPS client :)