The IoT services described in the previous blogs ensure trust management in IoT systems using computing resources provided by Fog Computing. For that reason, the feasibility of such an approach is heavily dependent on the devices already present in the local IoT networks. Performance tests of trust management services - FTA and FTP - are conducted to evaluate that.

The performance tests were conducted on multiple devices that are selected to represent Fog Nodes:

• Raspberry Pi4 model B [1], with 1.5GHz quad-core CPU and 4GB RAM;
• Raspberry Pi3 model B [2], with 1.4GHz quad-core CPU and 1GB RAM;
• Raspberry Pi Zero [3], with 1GHz single-core CPU and 512MB RAM.

FTA and FTP services were deployed using Docker [4], a container-based plaform for services and platform deployment. The deployed services were implemented using Java programming language and SpringBoot framework [5].

Storage requirements

The storage requirements are measured regarding the HDD, SSD, or SD Card space needed to download and install the Docker images holding the deployed services. For that purpose, images FTA, FTP, and the accompanying services for database and AMQP messaging are measured, providing the following results:

• AMQP Broker -  217 MB
• Database service -  424 MB 
• FTA - 270 MB
• FTP - 270 MB

This results in 1181 MB ~ 1.1 GB of space required for the initial download of the Docker images. It is worth mentioning that additional space will be required once the services are running since the services persist data, at least in the database server. Nevertheless, the requirement of 1.1 GB is easily satisfied by the vast majority of the HDD, SSD, or SD cards on the market, implying that storage requirements can also be satisfied by Fog Nodes.

Startup latency

The second evaluation aspect measures service startup times. Startup time involves a period from starting the Docker container from the preinstalled image until component logs print out information that the service executable is fully deployed. For this purpose, FTP has been taken as a service representative. Measured startup times for each FN device are:

• Raspberry Pi 4 = 19.41 seconds
• Raspberry Pi 3 = 33.031 seconds
• Raspberry Pi Zero = 9538.474 seconds = 159 minutes = 2.65 hours

Startup times indicate that Raspberry Pi 3 and 4 devices can start a single FC service in a reasonable period. A startup time of multiple hours, in the case of Raspberry Pi Zero, cannot be tolerated in IoT environments since this would mean that the IoT system is not functioning for 2.65 hours once plugged in.  

Services' responsiveness

The third evaluation focuses on responsiveness, that is, E2E (network + processing) latency present when invoking FTP services. For that purpose, every 10 seconds, 50 IoT devices invoked FTP services for encrypting and decrypting messages containing sensor measurements. The targeted FTP service is parametrized with a security profile on how the message should be handled, determining the encryption algorithm that will be applied and allowing evaluation of computational complexity on the E2E latency. These profiles are : (1) no security - no encryption, (2) symmetric encryption - AES-256, and (3) asymmetric encryption - ECC with 256 key length. 

Evaluation results presented in the table show that the FTP service on Raspberry Pi 4 and Raspberry Pi 3 devices handles requests in a couple of milliseconds, which is acceptable latency for the vast majority of IoT environments. Significant latency increase (above 10%) on these devices is present only for the asymmetric encryption on the Raspberry Pi 3 device. Nevertheless, provided results imply that these devices are promising candidates for Fog-based services that enable encryption capabilities for IoT devices. On the other hand, the Raspberry Pi Zero device introduces significant latency of a minimum of 1.8 seconds, which is hardly acceptable in most IoT environments. However, suppose the provided IoT service is not time-critical. In that case, the Raspberry Pi Zero device can be used for FTP service, reducing hardware costs in such an IoT network compared to the other observed Raspberry Pi devices.

The provided evaluation results prove the feasibility of the chosen approach for IoT services deployment using Fog computing. The noticed downside is bound to the Raspberry Pi Zero device, which introduces significant startup and E2E latency, making it not usable for time-critical IoT systems. The possible application of the Raspberry Pi Zero device would be as support for the background services so that the introduced latency is not noticeable to the users. On the other hand, Raspberry Pi 4 and Raspberry Pi 3 devices provide performant execution of services and open the door for novel Fog-based services in the future!

[1] https://www.raspberrypi.org/products/raspberry-pi-4-model-b/

[2] https://www.raspberrypi.org/products/raspberry-pi-3-model-b-plus/

[3] https://www.raspberrypi.org/products/raspberry-pi-zero/

[4] https://www.docker.com/

[5] https://spring.io/projects/spring-boot