Programming applications in fog-to-cloud platforms, the mF2C solution

Reading Time: 2 minutes

 

Posted by  Daniele Lezzi, Senior Researcher, Barcelona Supercomputing Center

 

The traditional cloud computing model, based on a centralized control of computing and data resources, does not provide the proper support to the requirements of big data applications that produce and consume volumes of data through IoT devices, fast mobile networks, AI applications, etc.

Fog computing has emerged as a complementary solution to overcome the issues related to real time processing, security, latency and transparent management of a decentralized, heterogeneous and dynamic set of resources.

Application partitioning, task scheduling, and offloading mechanisms are all problems widely explored in the field of distributed computing. The main differences between previous efforts related to cloud computing and mobile computing are due to issues related to the high mobility of the device, the limited availability of energy of the devices and the impact of the network (latency, monetary cost, bandwidth) on the performance of the entire framework.

mF2C takes on the yet-unaddressed challenge of fog-to-cloud and cloud-to-fog offloading. mF2C proposes a solution based on COMP Superscalar (COMPSs), a programming model and its associated run-time that on the one hand is able to handle distribution, parallelism and heterogeneity in the resources transparently to the application programmer, and on the other hand is able to handle data regardless of persistency by supporting a single and unified data model.

The architecture of this programming framework (Distributed Execution Runtime) is designed following the OpenFog Reference Architecture. The lowest layer represents the low processing capability devices, such as sensors or embedded devices that produce data, while the middle layer contains fog devices that have more processing power (as a smartphone or a tablet) and are able to deploy and orchestrate the execution of a distributed application using other fog devices as workers (fog-to-fog). Clouds are at the top layer, hosting services for the control of the entire stack or used for the execution of computing intensive applications started either from the same cloud layer or from a fog device. It is worth noting, indeed, that the framework can be used to instantiate applications on smart devices on the fog layer and to offload part of the computation to the cloud (fog-to-cloud) or use the fog devices as workers for a cloud application.

The runtime is integrated with the Data Management service (based on dataClay) in order to allow applications and services to access data transparently regardless of its location and of its format, thus hiding the complexities of data management. In particular, this integration allows the runtime to recover the execution of part of the application failed on a fog node (disappeared for low battery or because no longer in the fog area), retrieving the data already produced by a task and resubmitting it on another node.

The adoption of container technologies allows an easy deployment of the DER component which is executed as a microservice.