mF2C 3rd Newsletter, Summer 2018
This is the third mF2C newsletter aimed at disseminating project activities and progress to a broad audience. Previous newsletters are also available.
The mF2C project is abuzz with activity as it’s approaching its first review, half way through the project. Software is tested, broken, fixed, and tested again; deliverables are written, presentations are updated and rehearsed. A busy time for insiders, but how useful is this process for anyone else?
The review will demonstrate all three use cases: smart cities, smart boats, and a smart hub deployed in an airport. It is difficult to get boats into a room – a well-known fashion house recently had a go at it, but they could get away with a half boat; we would need several whole boats. So the boats in the smart boat use case are simulated, typically on laptops, as is the airport and the city from the other use cases. The mF2C project has developed a test environment which would allow us to run a whole IoT environment inside of a single laptop, but that also looks less convincing to the reviewer: how do you know we’re not cheating? So a deployment across several laptops is used.
The review is an opportunity to take stock of the existing software. As is best practice in development of larger software projects, the software is split into components which are loosely coupled, that is, they exist as independent components but communicate with each other, usually using web services protocols. Having the software developed as independent components helps developers these components individually, but also promotes reuse of them as they can in general be plucked out of their original context and deployed elsewhere. All that is needed is that components are tested and available, and have their interfaces documented. Getting things ready for the review ensures that all the components are ready at the same time, and will talk to each other. There are twenty-five different components in IT-1.
| Moreover, a decision early in the project to deploy components in containers – essentially lightweight virtual hosting environments – has paid off. Older software projects would have been content with providing packages to install alongside those of the host operating system, but the process of deploying the component in a container makes the deployment even easier, and adds the benefit that we can control the interfaces available to the container and the resources that are allocated to it – even if it adds more complexity in preparing the software for distribution than the traditional packaging approach. The containerization process can help improve security, if used carefully and correctly, but one has to be careful with the privileges allocated to the container. |  |
Moreover, a decision early in the project to deploy components in containers – essentially lightweight virtual hosting environments – has paid off. Older software projects would have been content with providing packages to install alongside those of the host operating system, but the process of deploying the component in a container makes the deployment even easier, and adds the benefit that we can control the interfaces available to the container and the resources that are allocated to it – even if it adds more complexity in preparing the software for distribution than the traditional packaging approach. The containerization process can help improve security, if used carefully and correctly, but one has to be careful with the privileges allocated to the container.
In another improvement over past practices, not only are the containers readily available but also the source code. While Horizon 2020 projects have in general always produced open source software, actually finding the source code could sometimes be challenging – it would be hidden away in private repositories. Not so for mF2C: source code for all the fog components is available in github. As documentation is essential for use and reuse of the software, the documentation is of course also available publicly, in readthedocs. For a more general audience, the project has also recently published four friendly videos to introduce people to the project and its goals. These videos explain the architecture, as well as the critical processes of registration, resource categorisation, and discovery.
Looking ahead, other than the three use cases, who else would look to mF2C for software, guidance, or knowledge? The diagram above shows the different aspects of the value added by the mF2C developments. On the left, resource and service providers are interested in providing services, and mF2C can bring additional customers because it adds means to connect entities in the fog to services in the cloud. On the right of the diagram, value is added for the applications developers, because mF2C provides a platform for developing fog/fog-to-cloud applications. Having an open source platform that makes it easier to bring together applications developers and resource providers will be hugely advantageous: applications are not locked in to a specific commercial cloud service provider.
In the communications – shown in the diagram above – the mF2C software is responsible for the control communications that manage the execution of application tasks. The red highlights focus on both the restrictions and innovations that can be found in the communications in IT-1. One of the early architectural choices was to focus on a leader agent, located in the fog area. Once a device – say, a smartphone – joins the fog, there is a security bootstrap mechanism (not shown in image) where the agent connects to a service in the cloud, through a secure fog-to-cloud gateway, in order to obtain a credential. It then starts a process of discovery in which a leader is found and authenticated. Once the leader is discovered, the device can start participating through its agent, asking for tasks to be run, or discovering peer agents in the fog. The red highlights in the diagram show communications – going from left to right, control data is used within the platform to manage metadata about available services, applications communicate with their peers through agents, or the agents with their leader, the leader communicates with the backup leader, and the whole thing together providers edge-fog-cloud communications.
What is the role of the backup leader? Obviously, the leader is an important participant in the fog: without it, there would be no functioning mF2C fog, as the platform would not be able to register services. We had three options: accept that leaders can fail, have a backup leader available, or run an “election” process in a fog that does not yet have a leader. While we have designs for the latter, we decided for IT-1 to follow the backup leader strategy. A part of the demonstration for the review will indeed show a leader being removed, and the backup leader taking over. Through synchronisation in the data management layer, the backup leader will have recent data from the current leader, but another approach is to rebuild the leader database by rediscovering the agents that are currently present in the fog.
Software is not the only thing the project has produced. Apart from the sixteen deliverables produced so far, and several publications, the project is also extremely excited to be organising two high profile workshops in the coming six months. The first will take place in August in Torino, Italy, at the Euro-Par 2018, the 24th European conference on parallel and distributed computing. The second workshop will take place at the IEEE/ACM Utility and Cloud Computing conference which will take place in December, in Zurich. In both cases, the project is organizing workshops on fog-to-cloud and edge computing, the first of their kind at these conferences.
 |
Since the last newsletter, the project has also organised two general assembly meetings, with dedicated time for developers to turn coffee into working code and to work on the integration of services. Hosted by cloud and IoT services experts SixSq, the first was held in January in Geneva. The second such meeting was held in May 19-20, 2018 in Vilanova i la Geltrú, Spain and was organized by WorldSensing at UPC premises: here, code for the selected components for the IT-1 release was demonstrated. As an additional benefit of this process, a demonstrator has been made available for everyone to try: |  |
 |
Following up on an internal workshop in November 2017, TUBS have organized a training session in April 2018 on Machine Learning concepts and how to use them for improving decision processes in the mF2C core components. Initially, machine learning is being used for monitoring service level agreements (SLA), to quite simply assess whether a resource has delivered what it should in time, or not. Once this simple approach is working, the project will expand to a larger parameter space, so the algorithms will process many more dimensions of input data, thus leading to more sophisticated predictions, and thus better placement of tasks. |
| Beyond performance monitoring, machine learning is also of interest in security: there is a lot of interest both in academia and industry in using machine learning to adapt and react to security threats, in recognition of the fact that an automated reaction is more timely than waiting for a human to respond. |  |
Finally, we remind the reader of our blog, and, for the reader interested in further details, our public deliverables and scientific publications.