Posted by: Giuliana Carullo, Senior Research Scientist, Intel Labs Europe
The mF2C project is focused on providing as much autonomy as possible at the edge of the network. As part of properly managing resources from cloud to edge in a seamless way, providing smart placing recommendations has a critical role. We took a step toward this ambitious fog-to-cloud vision by releasing the Analytics Engine into the open source community. It is released under Apache 2.0 and it is available for download here.
Do we need it? Why? Data is a fundamental element needed to optimize workload deployment, scan for deviations and flaws as well as grasping any possible SLA violation. But it is not enough, data per se would not make any difference without a framework that simplifies the way we look at it and how it is (machine) learned. And this is where Analytics Engine enters the picture.
The Analytics Engine is a framework designed to enable and support data analysis of services deployed in a cloud/edge environment. Specifically, developers and data scientists can use this framework to automate their data analysis tasks and gather insights needed to perform better orchestration choices. To this end, telemetry and infrastructure topology are used as enablers of different (and flexible) analytics modules, models and knowledge base construction.
Telemetry is gathered by using Snap: our open telemetry framework. It is designed to simplify the collection and processing of telemetry data by allowing systems to expose a consistent set of monitored metrics through a single API. Infrastructure topology is retrieved by our Landscaper, which generates a graph-based model of the entire infrastructure.
Built on top of our Landscaper and Snap, the analytics engine provides a much more comprehensive and powerful way of analyzing infrastructure capabilities and behaviour (telemetry) over time. But it does not end here. Smart reasoning can be easily embedded within the framework to match all of your needs when dealing with the infrastructure. How? Implement your algorithms within the framework, tell the engine which infrastructure to look at and it will be more than happy to do the remaining job for you.
The analytics engine’s main contributions of this project are:
- Provide a powerful, lightweight tool able to collect infrastructure capabilities
- Simplify the way analytics tasks are performed
- Allow scalable execution of diverse analytics tasks
- Technology agnostic, by means of aiming at supporting analytics tasks without the tight link with the technology used (e.g., OpenStack and Docker)
- Highly customizable, we give you the core, you turn it into beautiful analytics pipelines.
Based on it, within mF2C, our consortium looks forward to contributing insights and solutions to make smarter infrastructural decisions for a flawless edge to cloud service integration.