Federated Learning

What is Federated Learning?

Federated learning, also known as collaborative learning, is a novel approach to machine learning (ML) that leverages the power of decentralized data sources, allowing models to be trained collaboratively across devices or nodes while keeping data localized, thus enabling privacy-preserving and efficient model development.

Under a federated learning approach, each connected device will use the artificial intelligence (AI) model to process data stored locally, which it uses to update the model’s parameters before sending the results back to the central server. The main model then aggregates the results alongside the output forwarded by other devices in the network.

Processing data in this fashion means that AI models don’t need to be trained on a single dataset located on a single server, data warehouse, or data lake.

Google first popularized federated learning in 2016 as an alternative approach to machine learning with the release of Communication-Efficient Learning of Deep Networks from Decentralized Data, a research paper authored by a team of Google research scientists.

How Does Federated Learning Work?

In its first research paper on the topic, Google explained that with federated learning, “each client has a local training dataset which is never uploaded to the server. Instead, each client computes an update to the current global model maintained by the server, and only this update is communicated.”

A basic outline of the interaction between the central server and downstream devices is broken down below:

• An organization deploys a pre-trained or untrained model to a central server;
• They then distribute the global AI model to downstream clients, devices, or servers;
• The clients proceed to train the model on data stored locally without sending it back to the cloud;
• The client sends the updated model parameters back to the central server (this process can be encrypted for extra security);
• The global AI model aggregates the parameters forwarded by the clients and updates its decision-making process;
• The server sends the updated model back to all downstream devices and servers.

Conducting ML in this manner means that AI models can be trained continuously on a decentralized dataset based on data generated in real time by end-user devices, even if they aren’t currently connected to the Internet.

It also means that organizations can combine the computational power of distributed devices to accelerate the speed and performance of model training.

Why is Federated Learning Important?

Federated learning is an important innovation in machine learning for a number of reasons.

One of the main reasons is that it enables organizations to move AI model training to the network’s edge. Training a centralized AI model on decentralized data stored across multiple devices means that insights can be extracted from edge devices, such as servers, smartphones, IoT devices, and wearables.

At the same time, federated learning is valuable because its lack of centralized data processing on a cloud server helps minimize the amount of personal data transferred to and processed by third parties. The lack of centralized data storage maintains privacy while giving users more control over how their data is used and processed.

In this sense, organizations can use federated learning to reduce the chance of non-compliance with data protection regulations, such as the EU’s General Data Protection Regulation (GDPR) or the Health Insurance Portability and Accountability Act (HIPAA).

It is also very useful for organizations looking to conduct machine learning in heavily-regulated sectors, such as the finance or healthcare sectors, which need to be extremely cautious about processing personally identifiable information (PII), patient health information, payment details, or other regulated data.

Centralized vs. Decentralised Federated Learning

There are two main types of federated learning; centralized and decentralized. Centralized federated learning (the most common type of federated learning outlined above) is where devices at the network’s edge use a centralized model to process data stored locally, sending updates to a centralized server.

In contrast, instead of using a central server to aggregate an AI model, decentralized federated learning uses a network of connected devices to collectively aggregate parameters. Essentially, each device downloads a predeveloped AI model and uses it to process local data before sending the results to other devices for aggregation.

What are the Benefits of Federated Learning?

There are a number of core benefits offered by federated learning to modern organizations. These include:

• Organizations can build centralized AI models while meeting data privacy compliance regulations;
• Generate insights from devices at the network’s edge;
• Scale to collect data from millions of devices;
• The lack of a need to connect to a central server leads to faster model training;
• Data processing can take place locally on devices without an internet connection;
• Lower the risk of a central server being targeted or compromised.

The Bottom Line

Federated learning has become a critical approach to AI development for organizations that want to collect insights from the network’s edge while protecting themselves from regulatory risk.

If deployed correctly, federated learning can help organizations have the confidence to extract insights from their data without risking violating local or international data protection regulations.