Flower (flwr): A Complete Guide to Federated Learning in Python
Federated learning is a machine learning paradigm that allows multiple participants to collaboratively train a model without sharing their raw data. Instead of centralizing data on a single server, each participant trains a local model on their own data and only shares model updates (such as gradients or weights) with a central server. This approach preserves data privacy, reduces bandwidth requirements, and enables learning from distributed datasets that cannot be combined due to regulatory or practical constraints.
Flower, also known by its package name flwr, is an open-source framework that makes federated learning accessible and practical. It provides a clean, flexible API for building federated learning systems in Python, supporting popular ML frameworks like PyTorch, TensorFlow, and JAX. Whether you are running a quick simulation on your laptop or deploying a production system across thousands of devices, Flower gives you the tools to do it.
In this tutorial, I will walk you through Flower from the ground up. We will start with installation and core concepts, build a working federated learning system with PyTorch and CIFAR-10, explore advanced features like custom strategies and secure aggregation, and discuss best practices for real-world deployments. By the end, you should be comfortable using Flower for your own federated learning projects.
Installation
Flower requires Python 3.8 or later. The recommended way to install it is via pip. For a basic setup, you only need the core Flower package. If you plan to run simulations (which we will do in this tutorial), install the simulation extras as well.
pip install flwr
For simulation support:
pip install "flwr[simulation]"
Since we will use PyTorch for our model, install it along with torchvision:
pip install torch torchvision
You can verify that everything is installed correctly by running:
import flwr as fl
print(fl.version)
import torch
print(torch.version)
If both imports succeed and print version numbers, you are ready to proceed.
Federated Learning Fundamentals
Before writing code, let us understand the key concepts behind federated learning and how Flower implements them.
The Federated Learning Process
A typical federated learning round consists of these steps:
Flower Architecture
Flower organizes federated learning around three main components:
Server: Orchestrates the federated learning process. It manages client connections, distributes model parameters, collects updates, and runs the aggregation strategy. In Flower, you start a server withfl.server.startserver() or use the simulation engine.
Client: Represents a participant in the federated learning process. Each client holds its own local dataset and performs training. In Flower, you implement a client by subclassing fl.client.NumPyClient, which provides a simple interface based on NumPy arrays.
Strategy: Defines how the server aggregates client updates. Flower ships with several built-in strategies, including FedAvg (Federated Averaging), FedProx, FedAdam, and others. You can also implement your own custom strategy.
FedAvg: The Baseline Strategy
Federated Averaging (FedAvg), proposed by McMahan et al. in 2017, is the most widely used federated learning algorithm. It works as follows:
This simple approach works surprisingly well in practice and serves as the baseline for most federated learning research.
Basic Usage: Federated CIFAR-10 with PyTorch
Let us build a complete federated learning system that trains an image classifier on the CIFAR-10 dataset distributed across multiple clients.
Step 1: Define the Model
We will use a simple convolutional neural network suitable for CIFAR-10 classification.
# model.py
import torch
import torch.nn as nn
import torch.nn.functional as F
class CifarNet(nn.Module):
def init(self):
super(CifarNet, self).init()
self.conv1 = nn.Conv2d(3, 32, 3, padding=1)
self.conv2 = nn.Conv2d(32, 64, 3, padding=1)
self.pool = nn.MaxPool2d(2, 2)
self.fc1 = nn.Linear(64 8 8, 256)
self.fc2 = nn.Linear(256, 10)
self.dropout = nn.Dropout(0.25)
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = x.view(-1, 64 8 8)
x = self.dropout(F.relu(self.fc1(x)))
x = self.fc2(x)
return x
Step 2: Prepare the Data
We need a function that loads CIFAR-10 and partitions it across clients. In a real federated scenario, each client would have its own data. For simulation, we split the dataset into non-overlapping shards.
# data.py
import torch
from torch.utils.data import DataLoader, Subset
from torchvision import datasets, transforms
def loaddatasets(numclients: int, batchsize: int = 32):
"""Load CIFAR-10 and partition it across clients."""
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.4914, 0.4822, 0.4465),
(0.2470, 0.2435, 0.2616)),
])
trainset = datasets.CIFAR10(
root="./data", train=True, download=True, transform=transform
)
testset = datasets.CIFAR10(
root="./data", train=False, download=True, transform=transform
)
# Partition training data into equal shards for each client
totalsize = len(trainset)
shardsize = totalsize // numclients
indices = list(range(totalsize))
clientloaders = []
for i in range(numclients):
start = i shardsize
end = start + shardsize
subset = Subset(trainset, indices[start:end])
loader = DataLoader(subset, batchsize=batchsize, shuffle=True)
clientloaders.append(loader)
testloader = DataLoader(testset, batchsize=batchsize, shuffle=False)
return clientloaders, testloader
Step 3: Implement the Flower Client
The Flower client wraps our PyTorch model and data, implementing the NumPyClient interface. This interface uses NumPy arrays for parameter exchange, which Flower handles across the network.
# client.py
import flwr as fl
import torch
import numpy as np
from collections import OrderedDict
from model import CifarNet
DEVICE = torch.device("cuda" if torch.cuda.isavailable() else "cpu")
class CifarClient(fl.client.NumPyClient):
def init(self, trainloader, testloader):
self.model = CifarNet().to(DEVICE)
self.trainloader = trainloader
self.testloader = testloader
def getparameters(self, config):
"""Return model parameters as a list of NumPy arrays."""
return [val.cpu().numpy() for , val in self.model.statedict().items()]
def setparameters(self, parameters):
"""Set model parameters from a list of NumPy arrays."""
paramsdict = zip(self.model.statedict().keys(), parameters)
statedict = OrderedDict(
{k: torch.tensor(v) for k, v in paramsdict}
)
self.model.loadstatedict(statedict, strict=True)
def fit(self, parameters, config):
"""Train the model on local data."""
self.setparameters(parameters)
optimizer = torch.optim.Adam(self.model.parameters(), lr=1e-3)
criterion = torch.nn.CrossEntropyLoss()
self.model.train()
for epoch in range(1): # 1 local epoch per round
for images, labels in self.trainloader:
images, labels = images.to(DEVICE), labels.to(DEVICE)
optimizer.zerograd()
outputs = self.model(images)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
return self.getparameters(config={}), len(self.trainloader.dataset), {}
def evaluate(self, parameters, config):
"""Evaluate the model on local test data."""
self.setparameters(parameters)
criterion = torch.nn.CrossEntropyLoss()
correct, total, totalloss = 0, 0, 0.0
self.model.eval()
with torch.nograd():
for images, labels in self.testloader:
images, labels = images.to(DEVICE), labels.to(DEVICE)
outputs = self.model(images)
totalloss += criterion(outputs, labels).item()
, predicted = torch.max(outputs, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
accuracy = correct / total
avgloss = totalloss / len(self.testloader)
return avgloss, total, {"accuracy": accuracy}
The three key methods are:
getparameters: Returns the current model weights as NumPy arrays.fit: Receives global parameters, trains locally, and returns updated parameters along with the number of training examples.evaluate: Receives global parameters and evaluates the model, returning loss, number of test examples, and accuracy metrics.
Step 4: Set Up the Server
The server configuration specifies the strategy and the number of federated learning rounds.
# server.py
import flwr as fl
def startserver():
strategy = fl.server.strategy.FedAvg(
fractionfit=1.0, # Select all available clients for training
fractionevaluate=1.0, # Select all clients for evaluation
minfitclients=2, # Minimum 2 clients for training
minevaluateclients=2, # Minimum 2 clients for evaluation
minavailableclients=2, # Wait for at least 2 clients to connect
)
fl.server.startserver(
serveraddress="0.0.0.0:8080",
config=fl.server.ServerConfig(numrounds=10),
strategy=strategy,
)
if name == "main":
startserver()
Step 5: Run with Simulation
For development and experimentation, Flower provides a simulation engine that runs the server and all clients in a single process. This is much more convenient than starting separate processes.
# simulation.py
import flwr as fl
from model import CifarNet
from data import loaddatasets
from client import CifarClient
NUMCLIENTS = 5
def clientfn(cid: str):
"""Create a Flower client for simulation."""
trainloaders, testloader = loaddatasets(NUMCLIENTS)
clientid = int(cid)
return CifarClient(trainloaders[clientid], testloader).toclient()
if name == "main":
# Preload data to share across clients
trainloaders, testloader = loaddatasets(NUMCLIENTS)
def makeclient(cid: str):
clientid = int(cid)
return CifarClient(trainloaders[clientid], testloader).toclient()
strategy = fl.server.strategy.FedAvg(
fractionfit=0.5, # Use 50% of clients per round
fractionevaluate=0.5,
minfitclients=2,
minevaluateclients=2,
minavailableclients=NUMCLIENTS,
)
fl.simulation.startsimulation(
clientfn=makeclient,
numclients=NUMCLIENTS,
config=fl.server.ServerConfig(numrounds=10),
strategy=strategy,
)
Run the simulation with:
python simulation.py
You will see Flower print logs for each round, showing how many clients were selected, the aggregated loss, and evaluation metrics. After 10 rounds, you should see the accuracy climbing as the federated model improves.
Step 6: Running with Separate Processes
For a more realistic setup, you can run the server and clients as separate processes. First, start the server:
python server.py
Then, in separate terminals, start multiple clients:
# runclient.py
import sys
import flwr as fl
from model import CifarNet
from data import loaddatasets
from client import CifarClient
NUMCLIENTS = 5
CLIENTID = int(sys.argv[1])
trainloaders, testloader = loaddatasets(NUMCLIENTS)
client = CifarClient(trainloaders[CLIENTID], testloader)
fl.client.startclient(
serveraddress="127.0.0.1:8080",
client=client.toclient(),
)
python runclient.py 0
python runclient.py 1
... and so on
Advanced Usage
Custom Strategies
While FedAvg works well for many scenarios, you may need a custom aggregation strategy. Flower makes this straightforward by letting you subclass the Strategy base class or extend an existing strategy.
Here is an example of a custom strategy that adds server-side momentum to the aggregation step, which can improve convergence:
# customstrategy.py
import flwr as fl
from flwr.server.strategy import FedAvg
from flwr.common import (
Parameters,
Scalar,
FitRes,
ndarraystoparameters,
parameterstondarrays,
)
from typing import Dict, List, Optional, Tuple, Union
import numpy as np
class FedAvgWithMomentum(FedAvg):
"""FedAvg with server-side momentum for faster convergence."""
def init(self, servermomentum: float = 0.9, kwargs):
super().init(kwargs)
self.servermomentum = servermomentum
self.velocity = None
def aggregatefit(
self,
serverround: int,
results: List[Tuple[fl.server.clientproxy.ClientProxy, FitRes]],
failures: List[Union[Tuple[fl.server.clientproxy.ClientProxy, FitRes], BaseException]],
) -> Tuple[Optional[Parameters], Dict[str, Scalar]]:
"""Aggregate fit results with momentum."""
# Call parent FedAvg aggregation
aggregatedparameters, metrics = super().aggregatefit(
serverround, results, failures
)
if aggregatedparameters is None:
return None, {}
# Get the aggregated weights as numpy arrays
aggregatedndarrays = parameterstondarrays(aggregatedparameters)
# Initialize velocity on first round
if self.velocity is None:
self.velocity = [np.zeroslike(arr) for arr in aggregatedndarrays]
# Apply momentum: velocity = momentum velocity + (1 - momentum) update
# Here the "update" is the aggregated parameters themselves
updatedndarrays = []
for i, (agg, vel) in enumerate(zip(aggregatedndarrays, self.velocity)):
newvel = self.servermomentum vel + (1 - self.servermomentum) * agg
self.velocity[i] = newvel
updatedndarrays.append(newvel)
return ndarraystoparameters(updatedndarrays), metrics
Use it by replacing the strategy in your server or simulation:
strategy = FedAvgWithMomentum(
servermomentum=0.9,
fractionfit=0.5,
minfitclients=2,
minavailableclients=NUMCLIENTS,
)
Configuring Per-Round Client Behavior
You can pass configuration to clients on a per-round basis. This is useful for adjusting the learning rate, number of local epochs, or batch size as training progresses.
from flwr.common import Scalar
from typing import Dict
def fitconfig(serverround: int) -> Dict[str, Scalar]:
"""Return training configuration for each round."""
config = {
"localepochs": 1 if serverround < 5 else 2,
"learningrate": 1e-3 if serverround < 5 else 5e-4,
"batchsize": 32,
}
return config
strategy = fl.server.strategy.FedAvg(
fractionfit=0.5,
minfitclients=2,
minavailableclients=NUMCLIENTS,
onfitconfigfn=fitconfig,
)
Then, in your client's fit method, read the config:
def fit(self, parameters, config):
self.setparameters(parameters)
localepochs = int(config.get("localepochs", 1))
lr = float(config.get("learningrate", 1e-3))
batchsize = int(config.get("batchsize", 32))
optimizer = torch.optim.Adam(self.model.parameters(), lr=lr)
criterion = torch.nn.CrossEntropyLoss()
self.model.train()
for epoch in range(localepochs):
for images, labels in self.trainloader:
images, labels = images.to(DEVICE), labels.to(DEVICE)
optimizer.zerograd()
outputs = self.model(images)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
return self.getparameters(config={}), len(self.trainloader.dataset), {}
Secure Aggregation
In scenarios where even model updates could leak information about client data, secure aggregation ensures that the server can only see the aggregated result, not individual client updates. Flower supports secure aggregation through its SecAgg and SecAgg+ protocols.
To enable secure aggregation, you configure it on both the server and client side:
# Server-side: enable SecAgg+ in the strategy
from flwr.server.strategy import FedAvg
from flwr.server.workflow import SecAggPlusWorkflow
strategy = FedAvg(
fractionfit=1.0,
minfitclients=3,
minavailableclients=3,
)
When using the new Flower Next API:
SecAgg+ requires at least 3 clients
Configure with appropriate number of shares and reconstruction threshold
For practical secure aggregation, you can also implement a lightweight version using differential privacy. This adds calibrated noise to client updates before sending them to the server:
# clientwithdp.py
import numpy as np
class DPCifarClient(CifarClient):
"""Client with local differential privacy."""
def init(self, trainloader, testloader, noisescale=0.01):
super().init(trainloader, testloader)
self.noisescale = noisescale
def fit(self, parameters, config):
# Train normally
updatedparams, numexamples, metrics = super().fit(parameters, config)
# Add Gaussian noise to parameters before sending
noisyparams = []
for param in updatedparams:
noise = np.random.normal(0, self.noisescale, param.shape)
noisyparams.append(param + noise.astype(param.dtype))
return noisyparams, numexamples, metrics
The trade-off with differential privacy is between privacy and model accuracy. A larger noise scale provides stronger privacy guarantees but may slow down convergence. Tuning this parameter is important for your specific use case.
Using FedProx for Heterogeneous Data
When clients have non-IID (non-identically distributed) data, FedAvg can struggle. FedProx adds a proximal term to the local training objective that penalizes large deviations from the global model, which helps stabilize training.
# Using built-in FedProx strategy
strategy = fl.server.strategy.FedProx(
fractionfit=0.5,
minfitclients=2,
minavailableclients=NUMCLIENTS,
proximalmu=0.1, # Proximal term coefficient
)
The proximalmu parameter controls how strongly clients are pulled toward the global model. A value of 0.0 reduces FedProx to FedAvg, while larger values enforce stronger consistency at the cost of slower local adaptation.
Saving and Loading Checkpoints
For long-running federated training, you want to save checkpoints so you can resume if something fails. You can implement this in a custom strategy:
import pickle
from pathlib import Path
class FedAvgWithCheckpoints(FedAvg):
def init(self, checkpointdir: str = "./checkpoints", kwargs):
super().init(kwargs)
self.checkpointdir = Path(checkpointdir)
self.checkpointdir.mkdir(existok=True)
def aggregatefit(self, serverround, results, failures):
aggregatedparameters, metrics = super().aggregatefit(
serverround, results, failures
)
if aggregatedparameters is not None:
# Save checkpoint
ndarrays = parameterstondarrays(aggregatedparameters)
checkpointpath = self.checkpointdir / f"round{serverround}.pkl"
with open(checkpointpath, "wb") as f:
pickle.dump(ndarrays, f)
print(f"Saved checkpoint: {checkpointpath}")
return aggregatedparameters, metrics
Deployment Considerations
Running in Production
When deploying Flower in production, you should consider several factors beyond what a simulation provides.
TLS/SSL encryption: Always enable TLS for communication between the server and clients in production environments. Flower supports this through gRPC's built-in TLS.# Server with TLS
fl.server.startserver(
serveraddress="0.0.0.0:8080",
config=fl.server.ServerConfig(numrounds=10),
strategy=strategy,
certificates=(
Path("ca.crt").readbytes(),
Path("server.crt").readbytes(),
Path("server.key").readbytes(),
),
)
Client with TLS
fl.client.startclient(
serveraddress="your-server:8080",
client=client.toclient(),
rootcertificates=Path("ca.crt").readbytes(),
)
Client authentication: In a production system, you need to verify that only authorized clients can participate. Use mutual TLS (mTLS) where both server and clients present certificates.
Resource management: Flower's simulation engine can manage GPU resources across simulated clients. Use the clientresources parameter to control allocation:
fl.simulation.startsimulation(
client
fn=makeclient,
num
clients=NUMCLIENTS,
config=fl.server.ServerConfig(num
rounds=10),
strategy=strategy,
clientresources={"numcpus": 2, "numgpus": 0.25}, # Each client gets 1/4 GPU
)
Flower Next (SuperNode/SuperLink Architecture)
Recent versions of Flower introduce the SuperNode and SuperLink architecture, designed for production deployments. The SuperLink acts as the central coordinator (replacing the traditional server), while SuperNodes are long-running processes on client machines that manage client applications.
# Start the SuperLink (server side)
flower-superlink --insecure
Start SuperNodes (client side)
flower-supernode --insecure --superlink="127.0.0.1:9092"
This architecture supports dynamic client participation, where clients can join and leave without disrupting the overall training process.
Best Practices
Start with simulation: Always prototype and debug your federated learning setup using Flower's simulation mode. It is much faster to iterate when everything runs in a single process. Only move to distributed execution once your logic is validated. Monitor convergence carefully: Federated learning converges differently from centralized training. Log per-round metrics (loss, accuracy, number of participating clients) and plot them over time. Watch for signs of divergence, especially with non-IID data. Handle data heterogeneity: Real-world federated datasets are almost never IID. Test your system with skewed data partitions during development. Consider strategies like FedProx or use data augmentation on clients to mitigate the effects of non-IID data. Minimize communication overhead: Communication is often the bottleneck in federated learning. Reduce the number of parameters transmitted by using model compression, quantization, or sending only the gradient updates (delta) rather than full model weights. Set appropriate local training epochs: More local epochs mean fewer communication rounds, but too many local epochs can cause client drift where individual models diverge significantly from the global model. One to five local epochs per round is a common range. Use learning rate scheduling: Decay the server-side or client-side learning rate over rounds. This helps stabilize training as the model approaches convergence. Theonfitconfigfn callback shown earlier is the right place to implement this.
Implement fault tolerance: In real deployments, clients may drop out at any time. Configure your strategy with appropriate minimum client thresholds and ensure the server can proceed even if some clients fail to respond.
Version your experiments: Track your strategy configuration, number of clients, data partitioning scheme, and model architecture for each experiment. Federated learning has many more hyperparameters than centralized training, and reproducibility requires careful record-keeping.
Test with realistic conditions: Before deploying, simulate realistic network conditions including latency, bandwidth limitations, and client dropouts. Flower's simulation mode does not model network conditions by default, so you may need to add artificial delays in your client code during testing.
Keep security in mind from the start: Even model updates can leak information about training data through model inversion or membership inference attacks. Consider differential privacy or secure aggregation from the beginning of your project, not as an afterthought.
Conclusion
Flower provides a well-designed and practical framework for federated learning in Python. Its clean API, built-in strategies, simulation mode, and support for production deployment make it a strong choice for both research and real-world applications. In this tutorial, we covered the fundamentals of federated learning, built a complete system for training a CIFAR-10 classifier across multiple federated clients, explored advanced features like custom strategies and secure aggregation, and discussed deployment and best practices.
The federated learning landscape is evolving rapidly, and Flower is keeping pace with features like the SuperNode/SuperLink architecture, improved simulation capabilities, and integration with modern ML frameworks. Whether you are building a privacy-preserving healthcare application, a collaborative recommendation system, or a distributed sensor analytics pipeline, Flower gives you a solid foundation to build on.
To learn more, visit the Flower documentation at flower.ai and explore the extensive collection of example projects in the Flower GitHub repository. The community is active and welcoming, making it a great resource for getting help and contributing back to the ecosystem.