FedProx

class FedProx(*, fraction_fit: float = 1.0, fraction_evaluate: float = 1.0, min_fit_clients: int = 2, min_evaluate_clients: int = 2, min_available_clients: int = 2, evaluate_fn: Callable[[int, list[ndarray[Any, dtype[Any]]], dict[str, bool | bytes | float | int | str]], tuple[float, dict[str, bool | bytes | float | int | str]] | None] | None = None, on_fit_config_fn: Callable[[int], dict[str, bool | bytes | float | int | str]] | None = None, on_evaluate_config_fn: Callable[[int], dict[str, bool | bytes | float | int | str]] | None = None, accept_failures: bool = True, initial_parameters: Parameters | None = None, fit_metrics_aggregation_fn: Callable[[list[tuple[int, dict[str, bool | bytes | float | int | str]]]], dict[str, bool | bytes | float | int | str]] | None = None, evaluate_metrics_aggregation_fn: Callable[[list[tuple[int, dict[str, bool | bytes | float | int | str]]]], dict[str, bool | bytes | float | int | str]] | None = None, proximal_mu: float)[source]

Bases : FedAvg

Federated Optimization strategy.

Implementation based on https://arxiv.org/abs/1812.06127

The strategy in itself will not be different than FedAvg, the client needs to be adjusted. A proximal term needs to be added to the loss function during the training:

\[\begin{split}\\frac{\\mu}{2} || w - w^t ||^2\end{split}\]

Where $w^t$ are the global parameters and $w$ are the local weights the function will be optimized with.

In PyTorch, for example, the loss would go from:

loss = criterion(net(inputs), labels)

To:

for local_weights, global_weights in zip(net.parameters(), global_params):
    proximal_term += (local_weights - global_weights).norm(2)
loss = criterion(net(inputs), labels) + (config["proximal_mu"] / 2) *
proximal_term

With global_params being a copy of the parameters before the training takes place.

global_params = copy.deepcopy(net).parameters()
Paramètres:

  • fraction_fit (float, optional) – Fraction of clients used during training. In case min_fit_clients is larger than fraction_fit * available_clients, min_fit_clients will still be sampled. Defaults to 1.0.

  • fraction_evaluate (float, optional) – Fraction of clients used during validation. In case min_evaluate_clients is larger than fraction_evaluate * available_clients, min_evaluate_clients will still be sampled. Defaults to 1.0.

  • min_fit_clients (int, optional) – Minimum number of clients used during training. Defaults to 2.

  • min_evaluate_clients (int, optional) – Minimum number of clients used during validation. Defaults to 2.

  • min_available_clients (int, optional) – Minimum number of total clients in the system. Defaults to 2.

  • evaluate_fn (Optional[Callable[[int, NDArrays, Dict[str, Scalar]], Optional[Tuple[float, Dict[str, Scalar]]]]]) – Optional function used for validation. Defaults to None.

  • on_fit_config_fn (Callable[[int], Dict[str, Scalar]], optional) – Function used to configure training. Defaults to None.

  • on_evaluate_config_fn (Callable[[int], Dict[str, Scalar]], optional) – Function used to configure validation. Defaults to None.

  • accept_failures (bool, optional) – Whether or not accept rounds containing failures. Defaults to True.

  • initial_parameters (Parameters, optional) – Initial global model parameters.

  • fit_metrics_aggregation_fn (Optional[MetricsAggregationFn]) – Metrics aggregation function, optional.

  • evaluate_metrics_aggregation_fn (Optional[MetricsAggregationFn]) – Metrics aggregation function, optional.

  • proximal_mu (float) – The weight of the proximal term used in the optimization. 0.0 makes this strategy equivalent to FedAvg, and the higher the coefficient, the more regularization will be used (that is, the client parameters will need to be closer to the server parameters during training).

Methods

aggregate_evaluate(server_round, results, ...)

Aggregate evaluation losses using weighted average.

aggregate_fit(server_round, results, failures)

Aggregate fit results using weighted average.

configure_evaluate(server_round, parameters, ...)

Configure the next round of evaluation.

configure_fit(server_round, parameters, ...)

Configure the next round of training.

evaluate(server_round, parameters)

Evaluate model parameters using an evaluation function.

initialize_parameters(client_manager)

Initialize global model parameters.

num_evaluation_clients(num_available_clients)

Use a fraction of available clients for evaluation.

num_fit_clients(num_available_clients)

Return the sample size and the required number of available clients.
aggregate_evaluate(server_round: int, results: list[tuple[ClientProxy, EvaluateRes]], failures: list[tuple[ClientProxy, EvaluateRes] | BaseException]) → tuple[float | None, dict[str, bool | bytes | float | int | str]]

Aggregate evaluation losses using weighted average.

aggregate_fit(server_round: int, results: list[tuple[ClientProxy, FitRes]], failures: list[tuple[ClientProxy, FitRes] | BaseException]) → tuple[Parameters | None, dict[str, bool | bytes | float | int | str]]

Aggregate fit results using weighted average.

configure_evaluate(server_round: int, parameters: Parameters, client_manager: ClientManager) → list[tuple[ClientProxy, EvaluateIns]]

Configure the next round of evaluation.

configure_fit(server_round: int, parameters: Parameters, client_manager: ClientManager) → list[tuple[ClientProxy, FitIns]][source]

Configure the next round of training.

Sends the proximal factor mu to the clients

evaluate(server_round: int, parameters: Parameters) → tuple[float, dict[str, bool | bytes | float | int | str]] | None

Evaluate model parameters using an evaluation function.

initialize_parameters(client_manager: ClientManager) → Parameters | None

Initialize global model parameters.

num_evaluation_clients(num_available_clients: int) → tuple[int, int]

Use a fraction of available clients for evaluation.

num_fit_clients(num_available_clients: int) → tuple[int, int]

Return the sample size and the required number of available clients.