FedProx

class FedProx(*, fraction_fit: float = 1.0, fraction_evaluate: float = 1.0, min_fit_clients: int = 2, min_evaluate_clients: int = 2, min_available_clients: int = 2, evaluate_fn: Callable[[int, list[ndarray[tuple[Any, ...], dtype[Any]]], dict[str, bool | bytes | float | int | str]], tuple[float, dict[str, bool | bytes | float | int | str]] | None] | None = None, on_fit_config_fn: Callable[[int], dict[str, bool | bytes | float | int | str]] | None = None, on_evaluate_config_fn: Callable[[int], dict[str, bool | bytes | float | int | str]] | None = None, accept_failures: bool = True, initial_parameters: Parameters | None = None, fit_metrics_aggregation_fn: Callable[[list[tuple[int, dict[str, bool | bytes | float | int | str]]]], dict[str, bool | bytes | float | int | str]] | None = None, evaluate_metrics_aggregation_fn: Callable[[list[tuple[int, dict[str, bool | bytes | float | int | str]]]], dict[str, bool | bytes | float | int | str]] | None = None, proximal_mu: float)[source]

Bases : FedAvg

Federated Optimization strategy.

Implémentation basée sur https://arxiv.org/abs/1812.06127

The strategy in itself will not be different than FedAvg, the client needs to be adjusted. A proximal term needs to be added to the loss function during the training:

\[\begin{split}\\frac{\\mu}{2} || w - w^t ||^2\end{split}\]

Where $w^t$ are the global parameters and $w$ are the local weights the function will be optimized with.

Dans PyTorch, par exemple, la perte passerait de :

loss = criterion(net(inputs), labels)

En :

for local_weights, global_weights in zip(net.parameters(), global_params):
    proximal_term += (local_weights - global_weights).norm(2)
loss = criterion(net(inputs), labels) + (config["proximal_mu"] / 2) *
proximal_term

With global_params being a copy of the parameters before the training takes place.

global_params = copy.deepcopy(net).parameters()
Paramètres:

  • fraction_fit (float, optional) – Fraction of clients used during training. In case min_fit_clients is larger than fraction_fit * available_clients, min_fit_clients will still be sampled. Defaults to 1.0.

  • fraction_evaluate (float, optional) – Fraction of clients used during validation. In case min_evaluate_clients is larger than fraction_evaluate * available_clients, min_evaluate_clients will still be sampled. Defaults to 1.0.

  • min_fit_clients (int, optional) – Minimum number of clients used during training. Defaults to 2.

  • min_evaluate_clients (int, optional) – Minimum number of clients used during validation. Defaults to 2.

  • min_available_clients (int, optional) – Minimum number of total clients in the system. Defaults to 2.

  • evaluate_fn (Optional[Callable[[int, NDArrays, Dict[str, Scalar]], Optional[Tuple[float, Dict[str, Scalar]]]]]) – Optional function used for validation. Defaults to None.

  • on_fit_config_fn (Callable[[int], Dict[str, Scalar]], optional) – Function used to configure training. Defaults to None.

  • on_evaluate_config_fn (Callable[[int], Dict[str, Scalar]], optional) – Function used to configure validation. Defaults to None.

  • accept_failures (bool, optional) – Whether or not accept rounds containing failures. Defaults to True.

  • initial_parameters (Parameters, optional) – Initial global model parameters.

  • fit_metrics_aggregation_fn (Optional[MetricsAggregationFn]) – Metrics aggregation function, optional.

  • evaluate_metrics_aggregation_fn (Optional[MetricsAggregationFn]) – Metrics aggregation function, optional.

  • proximal_mu (float) – La pondération du terme proximal utilisé dans l’optimisation. 0,0 fait que cette stratégie est équivalente à FedAvg, et plus la coefficient est élevée, plus la régularisation sera utilisée (c’est-à-dire que les paramètres des clients devront être plus proches des paramètres du serveur pendant l’entraînement).

Methods

aggregate_evaluate(server_round, results, ...)

Aggregate evaluation losses using weighted average.

aggregate_fit(server_round, results, failures)

Aggregate fit results using weighted average.

configure_evaluate(server_round, parameters, ...)

Configure the next round of evaluation.

configure_fit(server_round, parameters, ...)

Configure le prochain tour d'entraînement.

evaluate(server_round, parameters)

Evaluate model parameters using an evaluation function.

initialize_parameters(client_manager)

Initialize global model parameters.

num_evaluation_clients(num_available_clients)

Use a fraction of available clients for evaluation.

num_fit_clients(num_available_clients)

Return the sample size and the required number of available clients.
aggregate_evaluate(server_round: int, results: list[tuple[ClientProxy, EvaluateRes]], failures: list[tuple[ClientProxy, EvaluateRes] | BaseException]) → tuple[float | None, dict[str, bool | bytes | float | int | str]]

Aggregate evaluation losses using weighted average.

aggregate_fit(server_round: int, results: list[tuple[ClientProxy, FitRes]], failures: list[tuple[ClientProxy, FitRes] | BaseException]) → tuple[Parameters | None, dict[str, bool | bytes | float | int | str]]

Aggregate fit results using weighted average.

configure_evaluate(server_round: int, parameters: Parameters, client_manager: ClientManager) → list[tuple[ClientProxy, EvaluateIns]]

Configure the next round of evaluation.

configure_fit(server_round: int, parameters: Parameters, client_manager: ClientManager) → list[tuple[ClientProxy, FitIns]][source]

Configure le prochain tour d’entraînement.

Sends the proximal factor mu to the clients

evaluate(server_round: int, parameters: Parameters) → tuple[float, dict[str, bool | bytes | float | int | str]] | None

Evaluate model parameters using an evaluation function.

initialize_parameters(client_manager: ClientManager) → Parameters | None

Initialize global model parameters.

num_evaluation_clients(num_available_clients: int) → tuple[int, int]

Use a fraction of available clients for evaluation.

num_fit_clients(num_available_clients: int) → tuple[int, int]

Return the sample size and the required number of available clients.