Quickstart with Docker Compose¶
This quickstart shows you how to set up Flower using Docker Compose in a single command, allowing you to focus on developing your application without worrying about the underlying infrastructure.
You will also learn how to easily enable TLS encryption and persist application state locally, giving you the freedom to choose the configuration that best suits your project’s needs.
전제 조건¶
Before you start, make sure that:
Step 1: Set Up¶
Clone the Docker Compose
complete
directory:$ git clone --depth=1 https://github.com/adap/flower.git _tmp \ && mv _tmp/src/docker/complete . \ && rm -rf _tmp && cd complete
Create a new Flower project (PyTorch):
$ flwr new quickstart-compose --framework PyTorch --username flower
Export the path of the newly created project. The path should be relative to the location of the Docker Compose files:
$ export PROJECT_DIR=quickstart-compose
Setting the
PROJECT_DIR
helps Docker Compose locate thepyproject.toml
file, allowing it to install dependencies in the SuperExec and SuperNode images correctly.
Step 2: Run Flower in Insecure Mode¶
To begin, start Flower with the most basic configuration. In this setup, Flower will run without TLS and without persisting the state.
참고
Without TLS, the data sent between the services remains unencrypted. Use it only for development purposes.
For production-oriented use cases, enable TLS for secure data transmission.
Open your terminal and run:
$ docker compose -f compose.yml up --build -d
Understand the command
docker compose
: The Docker command to run the Docker Compose tool.-f compose.yml
: Specify the YAML file that contains the basic Flower service definitions.--build
: Rebuild the images for each service if they don’t already exist.-d
: Detach the containers from the terminal and run them in the background.
Step 3: Run the Quickstart Project¶
Now that the Flower services have been started via Docker Compose, it is time to run the quickstart example.
To ensure the flwr
CLI connects to the SuperExec, you need to specify the SuperExec
addresses in the pyproject.toml
file.
Add the following lines to the
quickstart-compose/pyproject.toml
:[tool.flwr.federations.docker-compose] address = "127.0.0.1:9093" insecure = true
Execute the command to run the quickstart example:
$ flwr run quickstart-compose docker-compose
Monitor the SuperExec logs and wait for the summary to appear:
$ docker compose logs superexec -f
Step 4: Update the Application¶
In the next step, change the application code.
For example, go to the
task.py
file in thequickstart-compose/quickstart_compose/
directory and add aprint
call in theget_weights
function:# ... def get_weights(net): print("Get weights") return [val.cpu().numpy() for _, val in net.state_dict().items()] # ...
Rebuild and restart the services.
참고
If you have modified the dependencies listed in your
pyproject.toml
file, it is essential to rebuild images.If you haven’t made any changes, you can skip this step.
Run the following command to rebuild and restart the services:
$ docker compose -f compose.yml up --build -d
Run the updated quickstart example:
$ flwr run quickstart-compose docker-compose $ docker compose logs superexec -f
In the SuperExec logs, you should find the
Get weights
line:superexec-1 | INFO : Starting Flower SuperExec superexec-1 | WARNING : Option `--insecure` was set. Starting insecure HTTP server. superexec-1 | INFO : Starting Flower SuperExec gRPC server on 0.0.0.0:9093 superexec-1 | INFO : ExecServicer.StartRun superexec-1 | 🎊 Successfully installed quickstart-compose to /app/.flwr/apps/flower/quickstart-compose/1.0.0. superexec-1 | INFO : Created run -6767165609169293507 superexec-1 | INFO : Started run -6767165609169293507 superexec-1 | WARNING : Option `--insecure` was set. Starting insecure HTTP client connected to superlink:9091. superexec-1 | Get weights superexec-1 | INFO : Starting Flower ServerApp, config: num_rounds=3, no round_timeout
Step 5: Persisting the SuperLink State¶
In this step, Flower services are configured to persist the state of the SuperLink service, ensuring that it maintains its state even after a restart.
참고
When working with Docker Compose on Linux, you may need to create the state
directory first and change its ownership to ensure proper access and permissions.
For more information, consult the following page: Persist the State of the SuperLink.
Run the command:
$ docker compose -f compose.yml -f with-state.yml up --build -d
Understand the command
docker compose
: The Docker command to run the Docker Compose tool.-f compose.yml
: Specify the YAML file that contains the basic Flower service definitions.-f with-state.yml
: Specifies the path to an additional Docker Compose file thatcontains the configuration for persisting the SuperLink state.Docker merges Compose files according to merging rules.--build
: Rebuild the images for each service if they don’t already exist.-d
: Detach the containers from the terminal and run them in the background.
Rerun the
quickstart-compose
project:$ flwr run quickstart-compose docker-compose
Check the content of the
state
directory:$ ls state/ state.db
You should see a
state.db
file in thestate
directory. If you restart the service, the state file will be used to restore the state from the previously saved data. This ensures that the data persists even if the containers are stopped and started again.
Step 6: Run Flower with TLS¶
To demonstrate how to enable TLS, generate self-signed certificates using the
certs.yml
Compose file.중요
These certificates should be used only for development purposes.
For production environments, use a service like Let’s Encrypt to obtain your certificates.
Run the command:
$ docker compose -f certs.yml up --build
Add the following lines to the
quickstart-compose/pyproject.toml
:[tool.flwr.federations.docker-compose-tls] address = "127.0.0.1:9093" root-certificates = "../superexec-certificates/ca.crt"
Restart the services with TLS enabled:
$ docker compose -f compose.yml -f with-tls.yml up --build -d
Rerun the
quickstart-compose
project:$ flwr run quickstart-compose docker-compose-tls $ docker compose logs superexec -f
Step 7: Add another SuperNode¶
You can add more SuperNodes and ClientApps by duplicating their definitions in the
compose.yml
file.
Just give each new SuperNode and ClientApp service a unique service name like
supernode-3
, clientapp-3
, etc.
In compose.yml
, add the following:
# other service definitions
supernode-3:
image: flwr/supernode:${FLWR_VERSION:-1.11.1}
command:
- --insecure
- --superlink
- superlink:9092
- --supernode-address
- 0.0.0.0:9096
- --isolation
- process
- --node-config
- "partition-id=1 num-partitions=2"
depends_on:
- superlink
clientapp-3:
build:
context: ${PROJECT_DIR:-.}
dockerfile_inline: |
FROM flwr/clientapp:${FLWR_VERSION:-1.11.1}
USER root
RUN apt-get update \
&& apt-get -y --no-install-recommends install \
build-essential \
&& rm -rf /var/lib/apt/lists/*
USER app
WORKDIR /app
COPY --chown=app:app pyproject.toml .
RUN sed -i 's/.*flwr\[simulation\].*//' pyproject.toml \
&& python -m pip install -U --no-cache-dir .
ENTRYPOINT ["flwr-clientapp"]
command:
- --supernode
- supernode-3:9096
deploy:
resources:
limits:
cpus: "2"
stop_signal: SIGINT
depends_on:
- supernode-3
If you also want to enable TLS for the new SuperNodes, duplicate the SuperNode
definition for each new SuperNode service in the with-tls.yml
file.
Make sure that the names of the services match with the one in the compose.yml
file.
In with-tls.yml
, add the following:
# other service definitions
supernode-3:
command:
- --superlink
- superlink:9092
- --supernode-address
- 0.0.0.0:9096
- --isolation
- process
- --node-config
- "partition-id=1 num-partitions=2"
- --root-certificates
- certificates/ca.crt
secrets:
- source: superlink-ca-certfile
target: /app/certificates/ca.crt
Step 8: Persisting the SuperLink State and Enabling TLS¶
To run Flower with persisted SuperLink state and enabled TLS, a slight change in the
with-state.yml
file is required:
Comment out the lines 2-4 and uncomment the lines 5-9:
1 superlink: 2 # command: 3 # - --insecure 4 # - --database=state/state.db 5 command: 6 - --ssl-ca-certfile=certificates/ca.crt 7 - --ssl-certfile=certificates/server.pem 8 - --ssl-keyfile=certificates/server.key 9 - --database=state/state.db 10 volumes: 11 - ./state/:/app/state/:rw
Restart the services:
$ docker compose -f compose.yml -f with-tls.yml -f with-state.yml up --build -d
Rerun the
quickstart-compose
project:$ flwr run quickstart-compose docker-compose-tls $ docker compose logs superexec -f
Step 9: Merge Multiple Compose Files¶
You can merge multiple Compose files into a single file. For instance, if you wish to combine the basic configuration with the TLS configuration, execute the following command:
$ docker compose -f compose.yml \
-f with-tls.yml config --no-path-resolution > my_compose.yml
This will merge the contents of compose.yml
and with-tls.yml
into a new file
called my_compose.yml
.
Step 10: Clean Up¶
Remove all services and volumes:
$ docker compose down -v
$ docker compose -f certs.yml down -v