Quickstart with Docker

This quickstart aims to guide you through the process of containerizing a Flower project and running it end to end using Docker on your local machine.

This tutorial does not use production-ready settings, so you can focus on understanding the basic workflow that uses the minimum configurations.

전제 조건

Before you start, make sure that:

• The flwr CLI is installed locally.

• The Docker daemon is running.

Step 1: Set Up

1. Create a new Flower project (PyTorch):

   $ flwr new quickstart-docker --framework PyTorch --username flower
   
   🔨 Creating Flower project quickstart-docker...
   🎊 Project creation successful.
   
   Use the following command to run your project:
   
         cd quickstart-docker
         pip install -e .
         flwr run
   
   $ cd quickstart-docker
   

2. Create a new Docker bridge network called flwr-network:

   $ docker network create --driver bridge flwr-network
   

   User-defined networks, such as flwr-network, enable IP resolution of container names, a feature absent in the default bridge network. This simplifies quickstart example by avoiding the need to determine host IP first.

Step 2: Start the SuperLink

Open your terminal and run:

$ docker run --rm \
      -p 9091:9091 -p 9092:9092 -p 9093:9093 \
      --network flwr-network \
      --name superlink \
      --detach \
      flwr/superlink:1.14.0 \
      --insecure \
      --isolation \
      process

Understand the command

• docker run: This tells Docker to run a container from an image.

• --rm: Remove the container once it is stopped or the command exits.

• -p 9091:9091 -p 9092:9092 -p 9093:9093: Map port 9091, 9092 and 9093 of the

  container to the same port of the host machine, allowing other services to access the

  ServerAppIO API on http://localhost:9091, the Fleet API on http://localhost:9092 and

  the Exec API on http://localhost:9093.

• --network flwr-network: Make the container join the network named flwr-network.

• --name superlink: Assign the name superlink to the container.

• --detach: Run the container in the background, freeing up the terminal.

• flwr/superlink:1.14.0: The name of the image to be run and the specific

  tag of the image. The tag 1.14.0 represents a specific version of the image.
• --insecure: This flag tells the container to operate in an insecure mode, allowing

  unencrypted communication.
• --isolation process: Tells the SuperLink that the ServerApp is created by separate

  independent process. The SuperLink does not attempt to create it. You can learn more about

  the different process modes here: Run ServerApp or ClientApp as a Subprocess.

Step 3: Start the SuperNodes

Start two SuperNode containers.

1. Start the first container:

   $ docker run --rm \
       -p 9094:9094 \
       --network flwr-network \
       --name supernode-1 \
       --detach \
       flwr/supernode:1.14.0  \
       --insecure \
       --superlink superlink:9092 \
       --node-config "partition-id=0 num-partitions=2" \
       --clientappio-api-address 0.0.0.0:9094 \
       --isolation process
   

   Understand the command

   • docker run: This tells Docker to run a container from an image.

   • --rm: Remove the container once it is stopped or the command exits.

   • -p 9094:9094: Map port 9094 of the container to the same port of

     the host machine, allowing other services to access the SuperNode API on

     http://localhost:9094.

   • --network flwr-network: Make the container join the network named flwr-network.

   • --name supernode-1: Assign the name supernode-1 to the container.

   • --detach: Run the container in the background, freeing up the terminal.

   • flwr/supernode:1.14.0: This is the name of the

     image to be run and the specific tag of the image.
   • --insecure: This flag tells the container to operate in an insecure mode, allowing

     unencrypted communication.
   • --superlink superlink:9092: Connect to the SuperLink’s Fleet API at the address

     superlink:9092.
   • --node-config "partition-id=0 num-partitions=2": Set the partition ID to 0 and the

     number of partitions to 2 for the SuperNode configuration.
   • --clientappio-api-address 0.0.0.0:9094: Set the address and port number that the

     SuperNode is listening on to communicate with the ClientApp. If

     two SuperNodes are started on the same machine, set two different port numbers for each SuperNode.

     (E.g. In the next step, we set the second SuperNode container to listen on port 9095)
   • --isolation process: Tells the SuperNode that the ClientApp is created by separate

     independent process. The SuperNode does not attempt to create it.

2. Start the second container:

   $ docker run --rm \
       -p 9095:9095 \
       --network flwr-network \
       --name supernode-2 \
       --detach \
       flwr/supernode:1.14.0  \
       --insecure \
       --superlink superlink:9092 \
       --node-config "partition-id=1 num-partitions=2" \
       --clientappio-api-address 0.0.0.0:9095 \
       --isolation process
   

Step 4: Start a ServerApp

The ServerApp Docker image comes with a pre-installed version of Flower and serves as a base for building your own ServerApp image. In order to install the FAB dependencies, you will need to create a Dockerfile that extends the ServerApp image and installs the required dependencies.

1. Create a ServerApp Dockerfile called serverapp.Dockerfile and paste the following code in:

   serverapp.Dockerfile

   FROM flwr/serverapp:1.14.0
   
   WORKDIR /app
   
   COPY pyproject.toml .
   RUN sed -i 's/.*flwr\[simulation\].*//' pyproject.toml \
      && python -m pip install -U --no-cache-dir .
   
   ENTRYPOINT ["flwr-serverapp"]
   

   Understand the Dockerfile

   • FROM flwr/serverapp:1.14.0: This line specifies that the Docker image

     to be built from is the flwr/serverapp image, version 1.14.0.
   • WORKDIR /app: Set the working directory for the container to /app.

     Any subsequent commands that reference a directory will be relative to this directory.
   • COPY pyproject.toml .: Copy the pyproject.toml file

     from the current working directory into the container’s /app directory.
   • RUN sed -i 's/.*flwr\[simulation\].*//' pyproject.toml: Remove the flwr dependency

     from the pyproject.toml.
   • python -m pip install -U --no-cache-dir .: Run the pip install command to

     install the dependencies defined in the pyproject.toml file

     
     

     The -U flag indicates that any existing packages should be upgraded, and

     --no-cache-dir prevents pip from using the cache to speed up the installation.
   • ENTRYPOINT ["flwr-serverapp"]: Set the command flwr-serverapp to be

     the default command run when the container is started.

   중요

   Note that flwr is already installed in the flwr/clientapp base image, so only other package dependencies such as flwr-datasets, torch, etc., need to be installed. As a result, the flwr dependency is removed from the pyproject.toml after it has been copied into the Docker image (see line 5).

2. Afterward, in the directory that holds the Dockerfile, execute this Docker command to build the ServerApp image:

   $ docker build -f serverapp.Dockerfile -t flwr_serverapp:0.0.1 .
   

3. Start the ServerApp container:

   $ docker run --rm \
       --network flwr-network \
       --name serverapp \
       --detach \
       flwr_serverapp:0.0.1 \
       --insecure \
       --serverappio-api-address superlink:9091
   

   Understand the command

   • docker run: This tells Docker to run a container from an image.

   • --rm: Remove the container once it is stopped or the command exits.

   • --network flwr-network: Make the container join the network named flwr-network.

   • --name serverapp: Assign the name serverapp to the container.

   • --detach: Run the container in the background, freeing up the terminal.

   • flwr_serverapp:0.0.1: This is the name of the image to be run and the specific tag

     of the image.
   • --insecure: This flag tells the container to operate in an insecure mode, allowing

     unencrypted communication. Secure connections will be added in future releases.
   • --serverappio-api-address superlink:9091: Connect to the SuperLink’s ServerAppIO API

     at the address superlink:9091.

Step 5: Start the ClientApp

The procedure for building and running a ClientApp image is almost identical to the ServerApp image.

Similar to the ServerApp image, you will need to create a Dockerfile that extends the ClientApp image and installs the required FAB dependencies.

1. Create a ClientApp Dockerfile called clientapp.Dockerfile and paste the following code into it:

   clientapp.Dockerfile

   FROM flwr/clientapp:1.14.0
   
   WORKDIR /app
   COPY pyproject.toml .
   RUN sed -i 's/.*flwr\[simulation\].*//' pyproject.toml \
       && python -m pip install -U --no-cache-dir .
   
   ENTRYPOINT ["flwr-clientapp"]
   

   Understand the Dockerfile

   • FROM flwr/clientapp:1.14.0: This line specifies that the Docker image

     to be built from is the flwr/clientapp image, version 1.14.0.
   • WORKDIR /app: Set the working directory for the container to /app.

     Any subsequent commands that reference a directory will be relative to this directory.
   • COPY pyproject.toml .: Copy the pyproject.toml file

     from the current working directory into the container’s /app directory.
   • RUN sed -i 's/.*flwr\[simulation\].*//' pyproject.toml: Remove the flwr dependency

     from the pyproject.toml.
   • python -m pip install -U --no-cache-dir .: Run the pip install command to

     install the dependencies defined in the pyproject.toml file

     
     

     The -U flag indicates that any existing packages should be upgraded, and

     --no-cache-dir prevents pip from using the cache to speed up the installation.
   • ENTRYPOINT ["flwr-clientapp"]: Set the command flwr-clientapp to be

     the default command run when the container is started.

2. Next, build the ClientApp Docker image by running the following command in the directory where the Dockerfile is located:

   $ docker build -f clientapp.Dockerfile -t flwr_clientapp:0.0.1 .
   

   참고

   The image name was set as flwr_clientapp with the tag 0.0.1. Remember that these values are merely examples, and you can customize them according to your requirements.

3. Start the first ClientApp container:

   $ docker run --rm \
       --network flwr-network \
       --detach \
       flwr_clientapp:0.0.1  \
       --insecure \
       --clientappio-api-address supernode-1:9094
   

   Understand the command

   • docker run: This tells Docker to run a container from an image.

   • --rm: Remove the container once it is stopped or the command exits.

   • --network flwr-network: Make the container join the network named flwr-network.

   • --detach: Run the container in the background, freeing up the terminal.

   • --insecure: This flag tells the container to operate in an insecure mode, allowing

     unencrypted communication. Secure connections will be added in future releases.
   • flwr_clientapp:0.0.1: This is the name of the image to be run and the specific tag

     of the image.
   • --clientappio-api-address supernode-1:9094: Connect to the SuperNode’s ClientAppIO

     API at the address supernode-1:9094.

4. Start the second ClientApp container:

   $ docker run --rm \
       --network flwr-network \
       --detach \
       flwr_clientapp:0.0.1 \
       --insecure \
       --clientappio-api-address supernode-2:9095
   

Step 6: Run the Quickstart Project

1. Add the following lines to the pyproject.toml:

   pyproject.toml

   [tool.flwr.federations.local-deployment]
   address = "127.0.0.1:9093"
   insecure = true
   

2. Run the quickstart-docker project and follow the ServerApp logs to track the execution of the run:

   $ flwr run . local-deployment --stream
   

Step 7: Update the Application

1. Change the application code. For example, change the seed in quickstart_docker/task.py to 43 and save it:

   quickstart_docker/task.py

   # ...
   partition_train_test = partition.train_test_split(test_size=0.2, seed=43)
   # ...
   

2. Stop the current ServerApp and ClientApp containers:

   참고

   If you have modified the dependencies listed in your pyproject.toml file, it is essential to rebuild images.

   If you haven’t made any changes, you can skip steps 2 through 4.

   $ docker stop $(docker ps -a -q  --filter ancestor=flwr_clientapp:0.0.1) serverapp
   

3. Rebuild ServerApp and ClientApp images:

   $ docker build -f clientapp.Dockerfile -t flwr_clientapp:0.0.1 . && \
     docker build -f serverapp.Dockerfile -t flwr_serverapp:0.0.1 .
   

4. Launch one new ServerApp and two new ClientApp containers based on the newly built image:

   $ docker run --rm \
       --network flwr-network \
       --name serverapp \
       --detach \
       flwr_serverapp:0.0.1 \
       --insecure \
       --serverappio-api-address superlink:9091
   $ docker run --rm \
       --network flwr-network \
       --detach \
       flwr_clientapp:0.0.1  \
       --insecure \
       --clientappio-api-address supernode-1:9094
   $ docker run --rm \
       --network flwr-network \
       --detach \
       flwr_clientapp:0.0.1 \
       --insecure \
       --clientappio-api-address supernode-2:9095
   

5. Run the updated project:

   $ flwr run . local-deployment --stream
   

Step 8: Clean Up

Remove the containers and the bridge network:

$ docker stop $(docker ps -a -q  --filter ancestor=flwr_clientapp:0.0.1) \
   supernode-1 \
   supernode-2 \
   serverapp \
   superlink
$ docker network rm flwr-network

Where to Go Next

• Enable TLS for Secure Connections

• Persist the State of the SuperLink

• Quickstart with Docker Compose