Join the Summer
of Reproducibility
Contribute to the Flower Community; Merge and get Rewarded!
How to participate?
Become a Flower contributor, have your baseline added to Flower, and get rewarded!
Check out the rules
Remember to read the rules before contributing. We aim to ensure that the baselines meet high standards, which is why our review process will be stringent.
Read our FAQ
Got any questions? Check below this page. If you can't find what you're looking for, hop on the (SoR slack link here). The Flower team will be happy to assist you!
The SoR is a 3-month community sprint to improve reproducibility in Federated Learning. It's our goal to create 50 high quality baselines with the Flower Framework and make them the foundation of further FL research. Towards this goal, we are contributing a $100K USD prize pool to reward contributors. SoR will run from 1 July to 1 October 2023. Everyone is welcome to contribute! 🙌
_How to participate?
If you are interested in taking part in the Summer of Reproducibility, here are the instructions to follow:
- 01READY! Join the Summer of Reproducibility program
- Join the Flower Slack and say “Hi” in channel #summer-of-reproducibility.
- Pick a baseline from our curated list and open its GitHub issue.
- 02SET! Defining the scope of your contribution
- Add a comment to the baseline's issue and let us know what experiments you plan to reproduce.
- We'll be reaching out to you:
- to discuss your proposal (contribution eligibility).
- to learn a bit more about your: who you are, affiliation, location, and who's getting paid (reward eligibility).
- 03GO! You start working on your baseline of choice
- Follow the instructions in the “What happens next?” section of your GitHub issue.
- Create a Draft PR referring to your issue, commit frequently, and reach out to the Flower team if you encounter difficulties. We’d love to help you regarding Flower API questions!
- 04CHECK! Let us know when you are done, so we can verify your contribution.
- Mark the Draft PR as “Ready for review”:
- We’ll closely look into your code and run compatibility tests with the Flower API.
- If the results generated match those in the paper → excellent!! Your contribution is ready to be merged.
- If there are issues with your code and results, try your best to find the cause of the mismatch in performance, squash all the bugs, and let us know when you are done!
- 05DONE! Merge your baseline
- Your PR is merged into the main branch, and your issue is closed.
- Our finance team will reach out to you regarding your award.