Medical LLM Leaderboard
Embrace Federated LLM Fine-Tuning and Secure Your Spot on the Leaderboard!
| Rank | Team | Base Model | Comm. Costs | Average (↑) | PubMedQA | MedMCQA | MedQA | Code | Date |
|---|---|---|---|---|---|---|---|---|---|
1 | Gachon Cognitive Computing Lab | Bio-Medical-Llama-3-8B | 2.03 GB | 63.26 | 70.60 | 57.68 | 61.50 | link | 21.11.24 |
2 | Massimo R. Scamarcia | Qwen2.5-7B-Instruct | 45.14 GB | 46.37 | 44.60 | 39.06 | 55.46 | link | 23.11.24 |
3 | Massimo R. Scamarcia | Mistral-7B-Instruct-v0.3 | 45.14 GB | 43.08 | 63.80 | 25.60 | 39.83 | link | 23.11.24 |
4 | Baseline | Mistral-7B-v0.3 | 40.7 GB | 36.60 | 59.00 | 23.69 | 27.10 | link | 01.10.24 |
In healthcare, the accuracy of information extraction and decision support systems can significantly impact patient outcomes. Federated LLM fine-tuning on medical tasks addresses the critical need for models that are deeply familiar with medical terminologies, patient data, and clinical practices. By leveraging federated learning, hospitals and research institutions can collaboratively train a common model while maintaining the privacy of sensitive patient records. This method not only improves the model's capabilities in understanding and assisting with complex medical contexts but also ensures compliance with health data regulations like HIPAA, enhancing trust and adoption of AI in medicine.
