IOTAIRx announces release of their first medgemma-27b fine-tuned model

IOTAIRx is excited to announce a major milestone in our mission to build safe and effective medical AI: the creation of our first internally fine-tuned, specialized large language model.

Our journey begins with a powerful foundation: MedGemma-27B, a state-of-the-art, 27-billion-parameter model from Google Health. Our goal is to adapt this model for specific medical domains and further enhance its capabilities and accuracy.

The Fine-Tuning Process

Our first training phase was Supervised Fine-Tuning (SFT). In this stage, we trained the model on a proprietary dataset of high-quality medical question-and-answer pairs. This process is like having the model study for a specialty board exam by reviewing thousands of case-specific flashcards.

After this SFT phase, we evaluated the model on a standard clinical knowledge benchmark. The result was around 84% accuracy on a sample of mmlu clinical knowledge. We are further evaluating the model and testing accuracy across several benchmarks and specialities.

The Next Frontier: From 84% to 95% Accuracy

An 84% score is decent start, but in medicine, precision and reliability are everything. SFT is good at teaching the model the "correct" answer from a book. Our next phases, Reinforcement Learning (RL) and Test Time Scaling, will teach it why an answer is safe, helpful, and appropriate in a real-world clinical context.

Our plan involves leveraging these two advanced techniques to push performance toward our goal of 95% accuracy in selected medical domains.

This first fine-tuned model is a foundational step. By combining advanced AI techniques with invaluable human-expert guidance, we are on the path to creating a medical AI that is not just accurate, but truly reliable and trustworthy.

Hugging Face Model Release

The fine-tuned model is available on the Hugging Face Hub at the following repository:

https://huggingface.co/IOTAIRx/medgemma-27b-text-ft

A Note on Safety and Regulation

Important: This fine-tuned model is currently a research and development project. It is not available for real-world clinical use or patient care. Any deployment in a live clinical setting would first require rigorous validation, testing, and formal approval from regulatory bodies, including the FDA.