Machine Learning Algorithm for Healthcare
INDUSTRY
Healthcare/ML
SERVICE
Machine Learning
Machine Learning Algorithm for Healthcare
"A team of data scientists developed a Machine Learning diagnosis prediction system to enable Natural Language Processing in Healthcare."
Waverley developed a machine learning-based natural language processing system to automatically process patient clinical notes and predict diagnosis categories. Working with a California-based analytics software company, Waverley's data science team trained an ensemble of 29 specialized models to classify patient data across disease categories (Psychologic, Neurologic, Musculoskeletal, etc.) with 95% average accuracy. The system processes unstructured medical text through a sophisticated pipeline of filtering, term frequency analysis, word embeddings, and categorical routing, achieving results that demonstrate machine learning's potential in clinical decision support and medical record analysis.
The Client
Our Client
Our client is a Silicon Valley based software company that builds analytics solutions for businesses on cloud native, platform agnostic, open source technology. The company was founded by a group of technology veterans who have spent more than a decade building forward looking software in Silicon Valley.
Type of AI Project
Natural Language Processing and Machine Learning: multi model text classification for clinical diagnosis prediction.
What the System Does
The client asked Waverley for a system that could read clinical text and predict the likely diagnosis category behind it, at a scale no single model could handle on its own. The finished system takes in clinical text, determines which subtree of diagnosis categories it belongs to, routes it to the model trained for that subtree, and returns a predicted category with 95% average accuracy across the full data set.
Main Business Goal
Give the client a working, testable NLP diagnosis prediction engine as the technical foundation for a healthcare product: one that could move from research prototype to real patient data testing and, eventually, a user facing tool for hospitals.
Project Analysis
How Waverley Built It
Waverley engaged a dedicated team of data scientists to research and build the prediction pipeline from the ground up. Before any data reached a model, it moved through a preprocessing stage: input text was filtered, then scored for meaningful terms using a term frequency calculation drawn from the Google Books Ngram Dataset. Word embeddings were generated with the Universal Sentence Encoder, and a purpose built vocabulary handled the conversion of text to numerical indices and back.
With roughly 870,000 data entries in play, a single model could not hold the full problem in its storage capacity. Waverley's engineers addressed this by building 29 specialized prediction models, each responsible for one subtree of the classification space, plus one additional Classifier model that reads the incoming data first and decides which of the 29 models should handle it. That routing step is what let the system scale past the limits of a single model design while keeping every prediction specific to its category.
To keep predictions fast at that scale, the team ran the models across multiple GPUs in parallel and served predictions through TensorFlow Serving, which held throughput steady without giving up accuracy.
What We Delivered
Waverley's data science team, based in Ukraine, held weekly calls with the client in the United States to review progress and plan next steps. When the client's in house developers ran into issues reproducing the system on their own side, Waverley's engineers advised them directly. Because the project carried a heavy documentation load, Waverley also provided a Technical Writer on an on demand basis to keep the work reproducible and auditable.
Tools & Tech Stack
Considerations
Key Challenges
Handling a data set of roughly 870,000 entries too large for a single model's storage, which drove the move to a 29-model, subtree-routed architecture instead of one monolithic model.
Holding processing speed steady at scale without trading away the 95% accuracy baseline, addressed with multi-GPU execution and TensorFlow Serving.
Supporting the client's in-house developers, who were reproducing parts of the system in parallel, by advising on issues as they came up.
Meeting a heavy documentation requirement across a research-driven build, which is why Waverley staffed a Technical Writer on demand rather than leaving it to the data science team alone.
Interesting Technical Decisions
Adding a dedicated Classifier model as a routing layer, rather than training one large model, was the decision that made the 870,000-entry data set tractable in the first place.
Filtering for meaningful terms with a Google Books Ngram term-frequency score before embedding, which reduced noise reaching the Universal Sentence Encoder step.
Building a custom vocabulary to move data into numerical indices and back, keeping the pipeline consistent across 29 separate models.
Team Location
Ukraine (Waverley data science team); United States, Silicon Valley (client).
Outcomes & Impact
Key Results
Prediction accuracy: 95% average across the full model set
Data volume handled: ~870,000 clinical text entries
Model architecture: 29 specialized models plus 1 classifier for routing
Processing approach: Multi-GPU training with TensorFlow Serving for inference
Deployment status: In testing against real patient data ahead of a planned hospital rollout
Waverley delivered a working, 29-model NLP diagnosis prediction system that met its accuracy target and gave the client a technical foundation it could put in front of real patient data. The classifier and router architecture solved the client's core scale problem, turning a data set too large for one model into a set of specialized, coordinated models, and the multi-GPU serving setup kept predictions fast enough for practical use. With the model now in testing against real patient data, the client's next steps are to build out a more flexible system, automate data labeling, and design the user interface that will bring the tool in front of clinicians in hospitals.
This kind of engagement, standing up a production-track AI system from research through a working pipeline, is the core of Waverley's AI-Accelerated Product Development Pod: an embedded team that builds AI-native products and features at the client's pace, not a generic staffing pool.
Conclusion
The healthcare NLP project demonstrates Waverley's capability to apply machine learning to complex, high-impact medical problems. By thoughtfully designing the data processing pipeline, deploying an ensemble architecture optimized for diverse disease categories, and optimizing for computational efficiency, Waverley delivered a proof-of-concept that brings healthcare AI from theory into practical validation. The 95% accuracy achievement and detailed documentation show that AI-powered clinical tools are feasible when approached with rigor and domain understanding. For healthcare technology companies exploring AI-powered clinical decision support, medical record analysis, or diagnostic assistance, this case illustrates the importance of careful data science methodology and the potential for machine learning to improve clinical outcomes. Waverley's combination of data science depth and healthcare domain engagement enables partners to confidently invest in AI initiatives that deliver real clinical value.
Let's Build Something Great.
Tell us about your project — we'll find the right path forward.