We continue to share stories about the cases from the Summer School of the “Intellect” Foundation. Oleg Barabash, a student at the Faculty of Computational Mathematics and Cybernetics (CMC) at Moscow State University, worked with his team to develop a tool for quality control of medical documentation. The team created a model that analyzes a medical protocol at the level of a quality expert: it identifies errors, omissions, and inconsistencies, and thus helps specialists check documents faster and more accurately.
In the interview, Oleg explains how the team went beyond the original technical requirements and proposed a more ambitious solution. He also talks about the role played by the mentor, the support of the teachers, and the well‑coordinated teamwork, thanks to which the project became not just a summer school outcome, but a potential tool for improving the business processes of the “Be Healthy” clinic network.
– What business problem was your team solving at the summer school?
– Right away, we chose medical documentation processing as one of our focus areas – I already had some experience working with this type of data. We needed to create a tool to control the quality of document completion: a system that reads a medical protocol the way a quality expert would. The idea was to automatically check how correctly and completely the documentation is filled out, to find possible errors, omissions, or non‑compliance with requirements. Such a system could help reduce the workload on specialists, speed up protocol checks, and make quality control more transparent and consistent.
– How did your team organize the work on the case: where did you start, and what ideas did you discuss?
– I already knew the team beforehand and knew that my colleagues would handle the main technical part – we have different work profiles. But I didn’t get to sit idle either: at first, I developed the project concept. After several meetings with the mentor, we concluded that we shouldn’t limit ourselves to the initial case. We realized we could offer a bolder, more ambitious solution that went beyond the original terms of reference (ToR).
– What product did you manage to create by the end of the work?
– In the end, we created a system to help doctors avoid getting “buried” in documentation. Essentially, it’s two tools in one system.
The first is a doctor’s assistant used directly during a patient visit. It suggests what to fill in next: if the doctor enters a symptom or diagnosis, the system itself proposes logical next steps, and prescriptions (medication, dosage, treatment course) can be assembled in just one click, with everything immediately appearing in the electronic health record in ready‑made formulations. At the same time, the doctor can always rewrite everything in their own words.
The second is a dashboard for a supervisor that automatically checks the quality of completed protocols. Previously, a random sample of 5 % was checked; with our system, up to 100 % of protocols can be reviewed. And it’s not just a simple “bad/good” mark – the system highlights specific defects and explains why something is a defect, with a reference to the source, so the doctor understands the logic. It also collects analytics on typical errors, which can be used for staff training.
– What technologies did your team use in your solutions?
As a foundation, we used a large language model that analyzes the protocol and explains the problems found. At the very start, we made an important decision: not to fine‑tune it on synthetic data. Otherwise, the model might simply memorize the structure of artificial examples and fail to learn how to work with real medical documentation.
Instead, we divided defects into two groups. Technical errors – for example, unfilled fields, incorrect ICD codes, or incorrect prescription formatting – can be reliably and cost‑effectively detected using simple rules. For semantic defects, where it’s important to understand the meaning of the text and its compliance with clinical guidelines, we needed a model with RAG. In this approach, it relies not only on its own knowledge but also consults up‑to‑date Ministry of Health documents: clinical guidelines, standards, and drug reference books.
We tested several models of different sizes – with and without RAG – compared them using accuracy metrics, and ultimately selected the most effective combination.
– What was the most valuable thing for you personally in working on the case?
– Probably, the strongest motivation came from working with the mentor and the feeling that we were all “rowing in the same boat toward a common goal, gradually turning that boat into a powerful steamship.” In just one week, we actually managed to do what some teams spend months developing. The teachers’ support played an important role: we could always go to them for advice, and their expertise helped us find optimal solutions more than once. And, of course, I’d like to thank the team – it’s a real pleasure to work with them!
