AI Isn't the Hard Part

Patient-safety incidents are inevitable in healthcare. Learning from them is not, and in Finland the data you would need to learn from was scattered across a dozen systems that don't talk to each other.

These incidents (the wrong dose, an avoidable infection, a complication that should have been caught) happen in every health system. Reporting them and learning from them is how you stop the avoidable harm from happening again. It's mandatory, but in practice often neglected. International figures put the cost of errors and harm at up to 13% of healthcare spending, and in Finland safety incidents are estimated to cost around a billion euros a year. Reforming incident reporting is a key metric in Finland's national Client and Patient Safety Strategy for 2022–2026.

The data needed to drive that learning barely exists in usable form. Mandatory adverse events flow into Hilmo, the national care registry, coded in ICD-10 and fed automatically from records, but near-misses never reach it. Voluntary reporting happens in local systems like HaiPro that stay at the level of a single organisation and never aggregate nationally. The result is incomplete, inconsistent, uncomparable data: enough to file, but not enough to learn from or to steer a wellbeing services county's funding or safety work. That was the situation I was asked to study.

I ran the project end to end: the research and analysis, the process design, and the build. It was a research subproject inside Varha, the Wellbeing Services County of Southwest Finland, sitting under the national reform of incident reporting. Varha's data-services team provided the secure environment the work had to run inside, and a Varha surgeon evaluated the reports the system generated. Being one person across the whole thing is why it spans research, design and engineering.

I framed the work as design-science research, following the established six-stage model: identify the problem, define what a solution needs to do, design and build an artefact, demonstrate it, evaluate it, and communicate the result. In practice that meant interviewing my way to the requirements before I wrote a line of code, then building something working and putting it in front of a clinician to test.

The interviews ran to 14 people: six doctors at Varha, all in senior surgical or clinical-information roles, and eight people across the national authorities (the ministry, the national health institute, the social-insurance institution, and the agency running the national data platform). I analysed them with reflexive thematic analysis, then used what I learned to design the process and the proof-of-concept.

Four requirements came up again and again: the systems need to integrate so the same event isn't reported five times; classification needs to be standardised so data is comparable; AI could cut the reporting burden; and clinicians need feedback so reporting feels worth doing. The first three are technical. The fourth turned out to explain the others.

The sharpest recurring theme was that reporting feels futile. Clinicians are overworked and skip reporting unless an incident is clearly serious, partly because nothing visibly comes back when they do. You file into a system, never see it change anything, and next time you don't bother. In the interviewees' telling, underreporting is mostly a feedback problem rather than a discipline one.

Could a commercial LLM write the reports? Yes, and that turned out to be the simple bit. What it couldn't fix was the data beneath it: fragmented, uncomparable and underreported.

The work lands here: the bigger lever is a shift in the reporting culture. A model that writes tidy structured reports on top of data that is still entered five times, in five formats, into systems that never aggregate, just produces nicer fragments. The leverage is in recording the event once, in the record where it already happens, auto-structuring it, and only then letting it flow somewhere it can be compared. The exact national report structure can't be settled by one project; that has to be defined with the authorities and clinicians, reusing international templates. But the shape of the fix was clear: change the process, and use the model inside it.

The proof-of-concept ran on real, pseudonymised records inside Varha's environment: 40 patients, each with at least 10 free-text entries over five years, of which 21 had adverse-event codes and were used for the test. Per patient that was anywhere from 20,000 to over 700,000 characters of messy clinical text. The pipeline extracts the records flagged with adverse-event ICD-10 codes, combines them into one timestamped narrative per patient, and prompts GPT-4o with that text plus the required report structure, returning a structured incident report per distinct event.

The designed artefact is the process the model belongs in: a single-entry reporting flow across three lanes (the clinician, the LLM, and the record system). The clinician records the event once as part of normal documentation; the system detects a likely incident; the model generates a prefilled, standardised report; the clinician reviews, edits if needed, and signs it; and the report is stored with the patient record while an anonymised copy is sent to a national database. Human oversight stays in; duplicate entry comes out.

What landed: a 113-page master's thesis; the single-entry process model; the working proof-of-concept and its clinical evaluation; and an international peer-reviewed publication of the interview insights and the PoC study.

The model did its job; the bottleneck was the system around it: the fragmented data, the missing comparable structure, and a reporting culture where nothing comes back to the person who reported. That's still true for public healthcare.

I'd intended two or three clinicians to evaluate the output; delays meant a single evaluator, and that's the reliability gap I'd close first. The research drew on one organisation and a narrow set of specialist surgeons, so the findings may not transfer cleanly to primary care or elsewhere. And a large share of the effort went not into the model but into permissions and data protection: running a commercial LLM on real patient records under rules that partly pull against each other.

This was 2024 into 2025, when doing this on real patient data was still close to uncharted, which is a big part of why that work was so heavy. A couple of years on it already looks almost routine, and nobody would reach for GPT-4o for it now. The model was always the part most likely to date; the fragmented data and the reporting culture beneath it haven't dated the same way.