PCDCareHub
AI IN HEALTHCARE

Human-Centered AI: from healthcare hype to healthcare reality

An open digital backbone, with human-centered AI at its core, is essential to meeting the challenges facing the healthcare sector.

Updated 8 min read
Human-centered AI in healthcare — PCD CareHub

The growing pressure on the Dutch healthcare sector

The Dutch healthcare sector is under immense pressure. Professionals contend with long waiting lists, heavy administrative burdens, and a care demand that continues to rise due to an aging population. Forecasts point to a 30–40% increase in care demand by 2030.

At the same time, care institutions struggle with outdated and fragmented systems, leaving valuable data siloed and inaccessible.

We see intelligent AI agents not as a replacement, but as an amplifier of what remains inherently human in care. Through the CareHub, we make AI practically applicable within care organizations.

AI empowers healthcare professionals — it does not replace them.

The problem: AI hype versus healthcare reality

Healthcare professionals lose an average of 40% of their working time to administration and duplicate data entry caused by non-interoperable systems. In mental health and youth care, waiting times stretch to months.

70% of care institutions are seeking integrated solutions that guarantee interoperability in order to combat fragmented data.

AI is often presented as a silver bullet, but in practice its use remains confined to pilots or closed systems. Human-centered AI in healthcare requires a different approach: not black-box algorithms, but transparent AI agents.

The solution: CareHub with AI agents

PCD CareHub builds an open digital care hub with the CareHub, bringing complementary care-tech companies together. Interoperability is central: systems share real-time data through standards such as Wegiz, without duplicate data entry.

An AI agent analyzes intake data from EHRs, client portals, and wearables to propose appropriate care pathways. Rather than overriding professionals, the AI provides substantiation through explainable AI — transparent decision logic that complies with GDPR and NEN 7510.

Proven technologies such as workflow engines are connected to AI modules for 360° client views. The strength lies in the human layer: AI flags escalations early, but leaves the professional with the final word.

Real-world impact: from care to well-being

At intake in youth care, an AI agent matches the client based on hard criteria as well as soft factors. Clients receive a self-service portal, giving them greater control over their own care pathway. The care coordinator reviews the recommendation, can accept or override it, and the reasons are fed back to improve the system.

Professionals can focus on actual care delivery. Less administrative burden, reduced workload, higher satisfaction. In the elderly and long-term care sector, AI supports hybrid care through virtual check-ins and predictive monitoring — not as a replacement for human contact, but to direct that contact where it is needed most.

Organizations gain efficiency and can achieve greater reach without additional staffing. More important than the numbers: greater connectedness for clients, greater fulfillment for professionals. AI deployed in this way enhances the human character of care rather than eroding it.

What 'human-centered' concretely means in design

Human-centered AI is a directional term. But what does it mean in concrete design choices? Three principles recur consistently: the professional retains oversight, the client has transparency, and the organization retains accountability.

Professional oversight means: the AI suggests, the professional decides. This is not only a matter of UI (an 'approve' button is not sufficient) but of workflow: the AI must not do anything 'behind the scenes' that the professional cannot see or correct. This requires a design in which the AI is transparent and redirectable — and that is precisely where much SaaS-AI in healthcare falls short.

Client transparency means: clients can see what the AI says about them and can respond to it. Not as a compliance checkbox (GDPR already requires this for automated decision-making), but as a product principle. AI that is transparent to clients achieves faster adoption and better feedback.

Organizational accountability means: the care provider remains formally and operationally responsible for outcomes. AI vendors who contract away their responsibility through general terms and conditions offer no assurance; only vendors who participate in the governance chain belong in healthcare.

Adoption versus pilot: why so many AI projects stall

A recurring pattern: an AI pilot delivers promising results in a controlled environment, but stalls when scaled. Three reasons come up repeatedly.

One: workflow integration. An AI that exists in a separate location — a dedicated tab, a separate screen, a standalone report — gets forgotten. An AI that integrates into the existing workflow — within the EHR the professional already uses, within the scheduling tool already in use — gets deployed consistently. Adoption is a matter of placement, not performance.

Two: feedback loop. AI in a pilot is typically well monitored; in production, rarely. Without feedback — what was accepted, what was rejected, and for what reasons — the AI cannot learn and the team cannot evaluate. The pilot succeeded because it received attention; in production, that attention disappears.

Three: alignment with existing compliance. AI projects are often launched independently of NEN 7510 and GDPR practices, and later run into DPIA requirements, audit obligations, and consent flows. AI that is embedded in the existing compliance architecture from the design stage is simply adopted faster than AI that must catch up on compliance after the fact.

What this means for the structure of care organizations

Human-centered AI demands something from organizations: not only technical capacity, but also governance capacity. Who is mandated to make decisions about AI deployment? How is feedback from professionals systematically collected? Who periodically evaluates quality and bias?

A lightweight solution: an AI working group comprising a clinical lead, an IT lead, a compliance lead, and a patient representative. Not as a 'committee' that approves everything in advance, but as a group that conducts periodic evaluations and can intervene when signals arise. That is sufficient to avoid most pitfalls.

Organizations that do not have this structure in place by 2026 will find themselves three years behind — not because AI will have advanced so dramatically, but because organizations that do have it will have built their adoption capacity three times faster. AI in healthcare is not about who has the most models. It is about who has the best local governance to scale AI responsibly.

AGREE OR DISAGREE?

Let us know.

We grow from constructive pushback.