Hospital Chatbot Metrics: Containment vs Resolution

If your hospital chatbot reports a containment rate above 80 percent, treat that number as a warning, not a win. Containment measures how many conversations ended without a human. It does not measure how many patients got what they came for. In healthcare those two things diverge more sharply than in any other industry, and the gap is where cost, reputation and clinical risk hide.

This is the mistake most Indian hospital groups make in year one of a conversational AI programme. I made it too. The bot goes live, the dashboard lights up green, the vendor sends a case study, and six months later the call centre is busier than before. Here is why that happens and what to measure instead.

Why containment is the default metric

Containment is what the vendor can measure without your help. It sits entirely inside their platform: a session started, a session ended, no handoff triggered. It requires no integration with your hospital information system, no callback data, no appointment reconciliation. It is also the number that justifies the purchase order, because it maps directly to “calls we didn’t have to pay a person to handle”.

For a general-purpose retail bot that logic holds. A shopper who leaves the chat without buying was probably browsing. A patient who leaves the chat without booking was probably still sick.

The Indian context makes this worse. A large share of hospital conversations arrive on WhatsApp, often in Hinglish or a regional language, from a family member rather than the patient. The person typing may be standing in a corridor, using a shared phone, with a question they don’t know how to phrase. A bot that “contains” that conversation has frequently just exhausted the person, not helped them.

What containment hides

Three patterns show up in almost every hospital chatbot log once you look past the headline number.

1. The silent abandon

The patient asks about a doctor’s availability. The bot responds with a menu. The patient picks the wrong branch, gets a canned answer, and closes the window. Containment counts this as a success. The patient calls the front desk twenty minutes later, or calls a competitor, or does nothing and deteriorates at home. None of that shows up in the bot dashboard.

2. The loop

The patient asks a question the bot does not understand. The bot asks them to rephrase. They rephrase. The bot offers the main menu again. After the third loop they give up. Session ends, no handoff, contained. In our logs these loops were the single largest category of “contained” conversations once we actually classified them.

3. The wrong-channel deflection

The bot successfully redirects the patient to a phone number or a form. Technically the bot did its job. Practically, you moved a digital enquiry back into a manual channel and charged yourself the cost of two interactions instead of one.

Add these three together and a bot reporting 80 percent containment might be resolving 30 to 40 percent of what patients actually wanted. The remainder is not saved cost. It is deferred cost plus lost patients.

Resolution: the harder, honest metric

Resolution asks a single question: did the patient’s intent get fulfilled, in this channel, in this session? For a hospital, intents are finite and concrete:

Book, reschedule or cancel an appointment
Find a doctor by specialty, language or location
Get a report, invoice or discharge summary
Check insurance or TPA empanelment for a procedure
Ask a pre-visit logistics question: timings, parking, what to bring
Escalate something urgent

Each of these has an observable completion event. An appointment either exists in the HIS afterwards or it doesn’t. A report either got delivered or it didn’t. That means resolution can be measured, not surveyed. It just requires the bot to be wired into the systems where those events live, which is exactly the integration work that gets deferred when containment looks fine.

Once you measure resolution, the operating decisions change:

You stop optimising the menu tree and start optimising the top five intents.
You discover which intents the bot should never attempt, because a fast handoff resolves them in a fraction of the time.
You get a real cost-per-resolution that you can compare honestly against a human agent.

The Indian specifics that decide the number

Five things that separate a bot that resolves from one that only contains, in this market.

Language detection has to be per message, not per session. Families switch between English, Hindi and a regional language mid-conversation. A bot that locks language at the start will lose them at the second message.

Doctor discovery is the highest-volume intent and the hardest to get right. Patients search by symptom, by a half-remembered name, by “the lady doctor who did my mother’s knee”. The bot needs a fuzzy, forgiving doctor directory behind it, with specialty synonyms in multiple languages. This is a data problem before it is an AI problem.

Insurance questions are where trust is won or lost. “Is this covered?” is asked before almost every planned admission. A bot that says “please contact the TPA desk” has failed the patient at the moment of highest intent. Getting the empanelment data clean enough for the bot to answer confidently is worth more than any NLP upgrade.

Reports and bills must be self-served. These are the easiest intents to resolve fully, and the ones with the biggest call-volume payoff. If the bot cannot fetch a lab report against a registered mobile number and an OTP, it is not ready.

Handoff must be to a person with context. The escalation path that hands the patient to an agent who then asks “how can I help you?” from scratch destroys whatever goodwill the bot built. Pass the transcript. Pass the intent. Pass the patient’s identifier. This alone lifts resolution more than most model changes.

How to run the switch

You do not need to rip out the dashboard. You need to add a second layer and then let it take over.

Classify a sample. Take 500 contained conversations from last month. Have two people independently tag each as resolved, abandoned, looped or deflected. The first time you do this the result is uncomfortable. That is the point.
Define completion events per intent. Appointment created. Report delivered. Doctor profile viewed and slot selected. Callback logged with a ticket number. Write these down as system events, not as opinions.
Instrument the events. This is the integration work. It usually means the bot needs read access to the HIS or the appointment engine and write access to the ticketing system. Budget for it as a core deliverable, not a phase two.
Replace the headline. Resolution rate becomes the number on the weekly review. Containment moves to the second page as a diagnostic.
Re-price the business case. Cost per resolution versus cost per human interaction, for each intent. Some intents will show the bot losing. Route those to humans and stop pretending.

What gets harder

Resolution is a more demanding master. Your numbers will drop before they rise. Vendors will push back because their benchmarks are containment benchmarks. The integration work will surface data quality problems in the doctor directory, the insurance master and the appointment engine that the bot was politely hiding. Each of those is a real problem that was already costing you patients; the bot just stopped covering for it.

There is also an organisational cost. Once resolution is visible, the conversation with the contact-centre head changes from “the bot is deflecting your calls” to “the bot resolves these six intents and you own the rest”. That is a healthier division of labour, but it needs a sponsor senior enough to hold it.

The economics, honestly stated

A hospital contact centre in India handles a very high volume of low-complexity interactions at a cost per interaction that is already low by global standards. The case for a bot is therefore not “replace agents”. It is “resolve the repetitive intents completely, so that agents handle the ones that need judgement, and so that patients stop calling twice”. Containment measures the first half of that sentence. Resolution measures the whole thing.

Measured properly, a good hospital bot resolves a minority of total intents but a majority of total volume, because the high-volume intents are the simple ones. That is a strong result. It is also a smaller number than the containment rate, which is why nobody puts it in the vendor case study.

Reading a vendor proposal with resolution in mind

Every conversational AI proposal a hospital receives leads with a containment benchmark and a cost-per-conversation figure. Rewrite the arithmetic before you sign.

Ask for the platform’s cost per resolved intent, split by intent. If the vendor cannot produce it, ask what it would take to instrument it, and make that instrumentation a contractual deliverable with an acceptance test. A vendor who resists this is telling you their product has never been measured that way.

Ask what happens to a conversation the bot cannot resolve. If the answer is “it is deflected to a phone number”, you are being sold a cost transfer, not a cost saving. The acceptable answer is a warm handoff to a human with the transcript, the identified intent and the patient identifier attached.

Ask how language switching is handled per message, how the doctor directory is kept in sync with your HIS, and how insurance empanelment data gets into the bot. Each of these is a data pipeline, and each one that is missing is a category of conversation that will be contained rather than resolved.

Finally, insist on a pilot scoped to two or three high-volume intents rather than a broad launch. A narrow bot that resolves appointment booking and report retrieval at a high rate teaches you more, and earns more internal trust, than a broad bot that contains everything and resolves little.

A simple way to model the economics

For each intent, you need four numbers: monthly volume, the human cost per interaction, the bot’s resolution rate, and the bot’s fully loaded cost per conversation including integration and licence amortised over volume. The bot’s saving on that intent is volume multiplied by resolution rate multiplied by the difference between human and bot cost, minus the cost of handling the unresolved remainder twice.

Run that for the top six intents. Two or three will show a clear saving, usually reports, bills and simple bookings. One or two will show roughly break-even. At least one, typically anything involving clinical judgement or a complaint, will show the bot losing money because the unresolved remainder is expensive and the reputational cost is real. Route those intents to humans from the start. A bot that knows what not to attempt is worth more than one that attempts everything.

If you’re starting a chatbot programme next quarter

Write the list of intents before you write the RFP. Six to ten, ranked by volume.
Insist on completion-event integration in phase one. No integration, no launch.
Report resolution from day one, even if it’s ugly.
Design the handoff with as much care as the bot.
Fix the doctor directory and the insurance master first. The bot is only as good as the data behind it.

The metric you choose decides what your team optimises. Choose the one that describes what actually happened to the patient.