Containment vs resolution: the chatbot metric that misled us
Last reviewed
A hospital chatbot containment rate measures conversations that ended without a human, not whether the patient got what they came for, and the gap hides silent abandonment and looped, unresolved calls. The article argues for resolution as the honest metric and shows how to instrument, price and report it instead.
Containment vs resolution: at a glance
| Containment rate | Resolution rate | |
|---|---|---|
| What it counts | Conversations that end without a human handoff | Conversations where the patient’s need was actually met |
| What it rewards | Keeping patients away from staff | Solving the problem: booking made, question answered correctly |
| Main risk | Patients trapped in loops; failures look like success | Harder to measure, needs outcome data |
| How to measure | Bot logs | Booking records, repeat contacts, short follow-up surveys |
| Use it for | Cost and capacity planning | The primary quality KPI |
If your hospital chatbot reports a containment rate above 80 percent, treat that number as a warning, not a win. Containment measures how many conversations ended without a human. It does not measure how many patients got what they came for. In healthcare those two things diverge more sharply than in any other industry, and the gap is where cost, reputation and clinical risk hide.
New to the topic? Start here: AI in healthcare operations sets out the reading order.
This is the mistake most Indian hospital groups make in year one of a conversational AI programme. I made it too. The bot goes live, the dashboard lights up green, the vendor sends a case study, and six months later the call centre is busier than before. Here is why that happens and what to measure instead.
Why containment is the default metric
Containment is what the vendor can measure without your help. It sits entirely inside their platform: a session started, a session ended, no handoff triggered. It requires no integration with your hospital information system, no callback data, no appointment reconciliation. It is also the number that justifies the purchase order, because it maps directly to “calls we didn’t have to pay a person to handle”.
For a general-purpose retail bot that logic holds. A shopper who leaves the chat without buying was probably browsing. A patient who leaves the chat without booking was probably still sick.
The Indian context makes this worse. A large share of hospital conversations arrive on WhatsApp, often in Hinglish or a regional language, from a family member rather than the patient. The person typing may be standing in a corridor, using a shared phone, with a question they don’t know how to phrase. A bot that “contains” that conversation has frequently just exhausted the person, not helped them.
What containment hides
Three patterns show up in almost every hospital chatbot log once you look past the headline number.
1. The silent abandon
The patient asks about a doctor’s availability. The bot responds with a menu. The patient picks the wrong branch, gets a canned answer, and closes the window. Containment counts this as a success. The patient calls the front desk twenty minutes later, or calls a competitor, or does nothing and deteriorates at home. None of that shows up in the bot dashboard.
2. The loop
The patient asks a question the bot does not understand. The bot asks them to rephrase. They rephrase. The bot offers the main menu again. After the third loop they give up. Session ends, no handoff, contained. In our logs these loops were the single largest category of “contained” conversations once we actually classified them.
3. The wrong-channel deflection
The bot successfully redirects the patient to a phone number or a form. Technically the bot did its job. Practically, you moved a digital enquiry back into a manual channel and charged yourself the cost of two interactions instead of one.
Add these three together and a bot reporting 80 percent containment might be resolving 30 to 40 percent of what patients actually wanted. The remainder is not saved cost. It is deferred cost plus lost patients.
Resolution: the harder, honest metric
Resolution asks a single question: did the patient’s intent get fulfilled, in this channel, in this session? For a hospital, intents are finite and concrete:
- Book, reschedule or cancel an appointment
- Find a doctor by specialty, language or location
- Get a report, invoice or discharge summary
- Check insurance or TPA empanelment for a procedure
- Ask a pre-visit logistics question: timings, parking, what to bring
- Escalate something urgent
Each of these has an observable completion event. An appointment either exists in the HIS afterwards or it doesn’t. A report either got delivered or it didn’t. That means resolution can be measured, not surveyed. It just requires the bot to be wired into the systems where those events live, which is exactly the integration work that gets deferred when containment looks fine.
Once you measure resolution, the operating decisions change:
- You stop optimising the menu tree and start optimising the top five intents.
- You discover which intents the bot should never attempt, because a fast handoff resolves them in a fraction of the time.
- You get a real cost-per-resolution that you can compare honestly against a human agent.
The Indian specifics that decide the number
Five things that separate a bot that resolves from one that only contains, in this market.
Language detection has to be per message, not per session. Families switch between English, Hindi and a regional language mid-conversation. A bot that locks language at the start will lose them at the second message.
Doctor discovery is the highest-volume intent and the hardest to get right. Patients search by symptom, by a half-remembered name, by “the lady doctor who did my mother’s knee”. The bot needs a fuzzy, forgiving doctor directory behind it, with specialty synonyms in multiple languages. This is a data problem before it is an AI problem.
Insurance questions are where trust is won or lost. “Is this covered?” is asked before almost every planned admission. A bot that says “please contact the TPA desk” has failed the patient at the moment of highest intent. Getting the empanelment data clean enough for the bot to answer confidently is worth more than any NLP upgrade.
Reports and bills must be self-served. These are the easiest intents to resolve fully, and the ones with the biggest call-volume payoff. If the bot cannot fetch a lab report against a registered mobile number and an OTP, it is not ready.
Handoff must be to a person with context. The escalation path that hands the patient to an agent who then asks “how can I help you?” from scratch destroys whatever goodwill the bot built. Pass the transcript. Pass the intent. Pass the patient’s identifier. This alone lifts resolution more than most model changes.
How to run the switch
You do not need to rip out the dashboard. You need to add a second layer and then let it take over.
- Classify a sample. Take 500 contained conversations from last month. Have two people independently tag each as resolved, abandoned, looped or deflected. The first time you do this the result is uncomfortable. That is the point.
- Define completion events per intent. Appointment created. Report delivered. Doctor profile viewed and slot selected. Callback logged with a ticket number. Write these down as system events, not as opinions.
- Instrument the events. This is the integration work. It usually means the bot needs read access to the HIS or the appointment engine and write access to the ticketing system. Budget for it as a core deliverable, not a phase two.
- Replace the headline. Resolution rate becomes the number on the weekly review. Containment moves to the second page as a diagnostic.
- Re-price the business case. Cost per resolution versus cost per human interaction, for each intent. Some intents will show the bot losing. Route those to humans and stop pretending.
What gets harder
Resolution is a more demanding master. Your numbers will drop before they rise. Vendors will push back because their benchmarks are containment benchmarks. The integration work will surface data quality problems in the doctor directory, the insurance master and the appointment engine that the bot was politely hiding. Each of those is a real problem that was already costing you patients; the bot just stopped covering for it.
There is also an organisational cost. Once resolution is visible, the conversation with the contact-centre head changes from “the bot is deflecting your calls” to “the bot resolves these six intents and you own the rest”. That is a healthier division of labour, but it needs a sponsor senior enough to hold it.
The economics, honestly stated
A hospital contact centre in India handles a very high volume of low-complexity interactions at a cost per interaction that is already low by global standards. The case for a bot is therefore not “replace agents”. It is “resolve the repetitive intents completely, so that agents handle the ones that need judgement, and so that patients stop calling twice”. Containment measures the first half of that sentence. Resolution measures the whole thing.
Measured properly, a good hospital bot resolves a minority of total intents but a majority of total volume, because the high-volume intents are the simple ones. That is a strong result. It is also a smaller number than the containment rate, which is why nobody puts it in the vendor case study.
Reading a vendor proposal with resolution in mind
Every conversational AI proposal a hospital receives leads with a containment benchmark and a cost-per-conversation figure. Rewrite the arithmetic before you sign.
Ask for the platform’s cost per resolved intent, split by intent. If the vendor cannot produce it, ask what it would take to instrument it, and make that instrumentation a contractual deliverable with an acceptance test. A vendor who resists this is telling you their product has never been measured that way.
Ask what happens to a conversation the bot cannot resolve. If the answer is “it is deflected to a phone number”, you are being sold a cost transfer, not a cost saving. The acceptable answer is a warm handoff to a human with the transcript, the identified intent and the patient identifier attached.
Ask how language switching is handled per message, how the doctor directory is kept in sync with your HIS, and how insurance empanelment data gets into the bot. Each of these is a data pipeline, and each one that is missing is a category of conversation that will be contained rather than resolved.
Finally, insist on a pilot scoped to two or three high-volume intents rather than a broad launch. A narrow bot that resolves appointment booking and report retrieval at a high rate teaches you more, and earns more internal trust, than a broad bot that contains everything and resolves little.
A simple way to model the economics
For each intent, you need four numbers: monthly volume, the human cost per interaction, the bot’s resolution rate, and the bot’s fully loaded cost per conversation including integration and licence amortised over volume. The bot’s saving on that intent is volume multiplied by resolution rate multiplied by the difference between human and bot cost, minus the cost of handling the unresolved remainder twice.
Run that for the top six intents. Two or three will show a clear saving, usually reports, bills and simple bookings. One or two will show roughly break-even. At least one, typically anything involving clinical judgement or a complaint, will show the bot losing money because the unresolved remainder is expensive and the reputational cost is real. Route those intents to humans from the start. A bot that knows what not to attempt is worth more than one that attempts everything.
If you are hiring someone to own this kind of work, I have written separately about what a real AI marketing expert does and how to tell them from someone who has only watched the demos.
If you’re starting a chatbot programme next quarter
- Write the list of intents before you write the RFP. Six to ten, ranked by volume.
- Insist on completion-event integration in phase one. No integration, no launch.
- Report resolution from day one, even if it’s ugly.
- Design the handoff with as much care as the bot.
- Fix the doctor directory and the insurance master first. The bot is only as good as the data behind it.
The metric you choose decides what your team optimises. Choose the one that describes what actually happened to the patient.
Related: August AI turned 9 million free chats into a $39 care plan.
Further reading from primary sources
- WHO: Ethics and governance of artificial intelligence for health (2021): global principles for accountable AI in health.
- WhatsApp Business Messaging Policy: opt-in rules for messaging people.
- ICMR: Ethical guidelines for AI in biomedical research and healthcare (2023): India’s guidance on responsible AI in healthcare.
Questions people ask
Containment measures how many conversations ended without a handoff to a human. It sits entirely inside the vendor’s platform: a session started, a session ended, no escalation triggered. So it needs no integration with your hospital systems. That is why it is the default metric and why it justifies the purchase order. It does not measure whether the patient got what they came for, and in healthcare those two things diverge sharply.
Resolution asks one question: did the patient’s intent get fulfilled, in this channel, in this session? Hospital intents are finite (book or reschedule an appointment, find a doctor, get a report or bill, check insurance empanelment, ask a logistics question, escalate something urgent), and each has an observable completion event. An appointment either exists in the HIS afterwards or it does not. That means resolution can be measured from system events rather than surveyed.
Three patterns hide inside it. The silent abandon: the patient picks the wrong menu branch, gets a canned answer and closes the window, then calls the front desk or a competitor. The loop: the bot asks them to rephrase three times and they give up. The wrong-channel deflection: the bot hands them a phone number, so you pay for two interactions instead of one. All three count as contained. None resolved anything, and the call centre gets busier.
Reports and bills, because they are the easiest to resolve fully and carry the biggest call-volume payoff. Then simple appointment booking and rescheduling. Then doctor discovery, which is the highest-volume intent and the hardest to get right because patients search by symptom, half-remembered name or the lady doctor who did my mother’s knee. Start a pilot with two or three high-volume intents rather than a broad launch.
Define a completion event per intent as a system event, not an opinion: appointment created, report delivered, doctor profile viewed and slot selected, callback logged with a ticket number. Instrument those events by giving the bot read access to the HIS or appointment engine and write access to ticketing. Then classify a sample: take five hundred contained conversations and have two people independently tag each as resolved, abandoned, looped or deflected. The first result is uncomfortable.
You do not rip out the existing dashboard; you add a second layer and let it take over. Classifying a sample takes a couple of weeks. Defining completion events per intent takes a few more. The integration work is the long pole, usually a quarter or two depending on how open the HIS is. Report resolution from day one even if it is ugly, and expect the headline number to drop before it rises.
Read access to the HIS or appointment engine so it can confirm bookings and fetch reports against a registered mobile number and an OTP. A sync with the doctor directory, including specialty synonyms in multiple languages. Clean insurance and TPA empanelment data so it can answer whether a procedure is covered. Write access to ticketing for callbacks. And a handoff that passes transcript, intent and patient identifier to the agent. No integration, no launch.
Not by replacing agents. An Indian hospital contact centre already handles high volumes at a low cost per interaction by global standards. The case is that the bot resolves repetitive intents completely, so agents handle the ones needing judgement and patients stop calling twice. Model it per intent: volume multiplied by resolution rate multiplied by the cost difference, minus the cost of handling the unresolved remainder twice. Reports, bills and simple bookings usually show a clear saving.
Anything involving clinical judgement or a complaint. Run the per-intent economics and at least one of those will show the bot losing money, because the unresolved remainder is expensive and the reputational cost is real. Route those to humans from the start with a fast, warm handoff. A bot that knows what not to attempt is worth more than one that attempts everything, and clinicians trust the programme more when the boundary is explicit.
Ask for cost per resolved intent, split by intent; if they cannot produce it, make the instrumentation a contractual deliverable with an acceptance test. Ask what happens to a conversation the bot cannot resolve. Deflection to a phone number is a cost transfer, not a saving. Ask how language switching is handled per message, how the doctor directory syncs with your HIS, and how empanelment data gets in. Each missing pipeline is a category of contained conversations.
The conversation with the contact-centre head changes from the bot deflecting your calls to the bot resolving these six intents and you owning the rest. That is a healthier division of labour, but it needs a sponsor senior enough to hold it. Agents also need context on handoff: a patient passed to someone who asks how can I help you from scratch loses whatever goodwill the bot built. Pass the transcript, the intent and the identifier.
The doctor directory and the insurance master. Doctor discovery needs a fuzzy, forgiving directory with specialty synonyms in several languages, kept in sync with the HIS. Insurance questions are asked before almost every planned admission, and a bot that says please contact the TPA desk has failed the patient at the moment of highest intent. Both are data problems before they are AI problems, and the bot was politely hiding them.
Language detection has to be per message, not per session. Families switch between English, Hindi and a regional language mid-conversation, often typing Hinglish from a shared phone on behalf of the patient. A bot that locks language at the start loses them at the second message. This is a large share of hospital messaging traffic in India, and it is where containment quietly rises while resolution falls.
Resolution rate as the headline, with containment on the second page as a diagnostic. Cost per resolution versus cost per human interaction, by intent. And the honest shape of the result: a good hospital bot resolves a minority of total intents but a majority of total volume, because the high-volume intents are the simple ones. That is a strong outcome and a smaller number than any vendor case study, which is exactly why it is credible.
Read my takes first in Google Search
