By Jessica King (London School of Hygiene & Tropical Medicine)
The million-dollar question
How can we best measure quality of care? It’s the million-dollar question which those of us working in health systems and quality improvement would love to answer. So much of our time and effort is expended in trying to make the care that patients receive better, but it’s hard to say exactly what better care means, and then to measure the results.
That’s the void into which the standardised patient was introduced: a mystery shopper, but for health facilities. People pretending to be patients isn’t new; they’ve been used as a training tool for decades, and medical students will be familiar with standardised patients in their practical exams. What’s new is the idea that we can send standardised patients to doctors in real settings, and send them undercover- to measure how health workers behave when they don’t know they are being observed.
This raises a number of ethical, as well as practical, questions. Is there a danger to our fieldworkers when we send them to seek care in often basic settings? Does the deception of doctors by standardised patients violate norms of informed consent in medical research? Are we wasting the limited time and resources of health facilities by sending healthy people for consultations and to buy drugs?
Potential risks and challenges
Those of us who use standardised patients in our work would argue that the risks are outweighed by the huge benefit of good data on quality of care which no other measurement tool can provide. As well as avoiding the Hawthorne effect (when people change their behaviour because they’re being observed), they allow the researcher to control the patient and disease mix, avoiding confounding and meaning that the treatment of rarer conditions be investigated without the need of long observation periods waiting for the right patient to turn up in a health facility . And importantly, in our new Editor’s choice paper we explain how many of the risks can be mitigated and minimised.
We also discuss the practical challenges faced in organising a standardised patient survey. How can healthy people pretend to be ill? You have to carefully select conditions so that obvious visual symptoms aren’t required. We suggest scenarios which work: a person who reports a history of coughing, night sweats and weight loss should be treated as a potential TB patient and referred for testing, regardless of whether they look unwell or the doctor can hear ‘crackles’ on the chest.
A creative approach is sometimes required; for example, if you’re interested in the quality of malaria case management, you can’t send real malaria patients into health facilities. What we do instead is train our standardised patients to say they have a fever and think they have malaria- and the thing we test is whether or not they’re offered a test, and whether the test gives the correct (negative!) result.
Still, it’s important to ask how good standardised patients are at convincing doctors – is it ever obvious from their behaviour and appearance that they aren’t real? One way to answer this question is to conduct a detection survey. After the standardised patient visits are all finished, researchers contact the health facilities and ask if they suspected any recent patients were actually standardised patients. If the answer is yes, we take more details so we can confirm whether their suspicions were correct. In a recent study I carried out in Tanzania, 5% of our standardised patients were detected by health workers, but this varied by setting; in busy urban facilities detection rates were lower, but in rural settings where the presence of an outsider was more obvious, they were more likely to be detected.
Conclusion
Yes, standardised patients aren’t a perfect tool- they can’t be used to measure all quality of care (it’s hard to imagine how they could check on the quality of antenatal and delivery care, for example!) and they’re more challenging to implement than patient exit interviews. But the depth and richness of the data they give us make them well worth the extra effort.
Access the paper in Health Policy and Planning.
Image credit: Jessica King
