When the patient is the expert: measuring patient experience and satisfaction with care
Elysia Larson a, Jigyasa Sharma b, Meghan A Bohren c & Özge Tunçalp d
a. Department of Biostatistics, Harvard T.H. Chan School of Public Health, 655 Huntington Ave. Building II, 4th floor, Boston, MA 02115, United States of America (USA).
b. Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, USA.
c. Centre for Health Equity, University of Melbourne School of Population and Global Health, Melbourne, Australia.
d. Department of Reproductive Health and Research, World Health Organization, Geneva, Switzerland.
Correspondence to Elysia Larson (email: email@example.com).
(Submitted: 10 October 2018 – Revised version received: 10 May 2019 – Accepted: 15 May 2019 – Published online: 28 May 2019.)
Bulletin of the World Health Organization 2019;97:563-569. doi: http://dx.doi.org/10.2471/BLT.18.225201
In 2018, three independent reports1–3 propelled quality to the forefront of global conversations on health policy and practice. The reports were published by the United States National Academies of Sciences, Engineering, and Medicine;1 the World Health Organization, the Organisation for Economic Co-operation and Development and the World Bank;2 and the Lancet Global Health Commission on High-Quality Health Systems in the Sustainable Development Goals (SDG) era.3 All three reports emphasized that improvements to quality of health care are necessary to achieve effective universal health coverage, a central theme within SDG 3, that is, to ensure healthy lives and promote well-being for all at all ages.4 The reports defined quality of care as care that is effective in maintaining or improving health and is person-centred, meaning that it is “respectful of and responsive to individual preferences, needs, and values.”1
Person-centeredness is an essential aspect of quality for two reasons. First, it is intrinsically important because individuals have the right to be treated with dignity and respect when they are using health-care services. Second, it is instrumentally important as person-centred care is associated with improved health-care utilization and health outcomes.5 The focus on person-centred measures is not new; the Institute of Medicine’s landmark 2001 report on quality of care brought attention to what was then referred to as patient-centred care.6 Since then, many person-centred measures have been proposed in the research literature. In theory, the measures allow quality improvement efforts to be evaluated and health-care systems to be held accountable to those whom they aim to serve. However, in practice, these measures are easy to misuse, since they all rely on the patient’s report of their visit. The utility of the measures is thus limited by lack of clarity and precision in designing different types of measures.
In this paper, we discuss the important distinction between two broad categories of person-centred measures of quality of care: patient experience (the interactions that patients have with the health system) and patient satisfaction (patients’ evaluation of the care provided relative to their expectations). We provide positive examples from the maternal and child health literature to illustrate how these measures can be used.
Using quality measures effectively
The first step is to define how person-centred measures of health relate to one another and then to discuss clear steps that researchers, policy-makers and implementers can take to ensure that the measures can be used effectively. Fig. 1 illustrates the inter-relationship between measures of patients’ experiences of and satisfaction with care. This scheme builds on the frameworks developed by the Lancet Global Health Commission3 and the World Health Organization vision for quality of care for pregnant women and newborns.7
Fig. 1. Framework for person-centred measures of health system quality and responsiveness
Patient experience is a process indicator and reflects the interpersonal aspects of quality of care received. This indicator is broadly composed of three domains: effective communication; respect and dignity; and emotional support.7,8 These domains may be modified directly by factors, such as facility characteristics (e.g. number of patients seen, ratio of health-care providers to patients, availability of services and resources); patients’ characteristics (e.g. sociodemographic characteristics, clinical history, prior health care-seeking behaviour); and the type of service (e.g. preventive or non-emergency care versus emergency care). These modifiers themselves will depend on the country and health system. Alternatively, these modifiers can influence patients’ experiences more indirectly by shaping patients’ needs, expectations and values.
In contrast, patient satisfaction is an outcome measure of a patient’s experiences of care, along with health outcomes and confidence in the health system (Fig. 1), reflecting whether or not the care provided has met the patient’s needs and expectations.3 A patient’s needs and expectations are dynamic and may evolve depending on the care provided and the patient’s awareness of both facility-level (e.g. case fatality rates) and individual outcomes (e.g. health outcomes or patient satisfaction). Outcomes, including patient satisfaction, can both affect and be affected by patients’ needs, expectations and values. A patient’s experience of care may have a direct impact on the patient’s satisfaction, as well as an indirect impact through affecting the patient’s needs, expectations and values, which in turn affect satisfaction. Previous research has suggested that broader social factors, including patient characteristics, such as age and education, can explain variations in patients’ experiences of care, ability to evaluate the quality of care received, and satisfaction with care.9 Patient’s expectations and interpretations of their experiences of care are further shaped by the broader societal, community, and family contexts.
To produce evidence that can be acted on, we encourage researchers and implementers, (e.g. nongovernmental organizations delivering quality improvement programmes, local governments who manage care or private health-care providers) to consider three issues when using person-centred measures.1–3 First, because measures of patient experience and satisfaction are distinct, they represent different underlying constructs and are affected by different factors, choosing a measure based on how that measure will be used is essential. Second, because the reference standard for person-centred measures is the patient’s report, considering how subjectivity may play a role in the reporting is important. Third, we need to know whether the measures have been previously tested and validated.
Choosing person-centred measures
Defining the purpose
Person-centred measures are useful to policy-makers and implementers for guiding and evaluating quality improvement efforts and for holding the health system and its stakeholders accountable to the communities they serve. The choice of measures will depend on the purpose of the measurements, for example whether they will be used for improving quality of care or health-system accountability.
As process measures, patient experience measures may be sensitive to differences in quality care across different providers, institutions or time, and thus can be used to identify gaps or evaluate changes in quality resulting from interventions or policies.10 For example, in East Africa patient experience measures have been used to quantify types of disrespectful care during childbirth and inform targeted interventions towards improving care.11,12
Patient experience indicators are currently used to target quality improvement for maternal health care across nine countries within the Network for Improving Quality of Care for Maternal, Newborn and Child Health.13 One focus of the network is to improve support for women during labour and childbirth from a companion of her choice (such as a partner, sister or friend). Companionship in labour is associated with both improved patient experience, such as more positive experiences of childbirth, and better health and well-being outcomes, such as increased spontaneous vaginal birth, shorter duration of labour and higher 5-minute Apgar scores for the baby.14,15 By monitoring indicators of patient experience, such as the proportion of women wanting a labour companion compared to those who have one present, countries will be able to target areas in need of quality improvement interventions and evaluate the success of those interventions.
Measures of patient satisfaction can also be outcome indicators that reflect whether the care provided meets an individual’s needs and expectations. Satisfaction measures are useful for identifying areas of service provision that are important to individuals, or when aggregated for communities. However, the use of these measures requires caution, as changes in satisfaction level may be due to changes in quality of care or patient demand, values or expectations. Exploratory or qualitative research could help determine the underlying causes of changes in satisfaction. The role of expectations is discussed further in the next section.
Holding health systems and policy-makers accountable to the communities they serve is an instance where measures of both patient experience and satisfaction may be useful. A study conducted in government-managed primary care clinics in rural United Republic of Tanzania in 2018 assessed the ability of public feedback, through posters announcing facility performance on indicators of patient experience to hold health-care providers accountable for the quality of care they deliver.16 By using measures of patient experience, the study provided specific areas that health-care providers could target for improvement. Alternatively, if satisfaction measures are used for accountability, poor satisfaction scores might drive providers or policy-makers to identify aspects of services that are valuable to patients but where service provision is failing.
Person-centred care should be measured with a clear purpose. Patient experience measures can be used to evaluate quality of care, while satisfaction measures can track patients’ (or communities’) responses to care but not actual changes to care itself. While both measures can be used to hold health systems accountable, it must be clear whether the aim of accountability is to provide high quality care (patient experience) or be responsive to expectations of the population (patient satisfaction).
The success of person-centred measures for quality improvement or accountability depends on how directly the indicators measure the underlying construct as intended. Understanding and assessing patients’ experiences and satisfaction, by definition, requires asking patients, but these self-reports will inherently introduce subjectivity. To address subjectivity when assessing person-centred measures, researchers must consider the phrasing of the questions, response choices and whether the questions account for patients’ expectations.
How questions are framed determines the degree of subjectivity of measures. Questions that ask patients to provide a direct report of what happened, as is the case for measures of experiences of care, tend to be less subjective than those that ask patients to evaluate or rate their experience, as in the case for all satisfaction measures (and some experience measures).17 Within the dimension of patient experience of communication, consider for example the following question, found in a patient experience survey in the United States of America: “Before giving you any new medicine, how often did hospital staff describe possible side effects in a way you could understand?”18 By including the phrase “in a way you could understand,” the question changes from asking patients to report on care that was provided (or not) to asking them to evaluate their experience of care. This inclusion makes the question more subjective, which is important if researchers want to understand if providers are communicating in an effective manner for patients across a diverse population. Patients have different needs, and health systems must be responsive and adaptive to these variations. Subjectivity is also important when considering whether results should be adjusted for case-mix, for example, age, health status or type of care, a common practice when looking at health outcomes. When patient characteristics are strongly associated with patient experience measures, adjustment may be useful if the goal is to compare across facilities.19 However, we believe that case-mix adjustment should not be used as a method to create more positive scores or dismiss lower scores, but rather to understand which populations may be having sub-optimal experiences of care and how their experiences can be improved.
Similarly, carefully weighing response options to questions is important. Choices such as “yes, always”, “yes, sometimes” and “no” are more objective than responses such as “excellent”, “good”, “fair” and “poor.” Whereas the former elicits a factual description, the latter relies on an individual’s perception. To interpret and act from these more subjective questions and response options, the evaluation may need to use vignettes, as described later, or obtain additional information on patient characteristics.20
Another example is the case of mistreatment of women during childbirth, such as being slapped or pinched. A woman’s report of mistreatment (objective measure) is likely to align with other objective measures, such as actual observations of mistreatment. Whereas her experiences of stigma and discrimination (subjective measure) may depend on her expectations of the health system, the provider or her lived experiences of discrimination (Bohren MA et al., University of Melbourne School of Population and Global Health, Australia, unpublished data, 2019). This difference is important to consider when observation is used as a method of data collection, because it is difficult for an independent observer to capture stigma and discrimination (Bohren MA et al., University of Melbourne School of Population and Global Health, Australia, unpublished data, 2019).21
Satisfaction with care is inherently shaped by an individual’s values, expectations and experiences, such as expecting to have a health-care provider who includes them in decision-making, and thus is a highly subjective measure requiring a nuanced approach to its interpretation. Patients’ expectations and values are affected both by factors that are related to the health system, for example availability of care, and by factors outside of the health system, such as an individual’s social identity.22,23 Since dynamic factors can affect an individual’s expectations for care, and in turn her or his satisfaction with care and utilization of services, we must understand expectations and how these may change.9
Expectations can be assessed qualitatively or quantitatively. Qualitative research is useful to elucidate ideas that may be previously unknown to the researcher or to explore a person’s values and preferences. Anchoring vignettes can be useful; these are hypothetical situations or stories that the respondents evaluate, perhaps by rating their satisfaction with the care described. Such vignettes provide an opportunity to quantify an individual’s expectations.24 Since all respondents evaluate the same situation, differences between respondents’ judgements can be considered as due to different expectations, thus allowing for adjustment of their rating of their own care. By further assessing factors that affect satisfaction, including individuals’ expectations, values and awareness of care available to them, we can draw more informed conclusions about why satisfaction may differ or change across time and populations.
Validating and testing
For person-centred measures, the reference standard is patients’ self-reporting. This type of reporting makes validation and testing of measures and scales different from many health outcomes that have an objective reference standard, such as using blood pressure to diagnose hypertension. However, self-reporting does not exempt researchers from validating, or at least testing, their measures. Using validated measures can help researchers address the issue of subjectivity. We suggest focusing efforts on construct and content validity.
First, researchers should consider how well their measures reflect an established model or theory (construct validity). It is important to consider if these general constructs have been measured or tested before and if previously validated measures are available.
Second, researchers should assess how well the measure represents the probable range of patient experience or satisfaction (content validity). This validation should occur in the context where the research is taking place, considering both the type of care (e.g. primary care or hospital) and geographical or cultural context. Qualitative research with the population of interest is appropriate for content validation; for example, conducting focus group discussions with patients to understand if the proposed tools are measuring the experiences or satisfaction.
Finally, researchers should consider how a measure performs across populations. Not all measures need to be reliable across populations and settings, but this is an important consideration when assessing the generalizability of findings. When reliability across settings is important, for example, to allow data to be compared across populations, using existing measures is beneficial. Acknowledging this need, the World Health Organization is currently developing a guidance document on validation of indicators for monitoring maternal and newborn health.
A demonstration of these steps has been shown for validation of a person-centred maternity care index.25,26 Researchers used a literature review and interviews with experts to assess content validity and then interviewers administered surveys with postpartum women in both India and Kenya to assess the criterion validity and reliability of the index. This validation work has resulted in a 30-item scale that can be used to facilitate measurements of and eventually improvements to person-centred maternity care within populations that are similar to those for which the scale was validated.
Similarly, a multimethod, multistep approach has been used for developing tools to measure the mistreatment of women during childbirth.27 Based on a global systematic review28 and primary qualitative research in four countries,27 the researchers developed two measurement tools: observations of labour and childbirth, and a postpartum survey with women. The researchers conducted validity testing with maternal health experts and women who recently gave birth, and adjusted the tools based on the responses of the population of interest.27
Despite this focus on quantitative measurement of person-centred care, qualitative methods, including interviews and focus group discussions, can help provide more in-depth and descriptive information about a patient’s experience.29 For example, in-depth interviews, where patients are asked to detail their experiences with a health service, are a useful tool for designing better services.29 Qualitative methods can also be used to help validate and enhance information gained from quantitative measures.
We are currently in an age of renewed attention to quality, a necessary component of care for universal health coverage to be effective in improving peoples’ lives.30 Measurement of person-centred care is a key step towards ensuring accountability and action and quality of care improvement.3 When measures do not have a clear purpose or are incorrectly specified or interpreted, they risk conveying an inaccurate and unreliable assessment of quality of care. This inappropriate use of measures can waste time and resources, both in the initial collection of data and in initiatives and policies resulting from poor measurement. We have outlined questions that can guide the generation of clear, actionable evidence. Clarity in thinking and precision in using person-centred measures will advance the science and practice of delivering respectful and effective health care.
- Crossing the global quality chasm: improving health care worldwide. Washington, DC: National Academies of Science, Engineering, Medicine; 2018. Available from: http://www.nationalacademies.org/hmd/~/media/Files/Report%20Files/2001/Crossing-the-Quality-Chasm/Quality%20Chasm%202001%20%20report%20brief.pdf cited 2019 May 24].
- Delivering quality health services: a global imperative for universal health coverage. Geneva: World Health Organization, Organisation for Economic Co-operation and Development and The World Bank; 2018. Available from: https://apps.who.int/iris/bitstream/handle/10665/272465/9789241513906-eng.pdf [cited 2019 May 24].
- Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, et al. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Health. 2018 11;6(11):e1196–252. http://dx.doi.org/10.1016/S2214-109X(18)30386-3 pmid: 30196093
- Resolution A/RES/70/1. Transforming our world: the 2030 agenda for sustainable development. In: Seventieth United Nations General Assembly, New York, 25 September 2015. New York: United Nations; 2015. Available from: http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E [cited 2019 May 19].
- Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013 01 3;3(1):e001570. http://dx.doi.org/10.1136/bmjopen-2012-001570 pmid: 23293244
- Institute of Medicine Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington: National Academy Press; 2001.
- Tunçalp Ö, Were WM, MacLennan C, Oladapo OT, Gülmezoglu AM, Bahl R, et al. Quality of care for pregnant women and newborns – the WHO vision. BJOG. 2015 Jul;122(8):1045–9. http://dx.doi.org/10.1111/1471-0528.13451 pmid: 25929823
- Valentine N, Darby C, Bonsel GJ. Which aspects of non-clinical quality of care are most important? Results from WHO’s general population surveys of “health systems responsiveness” in 41 countries. Soc Sci Med. 2008 May;66(9):1939–50. http://dx.doi.org/10.1016/j.socscimed.2007.12.002 pmid: 18313822
- Bleich SN, Ozaltin E, Murray CK. How does satisfaction with the health-care system relate to patient experience? Bull World Health Organ. 2009 Apr;87(4):271–8. http://dx.doi.org/10.2471/BLT.07.050401 pmid: 19551235
- Mant J. Process versus outcome indicators in the assessment of quality of health care. Int J Qual Health Care. 2001 Dec;13(6):475–80. http://dx.doi.org/10.1093/intqhc/13.6.475 pmid: 11769750
- Abuya T, Ndwiga C, Ritter J, Kanya L, Bellows B, Binkin N, et al. The effect of a multi-component intervention on disrespect and abuse during childbirth in Kenya. BMC Pregnancy Childbirth. 2015 09 22;15(1):224. http://dx.doi.org/10.1186/s12884-015-0645-6 pmid: 26394616
- Kujawski SA, Freedman LP, Ramsey K, Mbaruku G, Mbuyita S, Moyo W, et al. Community and health system intervention to reduce disrespect and abuse during childbirth in Tanga Region, Tanzania: a comparative before-and-after study. PLoS Med. 2017 07 11;14(7):e1002341. http://dx.doi.org/10.1371/journal.pmed.1002341 pmid: 28700587
- Adeniran A, Likaka A, Knutsson A, Costello A, Daelmans B, Maliqi B, et al. Leadership, action, learning and accountability to deliver quality care for women, newborns and children. Bull World Health Organ. 2018 Mar 1;96(3):222–4. http://dx.doi.org/10.2471/BLT.17.197939 pmid: 29531422
- Bohren MA, Hofmeyr GJ, Sakala C, Fukuzawa RK, Cuthbert A. Continuous support for women during childbirth. Cochrane Database Syst Rev. 2017 07 6;7:CD003766. pmid: 28681500
- Sandall J, Tribe RM, Avery L, Mola G, Visser GH, Homer CS, et al. Short-term and long-term effects of caesarean section on the health of women and children. Lancet. 2018 10 13;392(10155):1349–57. http://dx.doi.org/10.1016/S0140-6736(18)31930-5 pmid: 30322585
- Mbatia R, Cohen J, Zuakulu M, Bukuku A, Chandarana S, Eliakimu E, et al. Basic Accountability to Stop Ill-Treatment (BASI): study protocol for a cluster randomized controlled trial in rural Tanzania. Front Public Health. 2018 09 24;6:273. http://dx.doi.org/10.3389/fpubh.2018.00273 pmid: 30320053
- Ahmed F, Burt J, Roland M. Measuring patient experience: concepts and methods. Patient. 2014;7(3):235–41. http://dx.doi.org/10.1007/s40271-014-0060-5 pmid: 24831941
- Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey [internet]. Baltimore: Centers for Medicare & Medicaid Services; 2018. Available from: http://www.hcahpsonline.org/en/survey-instruments/ [cited 2019 May 24].
- Paddison C, Elliott M, Parker R, Staetsky L, Lyratzopoulos G, Campbell JL, et al. Should measures of patient experience in primary care be adjusted for case mix? Evidence from the English General Practice Patient Survey. BMJ Qual Saf. 2012 Aug;21(8):634–40. http://dx.doi.org/10.1136/bmjqs-2011-000737 pmid: 22626735
- Gonyea RM. Self-reported data in institutional research: review and recommendations. New Dir Institutional Res. 2005;2005(127):73–89. http://dx.doi.org/10.1002/ir.156
- Freedman LP, Kujawski SA, Mbuyita S, Kuwawenaruwa A, Kruk ME, Ramsey K, et al. Eye of the beholder? Observation versus self-report in the measurement of disrespect and abuse during facility-based childbirth. Reprod Health Matters. 2018;26(53):107–22. http://dx.doi.org/10.1080/09688080.2018.1502024 pmid: 30199353
- Dawn AG, Lee PP. Patient expectations for medical and surgical care: a review of the literature and applications to ophthalmology. Surv Ophthalmol. 2004 Sep-Oct;49(5):513–24. http://dx.doi.org/10.1016/S0039-6257(04)00111-0 pmid: 15325196
- Peck BM, Asch DA, Goold SD, Roter DL, Ubel PA, McIntyre LM, et al. Measuring patient expectations: does the instrument affect satisfaction or expectations? Med Care. 2001 Jan;39(1):100–8. http://dx.doi.org/10.1097/00005650-200101000-00011 pmid: 11176547
- King G, Murray CJL, Salomon JA, Tandon A. Enhancing the validity and cross-cultural comparability of measurement in survey research. Am Polit Sci Rev. 2004;98(1):191–207. http://dx.doi.org/10.1017/S000305540400108X
- Afulani PA, Diamond-Smith N, Golub G, Sudhinaraset M. Development of a tool to measure person-centered maternity care in developing settings: validation in a rural and urban Kenyan population. Reprod Health. 2017 09 22;14(1):118. http://dx.doi.org/10.1186/s12978-017-0381-7 pmid: 28938885
- Afulani PA, Diamond-Smith N, Phillips B, Singhal S, Sudhinaraset M. Validation of the person-centered maternity care scale in India. Reprod Health. 2018 08 29;15(1):147. http://dx.doi.org/10.1186/s12978-018-0591-7 pmid: 30157877
- Bohren MA, Vogel JP, Fawole B, Maya ET, Maung TM, Baldé MD, et al. Methodological development of tools to measure how women are treated during facility-based childbirth in four countries: labor observation and community survey. BMC Med Res Methodol. 2018 11 15;18(1):132. http://dx.doi.org/10.1186/s12874-018-0603-x pmid: 30442102
- Bohren MA, Vogel JP, Hunter EC, Lutsiv O, Makh SK, Souza JP, et al. The mistreatment of women during childbirth in health facilities globally: a mixed-methods systematic review. PLoS Med. 2015 06 30;12(6):e1001847, discussion e1001847. http://dx.doi.org/10.1371/journal.pmed.1001847 pmid: 26126110
- de Silva D. Measuring patient experience. Evidence scan no. 18. London: The Health Foundation; 2013. Available from: https://www.health.org.uk/sites/default/files/MeasuringPatientExperience.pdf [cited 2019 May 24].
- Berwick DM, Kelley E, Kruk ME, Nishtar S, Pate MA. Three global health-care quality reports in 2018. Lancet. 2018 07 21;392(10143):194–5. http://dx.doi.org/10.1016/S0140-6736(18)31430-2 pmid: 30043742