State of the art metrics for nursing: a rapid appraisal more

State of the art metrics for nursing: a rapid appraisal Peter Griffiths with Simon Jones, Jill Maben and Trevor Murrells 2008 State of the art metrics for nursing Acknowledgements This work was commissioned and supported by the Department of Health in England as part of the Policy Research Programme which provides funding to the National Nursing Research Unit. The views expressed are those of the authors, not of the Department of Health. We thank the members of the Nursing Outcome Measures task and finish group chaired by Professor Anne Marie Rafferty and convened by Professor Dame Christine Beasley, the Chief Nursing Officer for England. These experts’ insights and experiences informed us as we undertook this review to support their work. The work presented here is our appraisal of the literature and is not intended to represent the full breadth of work undertaken by that group. We particularly thank members of the group and others who provided support and comments, and Janice Sigsworth, the Deputy Chief Nursing Officer who was directly responsible for commissioning this report. Contact address for further information: National Nursing Research Unit King’s College London James Clerk Maxwell Building 57 Waterloo Road London SE1 8WA nnru@kcl.ac.uk http://www.kcl.ac.uk/schools/nursing/nnru i State of the art metrics for nursing ii State of the art metrics for nursing Table of contents Acknowledgments Foreword Overview Introduction Possible indicators What makes a good indicator? Evidence base: associations between nursing and outcomes State of the art Conclusion and recommendations References Appendix 1: Core data sources Appendix 2: Indicators identified i v 1 2 6 10 14 17 23 26 31 34 iii State of the art metrics for nursing iv State of the art metrics for nursing Foreword The search for outcome measures for nursing has a long and distinguished history tracking back to Florence Nightingale herself. Her epidemiological approach to outcomes research and measurement has attracted admiration from statisticians and epidemiologists alike. Despite this prestigious pedigree, progress on refining metrics for the outcomes of nursing care has been slow. The prioritisation of quality within the Next Stage Review of the National Health Service in England and the commitment to hold trusts accountable for and to reward quality of care promises not only a renewed but relentless focus on quality of care. This report reviews the status of the evidence base on nursing metrics and provides a road map and set of recommendations to take nursing forward. I am grateful to members of the task and finish group who helped to identify candidate metrics in key domains of care and provided examples of good practice from their organisations. As Professor Griffiths and his team demonstrate, while there is much to be commended in such practice there is still much more to do. Developing metrics is only the first step in building a robust infrastructure for implementation and fully integrating nursing into the governance and management of the NHS at all levels. Such metrics can then enable the public to make informed decisions about their care based on criteria which matter to them as well as to managers and clinicians. We have a unique and unprecedented opportunity to make the quality of nursing care count. This report is an important step in that process. Professor Anne Marie Rafferty Dean and Head of the Florence Nightingale School of Nursing & Midwifery at King’s College London Chair of the Nursing Outcomes task and finish group v State of the art metrics for nursing vi State of the art metrics for nursing Overview • The Next Stage Review commits to quality measurement that reflects the compassion, safety and effectiveness of nursing care. As such measurement provides both a challenge and an opportunity for the profession, this report reviews ‘state of the art’ nursing quality measurement. Many possible indicators and existing indicator sets measure nursing’s contributions. Among the most widely used indicators are safety measures such as failure to rescue (death among patients with treatable complications), falls, healthcare associated infection and pressure ulcers. Neither effectiveness (positive contributions to well-being) nor compassion (elements of patient experience) are strongly represented in the existing measures. We did not set out to focus specifically on acute general inpatient nursing care, but examples from this setting dominate both indicators and evidence reviewed because there has been more development in these areas. Many themes apply equally to other areas even if specific indicators differ, and the lessons learned in acute care can assist in developing indicators for all specialties and care settings. To be useful, indicators must be measurable with available data at reasonable cost. There must be evidence that the quality or quantity of nursing substantially contributes to changes measured by the indicator. The indicator must be recognised as important (by the public, managers and nurses) and nursing’s contribution must be recognised (by nurses and others). Nurses must have responsibility for actions leading to outcomes in terms of legitimate authority, self-perception and sphere of practice. Measures should be chosen to minimise the risk of gaming, where improving performance on specific indicators detracts from overall improvement. Measures focussing on the performance of care (process) rather than outcome are most vulnerable to gaming. Not all existing indicators meet these requirements, and considerable work will be needed to develop practical, valid and useful indicators. Strong evidence supports an association between nurse staffing levels and mortality, but mortality is determined by many causes and is not likely to be a useful quality measure for nursing. This report identifies ‘best bets’ for indicator development, including measures of safety, effectiveness and compassion. Health care providers should form a quality coalition, facilitated by stakeholders includinf the Royal College of Nursing and The NHS Confederation, to share best practices and to move toward standard measurement of important nursing indicators. A programme of indicator development must include development of technical specifications for indicators, research to validate them and, crucially, patient involvement in identifying metrics for compassion. The development of metrics and the establishment of a National Quality Board and local quality observatories present a substantial opportunity for nursing to equip itself with the tools needed to deliver excellent care, and these initiatives should be welcomed within the profession. • • • • • • • 1 State of the art metrics for nursing Introduction This work was commissioned to inform the Nursing Outcome Measures task and finish group chaired by Professor Anne Marie Rafferty and convened by the Chief Nursing Officer for England as part of the nursing contribution to Lord Darzi’s Next Stage Review of the National Health Service. The group was tasked with identifying mechanisms for giving nurses tools, training and support to improve quality of care across the country, including: • • • Evidence-based metrics to measure nurse-delivered outcomes and patient experiences; National publication of performance data to identify best-practice examples and help nurses benchmark and improve their performance; and “Ward-to-board” accountability for the quality of nursing care. The Next Stage Review makes a commitment to develop an indicative set of metrics for nursing1 that comprises of indicators of quality reflecting the issues of safety, effectiveness and compassion identified by the task and finish group. This work will support a wider NHS initiative to establish regional quality observatories and a National Quality Board that will oversee the development of a quality measurement framework for all clinical services. Indicators serve to foster understanding of a system and how it can be improved, and to monitor performance against agreed standards or benchmarks. Crucially, indicators provide a mechanism by which care providers can be accountable for the quality of their nursing services. Accountability for nursing quality exists at many levels, from the point of care (where individual nurses are accountable to clinical managers and patients) to senior management, commissioners and beyond. While “ward–to-board” accountability for care is frequently referred to, a still wider view of accountability includes both the public and policy-makers, including health service users, the general public and funders of care2. A measuring system is needed, with a set of indicators that can: • • • quantify trends and characteristics; describe performance in achieving health service goals (in this case, elements to which nursing strongly contributes); and provide information to improve nursing care. Nursing-sensitive indicators The group was tasked with finding measures of “nurse-delivered outcomes and patient experience”. We take this to mean measures that directly reflect nursing’s end results in terms of impact upon patients. A large body of work describes quality measurement systems that focus on multiple aspects of quality in the nursing process3 or on nursing outcomes in terms of activities completed4. While occasionally the relationship between a particular process and an outcome is so strongly established that a measure of the process 2 State of the art metrics for nursing may suffice or even be preferable as a proxy indicator, our reference point has been patient outcome (including experience). Patient outcomes and experiences vary for many reasons and reflect the work of multiple professions. Indeed, in many cases the greatest determinant of outcome is the patient, whether because of underlying health status, behaviour or aspects of the wider social environment. In considering nursing-sensitive outcomes and experience, we must identify elements of variation that can be attributed largely to nursing care quality. To do this we must seek evidence of correlation with nursing as well as evidence that this correlation is a plausible consequence of variation in nursing rather than other factors. Determining such plausibility requires professional knowledge of possible mechanisms and technical knowledge, such as research evidence that adjusts for confounding factors. These types of indicators can be used5: • • • to improve quality in local settings by monitoring and managing performance; to support policy analysis and strategic decision-making, including commissioning, reimbursement systems and accreditation; and to research the role of nursing care in determining patient safety outcomes by examining structure-outcome, process-outcome and structure-process-outcome relationships. The first two uses signal that different information may be required and useful at different levels. The third purpose of indicators is not the immediate concern of this report, but such research is necessary to develop indicators fit for the other uses. Context This initiative comes at a time of widespread public perception that nursing quality is sometimes poor and lacking in essential elements6-8, as represented in the oft-repeated accusation of nurses being ‘too posh to wash’. Recent reports on infection outbreaks in UK hospitals have highlighted situations where underlying issues of nursing care quality were given a low priority in the face of competing productivity targets9-11. Concerns about nursing care are not limited to the general public or to the UK. There is ongoing professional concern that nursing’s contribution to quality health care is underrecognised, leaving nursing services vulnerable to cost-reducing efforts12 ,13. Nursing’s contribution to quality care is not consistently recognised; an Audit Commission report on hospital staffing variations14 concluded early in this decade that “Unless and until trusts that spend more [on staffing] can demonstrate a clear link with the quality of care that is delivered, movement towards a more even allocation of resources seems reasonable both for patients and staff.” (p15) This presumes there is no link to quality unless otherwise demonstrated, yet a document from the same Audit Commission programme15 states: “It is difficult to … avoid the conclusion that they [variations in staffing] must result in differences in the quality of care available to patients in different trusts and on different wards.” (p3) 3 State of the art metrics for nursing While varying nursing quality is clearly a concern, tangible ways to demonstrate nursing’s contribution to quality care are less clear. In the UK, responses to healthcare acquired infections may have been instrumental in reasserting nursing’s central and fundamental role in providing a safe environment for care16. Yet this represents a narrow view of nursing’s potential contribution to patients’ experiences, health and well-being. A burgeoning number of recent studies explore the impact of variation in the quality and quantity of nursing care on a wide range of outcomes and experiences17. An increasing number of indicator sets identified as nurse-sensitive5 are used by national and local governance or quality improvement programmes. The task and finish group examined differing systems used in a variety of NHS organisations. By bringing the nursing contribution to the fore, the Next Stage Review and the resulting commitment to measure the compassion, safety and effectiveness of nursing care provide a challenge and an opportunity for the profession. In this light it is timely to explore how nursing might demonstrate its contribution to quality outcomes and patient experiences, as well as how quality indicators can hold all service providers accountable. Developments must build on existing evidence and initiatives for consistency across settings and to ensure that best practice is used. This paper explores potential nursing-sensitive indicators identified from published literature and indicator sets currently in use. The requirements of a good set of indicators for nursing are explored and evidence for indicators’ validity is considered through an examination of evidence linking nursing contributions and patient outcomes. The conclusion assesses the current state of the art in nursing-sensitive indicators. We did not set out to focus on acute general inpatient nursing care, but both the indicators and the evidence reviewed are dominated by examples from this setting because there has been more indicator development in these areas. While the state of the art may be more advanced in acute care, many themes apply equally to other areas even though specific indicators may differ. Certainly the lessons learned in acute care can usefully underpin development of indicators for all specialties and care settings. 4 State of the art metrics for nursing Box 1. Introduction: summary • • • Evidence abounds of public and professional concerns that nursing care quality is variable and sometimes poor. Nursing care quality has often failed to receive high level-attention in the face of competing productivity targets. There is a demand for measures whereby nursing can demonstrate and be held to account for its contribution from point of care to the board room. Such measures also are useful to all sectors of society benefiting from and making policy for health care services, ranging from the public to politicians. The Next Stage Review and resulting commitment to quality measurement reflecting nursing care’s effectiveness, safety and compassion provides both a challenge and an opportunity for the profession. This paper explores potential nurse-sensitive indicators identified from published literature and sets of indicators currently in use. This evidence base is used to determine the current ‘state of the art’ of metrics for nursing. • • 5 State of the art metrics for nursing Possible indicators A wide range of potentially measurable indicators of nursing care quality can be indentified from nurse-sensitive outcomes. These outcomes are aspects of patient experience, behaviour or health status (patient outcomes) that are determined in whole or part by nursing care received and variations in its quality or quantity. The precise construct is variously defined, and while most definitions tend to focus upon outcomes as results of specific nursing interventions18 ,19, others have emphasised system characteristics20 such as team functioning, staffing levels and skill mix as important determinants of outcome. Quality indicators also can derive from known or widely presumed links between nursesensitive outcomes and nursing interventions or structural characteristics. For example, use of a nutritional risk assessment might be identified as an indicator of quality because it is identified as a nursing intervention leading to improved outcomes (improved nutrition). Similarly, workforce variables such as staff satisfaction or skill mix might be used as indicators because of such variables’ known or presumed relationships with important patient outcomes. Data sources used A number of sources yielded possible indicators for nursing, including Doran’s review of nurse-sensitive indicators21 and recent systematic reviews linking the ward environment, nurse staffing and patient outcomes22-25. Various indicator systems were selected on the basis of their strong potential to give a high coverage of nurse-sensitive indicators, and on the basis of their prominence and advocacy within high-profile bodies such as the American Nurses Association, the UK’s NHS and the US Joint Commission for the Accreditation of Healthcare Organisations (JCAHO). This coverage was generally identified because the indicator set was proposed as nurse–sensitive. Some relevant indicator sets were developed for care settings where nursing takes a significant active lead (e.g. community and home healthcare), even where the set was not proposed for nursing per se. Appendix 1 lists key sources used. To this set were added indicators identified using database and web searches of terms such as nurse-sensitive indicators and metrics. This yielded links to additional indicator sets and locally developed ‘dashboards’ of indicators. The intention was not to achieve a comprehensive list, but to give an accurate overview. The authors explored these issues within the task and finish group and gleaned further examples from members. We stopped searching when new searches failed to identify new domains and no new individual indicators emerged that were substantively different from those already identified. Few indicators were identified solely on our general web search, suggesting that we have successfully identified the bulk of the available content although alternative specifications of the indicators we have identified might exist. We have not 6 State of the art metrics for nursing considered systems such as the Nursing Outcomes Classification System26 because they intend to provide a comprehensive taxonomy for recording the outcomes of specific interventions rather than to serve as general indicators of quality. Indicators identified Appendix 2 lists the broad range of indicators identified. Depending on the level of precision of definition, a list of nursing quality indicators could run into the hundreds. The majority of sources identified were specific to acute general hospital care, although many indicators could apply across other settings in adapted forms. The level of definition offered varied considerably between sources. Reviews necessarily were less precise in specifying outcomes and data-collection protocols. Variables generally were clearly defined, although some (e.g. ICN*) defined variables in very general terms. Existing indicator sets were generally the product of an extensive development process and offered precise specifications for data (e.g. relevant ICD codes, exclusions and riskadjustment models such as NQF). Other sources such as the AUKUH listed specific indicators, but offered little in terms of data specification or operational definitions while the ICN indicated only broad areas. Essence of Care benchmarks conversely offered considerable conceptual detail but little detail on measurement. The broad topics most specific to nursing are incorporated here. A range of patient outcomes was identified including aspects of knowledge, function (including instrumental activities of daily living and continence), nutrition, experience (including communication, satisfaction and complaints), preventative care such as vaccinations, safety outcomes (e.g. failure to rescue, falls, infections, medication errors, mortality, pressure ulcers), symptoms such as pain and dyspnoea and utilisation outcomes such as hospital stay and unplanned admissions. Also identified as possible indicators were processes directly linked to these outcomes (e.g. pain assessment, risk assessment) or relating to general aspects of quality such as planning and care coordination. Nursing workforce characteristics featured heavily as well: possible indicators included staffing levels, skill mix (including qualified nurses’ levels of educational preparation), team expertise, staff turnover rates and indicators of team functioning, such as interprofessional relations and perceptions of practice environment quality. Staff outcomes such as wellbeing and injury rates also were identified. Table 1 lists indicators that four or more sources identified, and the assignment given to them by the task and finish group in terms of safety, effectiveness and compassion**. Similar indicators are grouped for brevity, even though sources’ precise definitions * See Appendix 1 for definition of the acronyms used to refer to indicator sets ** Safety refers to adverse effects of care, effectiveness refers to positive benefits and compassion refers to aspects of patient experience such as perceived dignity, respect and quality of communication. Appendix 2 gives a fuller list of indicators identified; these are classified into topic areas using an earlier schema focussing on content devised before the task and finish group devised its broad classification 7 State of the art metrics for nursing and specifications of indicators may differ. While the number of sources that identify a particular indicator hint at its level of support, this should not be taken as a literal measure because our sample of sources was somewhat arbitrary. In particular, we discuss below why number of sources does not indicate evidence strength. Table 1. Most frequently identified indicators Indicator Pressure ulcer Failure to rescue Staffing levels Falls Health care associated infection: pneumonia Staff satisfaction and well-being Health care associated infection: urinary tract infection Staffing, skill mix Medication administration errors Mortality Practice environment/perceived quality Satisfaction with (nursing) care Sickness rates Smoking advice Staffing bank or agency use Communication Staff experience, knowledge, skills and expertise Health care associated infection: surgical wound Instrumental activities of daily living and self–care Perception of adequate staffing Use of restraints Area Safety Safety Effectiveness Safety Safety Effectiveness Safety Effectiveness Safety Safety Effectiveness Compassion Effectiveness Effectiveness Effectiveness Compassion Effectiveness Safety Effectiveness Safety Compassion Number of sources 11 9 9 8 8 7 6 6 5 5 5 5 5 5 5 4 4 4 4 4 4 Adverse events dominate the field of nurse-sensitive indicators5. Despite our inclusion of broader-based reviews such as Doran’s21 this remains the case here and is reflected by the high proportion of indicators of patient safety, particularly in relation to outcomes*. Most effectiveness indicators stem from a decision to interpret structural staffing outcomes * The relative balance between the types of indicators is in part a product of the level at which they are grouped. For example, the OASIS indicators include a detailed list of instrumental activities of daily living which can be considered as several indicators or, as we did, one (IADL). 8 State of the art metrics for nursing or other measures of workforce experience as indicating an ‘effective’ workforce under this schema. Such indicators could equally well have been identified as safety measures, since much of the available evidence relates to the link between staffing and patient safety. Interpretation of items such as satisfaction under ‘compassion’ rests upon the potential of the largely patient reported measures (including satisfaction measures) to incorporate elements of patient experience that reflect upon this dimension. Even allowing for their differing natures and sources, the lack of overlap Savitz5 noted remains. Indeed, agreement between sources is exaggerated by our grouping of similar indicators. However, there does seem to be a degree of consistency in identifying as indicators in acute-care settings: • • • • • failure to rescue (death among patients with treatable complications27); falls (sometimes defined in terms of resulting injury); health care associated infection (although the precise infections/specification vary considerably); pressure ulcers; and staffing levels. Nonetheless, the range of candidate indicators and lack of consistency in their content and precise definition make it essential to identify desirable indicator characteristics. Box 2. Possible indicators: summary • • Many possible indicators and indicator sets exist, mainly but not exclusively in acute care. However, little consistency exists between indicator sets. The most widely supported outcome indicators include failure to rescue (death among patients with treatable complications), falls, healthcare associated infection and pressure ulcers. Other commonly advocated indicators were patient experience, including communication with staff and support in activities in daily living; medication administration errors; the practice environment’s perceived quality; and workforce aspects including staffing levels, skill mix and measures of well-being or satisfaction. • 9 State of the art metrics for nursing What makes a good indicator? The National Quality Forum require indicators to be:important, scientifically sound useable and feasible28. Importance is defined in terms of • • • • • • • • • national goals impact of outcomes on individual service users the societal burden of disease availability of strategies for improvement, substantial variation in quality, or quality which is consistently substandard. precise specificationof the indicator reliability and validity of measures and adequacy of risk adjustment. Scientific criteria include Usability is the extent to which intended audiences can understand results and are likely to find them useful in decision-making while feasibility relates to the ability to obtain quality data in a timely manner with a demand on resources that is proportionate to benefits.These criteria are generally reflected in the NHS Institute for Innovation and Improvement’s Good Indicator Guide2, which specifies that indicators should describe “as much about a system as possible in as few points as possible”. (p5) So nursing indicators must be measurable with available data at reasonable cost, coding and recording must be consistent and complete and measures must be valid. If an indicator is to be used to represent the quality of nursing it must be attributable to nursing in a number of senses, including: • • • • evidence of sensitivity to nursing recognition of the phenomenon’s importance recognition as a nursing contribution (owned by nurses and acknowledged by others) and recognition as nurses’ responsibility in terms of legitimate authority, self-perception and sphere of practice Usability requires that there is sufficient knowledge to inform action since identification of a particular level of achievement is not useful unless a strategy for improvement can be identified. Indicators must apply to many patients and types of clinical services if they are used to represent general quality across specialities. Similarly, variation in a nursingattributable outcome must be substantial if the outcome is used to indicate nursing quality as opposed to care in general. If an outcome is used to compare diverse populations and 10 State of the art metrics for nursing clinical settings, it must be possible to adjust for differing baseline risks to ensure fair comparison. The evidence behind any nursing process or structural indicator also must establish that it is linked to outcomes29,30. The choice of indicators must minimise the risk of gaming or perverse incentives. Collecting data can be seen as unwarrantedly intrusive and burdensome at any organisational level31 and proper organisational support and infrastructure for data collection and feedback are essential. To maintain commitment, attention should focus not just on the initial burden but also on the information’s perceived value. Timely and informative data must be available to provide behavioural incentives and if necessary allow remedial action to be taken. The challenge is to turn the data into “actionable information” 31 so feedback mechanisms and formats such as the increasingly popular graphics-based ‘dashboards’ and traffic light systems are important31,32. It is essential that actionable information be acted upon. As one member of the task and finish group noted: “…outputs have to be used and the emerging issues addressed, otherwise staff very quickly become cynical.” Perhaps the most difficult balance to strike is between the data collection burden and the need to ensure that performance on indicators represents broad achievement of the goals of nursing or processes that deliver those goals. The more indicators that are collected the greater the data collection burden but the danger of focussing on a few narrowly defined indicators lies in the creation of perverse incentives33 where maximising performance on indicators detracts from overall performance in other aspects or changes performance in relation to the indicator that invalidates it by removing its relationship to overall quality. Numerous examples of such gaming behaviours have been noted within the English NHS34. One notable example was the 48-hour target for GP appointments that led to practices refusing to offer appointments more than 48 hours in advance34. Indicator selection must therefore consider the potential for gaming and seek to minimise this. Essentially the aim is to identify a relatively small number of indicators that still relate clearly to the multifaceted and somewhat elusive concept of quality nursing care. All indicator sets are incomplete; the challenge is to identify indicators that are important in themselves and also strong indicators of overall quality. Bevan and Hood note that indicators must be selected from a narrowly defined subset of target areas where both data and measures of sufficiently high quality are available34 (see Figure 1). The fundamental issue that indicators are primarily decided by available measures cannot be avoided. The limitations of available good measures may point towards using imperfect measures, even though this may in effect “open a can of worms”34 because such measures provide an incomplete and inaccurate picture. While pragmatic decisions may influence the choice of indicators and imperfect measures may be used in sufficiently important areas, such decisions may create additional opportunities for gaming and perverse incentives. 11 State of the art metrics for nursing Figure 1. The relationship between the system and indicators (after Bevan & Hood 2006) Total domain Subset of interest Subset with imperfect measures Subset with good measures This risk is illustrated by two examples relating to possible nursing indicators. One relates to the use of routinely collected data for indicators such pressure ulcers or HCAI. Such secondary diagnoses are notoriously poorly recorded35. While the chances of a patient developing pressure ulcers may relate to the quality of nursing care, early detection and proper documentation of pressure ulcers is also a marker of quality care which could lead to higher rates of recording in good-quality settings than in lower-quality ones. Thus a perverse incentive might be created for lower levels of surveillance and documentation among highperforming teams. It is certainly hard to see the incentive for poor-performing teams to increase their reporting. Risk adjustment for pressure ulcers is further problematic and risk assessments are not routinely conducted, centrally recorded or particularly discrmininating36 so comparisons between units (such as wards or hospitals) are difficult to interpret. In the face of such problems, measures of process seem an appealing alternative. But the apparent ease that many process measures offer in resolving the difficult measurement issues associated with outcomes may be outweighed by the choice of a process indicator because it is measurable, but which has doubtful links to patient outcomes or experience. Process data are rarely routinely available5 (which removes a potential advantage) and it is hard to capture in routine audits the complex interpersonal care processes involved in many nursing interventions. Thus indicators such as completed pressure ulcer risk assessment tools may come to be adopted even though token compliance is possible (for example, if completion of the form takes priority above identifying need). Furthermore, the specific process may be adopted because it is measurable rather than because it is known to be effective. In the case of formal pressure ulcer risk assessment tools, accuracy and effectiveness are both uncertain36. Resolving these issues can never be straightforward and must be a matter of informed judgement about potential benefits from deploying an imperfect indicator as opposed to no 12 State of the art metrics for nursing indicator. Evidence of links between nursing and outcome must be further examined, since without this knowledge any indicator has questionable utility. Box 3. What makes a good indicator: summary • • • • • • • • Indicators must be measurable with available data at reasonable cost. There must be evidence of variability associated with nursing and this variability must be substantial. For process or structure measures, evidence must support links to important outcomes. The indicator must be recognised as important (by the public, managers and nurses) and the contribution of nursing must also be recognised by nurses and others. Nurses must have responsibility for actions that lead to the outcome in terms of legitimate authority, self-perception and sphere of practice. There must be sufficient knowledge to inform remedial action. Measures should be chosen to minimise the risk of gaming, where improving performance on the indicators detracts from overall performance. Measures, especially measures of outcome, generally need to be risk adjusted to ensure comparability across settings. 13 State of the art metrics for nursing Evidence base: associations between nursing and outcomes Kane et al.22 provide the strongest single source of evidence for a link between nursing and outcomes. This systematic review examines the impact of a general nursing variable, the quantity of nursing care available, and assesses the extent to which nursing influences the indicators. The review included 96 studies linking nurse staffing to patient outcomes. Increased RN staffing was associated with lower hospital-related mortality (per additional full-time equivalent nurse per patient day) in: • • • intensive care units (odds ratio 0.91, 95% confidence interval 0.86–0.96) surgical units (odds ratio 0.84; 95% confidence interval 0.80–0.89) medical patients (odds ratio 0.94; 95% confidence interval 0.94–0.95) Table 2. variance associated with outcomes (Kane 2007) Indicator Surgical wound infection (surgery) Unplanned extubation (ICU) Hospital acquired bloodstream infection (surgery Cardiac arrest (all groups) Length of stay (ICU, surgery) Hospital acquired pneumonia (all groups) Failure to rescue (surgery) Respiratory failure (all groups) Mortality % variance associated with nurse staffing 85% 51% 36% 28% 24% 19% 16% 6% 4.2% Hospital-wide, 4.2% of variation in mortality is attributable to nursing, assuming causality. Unsurprisingly, other variables show stronger association with nursing (Table 2), the strongest association being with surgical wound infection. The variation associated with nurse staffing is 85%. This seems implausibly high and raises the question of the extent to which the associations observed may be a result of confounding, where there is a strong association between nurse staffing and other patient or care quality variables which are wholly or partly responsible for the outcome observed. There is certainly a possibility that high nurse staffing is associated with other hospital characteristics that are in turn associated with quality of care (the ‘magnet’ phenomenon37). Kane takes a cautious epidemiological approach to interpreting causation. Causality is supported by evidence of a ‘dose-response relationship’ which appears curvilinear as increased staffing at the highest levels yields diminishing returns. The evidence is consistent across study designs (including the use of risk adjustment) and while different designs give some modifications in estimates, overall conclusions are unchanged. Evidence 14 State of the art metrics for nursing supports a temporal association as some studies demonstrate that adverse outcomes occur immediately after periods of low staffing, although there is a lower estimate of effect on failure to rescue in studies assessing this temporal association. There was no consistent association between nurse staffing and patient falls, pressure ulcers or urinary tract infections, outcomes among the indicators most frequently identified. Although there is some evidence to support these21, their appearance on lists of indicators is clearly more predicated upon a convincing theoretical proposition than the strength of the evidence. Of the four most prominent outcome indicators identified (failure to rescue, falls, HCAI [pneumonia], pressure ulcers), only failure to rescue and HCAI pneumonia are supported by Kane’s review. This creates a dilemma. Kane’s evidence of cause would be more convincing if these outcomes, which are presumed to be the most nurse-sensitive, were most closely associated with nurse staffing. Limitations of available data sets, and in particular poor coding of secondary diagnoses in the administrative databases used in most of the larger studies, provide a possible explanation for this finding. However, this is a presumption; even if it is correct and the outcome is indeed linked to nursing, it appears from Kane’s study that measures of these outcomes derived from administrative databases are not necessarily valid indicators of nursing quality. Although most studies reviewed were from North America, it is unlikely that coding of secondary diagnoses is better in the UK; indeed it is almost certainly worse. Studies on UK nurse staffing and patient outcomes have not utilised these outcomes38 and in general coding of complications and secondary diagnoses, although improving, is starting from a weaker baseline than in the US39. This evidence of a link between nurse staffing and patient outcomes supports using staffing and workforce variables as indicators. Evidence is stronger for levels of RN staffing and high skill mix than for total nurse staffing23,40. Other evidence tends to support a range of workforce variables such as job satisfaction and turnover21,22,24,38,41. Staff rated practice environment quality is linked to patient outcomes in a number of studies in Katz’s review and others24,42-45. Staffing levels, staff satisfaction and staff perception of the quality of the practice environment were among the most frequently identified indicators in our sources (see Table 1). Box 4. Evidence base: summary • • • Strong evidence associates nurse staffing levels and mortality. Evidence linking variation in nurse staffing to failure to rescue and hospital acquired pneumonia suggests the potential significance of such indicators. Indicators theoretically most closely associated with nursing, such as pressure ulcers, are not clearly supported by strong or consistent evidence linking variation to nurses’ work. Evidence links other aspects of the workforce (job satisfaction, quality of the work environment including leadership) with mortality. • 15 State of the art metrics for nursing Doran’s extensive review considers evidence for a wider range of possible indicators21. However, while the review process is extensive and rigorous, the synthesis is narrative and does not fully meet criteria for a systematic review46-48. The weight of evidence offered is considered in a qualitative fashion; crucially, much of it derives from studies in which nursing interventions made a positive impact on patient outcomes. Rejecting this as a core source of evidence might seem perverse but because specific interventions generate specific outcomes, a focus on interventions is unlikely to yield valid overall indicators of quality that apply to clinical nursing services as a whole. The identification and delivery of effective interventions and measurement of associated outcomes are important but indicators must be able to broadly reflect quality. Thus while Doran’s review may offer a guide to developing indicators for outcomes of specific nursing interventions, it cannot offer the most authoritative overall source of evidence for indicators that reflect the broader care environment and characteristics. Future research might identify the variation in outcome associated with quality nursing care from systematic reviews of nursing interventions along the lines proposed by Mantz49, who assessed variation in outcome associated with quality stroke care from a systematic review of stroke units. However, searches of the Cochrane Library as part of this rapid appraisal did not reveal any equivalent reference points for the most widely advocated potential indicators, including falls, pressure ulcers and urinary tract infections. 16 State of the art metrics for nursing State of the art Best bets While indicators cannot provide a complete picture or a complete solution, they can provide a powerful mechanism to incentivise quality by making the contribution of nursing more visible within the healthcare system. However, identifying and using indicators is by no means straightforward. Table 4 illustrates the trade-offs by considering the relative merits of two possible indicators: mortality (for which the evidence base is strong) and pressure ulcers (where there is broad consensus about the contribution of nursing). Table 4: Comparison of mortality and pressure ulcers as outcome indicators for nursing Criteria Importance Scientific basis Impact Variation in quality Evidence of sensitivity to nursing Risk adjustment Reliability of data collection Useability Variation attributable to nursing Ownership by nursing Knowledge to inform action Wide applicability Positive behavioural incentives Potential for gaming Feasability Timely availability of data Routinely collected data Mortality High High Strong Feasible Good Low Unclear Unclear Yes High Low Potentially Yes Pressure ulcers Medium Unsure Weak Problematic Problematic Problematic Unclear High Clear Yes Mixed High Challenging No Specification/definition of the outcome Clear The importance of mortality as an issue is unambiguously high. There is ample evidence that some unexplained variation probably relates to the quality of health services, and evidence that some of this is attributable to variation in the quality of nursing. Generally data collection is reliable because the outcome is unambiguous and easy to define. Risk adjustment is possible, if challenging. However, the actual variation attributable to nursing is low, and direct ownership by the profession is also likely to be low since the contribution of other professions to preventing (or causing) death is widely recognised. Similarly, specific actions required of nursing to rectify quality problems are often unclear. However mortality is an outcome that applies to a wide range of settings, and while negative behavioural incentives are possible (for example, refusing to admit sicker patients) they are relatively unlikely at the level of a nursing service. Neither is mortality readily amenable to 17 State of the art metrics for nursing gaming, and such information is potentially available from routinely collected and timely administrative data. By contrast, the importance of pressure ulcers is harder to asses. The problem is prevalent but its social significance is harder to quantify, although economic impact is potentially high50. While risk adjustment models do exist51, elements of coding for both the outcome (grade of ulcer) and risk adjustment (for example mobility) are highly subjective (introducing potential bias and unreliability) and are not generally available in UK data sets. Determination of incidence is also problematic when pressure ulcer data are collected in point prevalence surveys, and the amount of variation associated with quality nursing is unclear. Clearly there is considerable ownership of the problem by the profession, as evidenced by the sources we have reviewed, but the difference in outcomes between those receiving high-quality nursing and poor-quality nursing is hard to estimate. If most pressure ulcers are preventable it could be very high, but the evidence we reviewed is neither consistent nor clear. Similarly, there is considerable scope for strategies to remedy these problems to be developed and acted upon by nursing autonomously, although the evidence base for action is not entirely clear. The outcome applies to a wide range of patients in a range of settings, although it predominantly relates to people with protracted (acute or long-term) institutional stays. Recording and coding problems potentially incentivise negative behaviours (poor reporting) and a lack of routine outcome data leaves open the potential for gaming with process measures. Because data are likely to be available only from intermittent audit, timely feedback will be challenging (but not impossible). It is hard to foresee fully valid data being derived from administrative data sets in the near future, although this area holds potential52. It is beyond the current paper’s scope to give a detailed consideration of all these issues against all the possible indicators identified, but the preceding discussion illustrates the challenges. As part of this review we identified a number of potential indicators and identified the strongest evidence for an association with variation in the quality of nursing. Of the range of potential indicators, a number emerge as potential “front runners” either because of strength of evidence or strength of opinion supporting the indicator (Box 5). Failure to rescue, hospital acquired infection, pressure ulcers and falls were the most strongly advocated patient safety indicators among our sources. Of these, failure to rescue and health care associated pneumonia are most clearly supported by evidence of variation associated with nursing. Because failure to rescue is likely to be a rare outcome in most settings, its utility for local quality monitoring is likely to be low. Staffing levels (generally registered nurse staffing), staff satisfaction and perception of the practice environment are also strongly supported workforce quality indicators generally supported by evidence. While indicators of compassion were not strongly reflected either in the evidence reviewed or the sources identified, they remain important. We classified satisfaction as reflecting compassion but satisfaction is a complex construct. It is widely used and many measures of satisfaction do contain items relating to patient experience of compassion but general 18 State of the art metrics for nursing patient satisfaction surveys seem insensitive to important aspects of experience53. Elements of communication should be regarded as intrinsic to quality and fundamental to both compassionate and effective care, but specific questions must be identified to properly reflect patients’ experience of ‘compassionate’ nursing. Any indicator needs to be linked to a nursing unit (for example a ward) and available in a timely fashion to provide maximum impact on staff. Box 5. ‘Front runners’ in the indicator stakes Safety • • • • • • • • • • Failure to rescue Health care associated pneumonia Health care associated infection Pressure ulcers Falls Staffing levels and patterns Staff satisfaction Staff perception of the practice environment Experience of care (patient-reported) Communication (patient-reported) Effectiveness Compassion Key challenges For many of the possible outcome metrics, the extent of the contribution of nursing relative to other professions is questionable or controversial. For example, although functional status/self-care/activities of daily living (ADL) have been cited as ‘nursing’ outcomes, other professions might dispute this. In the case of mortality, despite strong evidence of an association with nursing variables it is hard to offer it as a measure of nursing quality per se although the nursing contribution to it should not be disregarded. Of the widely advocated outcomes we identified, the most strongly supported by evidence from Kane’s systematic review is failure to rescue. This is generally recognised as nurse-sensitive but staffing levels were associated with only 16% of the variation (and only for surgical patients). Hospital acquired pneumonia (as opposed to HCAI in general) was associated with a larger proportion of variation (31%) across all (acute) settings. While these proportions may seem relatively small, this variation is related to staffing levels only, not to other aspects of quality nursing care. Although both indicators show strong potential in recent reports 52, the feasibility of deriving them from UK administrative data sets is yet to be fully tested. 19 State of the art metrics for nursing Inclusion of outcome indicators such as pressure ulcers and falls would need to be tied to a strategy to incentivise surveillance, recording and coding. These specific indicators are unlikely to be suited to use of routinely collected administrative data sets, certainly in the short term. Even US-based systems (with supposed better secondary coding) do not use routine data for pressure ulcers and instead rely on intermittent surveys54. While some outcomes may be controversial because of the relative contribution of other professions to variation, all are subject to patient-level variation. In most cases the main determinants of outcome are patients themselves, not care inputs. Adjustment for patientlevel variation in risk is likely to be a formidable challenge, particularly when comparing across institutions. Existing indicator sets such as the NQF attempt to tackle this but rely on US administrative data specifications and levels of secondary coding, so their applicability to the UK is likely to be limited. Instead, these sets may provide a relatively strong starting point for developing UK indicators to be derived from routinely collected data. Risk adjustment does not present such a substantial obstacle for local quality monitoring where the comparison is with a unit’s own past performance, provided that there is no substantial change in the patient population to alter underlying risk. It is appealing to consider care processes as useful alternatives to measuring outcomes. For nursing this has often been represented by systems focussing on documentation and recording of assessments. However, there is a potential problem of token compliance to such process measures which then have no impact upon the care delivered, so such metrics should be used with caution. In some cases process measures also need risk adjustment, although it is generally less problematic55. The burden of the audit associated with process measures will be relatively high where data are not routinely recorded in a useable format, as is generally the case in the UK. Since an audit is likely to be required, patient outcome audit may be a better investment although risk adjustment remains problematic. Any indicator set must be sufficiently diverse to indicate overall quality. While isolated outcomes are poor reflections of overall quality29, a purely process-based system is not well suited to nursing. Such approaches do not work well in complex systems involving high degrees of individual skill and multiple actions, and where there is great uncertainty about specific processes that contribute to success. In these cases, individual aspects of care that are measurable do not reflect the complex packages of care that represent best practice49, 55 , 56. Workforce factors emerge as strong candidates because of their association with important outcomes and because they are clearly nursing-related. However, the considerable available evidence on staffing levels and skill mix does little to guide the benchmarking of staffing levels for settings outside the US as contexts of care differ significantly. Setting benchmarks based on historical workforce characteristics and ways of working also may ‘freeze’ the current configuration and stifle innovation. Preliminary economic modelling suggests that investment in highly skilled staff may deliver better returns than investment in total staff40 ,41, indicating potential to change the current balance between qualified and unqualified staff, providing qualified staff are trained to a higher 20 State of the art metrics for nursing level. Benchmarks used as measures of quality might create perverse incentives to employ more junior and less qualified staff at the expense of fewer more highly skilled but more expensive ones and thus stifle positive innovations in staffing. Although not directly supported by evidence, the JCAHO indicator of staffing in relation to the institution’s staffing plan57 may be a more satisfactory indicator in the short term, since it requires a staffing plan and some justification of staffing levels. Measures of staff satisfaction and perceived work environment quality are promising. Yet the content of the NWI’s Practice Environment Scale58, while containing items of clear face validity (relating to leadership and support, perceived standards and adequacy of care) and wide applicability, has items with questionable support or that advocate practices of questionable validity (e.g. use of nursing diagnoses). Staff-reported measures and perceptions are less amenable to gaming and perverse incentives than staffing quotas and benchmarks although by no means immune. Despite these challenges (Box 6), the motivation for developing indicators is compelling. Even the harshest critics of such systems allow the potential for benefit and acknowledge the lack of clear alternatives in the context of a publically funded health system34, although there is much argument over the relative merits of process and outcome measures29,59. Desires within the nursing profession for articulation of its important contribution and among the public for improved standards point toward outcomes as a significant component of any indicator system, as their importance is more clearly understood and harder to contest. Finally, our proposed indicators largely lack positive articulations of nursing’s impact. Patient experience of care did not emerge prominently, although ‘satisfaction’ with nursing is among the most widely used nursing ‘outcome’ measures60 and is a major determinant of overall satisfaction scores61, 62. Nursing’s positive contributions do not easily translate into specific objective outcomes either because nursing makes a small contribution to the overall outcome (such as functional status) or because the contribution is to the subjective experience of the patient. As patient-reported outcome measures (PROMS) provide patients opportunities to assess their treatment outcome63, they merit consideration for evaluation of specific nursing interventions or narrowly focussed nursing services. However, PROMS’ utility as nursing indicators may be limited because the contribution of nursing services to most specific PROMS across a patient population is still likely to be small in light of patients’ experiences with other health services personnel Some elements of care processes, such as the quality of communication as assessed by patients, are clearly on a pathway to effective care, because change in health behaviour and treatment concordance require successful communication. The task and finish group were also very clear about nurses’ contribution to patient dignity and experiences of care as humane and compassionate. Concerns about a loss of compassion and expressions of caring in the emotional sense underpin both professional and public concerns about the state of nursing noted earlier6-8. 21 State of the art metrics for nursing Therefore, in addition to key safety effectiveness and workforce indicators, any set of measures that seeks to represent the nursing system as a whole must seek to represent the experiences of people who are ‘nursed’. Our list of possible indicators touches on this in items such as dignity, communication and satisfaction, which often are measured as composite scores based on evaluations of aspects of experience. Such indicators have the potential to be intrinsically important if properly surveyed (irrespective of evidence linking them to objective outcome) and many aspects of experience, such as communication with professionals, may have important objective consequences. Certainly, a lack of evidence directly linking these indicators to other aspects of quality represents a challenge for future research but not an obstacle to the use of patient experiences as indicators of quality. While there are a number of possible candidates (such as items from the NHS patient survey) the selection of indicators should not be decided by professionals alone. Challenges include delivering results of such surveys in a timely fashion and attributing them to nursing units such as wards rather than hospitals as a whole. Yet most potential indicators face these challenges, and although different values and expectations complicate comparison across different populations, providers’ opportunity for gaming is limited. Box 6. Key challenges • • • • • • • Defining data and full specification of indicators Adjusting for risk Improving the quality of clinical coding Identifying indicators for nursing’s impacts in mental health, community, primary care and paediatric settings Identifying and defining indicators that cross care pathways and boundaries Timely reporting at the nursing unit level Delivering action to improve quality 22 State of the art metrics for nursing Conclusion and recommendations Developing indicators for nursing is challenging, but current circumstances provide both an opportunity and an imperative for the profession to embrace tangible measures of nursing’s contributions to patient care. In this report we have identified candidate indicators for further development. Some indicators have potential applications in a range of settings, but indicators suitable for acute hospital settings have more evidence and better specification. We have been wary of process and structure indicators because of the uncertain link of likely process measures to outcomes or, in the case of workforce measures, the danger of fixing current workforce patterns and stifling innovation. Both processes and structures are important, and staff-reported measures of work environment and patientreported measures of their care experiences are both potentially important indicators, yet there are compelling reasons to ensure that nursing outcomes will be prominent in any indicator set. Ideally these outcomes should reflect all the dimensions of effectiveness, safety and compassion. The promise of ‘no more national targets’ and the Next Stage Review64 commitment to reward quality are reassuring. We began by reflecting on concerns that the quality of nursing was being neglected in the face of performance targets focusing on productivity. But performance and output of care remain important, and quality measures should directly reflect elements of patient health outcome and experience that benefit from highquality nursing care, lest these elements be neglected in favor of performance targets. The Next Stage Review proposes rewarding quality to a much greater extent than at present. If the quality of nursing can be properly reflected in wider sets of metrics to be developed, nursing’s contributions can be more fully recognised at all levels of NHS governance and management. The amount of work needed to fully realise this opportunity should not be underestimated, but without such metrics nursing faces increasing invisibility within a performance-managed health service. Most important, feedback on performance that is based on important measures of nursing can improve nursing care quality by providing frontline staff with information on trends, emerging problems and successes. Such feedback can help senior clinical managers by giving overviews of performance and allowing them to both troubleshoot problems and recognise success. Feedback can help managers articulate nursing’s contributions in tangible terms when seeking resources to protect and enhance quality, and it allows boards to ‘see’ these contributions and to properly support nursing. Finally, such nursing metrics can empower the public to choose between care options using indicators based on an aspect of health services which clearly matters to them as much as it matters to the profession: the quality of nursing care. 23 State of the art metrics for nursing Box 7. Conclusions • Of the range of potential indicators, a number emerge as potential “front-runners”. Failure to rescue, health care associated infection, pressure ulcers and falls were the most strongly advocated patient safety indicators. Staffing and skill mix are linked to patient outcome but their use as indicators would stifle change and create perverse incentives. Use of “staffing matched to planned staffing” as part of a suite of indicators including outcomes has more potential. Process indicators should be used with caution because of potential for gaming and difficulty in linking specific processes and patient outcomes. Patient experience of compassionate care is an important outcome in its own right and may provide the best measure of the nursing contributions to shared outcome and evaluation of complex processes that are otherwise elusive. Despite these and other challenges, the case for developing metrics is strong. • • • • Recommendations We believe the nursing profession should embrace development of metrics and establishment of a National Quality Board and local quality observatories. These initiatives present a substantial opportunity for the profession to equip itself with tools needed to deliver excellent care into the future. Based on the evidence considered here, we recommend: The new National Quality Board should ensure that nursing’s contribution is properly represented in its programme of metrics development. In conjunction with care regulators and other stakeholders, it should develop and support a national standard set of nursing indicators including outcome measures for safety, effectiveness and compassion. As part of this programme we recommend: • • • • • • • • Existing data sources should be used where possible, but a process to develop and adapt data collection systems to increase indicator validity also is needed. Specifications for failure to rescue and HCA pneumonia should be developed as outcome indicators derived from routinely collected data. These ‘shadow’ indicators should be launched and any association with known markers of quality (e.g. risk-adjusted mortality) should be tested. Standards should be developed to collect and report data on falls and pressure ulcers, as should risk adjustment models for use in regional and national benchmarking. Staff surveys should include specific nursing work environment questions. Specifications are needed for a workforce planning indicator based on the JCAHO model. Core nursing-related indicators in patient surveys should be identified. Patient groups and professionals can cooperate to identify key indicators of compassion. 24 State of the art metrics for nursing The Department of Health should ensure that the Connecting for Health programme provides a suitable infrastructure to support the collection of clinical outcome and process data to provide nursing indicators and that a Minimum Dataset for Nursing (including patient dependency and staffing variables) can be derived from it. The Royal College of Nursing, NHS Confederation and other key stakeholders should consider facilitating a nursing quality coalition of organisations which currently use metrics with the aim of sharing best practice, standardising data collection where this will be useful, and sharing data to facilitate research on indicator validity. The California Nursing Outcomes Coalition (CalNOC) may provide a model for this. Professional bodies and senior members of the profession should reinforce and reiterate the significance of nursing contribution in all settings to both nursing-sensitive and shared outcomes, such as mortality and preventable admissions. Claims need to be based on sound evidence and the nursing research community must focus more effort on exploring outcomes. Health care providers must identify mechanisms to ensure timely feedback of indicators at the nursing unit level, and to ensure that positive action results from this information. Health care commissioners and regulators should use nursing metrics as part of their assessment of quality and the commissioning process, and should move toward publication of indicators. Further work is needed to develop indicator sets outside acute settings and for patient pathways across settings. 25 State of the art metrics for nursing References 1. 2. 3. 4. 5. Department of Health. NHS Next Stage Review: A High Quality Workforce. London: Department of Health, 2008. Pencheon D. The Good Indicators Guide: Understanding how to use and choose indicators. Warwick: NHS Institute for Innovation and Improvement, 2008. Redfern SJ, Norman IJ. Quality assessment instruments in nursing: towards validation. Int J Nurs Stud 1995;32(2):115. Higgins M, McCaughan D, Griffiths M, Carr-Hill R. Assessing the outcomes of nursing care. J Adv Nurs 1992;17:561. Savitz L, Jones C, Bernard S. Quality Indicators Sensitive to Nurse Staffing in Acute Care Settings. Advances in Patient Safety: From Research to Implementation (AHRQ Publication No. 050021). Rockville: Agency for Healthcare Research and Quality, 2005:375-85. Corbin J. Is caring a lost art in nursing? Int J Nurs Stud 2008;45(2):163. Griffiths P. Editorial: The art of losing...? A response to the question ‘is caring a lost art?’ Int J Nurs Stud 2008;45(2):329. Raynor C. Why gossiping nurses are a symptom of an NHS in crisis. Daily Mail 2008 online http://www.dailymail.co.uk/pages/live/articles/health/healthmain. html?in_article_id=561164&in_page_id=1774 (accessed April 20th 2008) Healthcare Commission. Investigation into outbreaks of Clostridium difficile at Stoke Mandeville Hospital, Buckinghamshire Hospitals NHS Trust. London: Commission for Healthcare Audit and Inspection, 2006. 6. 7. 8. 9. 10. Healthcare Commission. Investigation into outbreaks of Clostridium difficile at Maidstone and Tunbridge Wells NHS Trust. London: Commission for Healthcare Audit and Inspection, 2007. 11. Healthcare Commission. Learning from investigations. 2008. 12. Maben J. The art of caring: Invisible and subordinated: A response to Juliet Corbin: ‘Is caring a lost art in nursing’. Int J Nurs Stud 2008;45(3):335. 13. Clarke SJ. Research on Nurse Staffing and its Outcomes. In: Nelson S, Gordon S, editors. The Complexities of Care: Nursing Reconsidered. New York: Cornell University Press, 2006. 14. Audit Commission. Review of National Finding: Ward Staffing. Acute Hospital Portfolio. London: Audit Commission, 2001:15. 15. Audit Commission. Ward Staffing Management Tools: Practical Tips for Budget Setting. London: Audit Commission, 2001:3. 16. Black N. Rise and demise of the hospital: a reappraisal of nursing. BMJ 2005;331(7529):1394. 26 State of the art metrics for nursing 17. Clarke SPP. Registered Nurse Staffing and Patient Outcomes in Acute Care: Looking Back, Pushing Forward. Med Care 2007;45(12):1126. 18. Maas ML, Johnson M, Moorhead S. Classifying Nursing-Sensitive Patient Outcomes. J. Nurs. Sch. 1996;28(4):295. 19. Doran D. Preface. In: Doran D, editor. Nursing-sensitive outcomes: state of the science. Sudbury / London: Jones and Bartlett Pub., 2003:vii-ix. 20. Mitchell PH, Ferketich S, Jennings BM. Quality Health Outcomes Model. J. Nurs. Sch. 1998;30(1):43. 21. Doran D. Nursing-sensitive outcomes: state of the science. Sudbury / London: Jones and Bartlett Pub., 2003. 22. Kane R, Shamliyan T, Mueller C, Duval S, Wilt T. The Association of Registered Nurse Staffing Levels and Patient Outcomes: Systematic Review and MetaAnalysis. Med Care 2007;45(12):1195. 23. Lankshear AJ, Sheldon TA, Maynard A. Nurse Staffing and Healthcare Outcomes: A Systematic Review of the International Research Evidence. Advances in Nursing Science 2005;28(2):163. 24. Kazanjian A, Green C, Wong J, Reid R. Effect of the hospital nursing environment on patient mortality: a systematic review. J Health Serv Res Policy 2005;10:111. 25. Lang TA, Hodge M, Olson V, Romano PS, Kravitz RL. Nurse-patient ratios: a systematic review on the effects of nurse staffing on patient, nurse employee, and hospital outcomes. J Nurs Adm 2004;34(7-8):326. 26. Johnson M, Maas M. The Nursing Outcomes Classification. J Nurs Care Qual 1998;12(5):9. 27. Silber JH, Romano PS, Rosen AK, Wang Y, Even-Shoshan O, Volpp KG. Failure-to-Rescue: Comparing Definitions to Measure Quality of Care. Med Care 2007;45(10):918. 28. National Quality Forum. A Comprehensive Framework for Hospital Care Performance Evaluation: A consensus report. Washington: National Quality Forum, 2003. 29. Lilford R, Mohammed MA, Spiegelhalter D, Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet 2004;363(9415):1147. 30. Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003;15(6):523. 31. Riehle AI, Hanold LS, Sprenger SL, Loeb JM. Specifying and Standardizing Performance Measures for Use at a National Level: Implications for Nursingsensitive Care Performance Measures. Med Care Res Rev 2007;64(2_suppl):64S. 32. Donaldson N, Brown D, Aydin C, Bolton M, Rutledge D. Leveraging Nurse-related Dashboard Benchmarks to Expedite Performance Improvement and Document Excellence. J Nurs Adm 2005;35(4):163. 27 State of the art metrics for nursing 33. Boyne GA, Law J. Setting Public Service Outcome Targets: Lessons from Local Public Service Agreements. Pub Money and Man 2005;25(4):253. 34. Bevan G, Hood C. What’s Measured Is What Matters: Targets and Gaming in the English Public Health Care System. Pub Admin 2006;84(3):517. 35. Quan H, Parsons GA, Ghali WA. Assessing accuracy of diagnosis-type indicators for flagging complications in administrative data. J Clin Epidemiol 2004;57(4):366. 36. Pancorbo-Hidalgo PL, Garcia-Fernandez FP, Lopez-Medina IM, Alvarez-Nieto C. Risk assessment scales for pressure ulcer prevention: a systematic review. J Adv Nurs 2006;54(1):94. 37. Havens DS, Aiken LH. Shaping Systems to Promote Desired Outcomes: The Magnet Hospital Model. JONA 1999;29(2):14. 38. Rafferty AM, Clarke SP, Coles J, Ball J, James P, McKee M, et al. Outcomes of variation in hospital nurse staffing in English hospitals: Cross-sectional analysis of survey data and discharge records. Int J Nurs Stud 2006. 39. McKee M, Coles J, James P. ‘Failure to rescue’ as a measure of quality of hospital care: the limitations of secondary diagnosis coding in English hospital data. J Pub Health 1999;21(4):453. 40. Needleman J, Buerhaus PI, Stewart M, Zelevinsky K, Mattke S. Nurse Staffing In Hospitals: Is there a Business Case for Quality? Health Aff (Millwood) 2006;25(1):204. 41. Newbold D. The production economics of nursing: A discussion paper. Int J Nurs Stud 2008;45(1):120. 42. Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital Nurse Staffing and Patient Mortality, Nurse Burnout, and Job Dissatisfaction. JAMA 2002;288(16):1987. 43. West E, Maben J, Rafferty A. Nursing and Patient Outcomes: How Can Employers Provide the Right Environment for Nurses to Deliver High-Quality Care? Harvard Health Policy Review 2006;7(1):64. 44. Stone PW, Mooney-Kane CM, Larson EL, Horan TM, Glance LG, Zwanziger J, et al. Nurse Working Conditions and Patient Safety Outcomes. Med Care 2007;45(6):571. 45. Estabrooks CA, Midodzi WK, Cummings GG, Ricker KL, Giovannetti P. The impact of hospital nursing characteristics on 30-day mortality. Nurs Res 2005;54(2):74. 46. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Metaanalysis of Observational Studies in Epidemiology: A Proposal for Reporting. JAMA 2000;283(15):2008. 28 State of the art metrics for nursing 47. Moher D, Cook D, Eastwood S, Olkin I, Rennie D, Stroup D. Improving the quality of reports of meta-analyses and randomised controlled trials: the QUOROM statement. Lancet 1999;354:1896 48. Center for Reviews and Dissemination. What are the criteria for the inclusion of reviews on DARE? York: Center for Reviews and Dissemination online http://www. york.ac.uk/inst/crd/darefaq.htm (accessed 01/05/2008) 49. Mant J. Process versus outcome indicators in the assessment of quality of health care. Int J Qual Health Care 2001;13(6):475. 50. Bennett G, Dealey C, Posnett J. The cost of pressure ulcers in the UK. Age Ageing 2004;33(3):230. 51. Berlowitz DR, Brandeis GH, Morris JN, Ash AS, Anderson JJ, Kader B, et al. Deriving a Risk-Adjustment Model for Pressure Ulcer Development Using the Minimum Data Set. J Am Geriatr Soc 2001;49(7):866. 52. Bottle A, Aylin P. How NHS trusts could use patient safety indicators to help improve care. Health Care Risk Report 2008;May 2008:12. 53. Jenkinson C, Coulter A, Bruster S, Richards N, Chandola T. Patients’ experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. Qual Saf Health Care 2002;11(4):335. 54. National Quality Forum. National Voluntary Consensus Standards for NursingSensitive Care: An Initial Performance Measure Set. Washington: National Quality Forum, 2004. 55. Rubin HR, Pronovost P, Diette GB. The advantages and disadvantages of processbased measures of health care quality. Int J Qual Health Care 2001;13(6):469. 56. Kendrick S. Using all the evidence: towards a truly intelligent National Health Service. Health Bulletin 2001;59(2). 57. JCAHO. Staffing Effectiveness in Hospitals. Oak Brook: Joint Commission Resources, 2003. 58. Lake ET. Development of the practice environment scale of the nursing work index. Res Nurs Health 2002;25(3):176. 59. Donabedian A. Evaluating the Quality of Medical Care. Milbank Quarterly 2005;83(4):691. 60. Avis M, Bond M, Arthur A. Satisfying solutions? A review of some unresolved issues in the measurement of patient satisfaction. J Adv Nurs 1995;22(2):316. 61. Yellen E. The Influence of Nurse-Sensitive Variables on Patient Satisfaction. AORN 2003;78(5):783. 62. Abramowitz S, Coté A, Berry E. Analyzing patient satisfaction: a multianalytic approach. Quality Review Bulletin 1987;13(4):122. 63. Smith S, Cano S, Lamping D, Staniszewska S, Browne J, Lewsey J, et al. Patient-Reported Outcome Measures (PROMs) for routine use in Treatment Centres: recommendations based on a review of the scientific evidence. London: London School of Hygiene and Tropical Medicine, 2005. 29 State of the art metrics for nursing 64. Department of Health. High Quality Care for All: NHS Next Stage Review Final Report. Norwich: Stationary Office, 2008. 65. Healthcare Commission. Acute Hospital Portfolio Review: ward staffing topic guide. London: Healthcare Commission, 2004. 66. International Council of Nurses. Nursing Sensitive Outcome Indicators. Geneva: International Council of Nurses online http://www.icn.ch/matters_indicators.htm (accessed 01/05/2008) 67. Naylor MD. Advancing the Science in the Measurement of Health Care Quality Influenced by Nurses. Med Care Res Rev 2007;64(2_suppl):144S. 68. Van den Heede K, Clarke SP, Sermeus W, Vleugels A, Aiken LH. International Experts’ Perspectives on the State of the Nurse Staffing and Patient Outcomes Literature. J Nurs Schol 2007;39(4):290. 30 State of the art metrics for nursing Appendix 1: Core data sources Name Short ref Description Setting/ clinical group Ref/link Agency for Healthcare Quality and Research Patient safety Indicators (nursing subset) American Nurses Association AHRQ Patient safety indicators Acute care* http://www. qualityindicators.ahrq.gov/ psi_overview.htm 5 ANA Nursesensitive indicator set Acute, paediatric, long term care and psychiatric http://www. mnursingworld.org/ ainMenuCategories/ professionalNursing/ PatientSafetyQuality/ NDNQI/NDNQI_1/ ngSensitiveIndicators.aspx http://www.aukuh.org.uk/ members/PCP.htm Association of UK University Hospitals nurse-sensitive indicators Doran 2007 (review) Essence of care AUKUH Nursesensitive indicator set Acute care Doran Review of Acute care nurse-sensitive outcomes Benchmark statements for the quality of nursing care Not specified 21 EoC http://www.dh.gov.uk/en/ Publicationsandstatistics/ Publications/ publicationsPolicy AndGuidance/ DH_4005475 http://www. healthcarecommission. org.uk/erviceproviderin formation/reviewsand studies/servicereviews/ ahpmethodology/ wardstaffing.cfm 65 Healthcare Commission HC Factors assessed in ward staffing review Acute care 31 State of the art metrics for nursing Name Short ref Description Setting/ clinical group Ref/link International Council of Nurses Briefing Joint Commission staffing effectiveness Indicators Kane 2007 (review) ICN Review of Not nurse-sensitive specified outcome indicators Staffing Acute care effectiveness indicators including NQF indicators Review of evidence for staffing outcome relationship Acute care http://www.icn.ch/ matters_indicators.htm 66 JCA 31 Kane 22 Kazanjian 2005 (review) Kaz Acute care Review of evidence linking ward environment to mortality Review of evidence for staffing outcome relationship Review of evidence for staffing outcome relationship Long-term care/post– acute care indicator set Acute care 24 Lang 2004 (review) Lang 25 Lankshear 2005 (review) Lank Acute care 23 Nursing Home Quality Initiative Medicare/ Medicaid Quality Compare NHQI Long-term care http://www.medicare.gov/ NHCompare/ 32 State of the art metrics for nursing Name Short ref Description Setting/ clinical group Acute care Ref/link National Quality Forum Consensus Standards for Nursingsensitive Care OASIS (Medicare reporting subset) Other NQF Nursesensitive indicator set http://www.qualityforum. org/nursing/54 ,67 OASIS Home health care indicator set Other clinical indicator sets /dashboards identified on web searches Home/ community health care Generally acute http://www.cms.hhs.gov/ OASIS/ Oth N/A Quality Outcomes Framework (nursing items) QOF Primary General care practice quality indicators with specific items relating to nursing Not Expert specified consensus regarding nurse-sensitive outcomes http://www.dh.gov.uk/en/ Healthcare/Primarycare/ Primarycarecontracting/ QOF/index.htm Van den Heede 2007 VdH 68 33 Appendix 2: Indicators identified Area AHRQ ANA Doran EoC HC ICN JCA Kane Kaz Lang Lank NHQI NQF OASIS Oth QOF VdH AUKUH Count Indicator Safety √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ Pressure ulcer Safety Workforce Safety Safety Workforce Safety Workforce Safety Safety Workforce Experience Workforce Preventative Workforce Experience Workforce Safety Workforce Function Safety Experience 11 9 9 8 8 7 6 6 5 5 5 5 5 5 5 4 4 4 4 4 4 3 3 Failure to rescue Staffing levels (nurses/hours per patient) Falls HCAI (pneumonia) Staff satisfaction and well-being HCAI (UTI) Staffing, skill mix Medication administration errors Mortality Practice environment/perceived quality State of the art metrics for nursing 34 Satisfaction with (nursing) care Sickness rates Smoking advice Staffing bank/agency Communication Staff experience, knowledge, skills, expertise HCAI (surgical wound) Instrumental activities of daily living and self-care Function Perception of adequate staffing Use of restraints Arrest/shock Complaints Indicator AHRQ ANA Doran EoC HC ICN JCA Kane Kaz Lang Lank NHQI NQF OASIS Oth QOF VdH Count Area Experience √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ AUKUH Confidence and trust Function Safety Safety Safety Safety Safety (staff) Workforce Experience Utilisation Nutrition Safety Safety Safety Workforce Symptom Experience Planning Experience Safety Experience Utilisation 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 3 Continence Emergency equipement/drugs Extubation, unplanned HCAI (any) HCAI (bloodstream) Injuries to staff Interprofessional relations Knowledge of condition and treatment Length of stay Nutritional assessment/screening State of the art metrics for nursing 35 Pain assessment/assessment intervention cycles Symptom PE/DVT Record keeping/reporting systems Respiratory failure RN vacancies Symptom control (other, e.g. nausea) Ability to talk to nurse Care planning/assessment processes Cleanliness Confusion, delirium Dignity/respect Emergency care Indicator AHRQ ANA Doran EoC HC ICN JCA Kane Kaz Lang Lank NHQI NQF OASIS Oth QOF VdH Area Safety √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ AUKUH Count HCAI (c. diff.) Safety Safety Utilisation Workforce Symptom Experience Safety Safety Nutrition Preventative Workforce Workforce Safety Workforce Workforce Preventative Safety Workforce Safety Safety Workforce Safety 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1 1 HCAI (central line) HCAI (MRSA) Hospital admissions/readmissions Leadership Pain/pain control Patient involvement Post-operative complications Pressure ulcer risk assessment/planning Protected meal times policy implemented Risk assessments/other health promotion State of the art metrics for nursing 36 RN turnover Staff intent to leave Symptom control (dyspnea) Time spent in CPD Understaffing (compared to staffing plan) Vaccination rates Vascular access device ‘incidents’ Workload Accidents/incidents Allergies recorded Appraisal Cannula infiltration Indicator AHRQ ANA Doran EoC HC ICN JCA Kane Kaz Lang Lank NHQI NQF OASIS Oth QOF VdH Count Area Utilisation √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ AUKUH Discharge from caseload Planning Safety Workforce Safety Safety Workforce Safety Safety Experience Workforce Function Safety Safety Function Safety (staff) Nutrition 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Discharge planning/case management processes HCAI surveillance system Induction Mortality in low–mortality groups Patients absconding or lost Personal learning plan Pressure ulcer monitoring systems Psychiatric physical/sexual assault Psychological well-being Record of training and updating State of the art metrics for nursing 37 Restraint documentation Self-harm risk assessment Timely assessment/intervention for pneumonia Safety Upper GI bleed Use of long-term catheter Violence and aggression toward staff Weight loss A=acute care, C=home/community health care, L=long-term care, N=not specified, P=paediatric, Pc=primary care, Ps=psychiatric State of the art metrics for nursing 38 To order further copies Please contact: The National Nursing Research Unit at King’s College London Waterloo Campus, 57 Waterloo Road London SE1 8WA Telephone 020 7848 3057 Fax 020 7848 3069 http://www.kcl.ac.uk/schools/nursing/nnru PDF download: free Hard copy: administration charge £15 (including UK p&p) Cover picture: The Diagram [based on the “coxcomb” diagram used by Florence Nightingale to depict rate and cause of mortaliry among British soldiers during the Crimean war] Laura Potter 2007 Silver, red gold, yellow gold Approx. 600cm chain Photo: Tomás Valenzuela Part of the Culture & Care Project ISBN: 978-0-9555991-2-5 © King’s College London 2008