EBP Step 5

Evaluation of practice and outcomes

The final step in the EBP process is the evaluation of evidence uptake to determine the effectiveness of strategies that contributed to successful and sustainable implementation of evidence. Comprehensive evaluation of evidence uptake includes measures of processes as well as intermediate outcomes such as change in awareness, knowledge, attitudes and behaviour.

Hakkennes and Green (2006) grouped measures for assessing evidence uptake into three categories:

Patient level
- Measures of patient/symptom change: changes in patients’ health status, mortality, quality of life
  - Commonly used methods: patient interview, survey or questionnaire, disease-specific questionnaires; use of clinical tests
- Surrogate measures of patient change: patient satisfaction, length of hospital stay, number of health care visits, hospitalisations, test use, patient knowledge or attitudes, functional status and return to work, patient compliance
  - Commonly used methods: patient survey or questionnaire, patient interview, audit of medical record, medical practitioner interview, computerised database, log book/department record/register, encounter form/request slip/diary
Practitioner level
- Measures of practitioner change: change in the behaviour of the practitioner targeted by the intervention (e.g. compliance with guidelines)
  - Commonly used methods: medical record audit, computerised databases, encounter forms, request slips or diaries, practitioner interview or questionnaire, patient reporting (through an interview), video-recorded or audio-taped consultations
- Surrogate measures of practitioner change: generally involved the measurement of knowledge and attitudes (attitudes toward the information being implemented)
  - Commonly used methods: case scenarios, objective tests such as short answer or multiple choice questions; questionnaire
System/organisational or process level: measures of cost and/or items relating to the intervention itself; implementation strategy; changes in policies and procedures and changes in the organisational level such as the presence of the required equipment and the impact of intervention on the length of consultation with medical practitioner
- Commonly used methods: computerised database; survey or interview of the practitioner; encounter form, chart or diary

Reference

Hakkennes S and Green S. Measures for assessing practice change in medical practitioners. Implementation Sci 2006; 1:29.

top

Models of health care quality

The improvement of health care quality is the expected end result of EBP implementation. Without an effective system of measurement, one can never be sure whether the implementation led to any improvements in healthcare.

Donabedian’s Model

This framework consists of improving the quality of care into three fundamental parts of healthcare:

Donabedian's model

This model can be used to categorise quality indicators and frame the outcomes of an EBP implementation program. In the Donabedian model, structure refers to the setting in which care is delivered, and includes the attributes of material resources (e.g. facilities, equipment), of human resources (e.g. number and characteristics of personnel), and of organisational structure (e.g. medical staff organisation, methods of peer review). Process refers to the approaches or means of providing health care which includes the services and treatments the patients receive. Outcome refers to the result or impact of care on the health status of patients and populations. It may also involve improvements in patient’s knowledge & behaviour and degree of patient satisfaction.

References

Donabedian A. The quality of care: how can it be assessed? JAMA 1988; 260(12): 1743-1748.

Visnjic A, Velickovic V, Jovic S. Measures for improving the quality of health care. Scientific Journal of the Faculty of Medicine in Nis 2012; 29(2): 53-58.

Deming PDSA Cycle

The PDSA cycle is a continuous quality improvement model that consists of a logical sequence of four repetitive steps for continuous improvement: Plan, Do, Study and Act. This approach allows the evaluation process to deviate from traditional evaluation approaches, which should adhere to a strict protocol from beginning to end. As a result, evaluations undertaken in this framework should be sufficiently flexible in terms of design and measurements and allow refinements, as required, to appropriately address the aims of the quality improvement program.

Deming PDSA Cycle

Plan: The first step is to plan the change to be implemented. The function of planning is based on the set of quality requirements to be met at certain levels of the organisation and in certain parts of the organisational structure to improve quality.

Do: The next step is to work the plan. The function of execution (Do) is the implementation of previously planned quality improvement by applying appropriate techniques.

Study: The next step is to study the outcome and examine the results of the quality improvement. The function of consideration (Study) is to establish and analyse previously conducted quality using the methods of input, process and output quality control.

Act: The next step in the cycle is to take action based on the results of the analysis (Study). The function of introduction (Act) has the task of evaluating result considering the necessary decisions for implementation of quality improvement in the quality management process.

Reference

Visnjic A, Velickovic V, Jovic S. Measures for improving the quality of health care. Scientific Journal of the Faculty of Medicine in Nis 2012; 29(2): 53-58.

Further reading

NHS Institute for Innovation and Improvement. Quality and Service Improvement Tools 2008.

ASPIRE for quality framework

The ASPIRE for quality is an evidence-based framework developed by the International Centre for Allied Health Evidence to evaluate clinical service performance in allied health. ASPIRE was developed to assist allied health practitioners evaluate their clinical performance as a means for improving the quality of allied health services.

ASPIRE for Quality image