The Effectiveness of Different Mechanisms for Spreading Best Practice

A research project report by the Office for Public Management.

Contents

1. Introduction + Methodology

There are many different examples of how guidance information and databases are being used to help disseminate and spread best practice across public services. Ultimately, it is only through mechanisms of this nature that key changes in policy and practices can be conveyed to the largest possible audience. How teachers find out about a new way of teaching a subject area or how doctors find out the best way to treat a certain symptom, for example, will require, at the very least, some form of guidance, or a review placed on a data-base. Relatively few evaluations appear to have been undertaken of the effectiveness of these mechanisms in terms of their impact on changing practice, particularly where they are the main or sole method of dissemination Although guidance continues to be channelled through regular policy directives, publications and information leaflets, their effectiveness has not been fully assessed.

There have been some improvements however, in the way that guidance and database information is provided to practitioners. Feedback from those involved in developing strategies which combine several approaches to dissemination strongly suggests that guidance materials and databases work best if they are part of a wider, co-ordinated strategy and where efforts have been made to identify the data needs and preferences of target audiences. Our research identified several surveys of users of guidance and information materials which showed, for example, that specific types of material were most useful for (and most likely to be taken up by) specific categories of ‘recipient’.

Below are two widely acclaimed examples of evaluation in the health sector, followed by three examples where surveys of users have been carried out in relation to specific initiatives.

The York Centre is at the heart of attempts to help the NHS to improve the way it transfers best practice. It does this through a variety of mechanisms including regular publications, courses and databases. One of the main forms of communication it uses is the Effective Health Care bulletin (NHS Centre for Reviews and Dissemination 1999). This has been used to inform practitioners in a concise and accessible format about some of the latest best practice initiatives in healthcare. The most recent bulletin produced by the York Centre explicitly focuses on how evidence could be put into practice. It has also been cited by a number of the experts we interviewed as an excellent source of information about spreading best practice.

The aim of this bulletin is to approach changing practice from a range of different perspectives, including research evidence from empirical studies of behavioural change, theoretical evidence from models of behaviour change derived from psychology, marketing and health promotion; and insights from case studies which have attempted to change professional practice the NHS. The bulletin then goes on to summarise the results of an overview of over 100 systematic reviews of different disseminations and implementations and makes the following key points:

Back to Top

The Cochrane Collaboration

The Cochrane Collaboration is an international organisation which aims to help people make well informed decisions about health care by preparing, maintaining and ensuring accessibility of systemic reviews of the effects of health care interventions. Many of the people we interviewed about spreading best practice pointed to the Cochrane Collaboration as one of the leading organisations in disseminating and sharing best practice.

The Cochrane’s main information body, the reviews, are published electronically in successive issues of The Cochrane Database of Systematic Reviews. Preparation and maintenance of the reviews is the responsibility of international collaborative review groups. The members of these groups of researchers, health care professionals, consumers and others share an interest in generating reliable, up-to-date evidence relevant to the prevention, treatment and rehabilitation of particular health care problems or groups of problems.

The work of the Cochrane Collaboration is supported and facilitated by the work of Cochrane centres. Among other roles these centres are responsible for:

Back to Top

Research into the Information Requirements of Urban Regeneration Partnerships

The purpose of this research, carried out in 1994, was to identify partnerships’ information needs and help the department in preparing its dissemination strategy for the coming years (Department of the Environment, 1994). The study looked at five elements of dissemination strategy:

Included among a wide range of detailed findings were the following points:

Back to Top

This survey was carried out among firms which had ordered publications, to ascertain the types of use and impact of DTI Management Best Practice publications. The detailed findings included:

Analysis of those who felt that definite improvements could be made with the assistance of DTI materials showed that there appeared to be a particularly high correlation between those who were intending to use the materials to think through all the possible options and those who expected definite performance improvements from their use. There was also a strong correlation among those for whom the primary use was to sell the ideas more widely in their firm.

The report points out that in such cases the business benefits may have already been identified and the result may reflect the type of firm involved rather than the nature of the publication.

The study also found a very low level of discarded publications – less than two per cent of respondents – while nearly three quarters had kept the document themselves (59%) or passed it to a colleague.

Of those who felt that action had resulted from the materials (62%), a significant majority felt that the publications had added to the quality of the final project or activity and others felt that actions had been promoted at greater speed

The results of this study appear to suggest that publications can have a significant influence on the adoption of best practice – particularly where there is already a commitment to act and options are already being considered.

This evaluation looked at the Code’s use by LEA staff (160); teachers and governors (290) and parents of SEN children (296) (DfEE, 1998) Nearly two-thirds of LEA staff referred directly to the Code two or three times a month; as did three-quarters of school staff at least once a term. The Code was used in particular to check criteria and time scales for assessment and to examine or confirm procedures. The explanation of the 5-stage model for the identification and assessment of special needs, and suggested procedures, were the most highly rated parts.

Guidance information and databases are commonly used ways of presenting new ideas and practices to practitioners and continue to play an important role in dissemination strategies. They have advantages: their availability is long lasting (they can be used as a continuing reference base when required) and they fit well with the learning preferences of many user groups. If used effectively, they can provide the latest examples of best practices to the largest possible audience. One suggested way of improving their impact is through a system of coding or ‘flagging up’ to busy practitioners the main uses for which they are designed. Such categories could include a distinction between what you need to do (e.g. codes of practice) and what will help you to work effectively (guidance and best practice examples).

However, most data sources still suggest that information presented on its own will not change practice. Instead, in order to be effective, guidance and databases should be incorporated as part of an holistic approach.

The notion of creating champions as key agents of change in the public sector has emerged strongly in recent government policy and academic literature. However, the word, ‘champions’ has often been used interchangeably with the concepts of ‘leaders’ or ‘catalysts of change’ (Lloyd, 1999). One of the most recent references to champions was in the Wiring It Up report by the Performance and Innovation Unit which proposed the creation of ‘Ministerial Champions’ for cross-cutting policies (Cabinet Office, 2000).

Below is one of the latest strategies for taking this policy forward, led by the DfEE. In it, the DfEE describes how local champions will be created to spread best practice amongst their peers and across agencies and sectors. It describes how their roles will be developed through targeted support from the government, as well as from senior management.

The concept of local champions is a key element of the new strategy being developed by the DfEE to spread best practice (DfEE, 1998). Local champions are people who have key skills, knowledge or experiences which they are willing to spread to other peers within their profession. In other words, they are ‘leaders’ of their fields, able to advise others about the latest or most effective forms of practice, although they are not necessarily senior figures in the management hierarchy. The goal of the DfEE is to identify these champions and ensure that they are supported in their role. This can be done by:

The DfEE has also devised a clear strategy to enable organisations to gain the most from local champions. This involves:

If the strategy is implemented successfully, local champions will become key individuals in the spreading of new ideas and knowledge across the education sector. But there are ways in which their value could be drained if managers are not careful. Managers should therefore take care that the champions are not being used too much, exhausting their time and good will; that they do not become objects of envy within their organisation, reducing team spirit; and that they are kept informed of new practice, thereby making their contribution relevant and up to date.

Back to Top

Promoting champions as catalysts of change, like promoting leadership, has become a core concern of policy makers. The current belief is that champions:

However, in order for champions to work, they need to be supported, consulted and involved regularly by decision makers, thus updating their knowledge and strengthening their capacity to influence others.

Sharing decisions with patients: is the information good enough?

Although primarily about what forms of information patients, rather than practitioners, prefer, this article provides a good account of how information (regarding often-complex procedures and practices) can be disseminated and understood (Coulter, Entwistle and Gilbert, 1999).

Based on a study of patient information for 10 common treatments and conditions, the authors constructed the following ‘process’ checklist for policy-makers concerned with providing effective information: