Evidence based practice
Encyclopedia
Evidence-based practice is an interdisciplinary approach gaining ground after 1992. It started in medicine as evidence-based medicine (EBM) and spread to other fields such as nursing, psychology, education, library and information science and other fields. Its basic principles are that all practical decisions maken should 1) be based on research studies and 2) that these research studies are selected and interpreted according to some specific norms characteristic for EBP. Typically such norms disregards theoretical studies and qualitative studies and consider quantitative studies according to a narrow set of criteria of what counts as evidence. If such a narrow set of methodological criteria are not applied, it is better instead just to speak of research based practice.
Evidence-based practice (EBP) "entails making decisions about how to promote health or provide care by integrating the best available evidence with practitioner expertise and other resources, and with the characteristics, state, needs, values and preferences of those who will be affected. This is done in a manner that is compatible with the environmental and organizational context. Evidence is research findings derived from the systematic collection of data through observation and experiment and the formulation of questions and testing of hypotheses" (www.ebbp.org).
Empirically-supported treatments (ESTs) are defined as "clearly specified psychological treatments shown to be efficacious in controlled research with a delineated population" (Chambless & Hollon 1998).
Areas of professional practice, such as medicine, psychology, psychiatry, rehabilitation and so forth, have had periods in their pasts where practice was based on loose bodies of knowledge. Some of the knowledge was lore that drew upon the experiences of generations of practitioners, and much of it had no valid scientific evidence on which to justify various practices.
In the past, this has often left the door open to quackery
perpetrated by individuals who had no training at all in the domain, but who wished to convey the impression that they did for profit or other motives. As the scientific method became increasingly recognized as the means to provide sound validation for such methods, it became clear that there needed to be a way of excluding quack practitioners not only as a way of preserving the integrity of the field (particularly medicine), but also of protecting the public from the dangers of their "cures." Furthermore, even where overt quackery was not present, it was recognized that there was a value in identifying what actually does work so it could be improved and promoted.
The notion of evidence based practice has also had an influence in the field of education. Here, some commentators have suggested that the putative lack of any conspicuous progress is attributable to practice resting in the unconnected and noncumulative experience of thousands of individual teachers, each re-inventing the wheel and failing to learn from hard scientific evidence about 'what works'. Opponents of this view argue that hard scientific evidence is a misnomer in education; knowing that a drug works (in medicine) is entirely different from knowing that a teaching method works, for the latter will depend on a host of factors, not least those to do with the style, personality and beliefs of the teacher and the needs of the particular children. Thus, opponents of EBP in education suggest that all teachers do indeed need to develop their own personal practice, dependent on personal knowledge garnered through their own experience.
EBT uses various methods (e.g. carefully summarizing research, putting out accessible research summaries, educating professionals in how to understand and apply research findings) to encourage, and in some instances to force, professionals and other decision-makers to pay more attention to evidence that can inform their decision-making. Where EBT is applied, it encourages professionals to use the best evidence possible, i.e. the most appropriate information available.
Much of the credit for today’s EBP techniques belongs to Archie Cochrane
, an epidemiologist, author of the book, Effectiveness and Efficiency: Random Reflections on Health Services. Cochrane suggested that because resources would always be limited, they should be used to provide forms of health care which had been shown in properly designed evaluations to be effective. Cochrane maintained that the most reliable evidence was that which came from randomised controlled trials (RCTs).
EBP develops individualized guidelines of best practices to inform the improvement of whatever professional task is at hand. Evidence-based practice is a philosophical approach that is in opposition to rules of thumb, folklore, and tradition
. Examples of a reliance on "the way it was always done" can be found in almost every profession, even when those practices are contradicted by new and better information.
However, in spite of the enthusiasm for EBP over the last decade or two, some authors have redefined EBP in ways that contradict, or at least add other factors to, the original emphasis on empirical research foundations. For example, EBP may be defined as treatment choices based not only on outcome research but also on practice wisdom (the experience of the clinician) and on family values (the preferences and assumptions of a client and his or her family or subculture).
Research oriented scientists, as opposed to authors, test whether particular practices work better for different subcultures or personality types, rather than just accept received wisdom. For example, the MATCH Study run at many sites around the US by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) tested whether particular types of clients with alcohol dependence would benefit differentially from three different treatment approaches to which they were randomly assigned. The idea was not to test the approaches but the matching of clients to treatments, and though this missed the question of client choice, it did demonstrate a lack of difference between the different approaches regardless of most client characteristics, with the exception that clients with high anger scores did better with the non-confrontational Motivational Enhancement approach which has been demonstrated superior in a meta-analysis of alcohol treatment outcome research and only required four as opposed to twelve session within Project MATCH.
The theories of evidence based practice are becoming more commonplace in nursing care. Nurses who are “baccalaureate prepared are expected to seek out and collaborate with other types of nurses to demonstrate the positives of a practice that is based on evidence.” Looking at a few types of articles to examine how this type of practice has influenced the standard of care is important but rarely internally valid. None of the articles specify what their biases are. Evidence based practice has gotten its reputation by examining the reasons why any and all procedures, treatments, and medicines are given. This is important for refining practice so the goal of assuring patient safety is met.
and development decisions are made after reviewing information from repeated rigorous data gathering
instead of relying on rules
, single observations, or custom
. Evidence-based medicine
and evidence-based nursing practice are the two largest fields employing this approach. In psychiatry
and community mental health, evidence-based practice guides have been created by such organizations as the Substance Abuse and Mental Health Services Administration
and the Robert Wood Johnson Foundation
, in conjunction with the National Alliance on Mental Illness
. Evidence-based practice has now spread into a diverse range of areas outside of health where the same principles are known by names such as results-focused policy, managing for outcomes, evidence-informed practice etc.
This model of care has been studied for 30 years in universities and is gradually making its way into the public sector. It effectively moves away from the old “medical model” (You have a disease, take this pill.) to a “evidence presented model” using the patient as the starting point in diagnosis. EBPs are being employed in the fields of health care, juvenile justice, mental health and social services among others. The theories of evidence based practice are becoming more commonplace in the nursing care. Nurses who are “baccalaureate prepared are expected to seek out and collaborate with other types of nurses to demonstrate the positives of a practice that is based on evidence.”
Key elements in using the best evidence to guide the practice of any professional include the development of questions using research-based evidence, the level and types of evidence to be used, and the assessment of effectiveness after completing the task or effort. One obvious problem with EBP in any field is the use of poor quality, contradictory, or incomplete evidence. Evidence-based practice continues to be a developing body of work for professions as diverse as education
, psychology
, economics
, nursing
, social work
and architecture
.
therapy, past lives therapy, chiropractic
manipulation, and Erhard Seminars Training
. The limitation to the study was that some subjects may not have been discredited even though there was no evidence for efficacy. It was recommended that future polls take this into consideration, though the researchers concluded that the study does identify the dark side or "quack factor" of modern mental health practice.
A second major flaw in this survey was the lack of safeguards. As it stands there is no evidence, except self-report, as to what the respondents actually knew about the techniques they were "voting on". Thus the results could reflect nothing more reliable than the respondents' personal opinions and prejudices. There is no way of telling how far this is, or is not, the case.
Evaluation of research quality can be a difficult task requiring meticulous reading of research reports and background information. It may not be appropriate simply to accept the conclusion reported by the researchers; for example, in one investigation of outcome studies, 70% were found to have stated conclusions unjustified by their research design.
Although early consideration of EBP issues by psychologists provided a stringent but simple definition of EBP, requiring two independent randomized controlled trials supporting the effectiveness of a treatment, it became clear that additional factors needed to be considered. These included both the need for lower but still useful levels of evidence, and the need to require even the "gold standard" randomized trials to meet further criteria.
A number of protocols for the evaluation of research reports have been suggested and will be summarized here. Some of these divide research evidence dichotomously into EBP and non-EBP categories, while others employ multiple levels of evidence. As the reader will see, although the criteria used by the various protocols overlap to some extent, they do not do so completely.
The Kaufman Best Practices Project approach did not use an EBP category per se, but instead provided a protocol for selecting the most acceptable treatment from a group of interventions intended to treat the same problems. To be designated as "best practice", a treatment would need to have a sound theoretical base, general acceptance in clinical practice, and considerable anecdotal or clinical literature. This protocol also requires absence of evidence of harm, at least one randomized controlled study, descriptive publications, a reasonable amount of necessary training, and the possibility of being used in common settings. Missing from this protocol are the possibility of nonrandomized designs (in which clients or practitioners decide whether an individual will receive a certain treatment), the need to specify the type of comparison group used, the existence of confounding variables, the reliability or validity of outcome measures, the type of statistical analysis required, or a number of other factors required by some evaluation protocols.
A protocol suggested by Saunders et al. assigns research reports to six categories, on the basis of research design, theoretical background, evidence of possible harm, and general acceptance. To be classified under this protocol, there must be descriptive publications, including a manual or similar description of the intervention. This protocol does not consider the nature of any comparison group, the effect of confounding variables, the nature of the statistical analysis, or a number of other criteria. Interventions are assessed as belonging to Category 1, well-supported, efficacious treatments, if there are two or more randomized controlled outcome studies comparing the target treatment to an appropriate alternative treatment and showing a significant advantage to the target treatment. Interventions are assigned to Category 2, supported and probably efficacious treatment, based on positive outcomes of nonrandomized designs with some form of control, which may involve a non-treatment group. Category 3, supported and acceptable treatment, includes interventions supported by one controlled or uncontrolled study, or by a series of single-subject studies, or by work with a different population than the one of interest. Category 4, promising and acceptable treatment, includes interventions that have no support except general acceptance and clinical anecdotal literature; however, any evidence of possible harm excludes treatments from this category. Category 5, innovative and novel treatment, includes interventions that are not thought to be harmful, but are not widely used or discussed in the literature. Category 6, concerning treatment, is the classification for treatments that have the possibility of doing harm, as well as having unknown or inappropriate theoretical foundations.
A protocol for evaluation of research quality was suggested by a report from the Centre for Reviews and Dissemination, prepared by Khan et al. and intended as a general method for assessing both medical and psychosocial interventions. While strongly encouraging the use of randomized designs, this protocol noted that such designs were useful only if they met demanding criteria, such as true randomization and concealment of the assigned treatment group from the client and from others, including the individuals assessing the outcome. The Khan et al. protocol emphasized the need to make comparisons on the basis of "intention to treat" in order to avoid problems related to greater attrition in one group. The Khan et al. protocol also presented demanding criteria for nonrandomized studies, including matching of groups on potential confounding variables and adequate descriptions of groups and treatments at every stage, and concealment of treatment choice from persons assessing the outcomes. This protocol did not provide a classification of levels of evidence, but included or excluded treatments from classification as evidence-based depending on whether the research met the stated standards.
An assessment protocol has been developed by the U.S. National Registry of Evidence-Based Practices and Programs (NREPP). Evaluation under this protocol occurs only if an intervention has already had one or more positive outcomes, with a probability of less than .05, reported, if these have been published in a peer-reviewed journal or an evaluation report, and if documentation such as training materials has been made available. The NREPP evaluation, which assigns quality ratings from 1 to 4 to certain criteria, examines reliability and validity of outcome measures used in the research, evidence for intervention fidelity (predictable use of the treatment in the same way every time), levels of missing data and attrition, potential confounding variables, and the appropriateness of statistical handling, including sample size.
A protocol suggested by Mercer and Pignotti uses a taxonomy intended to classify on both research quality and other criteria. In this protocol, evidence-based interventions are those supported by work with randomized designs employing comparisons to established treatments, independent replications of results, blind evaluation of outcomes, and the existence of a manual. Evidence-supported interventions are those supported by nonrandomized designs, including within-subjects designs, and meeting the criteria for the previous category. Evidence-informed treatments involve case studies or interventions tested on populations other than the targeted group, without independent replications; a manual exists, and there is no evidence of harm or potential for harm. Belief-based interventions have no published research reports or reports based on composite cases; they may be based on religious or ideological principles or may claim a basis in accepted theory without an acceptable rationale; there may or may not be a manual, and there is no evidence of harm or potential for harm. Finally, the category of potentially harmful treatments includes interventions such that harmful mental or physical effects have been documented, or a manual or other source shows the potential for harm.
Protocols for evaluation of research quality are still in development. So far, the available protocols pay relatively little attention to whether outcome research is relevant to efficacy (the outcome of a treatment performed under ideal conditions) or to effectiveness (the outcome of the treatment performed under ordinary, expectable conditions).
In other situations, facts about a group of study outcomes may be gathered and discussed in the form of a systematic research synthesis (SRS). A SRS can be more or less useful, depending on the evaluation protocol chosen, and errors in choice or use of a protocol have led to fallacious reports. The meaningfulness of a SRS report on an intervention is limited by the quality of the research under consideration, but SRS reports can be helpful to readers seeking to understand EBP-related choices.
Miller et al. provide an excellent example and explication of the use of meta-analysis examining treatment outcome research, incorporating the principles of rigorous empirical research from the strong end of the continuum of levels of evidence. This textbook also explicates how the research included was selected (e.g. controlled study looking at two different approaches, appearing in a peer reviewed journal, sufficient power to find significant differences if they occurred) and how each study was checked for validity (how was the outcome measured?) and reliability (did the research do what they said they did?), etc. to create a Cumulative Evidence Score weighted by the quality of the study (and not by the outcome) such that better studies with "stronger designs" and better methodological quality ratings carry more weight than weaker studies.The results lead to a rank ordering of the 48 treatment modalities included and provide a basis for selecting supportable treatment approaches beyond anecdotes, traditions and lore.
directed at obtaining evidence about social programs of all types. A research collaboration called the Campbell Collaboration has been set up in the social policy area to provide evidence for evidence-based social policy decision-making. This collaboration follows the approach pioneered by the Cochrane Collaboration in the health sciences. Using an evidence-based approach to social policy has a number of advantages because it has the potential to decrease the tendency to run programs which are socially acceptable (e.g. drug education in schools) but which often prove to be ineffective when evaluated.
Evidence-based practice (EBP) "entails making decisions about how to promote health or provide care by integrating the best available evidence with practitioner expertise and other resources, and with the characteristics, state, needs, values and preferences of those who will be affected. This is done in a manner that is compatible with the environmental and organizational context. Evidence is research findings derived from the systematic collection of data through observation and experiment and the formulation of questions and testing of hypotheses" (www.ebbp.org).
Empirically-supported treatments (ESTs) are defined as "clearly specified psychological treatments shown to be efficacious in controlled research with a delineated population" (Chambless & Hollon 1998).
EBP and the history of medicine and education
In recent years, EBP has been stressed by professional organizations such as the American Psychological Association, the American Occupational Therapy Association, the American Nurses Association, and the American Physical Therapy Association, which have also strongly recommended their members to carry out investigations to provide evidence supporting or rejecting the use of specific interventions. Equivalent recommendations apply to the Canadian equivalent of these associations. Pressure toward EBP has also come from public and private health insurance providers, which have sometimes refused coverage of practices lacking in systematic evidence of usefulness.Areas of professional practice, such as medicine, psychology, psychiatry, rehabilitation and so forth, have had periods in their pasts where practice was based on loose bodies of knowledge. Some of the knowledge was lore that drew upon the experiences of generations of practitioners, and much of it had no valid scientific evidence on which to justify various practices.
In the past, this has often left the door open to quackery
Quackery
Quackery is a derogatory term used to describe the promotion of unproven or fraudulent medical practices. Random House Dictionary describes a "quack" as a "fraudulent or ignorant pretender to medical skill" or "a person who pretends, professionally or publicly, to have skill, knowledge, or...
perpetrated by individuals who had no training at all in the domain, but who wished to convey the impression that they did for profit or other motives. As the scientific method became increasingly recognized as the means to provide sound validation for such methods, it became clear that there needed to be a way of excluding quack practitioners not only as a way of preserving the integrity of the field (particularly medicine), but also of protecting the public from the dangers of their "cures." Furthermore, even where overt quackery was not present, it was recognized that there was a value in identifying what actually does work so it could be improved and promoted.
The notion of evidence based practice has also had an influence in the field of education. Here, some commentators have suggested that the putative lack of any conspicuous progress is attributable to practice resting in the unconnected and noncumulative experience of thousands of individual teachers, each re-inventing the wheel and failing to learn from hard scientific evidence about 'what works'. Opponents of this view argue that hard scientific evidence is a misnomer in education; knowing that a drug works (in medicine) is entirely different from knowing that a teaching method works, for the latter will depend on a host of factors, not least those to do with the style, personality and beliefs of the teacher and the needs of the particular children. Thus, opponents of EBP in education suggest that all teachers do indeed need to develop their own personal practice, dependent on personal knowledge garnered through their own experience.
Evidence Based Treatement
Evidence-based treatment (EBT) is an approach that tries to specify the way in which professionals or other decision-makers should make decisions by identifying such evidence that there may be for a practice, and rating it according to how scientifically sound it may be. Its goal is to eliminate unsound or excessively risky practices in favor of those that have better outcomes.EBT uses various methods (e.g. carefully summarizing research, putting out accessible research summaries, educating professionals in how to understand and apply research findings) to encourage, and in some instances to force, professionals and other decision-makers to pay more attention to evidence that can inform their decision-making. Where EBT is applied, it encourages professionals to use the best evidence possible, i.e. the most appropriate information available.
The current implementation of EBP
The core activities at the root of evidence-based medicine can be identified as:- a questioning approach to practice leading to scientific experimentation
- meticulous observation, enumeration, and analysis replacing anecdotal case description, for instance, EBSCO's Dynamed.
- recording and cataloguing the evidence for systematic retrieval.
Much of the credit for today’s EBP techniques belongs to Archie Cochrane
Archie Cochrane
Archibald Leman Cochrane was a Scottish doctor whose name is synonymous with scientific method in medicine.-Biography:Cochrane was born in Kirklands, Galashiels, Scotland...
, an epidemiologist, author of the book, Effectiveness and Efficiency: Random Reflections on Health Services. Cochrane suggested that because resources would always be limited, they should be used to provide forms of health care which had been shown in properly designed evaluations to be effective. Cochrane maintained that the most reliable evidence was that which came from randomised controlled trials (RCTs).
Evidence-based practice vs. intuition
Evidence-based practice (EBP) involves complex and conscientious decision-making which is based not only on the available evidence but also on patient characteristics, situations, and preferences. It recognizes that care is individualized and ever changing and involves uncertainties and probabilities.EBP develops individualized guidelines of best practices to inform the improvement of whatever professional task is at hand. Evidence-based practice is a philosophical approach that is in opposition to rules of thumb, folklore, and tradition
Tradition
A tradition is a ritual, belief or object passed down within a society, still maintained in the present, with origins in the past. Common examples include holidays or impractical but socially meaningful clothes , but the idea has also been applied to social norms such as greetings...
. Examples of a reliance on "the way it was always done" can be found in almost every profession, even when those practices are contradicted by new and better information.
However, in spite of the enthusiasm for EBP over the last decade or two, some authors have redefined EBP in ways that contradict, or at least add other factors to, the original emphasis on empirical research foundations. For example, EBP may be defined as treatment choices based not only on outcome research but also on practice wisdom (the experience of the clinician) and on family values (the preferences and assumptions of a client and his or her family or subculture).
Research oriented scientists, as opposed to authors, test whether particular practices work better for different subcultures or personality types, rather than just accept received wisdom. For example, the MATCH Study run at many sites around the US by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) tested whether particular types of clients with alcohol dependence would benefit differentially from three different treatment approaches to which they were randomly assigned. The idea was not to test the approaches but the matching of clients to treatments, and though this missed the question of client choice, it did demonstrate a lack of difference between the different approaches regardless of most client characteristics, with the exception that clients with high anger scores did better with the non-confrontational Motivational Enhancement approach which has been demonstrated superior in a meta-analysis of alcohol treatment outcome research and only required four as opposed to twelve session within Project MATCH.
The theories of evidence based practice are becoming more commonplace in nursing care. Nurses who are “baccalaureate prepared are expected to seek out and collaborate with other types of nurses to demonstrate the positives of a practice that is based on evidence.” Looking at a few types of articles to examine how this type of practice has influenced the standard of care is important but rarely internally valid. None of the articles specify what their biases are. Evidence based practice has gotten its reputation by examining the reasons why any and all procedures, treatments, and medicines are given. This is important for refining practice so the goal of assuring patient safety is met.
Research-based evidence
Evidence-based designEvidence-based design
Evidence-based design often shortened to EBD is a field of study that emphasizes the importance of using credible data in order to influence the design process. The approach has become popular in Healthcare Architecture in an effort to improve patient and staff well-being, patient healing...
and development decisions are made after reviewing information from repeated rigorous data gathering
Experiment
An experiment is a methodical procedure carried out with the goal of verifying, falsifying, or establishing the validity of a hypothesis. Experiments vary greatly in their goal and scale, but always rely on repeatable procedure and logical analysis of the results...
instead of relying on rules
Expert system
In artificial intelligence, an expert system is a computer system that emulates the decision-making ability of a human expert. Expert systems are designed to solve complex problems by reasoning about knowledge, like an expert, and not by following the procedure of a developer as is the case in...
, single observations, or custom
Norm (sociology)
Social norms are the accepted behaviors within a society or group. This sociological and social psychological term has been defined as "the rules that a group uses for appropriate and inappropriate values, beliefs, attitudes and behaviors. These rules may be explicit or implicit...
. Evidence-based medicine
Evidence-based medicine
Evidence-based medicine or evidence-based practice aims to apply the best available evidence gained from the scientific method to clinical decision making. It seeks to assess the strength of evidence of the risks and benefits of treatments and diagnostic tests...
and evidence-based nursing practice are the two largest fields employing this approach. In psychiatry
Psychiatry
Psychiatry is the medical specialty devoted to the study and treatment of mental disorders. These mental disorders include various affective, behavioural, cognitive and perceptual abnormalities...
and community mental health, evidence-based practice guides have been created by such organizations as the Substance Abuse and Mental Health Services Administration
Substance Abuse and Mental Health Services Administration
The Substance Abuse and Mental Health Services Administration is a branch of the U.S. Department of Health and Human Services. It is charged with improving the quality and availability of prevention, treatment, and rehabilitative services in order to reduce illness, death, disability, and cost to...
and the Robert Wood Johnson Foundation
Robert Wood Johnson Foundation
The Robert Wood Johnson Foundation is the United States' largest philanthropy devoted exclusively to health and health care; it is based in Princeton, New Jersey. The foundation's mission is to improve the health and health care of all Americans...
, in conjunction with the National Alliance on Mental Illness
National Alliance on Mental Illness
The National Alliance on Mental Illness was founded in 1979 as the National Alliance for the Mentally Ill. NAMI is a nation-wide American advocacy group, representing families and people affected by mental illness as a non-profit grass roots organization and has affiliates in every American state...
. Evidence-based practice has now spread into a diverse range of areas outside of health where the same principles are known by names such as results-focused policy, managing for outcomes, evidence-informed practice etc.
This model of care has been studied for 30 years in universities and is gradually making its way into the public sector. It effectively moves away from the old “medical model” (You have a disease, take this pill.) to a “evidence presented model” using the patient as the starting point in diagnosis. EBPs are being employed in the fields of health care, juvenile justice, mental health and social services among others. The theories of evidence based practice are becoming more commonplace in the nursing care. Nurses who are “baccalaureate prepared are expected to seek out and collaborate with other types of nurses to demonstrate the positives of a practice that is based on evidence.”
Key elements in using the best evidence to guide the practice of any professional include the development of questions using research-based evidence, the level and types of evidence to be used, and the assessment of effectiveness after completing the task or effort. One obvious problem with EBP in any field is the use of poor quality, contradictory, or incomplete evidence. Evidence-based practice continues to be a developing body of work for professions as diverse as education
Education
Education in its broadest, general sense is the means through which the aims and habits of a group of people lives on from one generation to the next. Generally, it occurs through any experience that has a formative effect on the way one thinks, feels, or acts...
, psychology
Psychology
Psychology is the study of the mind and behavior. Its immediate goal is to understand individuals and groups by both establishing general principles and researching specific cases. For many, the ultimate goal of psychology is to benefit society...
, economics
Economics
Economics is the social science that analyzes the production, distribution, and consumption of goods and services. The term economics comes from the Ancient Greek from + , hence "rules of the house"...
, nursing
Nursing
Nursing is a healthcare profession focused on the care of individuals, families, and communities so they may attain, maintain, or recover optimal health and quality of life from conception to death....
, social work
Social work
Social Work is a professional and academic discipline that seeks to improve the quality of life and wellbeing of an individual, group, or community by intervening through research, policy, community organizing, direct practice, and teaching on behalf of those afflicted with poverty or any real or...
and architecture
Architecture
Architecture is both the process and product of planning, designing and construction. Architectural works, in the material form of buildings, are often perceived as cultural and political symbols and as works of art...
.
Psychology
According to Norcross et al. (2006) "the burgeoning evidence based practice movement in mental health attempts to identify, implement, and disseminate treatments that have been proven demonstrably effective according to the empirical evidence". However, Norcross et al. (2006) also state that perhaps it is more useful to identify what does not work. They conducted a survey rating experts opinions of "not at all discredited" to "certainly discredited" for a range of treatments. Examples of the range of discredited psychotherapies includes: angel therapy, the use of pyramid structures, orgoneOrgone
Orgone energy is a theory originally proposed in the 1930s by Wilhelm Reich. Reich, originally part of Sigmund Freud's Vienna circle, extrapolated the Freudian concept of libido first as a biophysical and later as a universal life force...
therapy, past lives therapy, chiropractic
Chiropractic
Chiropractic is a health care profession concerned with the diagnosis, treatment and prevention of disorders of the neuromusculoskeletal system and the effects of these disorders on general health. It is generally categorized as complementary and alternative medicine...
manipulation, and Erhard Seminars Training
Erhard Seminars Training
Erhard Seminars Training, an organization founded by Werner H. Erhard, offered a two-weekend course known officially as "The est Standard Training"...
. The limitation to the study was that some subjects may not have been discredited even though there was no evidence for efficacy. It was recommended that future polls take this into consideration, though the researchers concluded that the study does identify the dark side or "quack factor" of modern mental health practice.
A second major flaw in this survey was the lack of safeguards. As it stands there is no evidence, except self-report, as to what the respondents actually knew about the techniques they were "voting on". Thus the results could reflect nothing more reliable than the respondents' personal opinions and prejudices. There is no way of telling how far this is, or is not, the case.
Levels of evidence and evaluation of research
Because conclusions about research results are made in a probabilistic manner, it is impossible to work with two simple categories of outcome research reports. Research evidence does not fall simply into "evidence-based" and "non-evidence-based" classes, but can be anywhere on a continuum from one to the other, depending on factors such as the way the study was designed and carried out. The existence of this continuum makes it necessary to think in terms of "levels of evidence", or categories of stronger or weaker evidence that a treatment is effective. To classify a research report as strong or weak evidence for a treatment, it is necessary to evaluate the quality of the research as well as the reported outcome.Evaluation of research quality can be a difficult task requiring meticulous reading of research reports and background information. It may not be appropriate simply to accept the conclusion reported by the researchers; for example, in one investigation of outcome studies, 70% were found to have stated conclusions unjustified by their research design.
Although early consideration of EBP issues by psychologists provided a stringent but simple definition of EBP, requiring two independent randomized controlled trials supporting the effectiveness of a treatment, it became clear that additional factors needed to be considered. These included both the need for lower but still useful levels of evidence, and the need to require even the "gold standard" randomized trials to meet further criteria.
A number of protocols for the evaluation of research reports have been suggested and will be summarized here. Some of these divide research evidence dichotomously into EBP and non-EBP categories, while others employ multiple levels of evidence. As the reader will see, although the criteria used by the various protocols overlap to some extent, they do not do so completely.
The Kaufman Best Practices Project approach did not use an EBP category per se, but instead provided a protocol for selecting the most acceptable treatment from a group of interventions intended to treat the same problems. To be designated as "best practice", a treatment would need to have a sound theoretical base, general acceptance in clinical practice, and considerable anecdotal or clinical literature. This protocol also requires absence of evidence of harm, at least one randomized controlled study, descriptive publications, a reasonable amount of necessary training, and the possibility of being used in common settings. Missing from this protocol are the possibility of nonrandomized designs (in which clients or practitioners decide whether an individual will receive a certain treatment), the need to specify the type of comparison group used, the existence of confounding variables, the reliability or validity of outcome measures, the type of statistical analysis required, or a number of other factors required by some evaluation protocols.
A protocol suggested by Saunders et al. assigns research reports to six categories, on the basis of research design, theoretical background, evidence of possible harm, and general acceptance. To be classified under this protocol, there must be descriptive publications, including a manual or similar description of the intervention. This protocol does not consider the nature of any comparison group, the effect of confounding variables, the nature of the statistical analysis, or a number of other criteria. Interventions are assessed as belonging to Category 1, well-supported, efficacious treatments, if there are two or more randomized controlled outcome studies comparing the target treatment to an appropriate alternative treatment and showing a significant advantage to the target treatment. Interventions are assigned to Category 2, supported and probably efficacious treatment, based on positive outcomes of nonrandomized designs with some form of control, which may involve a non-treatment group. Category 3, supported and acceptable treatment, includes interventions supported by one controlled or uncontrolled study, or by a series of single-subject studies, or by work with a different population than the one of interest. Category 4, promising and acceptable treatment, includes interventions that have no support except general acceptance and clinical anecdotal literature; however, any evidence of possible harm excludes treatments from this category. Category 5, innovative and novel treatment, includes interventions that are not thought to be harmful, but are not widely used or discussed in the literature. Category 6, concerning treatment, is the classification for treatments that have the possibility of doing harm, as well as having unknown or inappropriate theoretical foundations.
A protocol for evaluation of research quality was suggested by a report from the Centre for Reviews and Dissemination, prepared by Khan et al. and intended as a general method for assessing both medical and psychosocial interventions. While strongly encouraging the use of randomized designs, this protocol noted that such designs were useful only if they met demanding criteria, such as true randomization and concealment of the assigned treatment group from the client and from others, including the individuals assessing the outcome. The Khan et al. protocol emphasized the need to make comparisons on the basis of "intention to treat" in order to avoid problems related to greater attrition in one group. The Khan et al. protocol also presented demanding criteria for nonrandomized studies, including matching of groups on potential confounding variables and adequate descriptions of groups and treatments at every stage, and concealment of treatment choice from persons assessing the outcomes. This protocol did not provide a classification of levels of evidence, but included or excluded treatments from classification as evidence-based depending on whether the research met the stated standards.
An assessment protocol has been developed by the U.S. National Registry of Evidence-Based Practices and Programs (NREPP). Evaluation under this protocol occurs only if an intervention has already had one or more positive outcomes, with a probability of less than .05, reported, if these have been published in a peer-reviewed journal or an evaluation report, and if documentation such as training materials has been made available. The NREPP evaluation, which assigns quality ratings from 1 to 4 to certain criteria, examines reliability and validity of outcome measures used in the research, evidence for intervention fidelity (predictable use of the treatment in the same way every time), levels of missing data and attrition, potential confounding variables, and the appropriateness of statistical handling, including sample size.
A protocol suggested by Mercer and Pignotti uses a taxonomy intended to classify on both research quality and other criteria. In this protocol, evidence-based interventions are those supported by work with randomized designs employing comparisons to established treatments, independent replications of results, blind evaluation of outcomes, and the existence of a manual. Evidence-supported interventions are those supported by nonrandomized designs, including within-subjects designs, and meeting the criteria for the previous category. Evidence-informed treatments involve case studies or interventions tested on populations other than the targeted group, without independent replications; a manual exists, and there is no evidence of harm or potential for harm. Belief-based interventions have no published research reports or reports based on composite cases; they may be based on religious or ideological principles or may claim a basis in accepted theory without an acceptable rationale; there may or may not be a manual, and there is no evidence of harm or potential for harm. Finally, the category of potentially harmful treatments includes interventions such that harmful mental or physical effects have been documented, or a manual or other source shows the potential for harm.
Protocols for evaluation of research quality are still in development. So far, the available protocols pay relatively little attention to whether outcome research is relevant to efficacy (the outcome of a treatment performed under ideal conditions) or to effectiveness (the outcome of the treatment performed under ordinary, expectable conditions).
Meta-analyses and systematic research syntheses
When there are many small or weak studies of an intervention, a statistical meta-analysis can be used to co-ordinate the studies' results and to draw a stronger conclusion about the outcome of the treatment. This can be an important contribution to the establishment of a foundation of evidence about an intervention.In other situations, facts about a group of study outcomes may be gathered and discussed in the form of a systematic research synthesis (SRS). A SRS can be more or less useful, depending on the evaluation protocol chosen, and errors in choice or use of a protocol have led to fallacious reports. The meaningfulness of a SRS report on an intervention is limited by the quality of the research under consideration, but SRS reports can be helpful to readers seeking to understand EBP-related choices.
Miller et al. provide an excellent example and explication of the use of meta-analysis examining treatment outcome research, incorporating the principles of rigorous empirical research from the strong end of the continuum of levels of evidence. This textbook also explicates how the research included was selected (e.g. controlled study looking at two different approaches, appearing in a peer reviewed journal, sufficient power to find significant differences if they occurred) and how each study was checked for validity (how was the outcome measured?) and reliability (did the research do what they said they did?), etc. to create a Cumulative Evidence Score weighted by the quality of the study (and not by the outcome) such that better studies with "stronger designs" and better methodological quality ratings carry more weight than weaker studies.The results lead to a rank ordering of the 48 treatment modalities included and provide a basis for selecting supportable treatment approaches beyond anecdotes, traditions and lore.
Social policy
There are increasing demands for the whole range of social policy and other decisions and programs run by government and the NGO sector to be based on sound evidence as to their effectiveness. This has seen an increased emphasis on the use of a wide range of Evaluation approachesEvaluation approaches
Evaluation approaches are conceptually distinct ways of thinking about, designing and conducting evaluation efforts. Many of the evaluation approaches in use today make unique contributions to solving important problems, while others refine existing approaches in some way...
directed at obtaining evidence about social programs of all types. A research collaboration called the Campbell Collaboration has been set up in the social policy area to provide evidence for evidence-based social policy decision-making. This collaboration follows the approach pioneered by the Cochrane Collaboration in the health sciences. Using an evidence-based approach to social policy has a number of advantages because it has the potential to decrease the tendency to run programs which are socially acceptable (e.g. drug education in schools) but which often prove to be ineffective when evaluated.
See also
- EpidemiologyEpidemiologyEpidemiology is the study of health-event, health-characteristic, or health-determinant patterns in a population. It is the cornerstone method of public health research, and helps inform policy decisions and evidence-based medicine by identifying risk factors for disease and targets for preventive...
- Evidence-based designEvidence-based designEvidence-based design often shortened to EBD is a field of study that emphasizes the importance of using credible data in order to influence the design process. The approach has become popular in Healthcare Architecture in an effort to improve patient and staff well-being, patient healing...
- Evidence Based Library and Information PracticeEvidence based library and information practiceEvidence Based Library and Information Practice or evidence based librarianship is the application of the interdisciplinary approach known as evidence-based practice to problems in the field of library and information science...
- Evidence-based managementEvidence-based managementEvidence-based management is an emerging movement to explicitly use the current, best evidence in management decision-making. Its roots are in evidence-based medicine, a quality movement to apply the scientific method to medical practice....
- Evidence-based medicineEvidence-based medicineEvidence-based medicine or evidence-based practice aims to apply the best available evidence gained from the scientific method to clinical decision making. It seeks to assess the strength of evidence of the risks and benefits of treatments and diagnostic tests...
- Evidence-based nursingEvidence-Based NursingEvidence-Based Nursing or EBN is a type of evidence-based healthcare, drawing on some of the traditions of evidence-based medicine. It involves identifying solid research findings and implementing them in nursing practices, in order to increase the quality of patient care. The goal of EBN is to...
- Evidence-based pharmacy in developing countriesEvidence-based pharmacy in developing countriesMany developing nations have developed national drug policies, a concept that has been actively promoted by the WHO. For example, the national drug policy for Indonesia drawn up in 1983 had the following objectives:...
- Dynamic treatment regimesDynamic treatment regimesIn medical research, a dynamic treatment regime or adaptive treatment strategy is a set of rules for choosing effective treatments for individual patients. The treatment choices made for a particular patient are based on that individual's characteristics and history, with the goal of optimizing...
- Improvement Science Research NetworkImprovement Science Research NetworkThe ' is a large-scale practice based research network and coordinating center that was created to accelerate inter-professional improvement science in a systems context across multiple hospital sites...
External links
- http://www.ebbp.org Evidence-Based Behavioral Practice Project
- NREPP National Registry of Evidence-based Programs and Practices
- Evidence-Based Practice @ Keele
- Evidence Based Practice Definitions
- The Joanna Briggs Institute - International Collaborative on Evidence-based Practice in Nursing.
- Indiana Center for Evidence Based Nursing Practice: A JBI Collaborating Center
- Evidence Communicate leading research in order to promote international cooperation and evidence-based treatments
- Evidence based Practice clinic intitiatives
- Evidence Based Practice studies at the University of Massachusetts
- Center for the Advancement of Evidence-Based Practice (CAEP) at Arizona State University College of Nursing and Health Innovation
- The National Nursing Practice Network
- Website about Evidence-Based teaching
- Improvement Science Research Network
- Academic Center for Evidence-Based Practice