Anecdotal evidence
Encyclopedia
The expression anecdotal evidence refers to evidence
from anecdotes. Because of the small sample, there is a larger chance that it may be true but unreliable due to cherry-picked or otherwise unrepresentative of typical cases.
Also, anecdotal evidence can be inaccurate, sometimes based on anecdote
s, second-hand accounts of events or hearsay
.
Anecdotal evidence, which may itself be true and verifiable, can be used to deduce a conclusion which does not follow from it, usually by generalising from an insufficient amount of evidence. For example "my grandfather smoked like a chimney and died healthy in a car crash at the age of 99" does not disprove the proposition that "smoking markedly increases the probability of cancer and heart disease at a relatively early age". While the evidence is true, it does not warrant the conclusion made from it.
Conclusions made from anecdotal evidence might not be untrue, but they are unreliable because they don't follow from the evidence and might easily be incorrect.
You can also take a generalization and then search for anecdotal evidence that "confirms" it. For example "Yogurt prolongs life. I didn't believe it, but then I heard that a man in a mountain village who ate only yogurt lived to 120."
The term is often used in contrast to scientific evidence
, such as evidence-based medicine
, which are types of formal accounts. Some anecdotal evidence does not qualify as scientific evidence because its nature prevents it from being investigated using the scientific method
. Misuse of anecdotal evidence is a logical fallacy and is sometimes informally referred to as the "person who" fallacy ("I know a person who..."; "I know of a case where..." etc. Compare with hasty generalization
). Anecdotal evidence is not necessarily representative of a "typical" experience; statistical evidence can more accurately determine how typical something is.
Accounts of direct personal experience are commonly equated to anecdotal evidence where the evidence is anecdote
, hearsay
or represents a conclusion deduced from generalisation. Unlike anecdotal evidence, the reliability of accounts of personal experience is normally capable of assessment for legal proceedings.
When used in advertising
or promotion of a product, service, or idea, anecdotal reports are often called a testimonial
, which are banned in some jurisdictions. The term is also sometimes used in a legal context to describe certain kinds of testimony
. Psychologists have found that people are more likely to remember notable examples than typical examples.
Anecdotal evidence can have varying degrees of formality. For instance, in medicine, published anecdotal evidence is called a case report
, which is a more formalized type of evidence subjected to peer review
. Although such evidence is not seen as conclusive, it is sometimes regarded as an invitation to more rigorous scientific study of the phenomenon in question. For instance, one study found that 35 of 47 anecdotal reports of side effects were later sustained as "clearly correct."
Anecdotal evidence is considered the least certain type of scientific information. Researchers may use anecdotal evidence for suggesting new hypotheses
, but never as validating evidence.
because various forms of cognitive bias
may affect the collection or presentation of evidence. For instance, someone who claims to have had an encounter with a supernatural being or alien may present a very vivid story, but this is not falsifiable
. This phenomenon can also happen to large groups of people through subjective validation
.
Anecdotal evidence is also frequently misinterpreted via the availability heuristic
, which leads to an overestimation of prevalence. Where a cause can be easily linked to an effect, people overestimate the likelihood of the cause having that effect (availability). In particular, vivid, emotionally-charged anecdotes seem more plausible, and are given greater weight. A related issue is that it is usually impossible to assess for every piece of anecdotal evidence, the rate of people not reporting that anecdotal evidence in the population.
A common way anecdotal evidence becomes unscientific is through fallacious
reasoning such as the Post hoc ergo propter hoc
fallacy, the human tendency to assume that if one event happens after another, then the first must be the cause of the second. Another fallacy involves inductive reasoning
. For instance, if an anecdote illustrates a desired conclusion rather than a logical conclusion, it is considered a faulty
or hasty generalization
. For example, here is anecdotal evidence presented as proof of a desired conclusion:
Anecdotes like this do not prove anything. In any case where some factor affects the probability of an outcome, rather than uniquely determining it, selected individual cases prove nothing; e.g. "my grandfather smoked 40 a day until he died at 90" and "my sister never went near anyone who smoked but died of lung cancer". Anecdotes often refer to the exception, rather than the rule: "Anecdotes are useless precisely because they may point to idiosyncratic responses." Even when many anecdotes are collected to prove a point, "The plural of anecdote is not data." (Roger Brinner)
More generally, a statistical correlation between things does not in itself prove that one causes the other (a causal link). A study found that television viewing was strongly correlated with sugar consumption, but this does not prove that viewing causes sugar intake (or viceversa).
In medicine anecdotal evidence is also subject to placebo effect
s: it is well-established that a patient's (or doctor's) expectation can genuinely change the outcome of treatment. Only double-blind
randomized
placebo
-controlled clinical trial
s can confirm a hypothesis
about the effectiveness of a treatment independently of expectations.
Sites devoted to rhetoric
often give explanations along these lines:
By contrast, in science and logic, the "relative strength of an explanation" is based upon its ability to be tested, proven to be due to the stated cause, and verified under neutral conditions in a manner that other researchers will agree has been performed competently, and can check for themselves.
testimony
is a common form of evidence
in law, and law has mechanisms to test witness evidence for reliability or credibility. Legal processes for the taking and assessment of evidence are formalized. Some witness testimony could be described as anecdotal evidence, such as individual stories of harassment
as part of a class action lawsuit. However, witness testimony can be tested and assessed for reliability. Examples of approaches to testing and assessment include the use of questioning, evidence of corroborating witnesses, documents, video and forensic evidence. Where a court lacks suitable means to test and assess testimony of a particular witness, such as the absence of forms of corroboration or substantiation it may afford that testimony limited or no "weight" when making a decision on the facts.
. This ruling holds that before evidence is presented to witnesses by experts, the methodology must be "generally accepted" among scientists. In some situations, anecdotal evidence may meet this threshold (such as certain case reports which corroborate or refute other evidence).
Altman and Bland argue that the case report or statistical outlier cannot be dismissed as having no weight: "With rare and uncommonly occurring diseases, a nonsignificant finding in a randomized trial does not necessarily mean that there is no causal association between the agent in question and the disease."
Evidence
Evidence in its broadest sense includes everything that is used to determine or demonstrate the truth of an assertion. Giving or procuring evidence is the process of using those things that are either presumed to be true, or were themselves proven via evidence, to demonstrate an assertion's truth...
from anecdotes. Because of the small sample, there is a larger chance that it may be true but unreliable due to cherry-picked or otherwise unrepresentative of typical cases.
Also, anecdotal evidence can be inaccurate, sometimes based on anecdote
Anecdote
An anecdote is a short and amusing or interesting story about a real incident or person. It may be as brief as the setting and provocation of a bon mot. An anecdote is always presented as based on a real incident involving actual persons, whether famous or not, usually in an identifiable place...
s, second-hand accounts of events or hearsay
Hearsay
Hearsay is information gathered by one person from another person concerning some event, condition, or thing of which the first person had no direct experience. When submitted as evidence, such statements are called hearsay evidence. As a legal term, "hearsay" can also have the narrower meaning of...
.
Anecdotal evidence, which may itself be true and verifiable, can be used to deduce a conclusion which does not follow from it, usually by generalising from an insufficient amount of evidence. For example "my grandfather smoked like a chimney and died healthy in a car crash at the age of 99" does not disprove the proposition that "smoking markedly increases the probability of cancer and heart disease at a relatively early age". While the evidence is true, it does not warrant the conclusion made from it.
Conclusions made from anecdotal evidence might not be untrue, but they are unreliable because they don't follow from the evidence and might easily be incorrect.
You can also take a generalization and then search for anecdotal evidence that "confirms" it. For example "Yogurt prolongs life. I didn't believe it, but then I heard that a man in a mountain village who ate only yogurt lived to 120."
The term is often used in contrast to scientific evidence
Scientific evidence
Scientific evidence has no universally accepted definition but generally refers to evidence which serves to either support or counter a scientific theory or hypothesis. Such evidence is generally expected to be empirical and properly documented in accordance with scientific method such as is...
, such as evidence-based medicine
Evidence-based medicine
Evidence-based medicine or evidence-based practice aims to apply the best available evidence gained from the scientific method to clinical decision making. It seeks to assess the strength of evidence of the risks and benefits of treatments and diagnostic tests...
, which are types of formal accounts. Some anecdotal evidence does not qualify as scientific evidence because its nature prevents it from being investigated using the scientific method
Scientific method
Scientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge. To be termed scientific, a method of inquiry must be based on gathering empirical and measurable evidence subject to specific principles of...
. Misuse of anecdotal evidence is a logical fallacy and is sometimes informally referred to as the "person who" fallacy ("I know a person who..."; "I know of a case where..." etc. Compare with hasty generalization
Hasty generalization
Hasty generalization is a logical fallacy of faulty generalization by reaching an inductive generalization based on insufficient evidence essentially making a hasty conclusion without considering all of the variables...
). Anecdotal evidence is not necessarily representative of a "typical" experience; statistical evidence can more accurately determine how typical something is.
Accounts of direct personal experience are commonly equated to anecdotal evidence where the evidence is anecdote
Anecdote
An anecdote is a short and amusing or interesting story about a real incident or person. It may be as brief as the setting and provocation of a bon mot. An anecdote is always presented as based on a real incident involving actual persons, whether famous or not, usually in an identifiable place...
, hearsay
Hearsay
Hearsay is information gathered by one person from another person concerning some event, condition, or thing of which the first person had no direct experience. When submitted as evidence, such statements are called hearsay evidence. As a legal term, "hearsay" can also have the narrower meaning of...
or represents a conclusion deduced from generalisation. Unlike anecdotal evidence, the reliability of accounts of personal experience is normally capable of assessment for legal proceedings.
When used in advertising
Advertising
Advertising is a form of communication used to persuade an audience to take some action with respect to products, ideas, or services. Most commonly, the desired result is to drive consumer behavior with respect to a commercial offering, although political and ideological advertising is also common...
or promotion of a product, service, or idea, anecdotal reports are often called a testimonial
Testimonial
In promotion and of advertising, a testimonial or show consists of a written or spoken statement, sometimes from a person figure, sometimes from a private citizen, extolling the virtue of some product. The term "testimonial" most commonly applies to the sales-pitches attributed to ordinary...
, which are banned in some jurisdictions. The term is also sometimes used in a legal context to describe certain kinds of testimony
Testimony
In law and in religion, testimony is a solemn attestation as to the truth of a matter. All testimonies should be well thought out and truthful. It was the custom in Ancient Rome for the men to place their right hand on a Bible when taking an oath...
. Psychologists have found that people are more likely to remember notable examples than typical examples.
Introduction
In all forms of anecdotal evidence, its reliability by objective independent assessment may be in doubt. This is a consequence of the informal way the information is gathered, documented, presented, or any combination of the three. The term is often used to describe evidence for which there is an absence of documentation, leaving verification dependent on the credibility of the party presenting the evidence.Scientific context
In science, anecdotal evidence has been defined as:- "information that is not based on facts or careful study"
- "non-scientific observations or studies, which do not provide proof but may assist research efforts"
- "reports or observations of usually unscientific observers"
- "casual observations or indications rather than rigorous or scientific analysis"
- "information passed along by word-of-mouth but not documented scientifically"
Anecdotal evidence can have varying degrees of formality. For instance, in medicine, published anecdotal evidence is called a case report
Case report
In medicine, a case report is a detailed report of the symptoms, signs, diagnosis, treatment, and follow-up of an individual patient. Case reports may contain a demographic profile of the patient, but usually describe an unusual or novel occurrence....
, which is a more formalized type of evidence subjected to peer review
Peer review
Peer review is a process of self-regulation by a profession or a process of evaluation involving qualified individuals within the relevant field. Peer review methods are employed to maintain standards, improve performance and provide credibility...
. Although such evidence is not seen as conclusive, it is sometimes regarded as an invitation to more rigorous scientific study of the phenomenon in question. For instance, one study found that 35 of 47 anecdotal reports of side effects were later sustained as "clearly correct."
Anecdotal evidence is considered the least certain type of scientific information. Researchers may use anecdotal evidence for suggesting new hypotheses
Hypothesis
A hypothesis is a proposed explanation for a phenomenon. The term derives from the Greek, ὑποτιθέναι – hypotithenai meaning "to put under" or "to suppose". For a hypothesis to be put forward as a scientific hypothesis, the scientific method requires that one can test it...
, but never as validating evidence.
Anecdotal evidence and faulty logic
Anecdotal evidence is often unscientific or pseudoscientificPseudoscience
Pseudoscience is a claim, belief, or practice which is presented as scientific, but which does not adhere to a valid scientific method, lacks supporting evidence or plausibility, cannot be reliably tested, or otherwise lacks scientific status...
because various forms of cognitive bias
Cognitive bias
A cognitive bias is a pattern of deviation in judgment that occurs in particular situations. Implicit in the concept of a "pattern of deviation" is a standard of comparison; this may be the judgment of people outside those particular situations, or may be a set of independently verifiable...
may affect the collection or presentation of evidence. For instance, someone who claims to have had an encounter with a supernatural being or alien may present a very vivid story, but this is not falsifiable
Falsifiability
Falsifiability or refutability of an assertion, hypothesis or theory is the logical possibility that it can be contradicted by an observation or the outcome of a physical experiment...
. This phenomenon can also happen to large groups of people through subjective validation
Subjective validation
Subjective validation, sometimes called personal validation effect, is a cognitive bias by which a person will consider a statement or another piece of information to be correct if it has any personal meaning or significance to them...
.
Anecdotal evidence is also frequently misinterpreted via the availability heuristic
Availability heuristic
The availability heuristic is a phenomenon in which people predict the frequency of an event, or a proportion within a population, based on how easily an example can be brought to mind....
, which leads to an overestimation of prevalence. Where a cause can be easily linked to an effect, people overestimate the likelihood of the cause having that effect (availability). In particular, vivid, emotionally-charged anecdotes seem more plausible, and are given greater weight. A related issue is that it is usually impossible to assess for every piece of anecdotal evidence, the rate of people not reporting that anecdotal evidence in the population.
A common way anecdotal evidence becomes unscientific is through fallacious
Fallacy
In logic and rhetoric, a fallacy is usually an incorrect argumentation in reasoning resulting in a misconception or presumption. By accident or design, fallacies may exploit emotional triggers in the listener or interlocutor , or take advantage of social relationships between people...
reasoning such as the Post hoc ergo propter hoc
Post hoc ergo propter hoc
Post hoc ergo propter hoc, Latin for "after this, therefore because of this," is a logical fallacy that states, "Since that event followed this one, that event must have been caused by this one." It is often shortened to simply post hoc and is also sometimes referred to as false cause,...
fallacy, the human tendency to assume that if one event happens after another, then the first must be the cause of the second. Another fallacy involves inductive reasoning
Inductive reasoning
Inductive reasoning, also known as induction or inductive logic, is a kind of reasoning that constructs or evaluates propositions that are abstractions of observations. It is commonly construed as a form of reasoning that makes generalizations based on individual instances...
. For instance, if an anecdote illustrates a desired conclusion rather than a logical conclusion, it is considered a faulty
Faulty generalization
A fallacy of defective induction reaches a conclusion from weak premises. Unlike fallacies of relevance, in fallacies of defective induction, the premises are related to the conclusions yet only weakly buttress the conclusions. A faulty generalization is thus produced...
or hasty generalization
Hasty generalization
Hasty generalization is a logical fallacy of faulty generalization by reaching an inductive generalization based on insufficient evidence essentially making a hasty conclusion without considering all of the variables...
. For example, here is anecdotal evidence presented as proof of a desired conclusion:
There's abundant proof that drinking water cures cancer. Just last week I read about a girl who was dying of cancer. After drinking water she was cured.
Anecdotes like this do not prove anything. In any case where some factor affects the probability of an outcome, rather than uniquely determining it, selected individual cases prove nothing; e.g. "my grandfather smoked 40 a day until he died at 90" and "my sister never went near anyone who smoked but died of lung cancer". Anecdotes often refer to the exception, rather than the rule: "Anecdotes are useless precisely because they may point to idiosyncratic responses." Even when many anecdotes are collected to prove a point, "The plural of anecdote is not data." (Roger Brinner)
More generally, a statistical correlation between things does not in itself prove that one causes the other (a causal link). A study found that television viewing was strongly correlated with sugar consumption, but this does not prove that viewing causes sugar intake (or viceversa).
In medicine anecdotal evidence is also subject to placebo effect
Placebo effect
Placebo effect may refer to:* Placebo effect, the tendency of any medication or treatment, even an inert or ineffective one, to exhibit results simply because the recipient believes that it will work...
s: it is well-established that a patient's (or doctor's) expectation can genuinely change the outcome of treatment. Only double-blind
Double-blind
A blind or blinded experiment is a scientific experiment where some of the people involved are prevented from knowing certain information that might lead to conscious or subconscious bias on their part, invalidating the results....
randomized
Random sample
In statistics, a sample is a subject chosen from a population for investigation; a random sample is one chosen by a method involving an unpredictable component...
placebo
Placebo
A placebo is a simulated or otherwise medically ineffectual treatment for a disease or other medical condition intended to deceive the recipient...
-controlled clinical trial
Clinical trial
Clinical trials are a set of procedures in medical research and drug development that are conducted to allow safety and efficacy data to be collected for health interventions...
s can confirm a hypothesis
Hypothesis
A hypothesis is a proposed explanation for a phenomenon. The term derives from the Greek, ὑποτιθέναι – hypotithenai meaning "to put under" or "to suppose". For a hypothesis to be put forward as a scientific hypothesis, the scientific method requires that one can test it...
about the effectiveness of a treatment independently of expectations.
Sites devoted to rhetoric
Rhetoric
Rhetoric is the art of discourse, an art that aims to improve the facility of speakers or writers who attempt to inform, persuade, or motivate particular audiences in specific situations. As a subject of formal study and a productive civic practice, rhetoric has played a central role in the Western...
often give explanations along these lines:
By contrast, in science and logic, the "relative strength of an explanation" is based upon its ability to be tested, proven to be due to the stated cause, and verified under neutral conditions in a manner that other researchers will agree has been performed competently, and can check for themselves.
Law
WitnessWitness
A witness is someone who has firsthand knowledge about an event, or in the criminal justice systems usually a crime, through his or her senses and can help certify important considerations about the crime or event. A witness who has seen the event first hand is known as an eyewitness...
testimony
Testimony
In law and in religion, testimony is a solemn attestation as to the truth of a matter. All testimonies should be well thought out and truthful. It was the custom in Ancient Rome for the men to place their right hand on a Bible when taking an oath...
is a common form of evidence
Evidence (law)
The law of evidence encompasses the rules and legal principles that govern the proof of facts in a legal proceeding. These rules determine what evidence can be considered by the trier of fact in reaching its decision and, sometimes, the weight that may be given to that evidence...
in law, and law has mechanisms to test witness evidence for reliability or credibility. Legal processes for the taking and assessment of evidence are formalized. Some witness testimony could be described as anecdotal evidence, such as individual stories of harassment
Harassment
Harassment covers a wide range of behaviors of an offensive nature. It is commonly understood as behaviour intended to disturb or upset, and it is characteristically repetitive. In the legal sense, it is intentional behaviour which is found threatening or disturbing...
as part of a class action lawsuit. However, witness testimony can be tested and assessed for reliability. Examples of approaches to testing and assessment include the use of questioning, evidence of corroborating witnesses, documents, video and forensic evidence. Where a court lacks suitable means to test and assess testimony of a particular witness, such as the absence of forms of corroboration or substantiation it may afford that testimony limited or no "weight" when making a decision on the facts.
Scientific evidence as legal evidence
In certain situations, scientific evidence presented in court must also meet the legal requirements for evidence. For instance, in the United States, expert testimony of witnesses must meet the Daubert standardDaubert Standard
The Daubert standard is a rule of evidence regarding the admissibility of expert witnesses' testimony during United States federal legal proceedings. Pursuant to this standard, a party may raise a Daubert motion, which is a special case of motion in limine raised before or during trial to exclude...
. This ruling holds that before evidence is presented to witnesses by experts, the methodology must be "generally accepted" among scientists. In some situations, anecdotal evidence may meet this threshold (such as certain case reports which corroborate or refute other evidence).
Altman and Bland argue that the case report or statistical outlier cannot be dismissed as having no weight: "With rare and uncommonly occurring diseases, a nonsignificant finding in a randomized trial does not necessarily mean that there is no causal association between the agent in question and the disease."
See also
- Anecdotal valueAnecdotal valueIn economics, anecdotal value refers to the primarily social and political value of an anecdote or anecdotal evidence in promoting understanding of a social, cultural, or economic phenomenon...
- Confirmation biasConfirmation biasConfirmation bias is a tendency for people to favor information that confirms their preconceptions or hypotheses regardless of whether the information is true.David Perkins, a geneticist, coined the term "myside bias" referring to a preference for "my" side of an issue...
- Empirical evidence
- FallacyFallacyIn logic and rhetoric, a fallacy is usually an incorrect argumentation in reasoning resulting in a misconception or presumption. By accident or design, fallacies may exploit emotional triggers in the listener or interlocutor , or take advantage of social relationships between people...
- Faulty generalizationFaulty generalizationA fallacy of defective induction reaches a conclusion from weak premises. Unlike fallacies of relevance, in fallacies of defective induction, the premises are related to the conclusions yet only weakly buttress the conclusions. A faulty generalization is thus produced...
- Hasty generalizationHasty generalizationHasty generalization is a logical fallacy of faulty generalization by reaching an inductive generalization based on insufficient evidence essentially making a hasty conclusion without considering all of the variables...
- Scientific methodScientific methodScientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge. To be termed scientific, a method of inquiry must be based on gathering empirical and measurable evidence subject to specific principles of...
- Post hoc ergo propter hocPost hoc ergo propter hocPost hoc ergo propter hoc, Latin for "after this, therefore because of this," is a logical fallacy that states, "Since that event followed this one, that event must have been caused by this one." It is often shortened to simply post hoc and is also sometimes referred to as false cause,...
Further reading
- Overall Assessment of Approach and Analysis of the Law Commission Report Ministry of Economic Development, New Zealand
- Legal Terms & Definitions Sprenger & Lang, Attorneys
- Judge certifies Wal-Mart class action lawsuit MSBN, June 22, 2004
- Second Generation Disparity Study Final Report City of Phoenix study
- "Anecdotal Evidence" from a course in Critical thinkingCritical thinkingCritical thinking is the process or method of thinking that questions assumptions. It is a way of deciding whether a claim is true, false, or sometimes true and sometimes false, or partly true and partly false. The origins of critical thinking can be traced in Western thought to the Socratic...
at Santa Rosa Junior CollegeSanta Rosa Junior CollegeSanta Rosa Junior College is a community college located in the city of Santa Rosa in Sonoma County, California. Founded in 1918, it is the tenth oldest community college in the state. Santa Rosa Junior College was modeled as a "junior" version of nearby University of California at Berkeley...
. - Lindsay, Don. Anecdotal Evidence, via don-lindsay-archive.org
- Carroll, Robert Todd. Anecdotal (testimonial) evidence, from the Skeptic's DictionarySkeptic's DictionaryThe Skeptic's Dictionary is a collection of cross-referenced skeptical essays by Robert Todd Carroll, published on his website skepdic.com and in a printed book. The skepdic.com site was launched in 1994 and the book was published in 2003 with nearly 400 entries. As of January 2011 the website has...
.