Field experiment
Encyclopedia
A field experiment applies the scientific method to experiment
ally examine an intervention in the real world (or as many experimentalists like to say, naturally occurring environments) rather than in the laboratory
. Field experiments, like lab experiments, generally randomize
subjects (or other sampling units) into treatment and control groups and compare outcomes between these groups.
Examples include:
has a long history of field experiments, since the time of Avicenna
, while field experiments in anthropology
date back to Biruni's study of India. Social psychology
also has a history of field experiments, including work by pioneering figures Philip Zimbardo
, Kurt Lewin
and Stanley Milgram
. In economics, Peter Bohm, University of Stockholm, was one of the first economists to take the tools of experimental economic methods and attempt to try them with field subjects. In the area of development economics, the pioneer work of Hans Binswanger in the late 1970s conducting experiments in India on risk behavior http://ideas.repec.org/p/feb/artefa/0009.html should also be noted. The use of field experiments in economics has grown recently with the work of John A. List
, Jeff Carpenter, Juan-Camilo Cardenas, Abigail Barr, Catherine Eckel, Michael Kremer
, Paul Gertler
, Glenn Harrison, Colin Camerer
, Bradley Ruffle, Esther Duflo
, Dean Karlan
, Edward "Ted" Miguel, Sendhil Mullainathan
, David H. Reiley, among others.
(J-PAL) at the Massachusetts Institute of Technology
, the Center of Evaluation for Global Action
at the University of California
, and Innovations for Poverty Action (IPA) in particular have received attention for their use of randomized field experiments to evaluate development programs. The aim of field experiments used in development research is to find causal relationships between policy interventions and development outcomes. Field experiments are seen by some academics as a rigorous way of testing general theories about economic and political behavior and most recently, field experiments have been used by political scientists to study political behavior, institutional dynamics, and conflict in the developing world.
In a randomized field experiment on an international development intervention, researchers would separate participants into two or more groups: a treatment group (or groups) and a control group. Members of the treatment group(s) then receive a particular development intervention being evaluated while the control group does not. (Often the control group receives the intervention later in the roll out of the study.) Field experiments have gained popularity in the field because they allow researchers to guard against selection bias
, a problem present in many current studies of development interventions. Selection bias refers to the fact that, in non-experimental settings, the group receiving a development intervention is likely different from a group that is not receiving the intervention. This may occur because of characteristics that make some people more likely to opt in to a program, or because of program targeting. Some academics dispute the claim that findings from field experiments are sufficient for establishing and testing theories about behavior. In particular, a hotly contested issue with regards to field experiments is their external validity
. Given that field experiments necessarily take place in a specific geographic and political setting, the extent to which findings can be extrapolated to formulate a general theory regarding economic behavior is a concern.
Experiment
An experiment is a methodical procedure carried out with the goal of verifying, falsifying, or establishing the validity of a hypothesis. Experiments vary greatly in their goal and scale, but always rely on repeatable procedure and logical analysis of the results...
ally examine an intervention in the real world (or as many experimentalists like to say, naturally occurring environments) rather than in the laboratory
Laboratory
A laboratory is a facility that provides controlled conditions in which scientific research, experiments, and measurement may be performed. The title of laboratory is also used for certain other facilities where the processes or equipment used are similar to those in scientific laboratories...
. Field experiments, like lab experiments, generally randomize
Randomization
Randomization is the process of making something random; this means:* Generating a random permutation of a sequence .* Selecting a random sample of a population ....
subjects (or other sampling units) into treatment and control groups and compare outcomes between these groups.
Examples include:
- Clinical trials of pharmaceuticals are one example of field experiments.
- Economists have used field experiments to analyze discriminationDiscriminationDiscrimination is the prejudicial treatment of an individual based on their membership in a certain group or category. It involves the actual behaviors towards groups such as excluding or restricting members of one group from opportunities that are available to another group. The term began to be...
, health careHealth careHealth care is the diagnosis, treatment, and prevention of disease, illness, injury, and other physical and mental impairments in humans. Health care is delivered by practitioners in medicine, chiropractic, dentistry, nursing, pharmacy, allied health, and other care providers...
programs, charitable fundraising, educationEducationEducation in its broadest, general sense is the means through which the aims and habits of a group of people lives on from one generation to the next. Generally, it occurs through any experience that has a formative effect on the way one thinks, feels, or acts...
, information aggregation in markets, and microfinanceMicrofinanceMicrofinance is the provision of financial services to low-income clients or solidarity lending groups including consumers and the self-employed, who traditionally lack access to banking and related services....
programs. - EngineerEngineerAn engineer is a professional practitioner of engineering, concerned with applying scientific knowledge, mathematics and ingenuity to develop solutions for technical problems. Engineers design materials, structures, machines and systems while considering the limitations imposed by practicality,...
s often conduct field tests of prototypePrototypeA prototype is an early sample or model built to test a concept or process or to act as a thing to be replicated or learned from.The word prototype derives from the Greek πρωτότυπον , "primitive form", neutral of πρωτότυπος , "original, primitive", from πρῶτος , "first" and τύπος ,...
products to validate earlier laboratory tests and to obtain broader feedback.
History
Of course, the use of experiments in the lab and the field have a long history in the natural and life sciences. GeologyGeology
Geology is the science comprising the study of solid Earth, the rocks of which it is composed, and the processes by which it evolves. Geology gives insight into the history of the Earth, as it provides the primary evidence for plate tectonics, the evolutionary history of life, and past climates...
has a long history of field experiments, since the time of Avicenna
Avicenna
Abū ʿAlī al-Ḥusayn ibn ʿAbd Allāh ibn Sīnā , commonly known as Ibn Sīnā or by his Latinized name Avicenna, was a Persian polymath, who wrote almost 450 treatises on a wide range of subjects, of which around 240 have survived...
, while field experiments in anthropology
Anthropology
Anthropology is the study of humanity. It has origins in the humanities, the natural sciences, and the social sciences. The term "anthropology" is from the Greek anthrōpos , "man", understood to mean mankind or humanity, and -logia , "discourse" or "study", and was first used in 1501 by German...
date back to Biruni's study of India. Social psychology
Social psychology
Social psychology is the scientific study of how people's thoughts, feelings, and behaviors are influenced by the actual, imagined, or implied presence of others. By this definition, scientific refers to the empirical method of investigation. The terms thoughts, feelings, and behaviors include all...
also has a history of field experiments, including work by pioneering figures Philip Zimbardo
Philip Zimbardo
Philip George Zimbardo is an American psychologist and a professor emeritus at Stanford University. He is president of the Heroic Imagination Project...
, Kurt Lewin
Kurt Lewin
Kurt Zadek Lewin was a German-American psychologist, known as one of the modern pioneers of social, organizational, and applied psychology....
and Stanley Milgram
Stanley Milgram
Stanley Milgram was an American social psychologist most notable for his controversial study known as the Milgram Experiment. The study was conducted in the 1960s during Milgram's professorship at Yale...
. In economics, Peter Bohm, University of Stockholm, was one of the first economists to take the tools of experimental economic methods and attempt to try them with field subjects. In the area of development economics, the pioneer work of Hans Binswanger in the late 1970s conducting experiments in India on risk behavior http://ideas.repec.org/p/feb/artefa/0009.html should also be noted. The use of field experiments in economics has grown recently with the work of John A. List
John A. List
John August List is The Homer J. Livingston Professor in Economics at the University of Chicago. He received his B.A. from the University of Wisconsin–Stevens Point, and his Ph.D. from the University of Wyoming in 1996...
, Jeff Carpenter, Juan-Camilo Cardenas, Abigail Barr, Catherine Eckel, Michael Kremer
Michael Kremer
Michael Robert Kremer is a development economist and is currently the Gates Professor of Developing Societies at Harvard University. He is a Fellow of the American Academy of Arts and Sciences, a recipient of a MacArthur Fellowship and a Presidential Faculty Fellowship, and was named a Young Global...
, Paul Gertler
Paul Gertler
Paul Gertler is an American economist and the Li Ka Shing Distinguished Professor of Economics in the Haas School of Business and the School of Public Health at UC Berkeley....
, Glenn Harrison, Colin Camerer
Colin Camerer
Colin F. Camerer is an American behavioral economist and a Robert Kirby Professor of Behavioral Finance and Economics at the California Institute of Technology ....
, Bradley Ruffle, Esther Duflo
Esther Duflo
Esther Duflo is a French economist, currently the Abdul Latif Jameel Professor of Poverty Alleviation and Development Economics at the Massachusetts Institute of Technology. She is also co-founder and the Director of the Abdul Latif Jameel Poverty Action Lab...
, Dean Karlan
Dean Karlan
Dean Karlan is a Professor of Economics at Yale University and a Research Fellow at the Abdul Latif Jameel Poverty Action Lab at the Massachusetts Institute of Technology...
, Edward "Ted" Miguel, Sendhil Mullainathan
Sendhil Mullainathan
Sendhil Mullainathan is a Professor of Economics at Harvard University. He was hired with tenure by Harvard in 2004 after having spent six years at MIT, first as a junior faculty member and then as a full professor. He is a recipient of a MacArthur Foundation "genius grant" and conducts research on...
, David H. Reiley, among others.
Field Experiments in International Development Research
Development economists have used field experiments to measure the effectiveness of poverty and health programs in developing countries. Organizations such as the Abdul Latif Jameel Poverty Action LabAbdul Latif Jameel Poverty Action Lab
The Abdul Latif Jameel Poverty Action Lab is an academic center located at the Massachusetts Institute of Technology Department of Economics which is dedicated to evaluating the impact of anti-poverty programs through randomized controlled trials similar to those used in medical research...
(J-PAL) at the Massachusetts Institute of Technology
Massachusetts Institute of Technology
The Massachusetts Institute of Technology is a private research university located in Cambridge, Massachusetts. MIT has five schools and one college, containing a total of 32 academic departments, with a strong emphasis on scientific and technological education and research.Founded in 1861 in...
, the Center of Evaluation for Global Action
Center of Evaluation for Global Action
The Center of Evaluation for Global Action is a research network at the University of California that advances global health and development through impact evaluation and economic analysis...
at the University of California
University of California
The University of California is a public university system in the U.S. state of California. Under the California Master Plan for Higher Education, the University of California is a part of the state's three-tier public higher education system, which also includes the California State University...
, and Innovations for Poverty Action (IPA) in particular have received attention for their use of randomized field experiments to evaluate development programs. The aim of field experiments used in development research is to find causal relationships between policy interventions and development outcomes. Field experiments are seen by some academics as a rigorous way of testing general theories about economic and political behavior and most recently, field experiments have been used by political scientists to study political behavior, institutional dynamics, and conflict in the developing world.
In a randomized field experiment on an international development intervention, researchers would separate participants into two or more groups: a treatment group (or groups) and a control group. Members of the treatment group(s) then receive a particular development intervention being evaluated while the control group does not. (Often the control group receives the intervention later in the roll out of the study.) Field experiments have gained popularity in the field because they allow researchers to guard against selection bias
Selection bias
Selection bias is a statistical bias in which there is an error in choosing the individuals or groups to take part in a scientific study. It is sometimes referred to as the selection effect. The term "selection bias" most often refers to the distortion of a statistical analysis, resulting from the...
, a problem present in many current studies of development interventions. Selection bias refers to the fact that, in non-experimental settings, the group receiving a development intervention is likely different from a group that is not receiving the intervention. This may occur because of characteristics that make some people more likely to opt in to a program, or because of program targeting. Some academics dispute the claim that findings from field experiments are sufficient for establishing and testing theories about behavior. In particular, a hotly contested issue with regards to field experiments is their external validity
External validity
External validity is the validity of generalized inferences in scientific studies, usually based on experiments as experimental validity....
. Given that field experiments necessarily take place in a specific geographic and political setting, the extent to which findings can be extrapolated to formulate a general theory regarding economic behavior is a concern.
Caveats
- Fairness of randomization (e.g. in 'negative income taxNegative income taxIn economics, a negative income tax is a progressive income tax system where people earning below a certain amount receive supplemental pay from the government instead of paying taxes to the government. Such a system has been discussed by economists but never fully implemented...
' experiments communities may lobby for their community to get a cash transfer so the assignment is not purely random) - Contamination of the randomization
- General equilibrium and "scaling-up"
- Difficulty of replicability (field experiments often require special access or permission, or technical detail—e.g., the instructions for precisely how to replicate a field experiment are rarely if ever available in economics)
- Limits on ability to obtain informed consent of participants
- Field testing is always less controlled than laboratory testing. This increases the variability of the types and magnitudes of stress in field testing. The resulting dataDataThe term data refers to qualitative or quantitative attributes of a variable or set of variables. Data are typically the results of measurements and can be the basis of graphs, images, or observations of a set of variables. Data are often viewed as the lowest level of abstraction from which...
, therefore, are more varied: larger standard deviationStandard deviationStandard deviation is a widely used measure of variability or diversity used in statistics and probability theory. It shows how much variation or "dispersion" there is from the average...
, less precision and accuracy, etc. This leads to the use of larger sample sizeSample sizeSample size determination is the act of choosing the number of observations to include in a statistical sample. The sample size is an important feature of any empirical study in which the goal is to make inferences about a population from a sample...
s for field testing.
Sources
- "Field Experiments." GW Harrison, JA List. Journal of Economic Literature. Vol. XLII (December 2004) pp. 1009–1055.