Experimental research strategies – lab, field or natural?

by Research Methods & Statistics

Here we go another debate about research methods. It’s worse than
politics! This week I’m going to discuss the different type of experiments;
these are laboratory, field and natural. Like most aspects of research methods
there are pros and cons associated with applying a certain experimental design.
So basically another blogging session slagging off statistics. Fun!

Let’s start with laboratory experiments! Like they say on the tin,
they are conducted in a laboratory setting and by this I mean a controlled
environment. The independent variable is directly manipulated by researchers to
show cause and effect on the dependent variable. An example of this
experimental method is Loftus and Palmers (1974)* experiment into the eye
witness testimony. They asked participants to estimate the speed of cars in an
accident using different forms of questioning techniques. They were five
conditions as participants were asked how fast the cars were travelling when
they hit/smashed/collided/bumped/contacted. Here, the verb is being manipulated
by the experimenter thus, the independent variable was the verb used in the
question and the dependent variable was the speed reported by participants.
Researchers found that the estimated speed was affected by verb use. There we
have it, a sound piece of research, right? Well let’s discuss. ..

An advantage of lab experiments is that they provide psychologists
with the highest level of control over variables and participants can be
randomly assigned to the different conditions, in this case the different verbs
used. Lab experiments can be easily replicated due to standardised procedures
and indeed Loftus and Palmer replicated their study and found results strongly
correlated to their original research**. Other researchers can also replicate
this study very easily in the future by showing http://www.youtube.com/watch?v=2xuOtM169K0***
to participants. A major advantage of this type of study is that cause and
effect can be established so, Loftus and Palmer know that it was the different
types of verbs used which had an effect on participants speed judgement. Great
news! But now it’s time to get to the fun (I use this word lightly) stuff, the
weaknesses! Due to a high level of internal validity there is obviously a low
level of ecological and external validity due to the artificial setting. This
research example especially lacks mundame realism as the video clip shown to
participants does not have the same emotional impact as seeing a real life car
crash. Although this type of experiment should have high internal validity,
this can be reduced by investigator effects and demand characteristics. Bad
times indeed!

What about field experiments I hear you cry. Well, it may come as
a surprise to you but they are not set in a field – disappointing I know! These
experiments are similar to lab experiments by researchers manipulating the
independent variable however; they are conducted in a natural setting or
familiar surroundings such as a school, hospital or in the street (hopefully not
at night as there all sorts of dodgy folk around then!) An example comes from
Bickman (1974)****. People were approached in the street and asked to pick up
litter. The researchers who asked people to pick up litter were either dressed
as a milkman, a civilian (in a coat and tie) or a guard. So, the street was the
natural environment and the way the person was dressed was the independent
variable as this was manipulated by researchers. The dependent variable was
whether the people would pick the litter up (and therefore obey). It was found
people were more likely to obey the guard, showing the power of uniform and

Now time to evaluate! Field experiments are rich in ecological
validity due to the real life setting (in this case, the street). Because a lot
of aspects of the experiment are controlled (the different uniforms worn),
cause and effect can be established. An advantage over lab experiments is that
there is a less likelihood of demand characteristics as participants may not
even realise they are taking part (obviously, this has ethical issues).
Disadvantage times! Although experimental realism is high, results may not
generalise to other real life circumstances different from the ones being
investigated. They are also seen as time consuming and expensive to run
(especially compared to lab experiments).

Finally, let’s go au naturale. Natural experiments occur when a
researcher takes advantage of a naturally occurring event or difference. It is
not really a true experiment (but still worth talking about) as the independent
variable occurs naturally (surprise surprise!) rather than being manipulated by
researchers – the total opposite of laboratory experiments if you haven’t
already gathered that. An example comes from developmental psychology, Hogers
and Tizard (1989)***** tested the effect of institutional upbringing on later
attachment. They followed children (doesn’t sound creepy at all…) in a
longitudinal study from small babies to when they were 16. As babies, the
children were raised in an institution where formation of attachments was
discouraged. As they grew up, some children returned to their families, some
were adopted and some stayed in the institution (the naturally occurring
IVs).Researchers found the adopted children formed better relationships than
children who stayed in the institution.

Now, let’s get our debate on! Natural experiments are fairly easy
to conduct as the majority of natural studies are observational, a researcher
just sat with a pukka pad watching and making notes with a snazzy parka pen.
Also, they are very ethical! The results from the example highlight the
importance of natural studies. It would’ve been extremely unethical to take
children away from their parents and have them adopted for experimental
purposes (any parent willing to let this happen probably shouldn’t have
children!) Therefore the researchers took advantage of a naturally occurring
event observing children already in an institution, thus avoiding manipulating
variables. Unfortunately, they are not all rainbows and lollipops . They are
very hard (if not impossible) to replicate due to a low level of control and no
manipulation of variables. They also take ages to do, especially true in
longitudinal studies such as the example above. Researchers can observe for
hours and hours and hours. They are also prone to observer bias, a confounding
variable in observational research where an error is introduced when observers
overemphasize behavior they expect to find and don’t notice behavior they don’t
expect. This can be overcome by the double-blind procedure (Bateson
1950)******. Observer bias is also introduced when researchers witness a
behavior and interpret it according to how it means to them, however it may
mean something completely different to the person showing the behavior, this is
known as observer-expectancy effect.

Once again, I’ve gone off on one. I think I just love me some
research methods debates a bit too much! So, time to conclude. Lab experiments
are great if you want to establish cause and effect, not so great for
generalizing results to real life situations. Field experiments are high in
ecological and external validity (BOOM) they may not generalize to other
situations. Natural experiments observe real people, in the real world, doing real
things and generally being real (good times) but there are a whole host of
issues with observer bias (bad times). If you choose one method, the other two
will be back to bite you on your bum. It’s a very vicious cycle! I like to view
experiments as beauty, natural is best as nobody likes fake tan – it ruins
your bed sheets! Just like the girl at school with more faces than big ben, you
can never really trust lab experiments where as natural experiments show what
is really happening.