Experimental Design
Explanatory variable(s):
Response variable:
Research question:
“Is there a causal association between the type of vaccine a subject received and the subject getting ill with the flu?”
Explanatory variable(s):
Response variable:
Research question:
“Is there a causal association between the subject playing memory games and their improvement in the IQ score?”
In both studies, researchers found an association between the explanatory and response variable in the collected data:
The big question: Is this association causal?
In other words, did the change in the explanatory variable cause the change in the response variable?
or, could there be some other explanation for the differences in the response variable?
In both studies, we found some other variables that could have caused the differences in the response variable:
These are called confounding variables, or confounders.
A confounding variable is a variable, often not even considered in a study, that has a potential to influence the values of the response variable.
A lurking variable is a confounding variable, not considered in the study, that can influence both the explanatory variable and the response variable.
In this study, an association was found between the percentage of roads in each county that are paved, and the number of diagnosed cancer cases per 1000 residents.
The lurking variable in this case was the “urbanization” of the counties:
Confounding can be eliminated or at least reduced through a careful study design.
There are some general principles that can often be useful.
Suppose the vaccine study only had two subjects: one receiving the old vaccine, one the new vaccine. The first subject gets the flu, the second does not. What can you conclude?
In order to have any chance of eliminating confounding, we need a sufficiently large number of observations, preferably from large number of subjects.
It is also useful to be able to replicate the whole study again, with a different set of subjects, and possibly different group of researchers.
If the explanatory variable is a factor, subjects should be randomly divided into groups corresponding to different levels. That will assure that any potentially confounding variables will be more or less evenly distributed between the groups, and, if they influence the response variable, it will happen the same way for all the groups.
If the explanatory variable is numerical of string, subjects should be assigned the values of the variable randomly, for the same reason.
Sometimes it is easier to keep a potentially confounding variable from varying.
For example, the flu vaccine study could be done in just one hospital. Then the type of hospital would no longer be a confounding variable, because it would be the same for all subjects.
This is called controlling the variable.
This is not for elimination of confounding, but more for detection.
In the flu vaccine study, the researchers could have used 20 more patients from each of the two hospitals as control groups. These patients would not get any vaccine. If the number of patients getting ill with the flu in the control groups were similar to those in the vaccinated groups, that is about three times as many in Chicago, that would indicate that it is not the vaccine that makes the difference.
To eliminate “placebo effects”, the subjects must not know which group they belong to, and which treatment they are getting.
In some studies the researchers who are administering the treatment to the subjects could inadvertently communicate some information to the subjects. In order to prevent that, the researchers that are in contact with the patients must not know which treatments the subjects receive.
This is an advanced method of controlling a variable. Suppose you decide to control a variable, but you are still interested in knowing if and how do the different values of the variable influence the result.
You would run the same study multiple times, each time keeping the variable constant, but changing its values between different runs of the study.
For example, the same vaccine study could be done in two or more hospitals.
Are the researchers actually able to use these methods? It depends on how much control do they have over the study:
A doctor at a university affiliated hospital wants to do a study that compares two different procedures that are both designed to alleviate certain condition.
After securing all the necessary permissions, she randomly divides her patients into two groups, and performs one procedure on the patients in group 1, and the other procedure on the patients in group 2. She then compares the results.
A researcher at a College of Health and Human Services at certain university wants to do a study that compares two different medical procedures that are both designed to alleviate certain condition. They contact doctors at 200 different hospitals that perform these procedures, and ask the doctors to report the results of the procedures to them.
A researcher at a university wants to do a study that compares two different simple procedures that are both designed to alleviate certain condition.
They contact a doctor at a local hospital and ask if they can do the study there. The doctor gives them the permission, and lets them even perform the procedures under her supervision, but insists on deciding which procedure will each patient undergo.