In favor of online experiments

Shanto Iyengar makes some interesting observations in his 2002 paper in favor of online experiments. The primary advantages that he mentions are:

1) You can reach a wider population. One of the major problems facing typical experiments is that they are limited to college sophomores, who may not be readily comparable to “real people.”

2) The results you get online should be just as good as those obtained from the laboratory.

3) Your personal computer is a more “natural” setting than the laboratory, so the responses you give may be more realistic. There is, however, some trade-off where some people may be less likely to complete an online survey in earnest, as opposed to a survey where a woman in a lab coat was staring at their responses. Of course, you can toss these poor results out, but where do you cross the line into manipulating your results?

His paper primarily discusses political science research, but it generalizes to all types of online experiments. One example that he brought up was an experiment done by researchers at Stanford. They created a game where users were able to “bash” famous political figures, and drew conclusions based on how they manipulated the settings of the game. Google has started framing useful research in the context of a game as well, and it is obviously a fascinating technique.

Reference

S. Iyengar, Experimental designs for political communications research: from shopping molls to the internet, Stanford University Political Communications Lab (2002).