The importance of study design


Parrot Study by benji2505, Attribution-NonCommercial License

This post directs you to a research paper by Simmons et al (2011) that is freely available online, and is well worth reading for three reasons.

One reason is its amusement value. In this paper the authors “prove” that listening to Beatles music makes you approximately a year and a half younger than you were before. They do not discover, unfortunately, whether it is just something about the song “When I’m sixty-four” or whether it is any Beatles song. This is a pity. One can easily get sick of listening to that particular song.

Another reason is that it is very well written and it is interesting to see how the authors structure their writing. Take, for example, the section headed “Nonsolutions” on the 7th page. Note how the authors introduce the point they are making first, before going on to give the detail. It is very easy to follow this sort of writing and is a pleasure to read. Also note the frequent use of the first person “we” and “our”. This shortens and simplifies sentences and makes the writing more direct.

And lastly, the message of the paper is important. The authors show how easy it is to reach different conclusions by changing the research design as you go along. One take home message is to spend sufficient time planning and ensure, in your research deign, that you can justify every step you plan to take. The other take home message is that you must record your steps and accurately report them when you write up your findings. Not keeping a research diary in which you record your steps and your thinking? You should be. See this previous post on research diaries.

Simmons JP, Nelson LD and Simonsohn U. (2011) False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. doi:10.1177/0956797611417632. Available at SSRN:


Leave a comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s