A recent study by the British Medical Journal suggests that luck is all in the hands. Sort of.
The BMJ study titled "Giving science the finger" looks at whether luck (as measured by poker hands) could be linked to the ratio between ring and index fingers (a.k.a. the 2D:4D ratio). If that sounds like a rather frivolous topic to you, you're not wrong. The aim of the paper was to demonstrate how a badly constructed study might generate spurious results.
Every year, around Christmas, the BMJ publishes a study that is designed to make people laugh and make a point. In 2013, the BMJ had two doctors read the James Bond novels and assess 007's drinking habits. That study highlighted the dangers of alcohol abuse. One author even suggested that Bond might prefer his drinks shaken as a way of hiding his alcoholic hand tremors.
In 2016, the BMJ ran a study to determine if the naughtiness or niceness of patients at childrens' hospitals influenced whether Santa would visit the wards. The article is under the rosy cheeks, largely about income inequality.
The goal of this year's Christmas study is, as the abstract puts it, "to explore whether random chance, weak research methodology, or inappropriate reporting can lead to claims of statistically significant (yet, biologically meaningless) biomarker associations, using as a model the relation between a common surrogate of prenatal testosterone exposure, second-to-fourth digit ratio (2D:4D), and a random indicator of good luck."
In other words, do your fingers' lengths affect how strong your poker hands are.
Madness in the method
Each participant, of which there were 176, came into the lab at High Point University in North Carolina. They had their index (D2) and ring (D4) fingers measured externally and with an x-ray. This ratio is a predictor of a few physiological features. The main one is how much testosterone an individual was exposed to in the womb.
Then the scientists gave each participant two shuffled decks of cards. They then told participants to pick five cards from each deck. The highest of the participant's two five-card hands went on the clipboard. All 176 hands were ranked against each other, giving an ostensible hierarchy of luckiness.
Then the researchers went to town with their stats software.
"In our (facetious) effort to persuade less statistically savvy readers of the validity of our statistical analyses, we report as many P values as possible," the authors explain.
Many of the analyses were inconclusive. However, by mining a wide variety of parameters, the authors were able to find enough strong links to make their point.
Results oriented thinking
For example, the radiographic measurements showed noticeable correlations when split by gender. Higher D2:D4 ratios make for luckier men and unluckier women.
The results section ends by saying: "This study intended to explore whether researchers can get lucky in finding statistically significant associations between a biomarker and various outcomes of interest, and whether these relations might reflect random chance rather than biological cause and effect. Failure to recognize these common research pitfalls [...] can allow false-positive findings to masquerade as evidence to support unsound theories."
The last two years are a period has been — to risk the cliche of the moment — unprecedented. Especially in the sheer number of people engaging directly with primary research.
So now is a good time to remind people to check the methods section of a study before retweeting.
Featured image source: Flickr by Images Money, used under CC license
