Every year I watch for the Ig Noble prize winners announcement with a certain amount of glee. Science generally takes itself way too seriously, and the Ig Nobles are a chance to step back a bit and get in a good laugh. This year was no different. From the need for reports about reports about reports (shockingly from the U.S. government) or why leaning left makes the Eifel tower look smaller, there was a good crop of crazy publications to choose from. One that caught my eye though was:
Pretty much showing that you can use statistics and expensive machinery to find statistically significant brain activity in even a dead salmon. There are some absolutely great lines in the paper, including:
One mature Atlantic Salmon (Salmo salar) participated in the fMRI study. The salmon measured approximately 18 inches long, weighed 3.8 lbs, and was not alive at the time of scanning. It is not known if the salmon was male or female, but given the post-mortem state of the subject this was not thought to be a critical variable.
After showing the salmon a few pictures of humans interacting in social inclusive or socially exclusive ways (at least how I read it), and they measured brain activity. Two regions of the brain were significant for activity. After correction by multiple hypothesis correction methods FDR or FWER (even with loose thresholds) these regions dropped out. So what? Well the point, made strongly in the paper, is that somewhere between 60-75% of papers surveyed used a multiple hypothesis correction method. That means over 25% of papers did not. Conference posters were even worse, with only ~25% using such a method.
When you’re testing multiple hypothesis such a correction is absolutely critical. There have been some high profile examples of statistical mistakes in cancer research (my interest); for instance see this letter to the editors of Science about a 2007 study on Breast and Colorectal cancer. Although the salmon paper is generally light-hearted and tongue-and-cheek, it raises a really important point. Statistics are generally a black box for biologists. If the black box spits out a p-value less than 0.05 researchers strike out in cheer, and can tack the all important ‘statistically significant’ moniker to the top of their abstract. A deeper understanding is necessary to save yourself from a flurry of letters to the editor, and I’d highly recommend taking in this paper about FDR and genome-wide studies.