Last year, The Journal of the American Medical Association (JAMA) made waves when they released a study that found evidence of chronic traumatic encephalopathy — the degenerative brain disease known better by its acronym CTE, which is believed to be caused by repetitive head trauma — in 110 out of 111 brains it examined of deceased former NFL players.
The study garnered big-time headlines — 99 percent is an alarming rate, after all. But it was also the subject of criticism because of its inherent selection bias. Boston University could only study brains that were donated to its brain bank, and most who donated the brains already had reason to suspect the deceased player suffered from CTE when he was alive. Though it can only be diagnosed posthumously, the symptoms of CTE — ailments such as severe depression, dementia, vertigo, and drastic mood swings — tend to be tragically present at the surface.
Last week, Dr. Zachary O. Binney and Dr. Kathleen E. Bachynski published a study that tries to quantify and account for those biases, and provides us with a much better picture of the true prevalence of CTE among NFL players. The news is not good.
Their study found that the estimated prevalence of CTE among NFL players is at least 9.6 percent. That figure is no surprise, given that the initial JAMA study cited a similar figure as a floor when trying to account for selection bias. But Dr. Binney and Dr. Bachynski took their calculations further, in an attempt to establish an upper threshold of potential CTE diagnoses.
“I wasn’t surprised that he minimum was 10 percent,” Binney told ThinkProgress. “What did surprise me was that under very plausible scenarios, it is much higher.”
Reasonable calculations could put the prevalence across all NFL players at between 20 to 50 percent.
How did we get here? In order to fully examine bias, Dr. Binney and Dr. Bachynski first needed to establish how many deceased NFL players were able to participate in the JAMA study. Considering the JAMA study only included brains of NFL players who died between February 2008 and May 2016, they obtained a complete list of former NFL players who died during that time frame from Pro-Football-Reference.com: 1,142.
Using that data, Binney and Bachynski set the minimum threshold: if the 110 CTE-afflicted brains that were donated to the brain bank represented the only cases of CTE among the 1,147 deceased players, that would mean that 9.6 percent of all NFL players who died during that time period have CTE.
Of course, it’s unlikely that the selection bias was so extreme. Of the 1,037 brains that were not donated to the Boston University study, at least a few would likely exhibit signs of CTE. This discrepancy is what the Binney and Bachynski study tries to account for. If the brain bank captured between 90 percent and 50 percent of players whose brains exhibited signs of CTE, that would still put the prevalence of CTE at between 10.7 and 19.3 percent. But if the donation probability dips to 40 percent? That would put the prevalence of CTE around 25 percent — meaning 1 in 4 NFL players who died during that time frame had CTE.
At a donation rate of only 30 percent — which would still mean that those with CTE were 232 percent more likely to donate their brains to the brain bank than those without CTE — the ratio of CTE reaches about 1 in 3. And at a donation rate of 20 percent — which would be a 118.4 percent higher donation rate than the donation rate of those without CTE — the prevalence of CTE reached 50 percent.
“Ten percent is already terrifying,” Binney said. “But imagine going into the league and saying, ‘flip a coin.'”
Both Dr. Binney and Dr. Bachynski acknowledge that much more research still needs to be done on CTE, and have made their methods public so that the findings can continue to be updated as more data becomes available. But they hope that this study provides the necessary context for the conversation about CTE.
The need for more context was highlighted this fall, when ESPN commentator and former NFL player Merril Hoge and Dr. Peter Cummings released a book, “Brainwashed: The Bad Science Behind CTE and the Plot to Destroy Football.” Like a lot of other critics of the JAMA study, the book tries to frame some of the scientifically valid shortcomings of the JAMA study as evidence of a vast liberal conspiracy to take down football. That particularly irked the authors of the new study.
“I’m not waging a war on football, I’m a scientist. I haven’t missed an NFL game in 20 years, I referee high school football. I love football,” Binney said.
Bachynski was similarly bothered by an op-ed on Yahoo that accompanied the launch of the book, entitled, “How one flawed study and irresponsible reporting launched a wave of CTE hysteria.”
That last word — “hysteria” — really stuck with her.
“That word popped up over and over again,” she said. “This idea that there’s this war on football driven by hysteria, it’s a misrepresentation of science and based on sexist ideas about where concern about football comes from.”
Bachynski and Binney’s study shows that while it is irresponsible and incorrect to claim that 99 percent of all NFL players will develop CTE, that the actual prevalence of CTE is anything but trivial.