University of Texas at Austin professor Mark Regnerus, notorious for his failed efforts to prove that same-sex couples make inferior parents, is back with a new attempt to justify bigotry with science.
Regnerus is now challenging a 2014 study that found negative health outcomes for gay, lesbian, and bi (LGB) people who live in communities with significant anti-gay prejudice. According to Regnerus, he ran his own analysis of the data using ten different approaches, including a “more refined” strategy than was described in the original study, and could not replicate the results. Whereas the first study found that anti-gay attitudes significantly hampered life expectancy and had several physical and mental healthy consequences, Regnerus found no such correlations.
Anti-LGBT conservatives were quick to champion his findings. Naomi Schaefer Riley of the Independent Women’s Forum suggested the original study was the latest example of the “crisis of replication,” studies whose results cannot be recreated in subsequent experiments. At National Review, Maggie Gallagher compared the situation to a 2014 study that claimed profound results for changing voters’ minds on marriage equality with canvassing but was found to have been largely fabricated. And Focus on the Family’s Glenn T. Stanton tried to drive home the idea that LGBT people are inherently broken and their mental health challenges cannot be blamed on stigma. In his post at The Federalist, he conspicuously avoided identifying Regnerus by name, instead referring to him repeatedly only as “the author.”
Stanton’s instinct that his readers may not trust a study by Regnerus may have been well-informed.
The conflicting studies
Mark Hatzenbuehler, associate professor of sociomedical science at Columbia University’s Mailman School of Public Health and lead researcher on the original study, told ThinkProgress that he was not convinced by Regnerus’ findings. “My colleagues and I stand by our results,” he said, adding that they are working on a “detailed response” to Regnerus’ article that they will submit for peer review.
The original study that Hatzenbuehler and his colleagues conducted didn’t involve collecting any new data, but using information available from massive yearly national studies. The General Social Survey provided data about prejudicial attitudes and the National Death Index provided information about mortality rates. These were then juxtaposed, and the researchers found that LGB people living in high-prejudice communities had shorter life spans and experienced substantially elevated levels of suicide, homicide/violence, and cardiovascular diseases than those living in more welcoming communities.
Regnerus counters that he tried ten different methods of imputation of that data and could not produce the same results. Imputation is a statistical process for filling in missing data with estimates that help flesh out the usable results, and there are countless ways to do it. Hatzenbuehler explained to ThinkProgress that his team used an approach called “multiple imputation,” but he also noted, “We confirmed our analysis without imputing any missing data and our results were consistent.”
If Regnerus had truly replicated Hatzenbuehler’s study, he would have used the same approach, and indeed he claims to have tried. But according to Hatzenbuehler, that’s not what happened. “Regnerus claims to have failed to replicate our results. We have had limited time to evaluate his paper, but an initial review of his code revealed several differences between his approach and ours that we suspect led to the discrepancies in the findings.”
In other words, it seems Regnerus didn’t actually replicate the approach used in the original study. If that’s the case, it means that Regnerus, a researcher with a reputation of distorting data to fit his anti-gay bias, just did his own analysis and found different results.
“Setting aside the question of whether his study is therefore truly a replication,” Hatzenbuehler said, “we welcome attempts to replicate our work.” He pointed out that the data are all publicly available, so anyone can attempt a replication at any time. Furthermore, his team’s response to Regnerus’ article will include all of the code for the original analyses that they ran, as well as for the re-analyses that they are currently conducting.
There is no trend of pro-gay junk science
Conservatives have tried to point to Hatzenbuehler’s study to conjure a trend of pro-gay junk science — attempting to compare it to a fraudulent canvassing study that was retracted last year.
But this argument falls flat. Not only are there significant distinctions between the two, but many the conclusions of the fraudulent study were also ultimately demonstrated by a valid one.
In Michael LaCour’s now-retracted 2014 study, he claimed that if canvassers went out door-to-door and had thorough personal conversations with voters about marriage equality, they could dramatically improve voters’ support for same-sex marriage. Unfortunately, it turns out he fabricated the data that informed his impressive results. Even though canvassers had actually gone out to conduct the experiment, LaCour seemed to have completely invented the survey results that assessed voters’ reactions, essentially making all the canvassing work a waste of time, money, and manpower — at least in terms of the study.
When researchers David Broockman (Stanford University) and Joshua Kalla (UC Berkeley) sought to replicate LaCour’s study, they found discrepancies in what he had published. They reached out to LaCour for the original raw data, but he could not produce them, and by pulling on that thread, they found that he had actually fabricated many aspects of the study.
But Broockman and Kalla were still interested in researching canvassing. So they started LaCour’s research over — this time studying public attitudes on transgender issues with a whole new army of canvassers. To compensate for LaCour’s missteps, they were incredibly transparent about their data collection.
When they published their results, it turned out LaCour probably would have been right if he’d actually completed his study authentically. Canvassing conversations actually can change attitudes on LGBT issues in a significant way.
As Hatzenbuehler highlighted, Regnerus didn’t have to collect new data in the way that Broockman and Kalla did; he just had to reproduce the same analyses Hatzenbuehler and his colleagues had conducted for the original study. But he didn’t do that, and seemingly did not even attempt to reach out to Hatzenbuehler for a clear understanding of what statistical processes his team used so that he could actually replicate them.
Broockman told ThinkProgress that these “replication controversies usually take time and effort to understand,” cautioning against a rush to judgment about the merits of either side. Still, he emphasized the importance of researchers publicly sharing the code they use in their studies, noting, “It’s pretty difficult to evaluate their claims without it.”
As for Regnerus’ attempt to undermine Hatzenbuehler’s research, Broockman also observed, “It is not like this is the only study indicating that experiencing prejudice has negative consequences.”
The future of LGBT research
Conservatives trying to resist LGBT equality have been waging war in academia as much as in politics. This new challenge to research about the impacts of stigma joins debates over issues like same-sex parenting, transgender children, and ex-gay therapy still being fought by a few holdouts who refuse to join the scientific consensuses already in place.
On all of these issues, more information will provide more clarity, and Hatzenbuehler hopes that the impact of stigma is no exception. “One important aspect of that work is that it does not rely on any single study,” he said. “Separate research groups have used diverse methods to document the negative health effects of structural stigma among LGB individuals. Continuing to explore the link between structural stigma and health outcomes remains a vital topic for public health research.”
If Regnerus doubts just how influential stigma is on health outcomes, as his article suggests, his attempts to undermine those claims could ultimately prove the opposite. Researchers like Hatzenbuehler will shore up their results and others will be motivated to conduct new studies exploring the question.