REPOST: Lies, damned lies and the Washington Post
Substituting a common-sense statistical metric for a less obvious and intuitive one is almost always a red flag for deceit and obfuscation
UPDATE: I originally posted this article on April 1st. In October, brought an error in this article to my attention, which meant that a significant section of the article was based on invalid statistical conclusions. I’ve decided that putting entire paragraphs of the article in strikethrough text would be unsightly, so instead I’m reposting the article with the offending sections removed. For the sake of transparency, the original retracted article will remain up in its original form, and I’ve added it to my Mistakes page.
With the introduction of its Notes feature, Substack appears to be making a pivot towards being a Twitter X competitor, the management perhaps having detected a gap in the market following the Elon Musk takeover. While one can question the wisdom of that decision, I commend them wholeheartedly for differentiating themselves from X in one key respect, namely avoiding the echo chamber dynamics which plague it and essentially every other major social media platform. Given how social media algorithms usually work, one would expect my Notes feed to be a nonstop deluge of gender-critical posts and anti-woke one-liners. On the contrary: in addition to plenty of nature photos and boomer dad jokes, I see the full spectrum of political opinions represented, from beliefs I wholeheartedly endorse to ones I would never consider in a lifetime. This is good, because being exposed to contrary opinions is healthy (“he who knows only his own side of the case knows little of that” and all that), but also because I'm an argumentative, pedantic “to play devil's advocate” type by disposition, and find it hard to resist the opportunity to pull someone up on a weak argument or erroneous factual statement. (In a previous post when I described leftists as “ornery, confrontational types”, I meant it as a sincere compliment. It's possible, even common, to get so good at “reading the room” that you forget how to write.)
One such Note shared the graphic below, which claims that there has been a sharp increase in hate crimes targeting the LGBTQ community in schools across the United States, and that states with “anti-LGBTQ” laws have seen larger spikes than states without. The graphic was accompanied by a couple of paragraphs expounding that such an increase was both foreseen and intended by the homophobic, transphobic lawmakers behind the legislation.
The graphic itself is based on an article in the Washington Post from a few weeks ago. It features interviews with LGBTQ teenagers and their parents, describing the bullying and harassment they’ve experienced at their hands of their heteronormative peers, and contrasting these anecdotes with public statements from conservative politicians and lawmakers. The journalists more or less explicitly claim that LGBTQ people living in states with these kinds of legislation are at greater risk of being victimised because of their identity than LGBTQ people living elsewhere in the US.
I have a lot of thoughts on this article. The number of column inches dedicated to implying that Dagny “Nex” Benedict died as a direct result of transphobic bullying is unseemly (even if the article begrudgingly acknowledge that Benedict’s death was ultimately ruled a suicide).1 Some of the laws the journalists characterise as “anti-LGBTQ” are farcical. The most common type are those which “restrict trans student access to sports”, which is just a roundabout way of saying “forbid male students from competing in female sporting events”.2 Given that male athletes competing in female sporting events is manifestly, transparently unfair to anyone with even the most passing familiarity with sexual dimorphism, I support such laws without qualification. Oklahoma passed legislation in 2020 requiring that schools “teach that ‘a person’s sex is an immutable biological trait’ that cannot be changed”: given that this statement is unambigously true, I find the law no more objectionable than bans on public schools teaching creationism or geocentrism. The journalists clearly expect me to be horrified about a Virginia law requiring schools to notify parents whenever the school uses sexually explicit instructional material. Guys, seriously: if you want people to stop throwing the “groomer” accusation around left and right, you’re going to have to meet them halfway.
But my biggest problem with the article is its core thesis, as represented by the graphic above. I’m always deeply suspicious of statistical claims about “fastest growing” or “biggest increase”. Of all commonly quoted statistical observations, it seems like the most susceptible to random noise: if you’re measuring a value that started at a low baseline, some noise within normal variation can be truthfully (but misleadingly) claimed to represent a massive increase when it may be indicative of no such thing.
It also strikes me as a uniquely bad metric for the specific task of comparing which of two regions is the more dangerous. Imagine two neighbouring countries A and B, each with a population of 5 million. Last year, there were 100 murders in Country A and 5 in Country B. This year, there were 110 in Country A and 10 in Country B. It's unambiguously true that Country B’s murder rate increased by 100% year-on-year, while Country A’s “only” increased by 10%. It's also plainly true that you're 11 times more likely to get murdered in Country A than in Country B. If you were planning to book a holiday in either Country A or Country B and you were wondering which was safer, “murders per capita” will tell you far more than “rate of increase of the murder rate year-on-year”: there’s a very good reason that the term “the murder rate” refers to the former metric rather than the latter. In light of the above, if I read an article which tried to make Country B look bad by pointing out that its murder rate had increased by 10 times as much as Country A’s, I’d immediately wonder if the journalist had any undisclosed financial ties to Country A’s tourism board.
My suspicions thus raised, I decided to carry out a deep dive to check how accurate the narrative presented by the journalists was. Conveniently, the journalists based their statistical claims on FBI hate crimes data, which is a publicly available dataset. I downloaded the data and filtered it to only include hate crimes targeting the LGBT community and which occurred in the years 2015-22 (excluding 2020, as the journalists did). I then sorted these hate crimes based on whether they took place in a school or elsewhere, and whether or not they took place in a state with restrictive legislation concerning LGBTQ issues.3 Armed with the necessary data, I was ready to jump in and test the article’s key claims and suggestions.
It will come as no surprise that what I found departs from the narrative presented by the article rather sharply.
Anti-LGBTQ hate crimes in schools
To back up its claims, the article divides the fifty states of the union into those which have restrictive legislation concerning LGBTQ issues and those which don’t, which I shall hereafter refer to as Restrictive States and Non-Restrictive States, respectively. While you could be forgiven for assuming that all the Restrictive States are Republican strongholds (a misconception the journalists do little to discourage), the category includes a number of states with Democratic governors, including North Carolina, Kansas, Kentucky, Minnesota and Arizona, the latter two of which voted for Biden in 2020.
The article then sorts anti-LGBTQ hate crimes in schools (hereafter referred to as School Anti-LGBTQ Hate Crimes or SALHCs) based on whether they took place in Restrictive States or Non-Restrictive States. This is where they derive their headline claim that Restrictive States have seen a larger increase in SALHCs than Non-Restrictive States since 2015.
That specific claim appears to be true. But at a glance, you’ll notice that Restrictive States report significantly fewer SALHCs than Non-Restrictive States in absolute terms, in both 2015-19 and 2021-22. (The authors are honest enough to acknowledge this, with caveats, which we’ll come back to later.)
I presumed that the higher rate in Non-Restrictive States was an artifact of the differing population sizes between the two regions, and that the two regions would have similar rates of SALHCs once you controlled for population; I even thought that Restrictive States might have more SALHCs per capita. But apparently not: according to the 2020 census, the two regions have a conveniently symmetric share of the total US population, with a difference of less than three million people. When taking this into account, not only are there more SALHCs in Non-Restrictive States than Restrictive States in absolute terms, but Non-Restrictive States have significantly more SALHCs per capita as well: 0.052/100k versus 0.037/100k, or about 40% higher in the former than the latter.
But maybe this is a population artifact of a different kind. Intuitively, I would expect that a higher proportion of the population openly identifies as LGBTQ in Non-Restrictive States when compared to Restrictive States. If anti-LGBTQ hate crimes are only reported as such if the victim openly identifies as LGBTQ, it stands to reason that regions with a large LGBTQ population will see higher rates of such hate crimes than regions with a smaller LGBTQ population, all things being equal. The journalists themselves gesture at this interpretation: “In addition, it’s possible more kids are public about their identities in more liberal states, creating more targets for bullies, said Lanae Erickson…”.
To check this, I looked at this report from UCLA’s Williams Institute, which estimates the number of LGBT young adults (aged 13-17) living in each state. This is an imperfect metric, as SALHCs includes anti-LGBTQ hate crimes which were committed in kindergarten all the way up to 12th grade, and therefore most likely includes some number of hate crimes in which the victim was under 13. It’s also likely that that some of the people who were victims of hate crimes in a school were not themselves children or young adults (e.g. an openly gay schoolteacher’s car gets vandalised by homophobic students). The report is also from September 2020, and I imagine the number of young adults openly identifying as LGBT has changed significantly since then. These caveats aside, I think the estimate is good enough for our purposes.
Unfortunately for the journalists, not only do more LGBT young adults reside in Restrictive States than in Non-Restrictive States (a difference of about 50,000 individuals), but Non-Restrictive States still report more SALHCs per capita than Restrictive States. The gap is even wider than in our previous table: 8.866/100k versus 5.969/100k, a difference of 49%.
I really don’t know that I’m saying anything terribly controversial here. Put yourself in the shoes of the parent of an LGBTQ child. Obviously you’re going to be concerned about your child getting bullied in school, and would prefer to send them to a school where they won’t be bullied on account of their sexuality or gender identity (or at all, for that matter). When assessing which school to send your child to, which of these two metrics would be most important to you: the rate of homophobic or transphobic bullying at the school per capita, or how big an increase the school has seen in the rate of homophobic or transphobic bullying over the last few years? I find it hard to imagine any circumstance in which a sensible, caring parent would prefer, all things being equal, to send their child to a school which had a high rate of homophobic bullying over a school which had a dramatically lower rate of homophobic bullying, but which had recently seen a significant increase.
In fairness to the journalists, the article isn’t quite as one-sided as I’m making out, and they were balanced enough to include one interview with a California teenager who’s been bullied because of their gender identity. But there’s something so dishonest about including this anecdote alongside paragraph after paragraph of editorialising about how cruel the legislation is in Virginia, Oklahoma and Mississippi - without even acknowledging that California reports about three times as many SALHCs per capita than Virginia, Oklahoma or Mississippi.
To sum up:
When comparing states with legislation governing bathrooms, sports and sex education with states without such legislation:
The latter region reports more hate crimes against LGBTQ people than the former, in absolute terms
The latter region reports more hate crimes against LGBTQ people than the former, per capita
The latter region reports more hate crimes against LGBTQ people than the former, per capita that identifies as LGBTQ
The latter region reports more hate crimes against LGBTQ people in schools than the former, in absolute terms
The latter region reports more hate crimes against LGBTQ people in schools than the former, per capita
The latter region reports more hate crimes against LGBTQ people in schools than the former, per capita that identifies as LGBTQ
However, hate crimes targeting LGBTQ people have increased more in the former region than the latter
And what is the reader intended to take away from all of the above? That it’s uniquely difficult to be a young LGBTQ person living in a state with legislation governing bathrooms, sports and sex education.
Conclusion
I feel more than a little resentful for having to go to the trouble of carrying out all this statistical analysis, because I know I’m double-jobbing. I’m extremely confident that the journalists who wrote this story have already carried most or all of the calculations listed above. They pitched this great story to their editor about a surge in hate crimes targeting LGBTQ people following the passing of anti-trans legislation. They went into this dataset with high hopes, confident that it would back up their thesis, plain as day. Then they dug into it and found, to their horror, that the data painted precisely the opposite story. I can almost see their brows furrowing in confusion and panic as they go down their wishlist of statistical metrics in order of preference, discovering that none of them paint the desired picture. Hate crimes more common in Restrictive States - nope. Hate crimes in schools more common in Restrictive States - nope.
But they can’t just kill the story, not when they’ve already written hundreds of words and secured interviews with an impressive collection of intelligent, articulate teenagers. Without hard data to back up the testimony of the interviewees, the story is relegated to mere anecdote - it’s not serious political journalism, it’s just a culture piece, a human interest story. So instead, they spent ages digging through this dataset, twisting it, contorting it, pleading with it to give them any relevant-ish metric which would back up their narrative. And this was the best metric they could find. Stories like this don’t exist because of honest mistakes: they only come into being through deceit and manipulation.
As I mentioned above, the journalists were honest enough to acknowledge that anti-LGBTQ hate crimes in schools are more common in Non-Restrictive States than Restrictive States, in absolute terms. They’re quick to explain away this inconvenient finding by claiming that it’s a reporting artifact. Hate crimes targeting LGBTQ people are more common in Restrictive States, they argue, but are more likely to go unreported because of a culture of silence, whereas LGBTQ people who’ve been victimised because of their identity in Non-Restrictive States are more likely to report it to the relevant authorities.
I’m sure this is a contributing factor to the differing rates of hate crimes reported in the two regions (although I very much doubt it’s sufficient to explain the disparity on its own). At the same time - come on. Does anyone really doubt that if the data had told the story that the journalists wanted it to tell - that anti-LGBTQ hate crimes are more common in Restrictive States than Non-Restrictive States - they wouldn’t be shouting that from the rooftops? No way in hell would they be claiming that a higher rate of reported hate crimes in a region is good, actually, if the shoe had been on the other foot. But the data didn’t give them the answer they wanted, so they’re forced to play this tiresome game of “Anti-LGBTQ hate crimes are higher in blue states than in red states - and that’s a good thing.”
Why does this article exist? Personally, I very much doubt that any would-be criminal hears a story on the radio about his state congress banning male students from competing in female sporting events, and immediately thinks to himself “Boy howdy, time to beat up some queers!” I doubt that even the journalists really think that any kind of causal relationship exists between legislation like this and the incidence of anti-LGBTQ hate crimes, so why use such a weak argument when the data don’t support your conclusions? I suspect that they’re practising what Scott Alexander calls fake consequentialism. The journalists obviously think that trans girls should be allowed to compete in female sporting events, but this is a very difficult policy to defend: doing so requires one to deny the very concept of sexual dimorphism and differences in strength and speed between males and females (something which is obvious to toddlers) - the denial of which makes you look like a crazy person. Instead, rather than getting into a debate over whether males competing in female sporting events is fair (which they’re sure to lose, because it isn’t), they shift the conversation to the consequentialist claim that banning male students from competing in female sporting events causes a spike in hate crimes against the LGBTQ community. But as I hope I’ve made abundantly clear, even this argument doesn’t check out.
Maybe I’m mind-reading, maybe this isn’t what the journalists are doing at all. But regardless of their motivations, they made at least one provably false assertion and a batch of true-but-misleading ones. This article is an insult to their readers’ intelligence, it’s bad, and they should feel bad.
A police officer interviewed Benedict in the hospital after the bathroom fight which was initially cited (erroneously) as the cause of death. The bodycam footage reveals that a) Benedict admits to having started the fight; b) Benedict freely responds to the name “Dagny” and seems entirely at ease being referred to with female pronouns; and c) Benedict never requests to be addressed as “Nex”, or referred to with gender-neutral pronouns. This bodycam footage was released several weeks prior to the Post’s article, making the journalists’ decision to use Benedict’s death as an example of the harms wrought by transphobic bullying all the more distasteful (particularly given that certain journalists working for the Post almost certainly knew that Benedict’s father is currently serving time for repeatedly raping her as a prepubescent child).
The journalists know full well that their readers will hear about laws which “restrict trans student access to sports” and will think “oh my god, trans high schoolers in Mississippi are actually banned from playing sports!” as opposed to “male students in Mississippi may not compete in female sporting events, regardless of how they identify”. No matter how many times I encounter “respectable” journalists brazenly attempting to hoodwink their readers like this, I never feel any less insulted or disgusted. Truly, have they no shame?
To ensure I was looking at the same basic dataset as that on which the journalist based their findings, I performed a quick sense-check by comparing the total number of anti-LGBTQ hate crimes which were committed in schools according to my dataset and according to the figure cited in the article: the two figures were almost identical. Why “almost”? Well, the Post article asserts that 251 anti-LGBTQ hate crimes were committed in schools or colleges in 2022. According to my dataset, the figure was 245. I don’t think the discrepancy is an error on the part of the journalists, and assume the figure of 251 was accurate at the time of writing. I think what happened is that the FBI’s database is constantly being updated, and some crimes are either removed from the dataset (e.g. the victim withdraws their complaint) or reclassified as something other than a hate crime (e.g. further investigation determines that the perpetrator of the crime was not motivated by homophobia). In other cases the figure in my dataset matches the figure cited in the article exactly e.g. I found 114 anti-LGBTQ hate crimes were committed in schools or colleges in 2018, just as the journalists did.
I think your subheadline is a law that should be widely taught and recognized. If a statistic used is not the most obvious or expected one it implies the journalist already checked that one and discarded it as it didn’t say what they wanted it to
I feel like you did quite the high-integrity move, especially in comparison to The Washington Post, who likely would've done a stealth edit and moved on.