I did some digging into the question of the effectiveness of “red flag” laws in preventing gun deaths.  It appears that the technical term for these “red flag” laws is “extreme risk protection orders” (“ERPO”) laws.

My methodology was simple — I googled the phrase “empirical study red flag laws,” and reviewed the results that appeared promising.

My conclusion is that there is no credible evidence of the effectiveness of these laws.  In addition to presenting some information about this, I’ll give an example of how the media spreads false information about this, claiming that a study demonstrated the effectiveness of “red flag” laws when the study itself states that it does not reach this conclusion.

I.  The Rand Corporation Report

I found a report (here) by the Rand Corporation dated April 22, 2020, titled The Effects of Extreme Risk Protection Orders.  It notes that as of January 1, 2020, 16 states and the District of Columbia had enacted “red flag” laws.  This report appears to be an attempt at a literature review, as it does not present empirical data.

The Rand authors investigated eight outcomes:

• Suicide
• Defensive Gun Use
• Gun Industry Outcomes
• Hunting and Recreation
• Mass Shootings
• Officer-Involved Shootings
• Unintentional Injuries and Deaths
• Violent Crime

A separate web page (here) describes their “Research Review Methodology,” which appears to present a careful methodology consisting of “five steps: framing questions for review, identifying relevant literature, assessing the quality of the literature, summarizing the evidence, and interpreting the findings.”

Their main conclusions are:

• Evidence that ERPOs “may increase” an outcome: “We found no qualifying studies showing that extreme risk protection orders increased any of the eight outcomes we investigated.”
• Evidence that ERPOs “may decrease” an outcome: “We found no qualifying studies showing that extreme risk protection orders decreased any of the eight outcomes we investigated.”

Colloquially, this appears to mean there was no useful research that would support any conclusion whatsoever about the effectiveness of “red flag” laws.  Not that this stopped 16 states, and the District of Columbia, from adopting such laws.

There was a bit more information in this Rand report.  They found “inconclusive evidence” of the effect of ERPOs on suicide.  With respect to the other seven outcomes studied, “no studies met our criteria.”

II.  The Loyola University Chicago Law Journal Article

I found this article titled An Empirical Assessment of Homicide and Suicide Outcomes with Red Flag Laws, published in 2021 in the Loyola University Chicago Law Journal.  The author is Rachel Dalafave, identified as a J.D./Ph.D. candidate in Law and Economics, expected to receive her degrees in 2022 from Vanderbilt Law School.

As an aside, I felt a bit of a kinship with Delafave.  I, too, was once a law student with an educational background in economics and econometrics.  I thought that her article was a pretty good analysis.

Delafave evaluated changes in homicide and suicide rates in states that had adopted, or not adopted, red flag laws.  Incidentally, according to her article, the number of states that had adopted red flag laws had increased to 19.  Her principal conclusions were:

• “Red flag laws have no statistically significant effect on overall or firearm-related homicides.”
• “[R]ed flag laws reduce firearm-related suicides by about 6.4%, with no statistically significant substitution to non-firearm suicides.”

To me, the first conclusion appears solidly supported by her analysis.  The second one seems a bit strange, so I want to address it further.  Her study does provide support for the proposition that red flag laws reduce suicides.

It appears to me that a limitation of this study is that Delafave’s analysis implicitly assumes that there were no other changes to the law, or other social changes, that would have affected homicide or suicide rates.  Changes in suicide or homicide rates that occurred after the adoption of a red flag law are assumed to have been the result of the red flag law.  I do not object to performing this type of correlation, but we should realize that there could be other explanations.  Post hoc ergo propter hoc is a logical fallacy, though it may be true in some instances.

I have a technical concern about Delafave’s reporting regarding suicide rates.  She found the following, which don’t appear consistent:

• “[A] red flag law decreases firearm-related suicides by about 6.4% overall.”
•  “The effect on suicides overall is therefore a statistically significant 3.7% for all groups.”
• “I found no evidence of a statistically significant relationship between the red flag laws and non-firearm-related suicides.”

These statements are all supported by her results, but they are strange.  The decrease in firearms-related suicides (6.4%) is quite a bit larger than the decline in overall suicides (3.7%), and both are statistically significant.  However, the change in non-firearm-related suicides, while positive — meaning that they increased — is not statistically significant.

I don’t think that the correct conclusion to draw is that there was no substitution from firearm suicides to non-firearm suicides.  I think that the correct conclusion to draw is that if there was such a substitution, it was too small to be statistically significant with this sample size.

Part of my confusion, I think, results from Delafave’s reporting of point estimates.  It turned out that the standard error for the estimated decline in firearm-related suicides was larger than for total suicides.  Instead of reporting the point estimates of a 6.4% decline in firearm-related suicides and a 3.7% decline in total suicides, she could have reported 95% confidence intervals:

• Red flag laws are correlated with a decrease in total suicides of 1.3-5.1%.
• Red flag laws are correlated with a decrease in firearm-related suicides of 2.2-10.6%.

I want to quote her conclusion in full (my emphasis):

This Article exploits state-level variation across time in the existence of red flag laws—gun control laws that permit police or family members to petition a state court to order the temporary removal of firearms from a person who may present a danger to others or themselves—to examine their effect on homicides and suicides. The existence of a red flag law reduces firearm-related suicides by 6.4% and overall suicides by 3.7%, with no substitution to non-firearm suicides. Red flag laws are not associated with statistically significant changes in homicides rates. Policymakers should consider red flag laws an effective method to prevent firearm-related suicide, one of the most deadly and prevalent potential causes of death in the United States. In light of this evidence, red flag laws should be more politically palatable than other forms of gun legislation because of their targeted nature and potential to balance the interests of gun owners against the negative externalities of gun violence.

There is something missing from this paper, which is a consideration of the costs.  In addition to enforcement costs, a red flag law risks depriving people of their right to bear arms.  Policy-makers may or may not consider it worthwhile to do so, in order to reduce overall suicides by a small percentage.  However, that evaluation should consider how many guns are confiscated, albeit perhaps temporarily, to obtain that benefit.  This cost is not addressed in Delafave’s paper.

It is a good paper, in my view.  I think that she should have been more clear in making the distinction between correlation and causation.

This paper does not support the use of red flag laws to prevent mass shootings, or to address homicides in general.  The beneficial effect suggested by Delafave’s paper involves suicides.  Tragic as a suicide may be, I find it a less compelling argument for restricting gun rights than homicides, which sometimes involve an innocent victim.

III.  The Violence Prevention Research Project Paper

This is the bad one, in my view.

I found this paper from November 2019 in the Annals of Internal Medicine, titled Extreme Risk Protection Orders Intended to Prevent Mass Shootings: A Case Series.”   The principal author is Dr. Garen J. Wintemute, whose bio (here) indicates that he is the Director of the Violence Prevention Research Program at UC Davis.

The paper presents information “for a preliminary series of 21 cases in which ERPOs were used in efforts to prevent mass shootings.”  It appears to be an initial report of the researchers’ effort to evaluate all 414 cases in which they report that ERPOs were used in California in 2016-2018.  They report having received 159 records thus far, and for some reason, provide a summary and case information for 21 of these.

For the life of me, I cannot see any reason for the publication of this paper.  It provides no useful information that I can discern.  It expressly states that it provides no evidence of the effectiveness of red flag laws — at least, that is my interpretation of the authors’ words, which are (my emphasis):

As of early August 2019, none of the threatened shootings had occurred, and no other homicides or suicides by persons subject to the orders were identified.  It is impossible to know whether violence would have occurred had ERPOs not been issued, and the authors make no claim of a causal relationship.  Nonetheless, the cases suggest that this urgent, individualized intervention can play a role in efforts to prevent mass shootings, in health care settings and elsewhere.  Further evaluation would be helpful.

I find this appalling.

I agree with the two highlighted sentences.  This paper presents no useful information, whatsoever, for evaluating the effectiveness of red flag laws.

The appalling part is that third sentence, which I did not highlight.  After admitting that it was “impossible to know” whether the ERPOs were effective, and admitting that they make no claim of a causal relationship, they make a completely unsubstantiated assertion: “Nonetheless, the cases suggest that this urgent, individualized intervention can play a role in efforts to prevent mass shootings . . ..”

No, they don’t.  These case studies show nothing useful whatsoever, as the authors admitted in the prior darned sentence!

I discovered this particular paper through a link from a Washington Post article summarizing the findings.  It followed the typical practice in such an article — a misleading headline, a misleading opening, with the inconclusive nature of the study not mentioned until later.  In this case, the headline and opening were:

‘Red flag’ laws may play role in preventing shootings, study finds

State laws that allow the removal of guns from people who present a threat to themselves or others may play a role in preventing mass shootings, according to a new study, a finding that could buttress support for “red flag” legislation being debated in Congress.

As quoted earlier, the study itself admitted that it made “no claim of a causal relationship.”  The disclaimers appear later in the article.

I find it very troubling that both the journal paper and the WaPo article drew unwarranted conclusions from essentially useless data.

IV.  My conclusion

I’d appreciate links to any other credible sources about the effectiveness of red flag laws.  My own conclusion, at present, is the same as the Rand Corporation Report.  I do not see any credible evidence that red flag laws prevent homicides, whether they are “mass shootings” or otherwise.  The evidence is inconclusive regarding suicides.

At this time, even if red flag laws were effective in reducing suicides by a small amount, I do not find this benefit to be a sufficient cause to give government the power to take guns away from law-abiding citizens.  This is not an empirical conclusion, but a political and policy judgment on my part.