Would it Kill the CBO to Show their Work?
August 9, 2017
This piece was published in the Washington Examiner on August 9, 2017.
The Congressional Budget Office (CBO) is a pretty darn good research institution. That doesn’t mean, though, that it couldn’t stand some improvement. The “CBO Show Your Work” Act, introduced by Sen. Mike Lee (R-UT) would do just that, forcing the Budget Office to show how they mathematically managed their methodologies. Having the CBO “open their books” would lead to a better debate over how best to model the world, and most importantly, give taxpayers a realistic glimpse into how much politicians’ promises would actually cost.
Criticism of the CBO is not new. In 2009, the CBO lowballed the costs of the Affordable Care Act (ACA) at less than $1 trillion. But, as The Weekly Standard pointed out, “For a variety of reasons, this tally doesn’t remotely reflect the bill’s real ten-year costs. First, it includes 2010 as the initial year. As most people are well aware, 2010 has now been underway for some time…If the clock were started in 2011, the first full year that the bill could possibly be in effect, the CBO says that the bill’s ten-year costs would be $1.2 trillion.”
As the budgetary “scoring” debate over the ACA and now the GOP alternatives shows, modeling transparency would save the CBO from a good deal of embarrassment. In the ACA repeal/replace kerfuffle, the CBO ascribed “near magical powers” to the mandate (as Forbes’s Avik Roy put it).
The “GOP would cut off 22 million from health insurance” talking point came from the CBO analysis of the plan, which assumed that if the mandate was kiboshed, the mandate would lead to a mass exodus from the individual insurance market. In fact, most empirical estimates of the mandate’s effect seem to show little evidence of the provision significantly increasing enrollment.
In a 2015 analysis, the Commonwealth Fund found that the CBO “overestimated marketplace enrollment by 30 percent and marketplace costs by 28 percent, while it underestimated Medicaid enrollment by about 14 percent.” The most telling part of the analysis, however, came when the Commonwealth researchers tried to determine how accurate the CBO would have been if it could correctly guess insurance premiums in advance. There were so many question marks about the parameters used by the CBO, though, that the researchers cautioned that their results were not “directly comparable to the CBO’s 2010 estimates.”
That’s not to say that the CBO’s estimates are worse than other forecasters, like the Lewin Group or Urban Institute. In fact, the aforementioned Commonwealth researchers believe that the CBO generally performed better than other institutions in the insurance enrollment guessing game.
But, as I was repeatedly implored to do as a research assistant, it’s important to “show all of your work” when making a claim based on data. If the CBO fully disclosed how they managed to create a model that showed such large effects from the mandate, a robust debate can develop about that model’s validity. And, everyone on both sides of the debate would be better off for it.
Nor is the CBO being asked to go above and beyond the call of duty. In the academic world, there’s been an increased trend towards documenting research step by step, and publishing various Excel and Stata sheets online showcasing the process. As Sen. Lee pointed out in his piece promoting the bill, the American Economic Association requires researchers to ensure their data “are readily available to any researcher for purposes of replication.” Allowing economic models to be out on full display allows the tinkering process that makes for more fulfilling discourse.
But as can be expected, any bill threatening to shake up the status-quo will encounter criticism from both sides of the aisle. From the left, any legislation trying to meddle in CBO processes undermines the legitimacy of a neutral arbiter of fiscal disputes. This hot take paints Sen. Lee’s bill as just the latest attack in an escalating war on a “nonpartisan, reality-based” institution.
This seems hysterical, considering that other countries have seamlessly managed to show their work in budgetary analysis without somehow impugning their reputation. Would it kill the CBO to have meticulous web-based documentation of their forecasting parameters, along the lines of the UK’s Office for Budget Responsibility?
In a world where basic spreadsheet errors can have immense research ramifications, it’s important for the CBO to open their research books to the public. If the Budget Office could only show their work, researchers and taxpayers alike would no longer be in the dark about lawmakers’ best-laid plans.