Traffic makes its way to Ambassador Bridge that connects Canada to the United States at Windsor, Ontario. Trying to analyze the benefits of a free-trade agreement, for example, requires more than just adding up imports and exports.Mark Spowart/The Canadian Press
Good economic analysis is surprisingly hard, as in many (but not all) cases it is impossible to run controlled experiments. To analyze the effects of a policy or event, economists must filter out everything unrelated to that event. They don't ask "how is the world different after that event?" but rather "how would the world be different had that event not taken place?" In economics speak, we need to consider the counterfactual.
Bad economic analysis, on the other hand, is quite straightforward; take a policy change, eyeball economic data before and after the change and jump to a conclusion. Three recent issues in the news are illustrative of this.
One example is Jim Stanford's piece on why the Canadian-U.S. Free Trade Agreement was a failure for Canada. Stanford includes a number of metrics to support his argument, such as: "[i]n the mid-1980s, 19 per cent of all U.S. imports came from Canada. Today, our share of their imports is just 14 per cent."
This statistical examination, while interesting on the surface, is ultimately meaningless. We cannot draw any conclusions from the data, as we need to ask "what would our share of imports be in the absence of the free trade deal". A great deal has changed since 1985, including the emergence of China and India as world trading powers. It is naive to think that international trade would have remained the same had we not signed a trade deal with the U.S.
We need to control for factors outside of the trade deal and consider the counterfactual where the deal had not been signed. In a 2004 study, economist Daniel Trefler does just this and finds a small, but positive, economic impact from the deal, including a 14 per cent increase in plant-level labour productivity. By failing to consider the counterfactual, Mr. Stanford comes to the wrong conclusion.
A second example is illustrated in Corporate tax revenues higher despite lower rate, as Bill Curry discusses how both supporters and detractors of the tax cuts will find evidence of their positions:
"Looking over the last four years of data, advocates of the tax cut could argue that cutting the rate has had virtually no impact on federal revenues. Critics of the tax cut could compare the latest numbers to the 2007-08 fiscal year and argue that revenues are down $8.9-billion."
Both positions are inherently flawed, as both fail to consider the counterfactual of what would have happened without the tax cuts. The lower corporate rate does effect investment, output and pre-tax corporate profits and therefore tax revenue (a fact often missed by detractors of the cut), but there are factors other than the tax cut that alter corporate revenue (a fact missed by those who would use the revenue change to prove the tax cuts pay for themselves). The best estimates economists have show that corporate tax revenues would be higher in Canada with higher federal rates, but the relationship is far from 1-to-1 (raising rates by 10 percent does not lead to 10 per cent more revenue for the government).
Finally, David Suzuki, in An economy is a means, not an end:
"But spending money on things that anyone would see as negative, or even horrendous, contributes to positive GDP growth – including disastrous oil spill cleanups, car accidents…"
This fallacious line of thinking has a name, the broken window fallacy. It is a fallacy because we need to consider how those resources would have been used if they were not cleaning up oil spills and how that activity would have increased economic activity. It is a fallacy because it fails to consider the counterfactual.
To truly understand the economic impact of a policy, we need to control for other factors which may have changed unrelated to that policy. If we do not consider the counterfactual, we are likely to be led astray.