Some Best Practices for Economic Impact Studies
Economic impact reports vary wildly in quality. To bring any level of credibility to the field, academics and journalists should make sure these reports conform to best practices. Here is a list of what I view as some best practices for economic impact reviews. Note: the first four items on this list summarize some recommendations given by Siegfried, Sanderson, and McHenry in their 2007 JEE article.
Here are 8 recommendations for economic impact studies:
1. Avoid stating both the monetary impact and the jobs impact. Doing this could easily mislead readers into thinking there is an $XX million dollar economic impact AND a certain number of jobs created.
2. Clearly define the geographic region for which you are estimating an economic impact. Related to this:
o An appropriate multiplier should be used.
o Discussions of what Siegfried, Sanderson, and McHenry (2007) call import and export substitutions. This means the extent to which people would still spend their money in a particular area, even if an institution ceased to exist.
3. Define the basis on which there’s an economic impact (Siegfried, Sanderson, and McHenry 2007 call this the “counterfactual”). If you’re determining the economic impact of an institution, you should be quite clear whether your economic impact is based on whether the institution ceased to exist, for example, or whether there is a small expansion of the institution.
4. Expenditures should only be counted once. (This one seems like an obvious one, right?)
The above recommendations simply summarize those from Siegfried, Sanderson, and McHenry (2007). Here are four more recommendations:
5. If possible, have an outside expert review and critique the economic impact statement prior to publishing. This reviewer should consent to putting his or her name to the report as in verifying that it uses appropriate methods. This is costly, of course, so for many smaller studies this won’t really fit into the budget. Having an outside reviewer also stake his/her reputation on the study helps lend additional credibility to the study.
Sometimes, budgets won’t allow for recommendation 5 to occur, or it would be too difficult to implement. Recommendations 6-8 help for cases when an outside reviewer is impractical. (But should be used even if an outside reviewer is obtained.)
6. Clearly show the full analytical methods for how you computed the economic impact for each activity. This should be clear enough that any other economist could see how you arrived at that particular dollar amount for an economic impact.
7. Describe how the economic impact that is being estimated compares to studies of similar institutions, situations, industries, etc. This will help the reader understand if the estimates are implausibly high or more realistic. Given the many problems with the state of this literature, there are no guarantees that a study that “seems reasonable” compared to the literature will have estimates that are “correct”.
8. Clearly disclose the source of the funding so the reader can make a judgment on which direction any bias might be occurring. Many times, this is so obvious it probably doesn’t need to be stated. For example, if one is conducting an economic impact analysis for a university, you probably don't have to state who's paying for the study, as anybody remotely logical will realize that the universities paid for the studies. For the economic impact for a Marcellus Shale drilling, however, the source of the funding may be more important.
There are other recommendations that could be useful, as well. However, this starting list of 8 recommendations should be easy enough for both practitioners and journalists to follow, and will go a long way to creating accurate economic impact statements based on credible methods.
For additional reading on economic impact study guidelines, please see Siegfried, Sanderson, and McHenry (2007). Link here to ungated version.