EVALUATING TECHNOLOGY INNOVATION PROGRAMS - THE USE OF COMPARISON GROUPS TO IDENTIFY IMPACTS

This paper discusses the pros and cons of alternative comparison groups for evaluating technology innovation programs, and focuses specifically on the selection of a comparison group for the evaluation of the U.S. Department of Energy's (DOE) Energy-Related Inventions Program (ERIP). The pros and cons of five alternative comparison group options are discussed, including: inventor societies, innovation and incubator centers, patent holders, near-participants, and program referrals. Program referrals are selected as a suitable comparison group for evaluating the Energy-Related Inventions Program. Data collected on ERIP participants and referrals provide strong evidence that ERIP-supported technologies achieved their considerable commercial success, at least in part because of the support provided by the DOE. There are large differences between the program referrals and the ERIP participants in terms of several indicators of commercial success. For example, average dollar sales by ERIP participants are an order of magnitude greater than the program referral group. This paper illustrates that the simultaneous tracking of program participants and a matched comparison group can enhance the evaluation of technology innovation programs by helping to isolate the effects of the government program from the host of other factors that influence the commercialization of inventions.