Science Progress | Where science, technology, and progressive policy meet

ARPA-E is Here to Stay

Comparing the New Energy Agency to 20th Century Federal Technology Programs

Citations and footnotes are available in the pdf version of this article.

Since its inception in 1958, countless groundbreaking innovations have been commercialized from the Department of Defense’s unique research arm: the Defense Advanced Research Projects Agency, or DARPA. Created in the aftermath of Russia’s launch of the first space satellite, Sputnik, DARPA’s primary goal was to ensure that the United States military maintained its technological superiority. As time progressed, inventions churned out of DARPA—from global positioning satellite systems, to the iPhone’s Siri, to the Internet—have become integral parts of out society and changed the way everything, from our daily life to the economy, operates.

In recent years a similar debate over whether the United States was losing its technological prowess in the realm of alternative-energy innovation has gained momentum. The National Academies report from 2005, “Rising Above the Gathering Storm”, highlighted the need for the United States government to stimulate high-impact, clean-energy innovation on our soil. One recommendation was to create an energy agency focused on catalyzing groundbreaking clean energy technologies towards market adoption.  Thus, in 2007 Congress authorized the creation of the Advanced Research Projects Agency – Energy, or ARPA-E, with the first appropriation coming in 2009 from the American Recovery and Reinvestment Act.  Housed within the Department of Energy, ARPA-E’s main goal is to invest, develop, and commercialize “transformational energy technologies” that “disrupt the status quo”.

In its first year ARPA-E received a staggering 3,700 concept papers (relatively brief overviews of the proposed technology’s merits) — well above the 500 to 800 expected.  This number underscores the need of ARPA-E in today’s advanced clean energy landscape.  The first funding-opportunity announcement led ARPA-E to fund 37 projects worth a collective $151 million.  ARPA-E plays an important role in the commercialization process of these capital-intensive energy innovations by bridging potential first “valley of death”— a common term used to describe the funding gap between a technological concept and a working prototype —for winning proposals.  It is in ARPA-E’s mission to find and fund the formation of advanced energy technologies, while filling a funding gap that the private sector usually finds too risky to undertake.

Due to the technical progress of ARPA-E’s early stage projects, millions in venture capital and corporate dollars have been drawn off the sidelines.  For example, OPX Biotechnologies, an ARPA-E project in the Electrofuels program armed with a $6 million grant, garnered $36.5 million in private funding last year after showing progress—a substantial return on the dollar for federal investment. Similarly, FloDesign Wind Turbine, an ARPA-E project developing a completely new wind turbine inspired by jet engines, announced a fundraising round of $27 million from private investors. As ARPA-E displays ability to bridge valleys of death for early stage clean energy companies, examples like OPX Biotechnologies and FloDesign Wind Turbine should become more common.

On the surface, and in name, ARPA-E resembles its predecessor, DARPA. In terms of similarities, the agencies have comparable structural organizations and envision concepts for technology programs in a similar manner. But beneath the hood, ARPA-E and DARPA differ in the way in which technologies move along the commercialization path. DARPA was able to move its innovations from concept to reality in large part because it leveraged the procurement power of the Department of Defense.  ARPA-E, however, lacks a similarly large and guaranteed source of demand or procurement ability in the Department of Energy.  Furthermore, given how complex and established the current energy landscape is in the United States, there is not a simple answer on how ARPA-E can directly transfer its technologies to energy markets.

Using DARPA as a relevant backdrop, the goal of this article will be to explain these salient differences, and identify potential strategies for ARPA-E technologies to experience similar commercial success. As this paper will explore, the differences between ARPA-E and DARPA include the availability of first adopters of technology; how their projects identify and penetrate markets; and whether the technologies each agency funds have to compete on price. By providing a comparative analysis of DARPA and ARPA-E, this paper examines the differences between the two agencies—and will glean from this analysis a few recommendations on how ARPA-E can amplify its efforts.


DARPA and the power of ‘market pull’

 As time progressed after DARPA’s launch in 1958, the agency developed a number of unique attributes for a government-funded R&D agency. DARPA’s organizational structure, mission objectives, and insulation from congressional inspection allowed the agency to gain a critical role in “seeding and encouraging” new technology fields in the United States.  Unlike most other R&D funding agencies, DARPA set its sights on a technological vision and would then source and fund technologies that had the potential to reach these goals. Since its onset, DARPA has been envisioning these “white spaces” in various transformative technology areas.

When thinking about next steps for the technologies DARPA funds, the agency streamlines the R&D process from discrete steps to more of a “connected R&D” method. In general, DARPA employs four stages of innovation to ensure that technologies are on the right track.  In an analysis done by William Bonvillian from MIT’s Washington Office and Richard Van Atta from Institute for Defense Analyses, this track is seemingly broken up into the following four stages:

1) breakthrough/R&D stage
2) prototype/demonstration stage
3) incremental advances stage
4) initial market-creation stage

Technologies developed by DARPA have consistently led to both military and commercial applications. However, because many DARPA projects originate to serve Defense Department needs, DARPA can effectively rely on military services’ procurement to provide the needed market pull to ensure successful commercialization of its technologies, as the fourth bullet above indicates. In the case of the budding information technology industry, both DARPA and the National Science Foundation contributed about 30 percent of overall federal R&D in 1990. But by 2005 DARPA represented only 6 percent while the National Science Foundation contributed 35 percent in overall federal R&D for information technology. In a time span of just 15 years, we can numerically see how DARPA’s catalytic role in the industry dwindled as the industry itself matured.

On the civilian side, DARPA-funded research can be traced to numerous technology products now sold by private-sector companies. Some examples include Xerox’s Ethernet, Apple’s desktop computing and graphical user interface, and Cisco’s Internet-protocol routers.  Even today’s internet giants, such as Facebook and Google, can trace some of their intellectual property back to what was originally DARPA-funded research, originally designed for military application and procurement.

In the case of Sun Microsystems, an information technology, computer software, and computer hardware company recently acquired by Oracle, DARPA was instrumental in its spinout to the private sector during its early stages.  Sun Microsystems had licensed its workstation-board technologies from DARPA to begin selling them commercially.  In 1982 Sun Microsystems had raised $4.5 million in venture capital funding, but DARPA was critical in engaging with academic institutions to encourage them to purchase these workstation computers — providing that critical market pull.  Stanford, UC Berkeley, and Carnegie Mellon University actually accounted for 80 percent of orders received in Sun Microsystems’ first year, in large part thanks to DARPA’s funding to these institutions.

DARPA also created opportunities for universities and laboratories to work together, with the hope that these partnerships would lead to the scaling of these technologies.

The Strategic Computing Initiative, a 10-year DARPA project that began in 1983, is another example of linking industrial, university, and research entities to work together and form “innovation networks.”  The goal of this initiative was to bridge together advances in a number of different technology areas and apply them to technological needs of the military.  In particular, the program looked to develop advanced machine-intelligence technology by leveraging advancements made with faster chips and other recent computing advances.

The Strategic Computing Initiative began out of the desire from the military to apply advanced computing strategies to their needs. Specifically, the Army desired an autonomous land vehicle, the Navy desired an aircraft carrier battle-management system, and the Air Force desired an automated pilot associate. At the time, folks at the Strategic Computing Initiative realized that recent advances in microelectronics, computer science, and artificial intelligence should be coupled to undertake these technological requests. Though the initiative had its challenges, the main point is that the integration of milestones was a central characteristic of the program. Visionary demand-pull factors played an important role with the initiative: The military applications led to requirements for intelligent functions, the intelligent functions drove the requirements for the system architectures, and the system architectures set forth the requirements for suppliers of microelectronics.

In order to ensure the long-term success of ARPA-E funded projects, one must not overlook the issues surrounding stage 4: initial market creation. Considering the agency is only three years old, the main focus thus far for ARPA-E has been to bring technologies from stage 1 to stage 2. Overall, the energy industry has a low appetite for risk and existing firms usually push for incremental advances. Furthermore, though utilities are now investing in renewable energy technologies, they are also diversifying their energy portfolio to spread their risk. In the above examples, DARPA’s synergistic orchestration of both technical and market forces to develop desired applications for the military with Sun Microsystems and the Strategic Computing Initiative are relevant models for the advanced energy space.


DARPA vs. ARPA-E: Compare and contrast



ARPA-E imported many of the important qualities that DARPA exhibited.  In regards to the organizational structure, programs developed within each agency have short timeframes (three to five years); these programs, all focused on different yet sometimes related technology areas, have to generate results or else they are terminated.  Within their respective departments, Defense and Energy, both agencies are insulated from bureaucracy in that they report directly to the Office of the Secretary.  This is a significant factor in eliminating bureaucratic lag time that many other offices have to go through, and enables both ARPA-E and DARPA to operate at a more efficient pace.

The unique organizational structure allows both agencies to act relatively swiftly in making personnel and project-management decisions, accelerating the motion of technologies down the innovation pipeline. ARPA-E followed DARPA’s footsteps in utilizing a challenge-based “right-left” research model, where the end-result goal sits on the right, and the technology pathways to get there begin on the left. Program directors contemplate technology solutions in different sectors (right), and then source cutting-edge technology research projects to achieve these results (left). This model contrasts the “peer-review” process, where proposals are reviewed by various experts of the particular research area, that most R&D agencies employ to deliver grant funds to qualified research projects.  Rather than focusing on incremental advances, ARPA-E and DARPA prioritize envisioning unique “white spaces” for future technological success.


But despite their similarities, there are also significant differences between the two agencies. (See Appendix 2) The projects ARPA-E funds are entering what can be described as a “complex established legacy sector.” The energy sector is a highly complicated and diversified industry, with energy sources, regulations, and infrastructure all varying at regional, state, and municipal levels.  While DARPA technologies were conceived with the idea that there were no current technological solutions, ARPA-E is developing technologies to alter the status quo of the existing energy economy, where much of the end product is a uniform commodity (liquid fuels or delivered electricity). These are two very different missions.

Case in point: The idea of unmanned air vehicles, now being heavily used by the United States for surveillance and attacks across the Middle East and Asia, was conceptualized at DARPA because there was no current technology with the capabilities desired.  On the other hand, current ARPA-E funded projects including modernizing existing grid infrastructure (Grid-Scale Rampable Intermittent Dispatchable Storage , or GRIDS), Green Electricity Network Integration, or GENI, programs, improving end-use energy efficiency (Building Energy Efficiency Through Innovative Thermodevices, or BEETIT), Agile Delivery of Electrical Power Technology, or ADEPT programs), altering transportation methods (Batteries for Electrical Energy Storage in Transportation , or BEEST, and Methane Opportunities for Vehicular Energy, or MOVE) programs), and changing the domestic fuel paradigm (Plants Engineered to Replace Oil, or PETRO, and Electrofuels programs). Thus, from a conceptual standpoint, unmanned aerial vehicles developed a drastically new technological direction, whereas ARPA-E programs are focused on drastically improving the existing way in which we “generate, store, and utilize” energy.

Another obstacle ARPA-E faces but that DARPA does not is that it works without a major first adopter. Technologies advanced through DARPA had a natural, albeit not guaranteed, customer in the Department of Defense. Since the Department of Energy isn’t a major direct purchaser or procurer of technology or equipment, ARPA-E doesn’t have a parallel source of demand for its technologies. This deprives ARPA-E technologies of a key “market-pull” effect so critical to commercialization.

The importance of market demand in pulling innovations out of development and into the market cannot be stressed enough. ARPA-E technologies must enter a crowded market and ultimately compete on price with the legacy fossil and nuclear sectors, while DARPA technologies do not suffer the same market competition. In the case of unmanned air vehicles, the primarily goal of this technology wasn’t necessarily to be cost competitive, but to give the military advanced surveillance and striking capabilities beyond what was currently available.  In the case of ARPA-E’s Batteries for Electrical Energy Storage in Transportation (BEEST) program, on the other hand, the goal of the program isn’t just to fund batteries with higher energy densities, but to also make sure that these batteries are 30 percent below today’s cost of vehicle batteries. In order for projects in ARPA-E’s BEEST portfolio to achieve eventual consumer adoption, they must display an ability to compete on price in existing markets. This cost component makes ARPA-E’s job considerably more challenging than that of DARPA.


Addressing the market challenge

Despite these challenges, ARPA-E is working to smooth the path to market and testing under real-world settings. In 2011,ARPA-E signed a memorandum of understanding with the Electric Power Research Institute and Duke Energy to provide a test-bed for many of its technologies. The Electric Power Research Institute’s main focus is to conduct research programs on electricity across a number of different energy sources. In addition, the institute works heavily with electric utilities and its members represent over 90 percent of the electricity generated in the United State. Duke Energy is one of the largest energy companies in the country and delivers electricity across five states to roughly four million customers.

This agreement is an important next step for ARPA-E projects, catered for (but not limited to) the GRIDS, GENI, and ADEPT programs.  Duke Energy plans to study the performance of ARPA-E technologies at the company’s own test-bed facilities for renewable energy, smart grid, and energy storage. The agreement should also pave the way for ARPA-E awardees and Electric Power Research Institute members to deploy and test technologies at agreed-upon sites and facilities. Testing and demonstration partnerships that can integrate multiple ARPA-E programs are a great way for these technologies to display their potential—all while getting critical feedback.

While this one-off deal is great, it represents only a subset of the market. Utilities, the obvious bulk purchasers of energy in the private sector, tend to be conservative and have a vested interest in providing the lowest possible rate for their ratepayers. Similarly, major corporations or government agencies can’t and won’t adopt next-generation energy technologies if there isn’t a positive bottom line effect for a return on investment. So despite the promise of ARPA-E technologies, the agency still has the additional challenge of navigating these barriers. Thus, despite the important role that ARPA-E plays in the innovation-commercialization pipeline, there is a need for additional federal policies to help foster demand for the clean energy technologies ARPA-E was designed to develop.


Prioritizing energy R&D: comparisons with the Manhattan Project and Apollo Program

The federal government has historically funded research to guide the innovation and development of a variety of technological and scientific advancements that meet critical public or national needs.  In defense, environmental protection, public health, and elsewhere, leaders of both parties have historically agreed that innovation and technological advancement have significant public benefits, and have invested public resources that reflect this common good.  In defense and in many other areas, the seeds that these federal investments have planted have grown to become major economic drivers in industry, cure fatal diseases in health, and iconize American ingenuity in space exploration. Without these sage investments, our country would likely not have enjoyed the economic prosperity that we have seen over the past 60 years.

As many scientists, policymakers, and economists have noted for years, our nation is at a critical juncture when it comes to the global clean-energy race. Unfortunately, annual allocation of federal R&D for energy has decreased proportionally when compared to other research areas over time. Federal R&D investments in health care and defense have grown consistently, while we have seen energy R&D fall to a fraction of overall federal R&D since 1980. (See Appendix 1)  If the United States desires to be a leader in the globalizing clean-energy landscape, a major commitment needs to be made now for funding advanced energy R&D.

Major government-led pushes for new technologies are nothing new. In the case of defense and space exploration, the federal government has promoted and directed major R&D efforts on scales far more significant than we’ve seen in energy. Specifically, the Manhattan Project was created to develop the atomic bomb in the context of World War II, while the Apollo Program was formed to send astronauts to the moon in the context of the Cold War.

Both received a level of focused public investment that we have not seen with energy, though they also had other attributes that made them different than in the case of energy. For example, both the Manhattan Project and the Apollo Program were major pushes to develop and implement relatively small number of physical machines for very specialized purposes—to end World War II and to put Americans on the moon, respectively. Furthermore, both were put on very strict timelines given that the United States was competing with other nations in both cases to develop those specialized capabilities. With the facts and effects of climate change still disputed by incumbent energy interests within Washington, the parallel imperative for funding alternative energy sources is not felt strongly in today’s polarized political climate. As a result, we haven’t seen a similar injection of funds — at such scales as the Manhattan and Apollo projects — into the renewable-energy sector.

But looking at funding amounts across various R&D initiatives isn’t a fair comparison in the case of energy.  Two major market failures have hindered widespread deployment of nascent clean energy technologies: an underinvestment in R&D and an absence of mechanisms to address environmental externalities. The former market failure relates to the aforementioned discussion of “technology-push” factors, such as increasing clean energy R&D investments; the latter market failure can be denoted as the “demand-pull” factor for clean energy technologies.

Many prominent academics, think tanks, politicians, and venture capitalists have called for appropriations to ARPA-E to significantly increase, and the benefits of having a larger and thus more influential agency are well discussed and understood.

However the amount of funding is only part of the battle for ARPA-E. Both the Manhattan Project and Apollo Program had concrete goals with an end-user of their respective technological developments. DARPA’s projects have enjoyed success with the military as a guaranteed customer for successful technologies. These comparisons suggest that ARPA-E’s technology push approach would be more effective if complemented with demand-pull effect.


Conclusions and recommendations

In general, policymakers must realize that the combination of climate change, energy security, and economic growth provide for a compelling reason for action to accelerate clean-energy innovation. Unfortunately, funding for energy R&D has both dropped over time and is incomparable to the massive undertakings of the likes of the Apollo Program or the Manhattan Project.

But technology-push factors in R&D can only be truly effective if they are complemented by market-pull factors. Considering ARPA-E funds capital-intensive projects that need to be incorporated into existing legacy energy markets and infrastructure in a way that DARPA does not, identifying partnerships and opportunities for follow-on funding and demonstration to advance these technologies in the short term is of utmost importance.

The above analysis gives insight on the origins of ARPA-E, and a present day comparison with DARPA. Again, ARPA-E’s primary focus thus far has been to find and fund technologies that are game changing and transformative—but that is only the beginning of the story. Because ARPA-E’s goal is to help its grantees translate technology for the first time from lab scale to market scale, there is a need for the agency to define commercialization roadmaps for the projects it funds. Though many of ARPA-E’s projects may have technological promise, coupling ARPA-E’s funding for early-stage innovation with strategic partnerships to create market demand, both in the public and private sector, is crucial to the longstanding success of its projects—and perhaps to the existence of the agency itself.

Though it was easier said than done, DARPA was able to strategically leverage its role in the Department of Defense to help prioritize its own research programs. Despite the fact that DARPA’s mission has changed throughout its history, its projects usually had narrowly defined objectives. This enabled the agency to take advantage of recent technological breakthroughs, build upon existing research, and strategically integrate technological milestones. The Strategic Computing Initiative was an example of DARPA’s role in solving unique military problems—with an eventual commercial application at the end. At the same time, having these interactions with end users allowed DARPA to display its advances to and iterate with decision makers in the military establishment.

For the first time, many ARPA-E projects are now in a transition period approaching options for manufacturing scalability and market viability for their technologies. Broadly speaking, the government can help facilitate this transition by providing beneficial standards, directly opting to procure advanced energy technologies, enacting a price on carbon, and/or setting a national clean energy standard. These broader demand-pull factors may have a more direct effect as ARPA-E-funded technologies mature down the line. Since ARPA-E is in its youth, it is timely to understand how the agency is approaching near-term ‘handoff’ mechanisms.

For these early-stage technologies, the near-term solutions include providing viable pathways to market by targeting first adopters, providing opportunities for technology demonstration, and identifying follow-on funding opportunities for potential customers as well as producers of the new energy technologies. Both the government and the private sector can help in this area. Furthermore, given that ARPA-E’s technology programs are diverse, from advanced electric vehicle batteries to innovative carbon-capture technologies, these pull-factors are unique to each of ARPA-E’s program areas.

In the energy landscape there is a clear gap in the government for additional de-risking of these advanced technologies. Specifically, moving these various technologies into demonstration and commercial pilot plant stages should be emphasized. Yet, ARPA-E itself may not be the place to do this work; the agency should stay true to its transition-stage role in the innovation pipeline. More formalized and recognized relationships with different aspects of the Department of Energy, such as the Office of Energy Efficiency and Renewable Energy, and more partnerships, such as the established agreement with Duke Energy and the Electric Power Research Institute, would allow ARPA-E to continue its role in finding and funding transformative technologies.

Even though ARPA-E may be making significant technological advances, there are clearly many hurdles on the roads to commercialization. Given the number of technology areas ARPA-E is investing in, there isn’t a cookie-cutter approach to understanding how these programs will succeed beyond the laboratory. Looking toward the future while building upon experience, larger scale market-pull factors must be coupled with the technology push and translational work occurring at ARPA-E. If ARPA-E can further cement linkages and handoff mechanisms all while focusing on the high-risk, high-payoff R&D it was built for, the agency will lay a foundation on which it can build a visionary and respected reputation of its own.

Varun Mehra has a degree in Mathematics/Economics from University of California, Los Angeles, and is currently a DOE Scholar with ARPA-E’s Technology-to-Market team. Opinions expressed in this article are those of the author and do not necessarily represent the views of ARPA-E or any government agency. Image via BigStockPhoto.

Download the full pdf with endotes here.

Comments on this article

By clicking and submitting a comment I acknowledge the Science Progress Privacy Policy and agree to the Science Progress Terms of Use. I understand that my comments are also being governed by Facebook's Terms of Use and Privacy Policy.