But the forecasters were not only wrong but stupendously, inexplicably wrong. The official US forecast of future oil prices, released by the Department of Energy’s Energy Information Administration in June of this year predicted oil prices for 2008 that are off by a factor of two. In fact, the June EIA forecast estimates that even in the highest price scenario, oil prices wouldn’t reach $100 a barrel by 2030. The EIA’s “reference case,” their best guess, shows oil at $60 a barrel in 2030. Energy plans around these estimates are bound to be just as reliable.

Estimating future oil prices is a thankless business since so many unpredictable events can shape them. Better intelligence would have helped, but we know from past experience that it is tough to predict a war in Sinai (1973), or the fall of the Shah of Iran (1978), events that sent oil prices soaring. Yet this time, the sharp increase in oil prices in 2008 did not result from any such shock. Prices were largely driven by the sheer pressure of growing international demand, including the exploding economies of China, India, and Brazil. Instead of a careful review of these enormous market forces, EIA analysis was obsessed by reviewing the impact of drilling in Alaska—issues that have almost no impact on the price of oil in the United States, or anywhere else for that matter.

As intelligence failures go, failure to anticipate the doubling of the price of a commodity central to the U.S. economy is a doozy. Blindsided by the sudden price increase we have no practical options to prevent the painful adjustments that have been forced on households and businesses around the country.

The lack of practical options, of course, doesn’t protect us from terrible ideas sold in the name of instant cures—such as offshore drilling or cutting federal gasoline taxes. Our ignorance also has long term effects by making it difficult to construct the kinds of energy policies we need.

Last year, for example, there was a wrenching debate over increasing fuel economy standards with lots of warring analysis—much of it based on EIA-derived oil price predictions that turned out to be absurdly low. If the Congress in 2007 knew that prices would reach $4 a gallon in 2008, it is likely that the standards chosen would have been much higher.

In fairness, the Department of Energy was not alone in missing the obvious. In a classic case of herd thinking, private energy forecasters were also badly wrong about oil. Global Insights Inc. estimated that crude oil prices would be $46 per in 2030. While closer to the mark, even the usually reliable Deutsche Bank was caught off guard, estimating $80 per barrel by 2030

So what should be done? First, we should take a careful look at why the forecasting was so colossally wrong. Second, we should find a way to build new teams to undertake forecasting—teams that combine specialized energy expertise with analysts able to recognize that the basic terms of international supply and demand have been upended by massive economic growth in China and elsewhere.

The rules will keep changing. Major decisions about how to manage climate change will drive fundamental changes in how energy is produced and used. We badly need analytical teams able to keep pace with these developments, evaluate alternatives, and provide us early warning of problems and opportunities. This will require combining the talents of several agencies, including the State Department and the intelligence community.

And perhaps most importantly, we need to give the Energy Information Agency the resources it needs to do its critical job. The agency’s statistical departments and analytical departments have both been cut back and starved for years. It’s tough to complain about a group that is expected to do so much with so little. The price we’re paying for saving nickels and dimes in analysis is very high. We’ll never get energy intelligence completely right, but we can’t again afford to be so wrong about something so important.

Henry Kelly is the President of the Federation of American Scientists, cofounder and Chair of the Board for Scientists and Engineers for America.

Traditional tests measure performance in situations that will seldom, if ever, occur in an actual job.

Instead, we should first engage in a national debate about the expertise students need to acquire in order to prosper in the 21^st century, and only then settle on how best to measure their progress. The interactive methods used in computer games represent some of the most powerful ways to test newly acquired skills, but understanding why they are so useful requires a clear recognition of why our current testing procedures are thoroughly outdated.

Despite all the complaints about the numerous tests mandated by the No Child Left Behind Act, the problem is not too many tests but too few tests. High stakes, standardized tests are an artifact of a mass production model imposed on education out of necessity during the last century. Traditional tests measure performance in situations that will seldom, if ever, occur in an actual job. Someone trained to solve problems working in isolation, with no access to reference material and no ability to consult experts, is largely useless in today’s economy—however many facts they may have mastered. We use such tests because they are inexpensive to implement.

But consider the ideal classroom scenario: An instructor able to spend plenty of time with each individual student, constantly challenging them, asking probing questions, and presenting increasingly complex challenges tailored for each student. By the time a test is taken the student should have run through the material enough times that they and their instructors have high confidence in success.

These powerful methods aren’t used in standard classrooms for two obvious reasons—they’re unaffordable and we continue to think of the classroom the same way they did 200 years ago. A solution, however, has emerged from an unexpected source—computer games.

The average U.S. teenage boy spends about 14 hours a week glued to computer games.^[1] Yet, we aren’t taking advantage of that. Most adults can’t imagine how the lessons of Super Mario could be applied to high school science or history. But consider that a good game will capture and hold a player’s attention with a series of compelling goals, each slightly beyond the player’s current abilities. A great game draws players in what designers call “the flow.” Once in it, they will try, fail and try again, working for hours to master the skills needed to win.

Surely it’s possible to create challenges in biology, history, or engineering that can capture and hold attention.

What’s striking, of course, is that they’re also being continuously tested. Tests are an integral part of winning, and players accept the fact that they will fail before they master the skills needed to move on. If you keep crashing your simulated aircraft you know that you’ve got to work harder—and want to. Winning at the most advanced levels of game play requires players to draw on a huge body of knowledge and experience.

Winning many games, moreover, often requires more than mastery of specific skills. They require precisely the skills that the Partnership for 21st Century Skills recently reported are in greatest demand in today’s economy: gathering evidence, making decisions under uncertainty, evaluating options, and (in the case of multiplayer games) working effectively as a member of a team.^[2]

The U.S. Department of Defense, which unlike most organizations is completely unembarrassed about having its employees play games (war games), has come to appreciate the power of simulation-based games to teach and test individuals and teams. They have convincing evidence that skills acquired through simulations translate into performance in the field.^[3]

Simulation-based instruction can reproduce the complexity, confusion, and tension of field conditions so faithfully that the success a soldier gains in the simulation translates directly into reliable performance during first real combat experience. This powerful transfer from simulation to practice has also been demonstrated for pilots and several areas of surgery.^[4] Surely it’s possible to create challenges in biology, history, or engineering that can capture and hold attention.

Building software to teach and test complex skills is expensive. Several billion dollars were invested and lost in education technologies towards the end of the dot-com boom a decade ago, and investors have been wary ever since. Schools and universities are a notoriously poor market for innovations, in part because of an understandable reluctance to take risks with unproven approaches. But as a result, an enormous opportunity is being lost.

We’ve confronted this kind of market failure before. The federal government has been able to fill gaps by funding basic science research, development, testing and evaluation that can be picked up by private investors. It can do this in new technologies for learning as well and create significant markets for robust new products…or we could just continue to fool ourselves that our education system can be fixed with ad hoc testing standards.

Henry Kelly, Ph.D., is the President of the Federation of American Scientists in Washington, DC and Chairman of the Board of Directors for Scientists and Engineers for America.

Notes

[1] Martin, Suzanne and Oppenheim, Koby. Video Gaming: General and Pathological Use. Trends & Tudes. Volume 6, Issue 3 March 2007 Harris Interactive Inc. http://www.harrisinteractive.com/news/newsletters/k12news/HI_TrendsTudes_2007_v06_i03.pdf

[2] Beyond the Three Rs, Voter Attitudes toward 21^st Century Skills. October 2007. Partnership for 21^st Century Skills.

[3] Fletcher, Dexter. Advanced Technology for Defense Training. Institute of Defense Analysis. June 2006. http://www.digitalpromise.org/newsite/Resources/Research/Dexter_Fletcher_Jun14.pdf

[4] Boosman, Frank. Simulation-Based Training: The Evidence is In. July 2007. Chief Learning Officer Magazine. http://www.clomedia.com/content/templates/clo_article.asp?articleid=1874&zoneid=162