Transparency Is Essential in Science Policy
Options for greater openness and transparency in science policy decision making by politicians and policymakers are presented to Congress largely in the following ways: reinstate the Office of Technology Assessment; elevate the status of the Office of Science and Technology Policy; harness “citizen scientists”; and introduce meaningful, participatory two-way public engagement.
The range of these suggestions underscores the intensifying challenge for policymakers in the face of rapid scientific and technological change. It’s the complexity of this change, the issues raised, the challenges posed to society, and the entangled morals and values in all of it that creates a new imperative to integrate methods of risk analysis, legislative practices, science advisory processes, and emerging forms of stakeholder engagement in decision making.
This requires engaging the public in new ways by encouraging broad deliberation. One way of thinking about this is to focus on “opening up” the decision making process by allowing a plurality of voices and lines of argument to enter into the discussion. This is in contrast to “closing down” the process through use of the same groups of experts, types of analysis, or singular, prescriptive framings of the policy problem. This “opening up” could be achieved through a unique appraisal methodology called Multicriteria Mapping, which through its novel approach to discussing different options in contested areas of technology policy might be up to the task of opening up decision making and engagement methods in the way just described.
Developed by Professor Andrew Stirling and others at the University of Sussex, MCM was first used in a study of stakeholder perspectives on GM crop policy in Britain, and has since been used to look at policy options for issues such as xenotransplantation (essentially the transplantation of cells, tissue, or organs from a nonhuman animal source into a human), future energy scenarios, and a multi-country European Union study on obesity policy. The methodology is based on principles of deliberation, participatory public engagement, and the integration of both “quantitative” and “qualitative” approaches to policy appraisal.
This requires some further explanation. MCM is a novel interview technique that allows for an array of views and opinions on a particular policy or technological issue to be expressed, and their specific effect identified. Like other types of multicriteria analysis, MCM allows one to understand the importance of individual criteria—the issues, beliefs, and values deemed relevant—upon a particular policy choice, or “option” in the language of the method. What makes MCM distinct is that it is highly transparent and, though mediated through a structured process, is sensitive and accommodating to the unique views of individual participants.
This is particularly important when it comes to deliberating on issues that involve scientific advances and technological solutions that have unknown costs and uncertainties associated with them—not to mention moral implications. Using MCM, an individual can analyse different decision options or ways forward in these situations, while still acknowledging the various influences and subjective viewpoints that affect them. It does not focus on identifying a single “best” option, but rather allows one to explore the diversity of perspectives that shape the perception of each option itself.
As shown in the nearby box, an MCM exercise is a five-staged interview process. The detailed interview, often lasting up to two hours, is mediated through an open-source computer software program. First, the interviewer defines a set of core options regarding the chosen policy issue. These are carefully developed by the interviewer after a thorough review of relevant literature and discussion with key stakeholders. For example, core options in a simple MCM exercise about energy policy might be nuclear power, coal burning, and wind energy. The options are discussed at the beginning of the interview with the participant and, if they wish, the participant can define his or her own options to add to the analysis.
Secondly, the participant is asked to define a set of criteria he or she will use to evaluate the options. Criteria are the different judgements, assumptions, technical views, or personal beliefs that a participant holds about the policy issue. In our example of energy policy, criteria might include public health effects, contribution to global warming, electricity cost, and amenity.
Once defined, these criteria are used to assess scores and then explore uncertainty for each of the options. Taking each criterion in turn, participants evaluate the options based on how well they “perform” under individual criterion. A pessimistic and optimistic value is used to evaluate this performance, usually on a scale of 1 to 10 or 1 to 100. This allows for participants to reflect any uncertainty they feel exists. For example, if the criterion “amenity” were used to evaluate wind energy, then the participant might give it a score of 3 to 7—depending on whether the wind turbines were located within an environmental area, rating a score of 3, or in an urban setting, which rated a 7. By enabling this expression of uncertainty, we can see the particular sensitivities of each option and the criteria that cause them.
The final stage of an MCM interview is weighting the different criteria. Participants assign 100 different points across the criteria so that they are able to express their relative importance of each. They are able to see how these weightings affect the final option rankings, and so can experiment until they get a proportion that accurately represents their views. At the end of the interview the final map of the performance rankings of the options is discussed with the participant (see box below).
An MCM exercise ultimately enables one to characterize an array of policy options for the technological issue at hand, and then analyze the options’ strengths and weaknesses under a set of evaluative criteria. One is left with an array of qualitative and quantitative information that combines to form a picture of the conditions, sensitivities, and framings associated with each policy option alongside the perspective of the participant.
It is this feature that makes it most appealing when it comes to engaging the public, politicians, specialists, and policy makers on pressing issues of science policy. Though MCM requires a particular level of specialist expertise in a given area to be most effective, it can be used as part of wider public consultation exercises. A good example of this was seen in the Deliberative Mapping project, which integrated expert and citizen assessments about options for treating end-stage kidney failure.
Here, the MCM exercise was modified for use in citizen panels where the panelists did not possess a detailed knowledge of the scientific issues at hand. The panelists worked together in moderated groups and jointly reached agreement about the evaluative criteria used and weighted. There were several points at which citizens and specialists came together to share views and exchange ideas. In other cases, an MCM exercise has been modified so that individual citizens work with pre-determined criteria and explore different weighting combinations in order to achieve a policy option ranking that best reflects individual views.
The key to success, however, in any deliberation or participatory engagement exercise is, as Rick Borchelt and Kathy Hudson argue in their Science Progress column “Engaging the Scientific Community with the Public,” about “agreeing up front to accommodate public input politically, not just to listen and nod politely.” To do this, we must open up policy and decision-making processes to the variety and plurality of views that exist about complex issues of science and technology policy. We must avoid processes that simply go through the motions of soliciting different views or engaging the public in order merely to “tick the box.”
Part of this is finding the right tools that encourage transparency and openness, and MCM is just one suggestion. But in order to realize effective, meaningful, engaged and deliberative dialogue among the body politic, we must move towards developing these and other approaches. Only then will science policymaking truly be opened up.
Molly E. Morgan is a Ph.D. student in Science and Technology Policy Research at the University of Sussex in the United Kingdom. Her doctoral research is using MCM to compare stakeholder views on the governance of human embryonic stem cell research in the U.S. and the U.K.
 A. Stirling, “Opening Up and Closing Down: power, participation and pluralism in the social appraisal of technology,” Science Technology and Human Values, 33 (2) (March 2008): 262-294.
 A. Stirling “Multi-criteria mapping: mitigating the problems of environmental valuation,” Valuing Nature?: Ethics, Economics and the Environment, John Foster, ed., (London: Routledge, 1997).
 Stirling A, Mayer S, “Rethinking Risk: a pilot multi-criteria mapping of a genetically modified crop in agricultural systems in the UK,” Report for the UK Roundtable on Genetic Modification, SPRU, (University of Sussex, August 1999).
 Burgess J, Stirling A, Clark J, Davies G, Eames M, Mayer M, Staley K, Williamson S, “Deliberative Mapping: developing an analytic-deliberative methodology to support contested science-policy decisions,” Public Understanding of Science, 16 (3) (2007): 299-322.
 McDowall W, Eames M. “Towards a Sustainable Hydrogen Economy: A Multi-criteria Mapping of the UKSHEC Hydrogen Futures,” (London: Policy Studies Institute, 2006).
 Stirling A, Lobstein T, Millstone E, “Methodology for obtaining stakeholder assessments of obesity policy options in the PorGrow project,” Obesity Review, 8 (2) (December 2006): 7-27; and other articles in this special issue.
 Multi-Criteria Analysis Manual, DTLR (2001) DTLR, (London, Department for Transport, Land and the Regions), available at http://www.communities.gov.uk/documents/corporate/pdf/146868.pdf.
 For more detailed information, various resources are available at http://www.multicriteria-mapping.org/.
Comments on this article