Climate-Science Communication and the Measurement Problem
Corresponding Author
Dan M. Kahan
Yale University
Correspondence concerning this article should be addressed to Dan M. Kahan, Elizabeth K. Dollard Professor of Law and Professor of Psychology, Yale Law School, Yale University, 127 Wall St., New Haven, CT, 06511. E-mail: [email protected]Search for more papers by this authorCorresponding Author
Dan M. Kahan
Yale University
Correspondence concerning this article should be addressed to Dan M. Kahan, Elizabeth K. Dollard Professor of Law and Professor of Psychology, Yale Law School, Yale University, 127 Wall St., New Haven, CT, 06511. E-mail: [email protected]Search for more papers by this authorAbstract
This article examines the science-of-science-communication measurement problem. In its simplest form, the problem reflects the use of externally invalid measures of the dynamics that generate cultural conflict over risk and other policy-relevant facts. But at a more fundamental level, the science-of-science-communication measurement problem inheres in the phenomena being measured themselves. The “beliefs” individuals form about a societal risk such as climate change are not of a piece; rather they reflect the distinct clusters of inferences that individuals draw as they engage information for two distinct ends: to gain access to the collective knowledge furnished by science and to enjoy the sense of identity enabled by membership in a community defined by particular cultural commitments. The article shows how appropriately designed “science comprehension” tests—one general and one specific to climate change—can be used to measure individuals’ reasoning proficiency as collective-knowledge acquirers independently of their reasoning proficiency as cultural-identity protectors. Doing so reveals that there is in fact little disagreement among culturally diverse citizens on what science knows about climate change. The source of the climate-change controversy and like disputes over societal risks is the contamination of the science-communication environment with forms of cultural status competition that make it impossible for diverse citizens to express their reason as both collective-knowledge acquirers and cultural-identity protectors at the same time.
References
- Anderegg, W. R., Prall, J. W., Harold, J., & Schneider, S. H. (2010). Expert credibility in climate change. Proceedings of the National Academy of Sciences, 107, 12107–12109.
- Ansolabehere, S., & Rivers, D. (2013). Cooperative survey research. Annual Review of Political Science, 16(1), 307–329. doi:10.1146/annurev-polisci-022811-160625
- Baron, J. (1993). Why teach thinking? An essay. Applied Psychology, 42, 191–214.
- Baron, J. (2008). Thinking and deciding. New York, NY: Cambridge University Press.
- Berinsky, A. J., & Druckman, J. N. (2007). The polls—Review: Public opinion research and support for the Iraq War. Public Opinion Quarterly, 71, 126–141.
- Bishop, B. A., & Anderson, C. W. (1990). Student conceptions of natural selection and its role in evolution. Journal of Research in Science Teaching, 27, 415–427.
- Bishop, G. F. (2005). The illusion of public opinion: Fact and artifact in American public opinion polls. Lanham, MD: Rowman & Littlefield.
- Buckland, P. (2014). “Resolved: Climate change is not a ‘crisis’”: Using cultural cognition research in high school ecology class. CCP Blog. Retrieved from http://www.culturalcognition.net/blog/2014/6/11/resolved-climate-change-is-not-a-crisis-using-cultural-cogni.html
- California Academy of Sciences. (2014). Anytime lesson plan: Global climate change and sea level rise. Retrieved from http://www.calacademy.org/teachers/resources/lessons/global-climate-change-and-sea-level-rise/
- Cohen, G. L. (2003). Party over Policy: The Dominating Impact of Group Influence on Political Beliefs. Journal of Personality & Social Psychology, 85(5), 808–822.
- Cook, J., Nuccitelli, D., Green, S. A., Richardson, M., Winkler, B., Painting, R., Way, R., Jacobs, P., & Skuce, A. (2013). Quantifying the consensus on anthropogenic global warming in the scientific literature. Environmental Research Letters, 8(2), 1–7.
- Corner, A., Whitmarsh, L., & Xenias, D. (2012). Uncertainty, skepticism and attitudes towards climate change: biased assimilation and attitude polarization. Climatic Change, 114, 463–478.
-
DeMars, C. (2010). Item response theory. Oxford, UK: Oxford University Press.
10.1093/acprof:oso/9780195377033.001.0001 Google Scholar
- Demastes, S. S., Settlage, J., & Good, R. (1995). Students' conceptions of natural selection and its role in evolution: Cases of replication and comparison. Journal of Research in Science Teaching, 32, 535–550.
- DeVellis, R. F. (2012). Scale development: Theory and applications. Thousand Oaks, CA: Sage.
- Dewey, J. (1910). Science as subject-matter and as method. Science, 31, 121–127.
- Dohmen, T., et al. (2011). Individual risk attitudes: Measurement, determinants, and behavioral consequences. Journal of the European Economic Association, 9, 522–550.
-
Doran, P. T., &
Zimmerman, M. K. (2009). Examining the scientific consensus on climate change. Eos, Transactions American Geophysical Union, 90, 22–23.
10.1029/2009EO030002 Google Scholar
-
Embretson, S. E., &
Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
10.1037/10519-153 Google Scholar
- Feynman, R. P., Leighton, R. B., & Sands, M. L. (1963). The Feynman lectures on physics. Reading, MA: Addison-Wesley.
- Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42.
- Ganzach, Y., Ellis, S., Pazy, A., & Ricci-Siag, T. (2008). On the perception and operationalization of risk perception. Judgment and Decision Making, 3, 317–324.
- Gliem, J. A., & Gliem, R. R. (2003). Calculating, interpreting, and reporting Cronbach's alpha reliability coefficient for Likert-type scales. Presented at the Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education, The Ohio State University. Retrieved from https://scholarworks.iupui.edu/bitstream/handle/1805/344/Gliem+&+Gliem.pdf?sequence=1
- Goebbert, K., Jenkins-Smith, H. C., Klockow, K., Nowlin, M. C., & Silva, C. L. (2012). Weather, climate and worldviews: The sources and consequences of public perceptions of changes in local weather patterns. Weather, Climate, and Society, 4, 132–144.
- Gore, A. (2007). An inconvenient truth: The crisis of global warming. Rev. ed. New York, NY: Viking.
- Hamilton, L. C. (2011). Education, politics and opinions about climate change evidence for interaction effects. Climatic Change, 104, 231–242.
-
Hamilton, L. C.,
Cutler, M. J., &
Schaefer, A. (2012). Public knowledge and concern about polar-region warming. Polar Geography, 35, 155–168.
10.1080/1088937X.2012.684155 Google Scholar
- Haran, U., Ritov, I., & Mellers, B. A. (2013). The role of actively open-minded thinking in information acquisition, accuracy and calibration. Judgment and Decision Making, 8, 188–201.
- IPCC. (2013). Climate Change 2013: The physical science basis, working group in contribution to the fifth assessment report of the intergovernmental panel on climate change. Cambridge, UK: Cambridge University Press.
- Kahan, D. M. (2007). The cognitively illiberal state. Stanford Law Review, 60, 115–154.
- Kahan, D. (2011). Industrial strength risk perception measure. CCP Blog. December 31. Retrieved from http://www.culturalcognition.net/blog/2011/12/31/industrial-strength-risk-perception-measure.html
- Kahan, D. (2012). Why we are poles apart on climate change. Nature, 488, 255.
- Kahan, D. M. (2013a). A risky science communication environment for vaccines. Science, 342, 53–54.
- Kahan, D. M. (2013b). Ideology, motivated reasoning, and cognitive reflection. Judgment and Decision Making, 8, 407–424.
- Kahan, D. M. (2014a). “Ordinary science intelligence”: A science comprehension measure for use in the study of risk perception and science communication. Cultural Cognition Project Working Paper No. 112.
- Kahan, D. M. (2014b). Making climate-science communication evidence-based—all the way down. In M. Boykoff & D. Crow (Eds.), Culture, politics and climate change (pp. 203–220). New York, NY: Routledge Press.
-
Kahan, D. M.,
Braman, D.,
Gastil, J.,
Slovic, P., &
Mertz, C. K. (2007). Culture and identity-protective cognition: Explaining the White-male effect in risk perception. Journal of Empirical Legal Studies, 4, 465–505. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=995634
10.1111/j.1740-1461.2007.00097.x Google Scholar
- Kahan, D. M., Braman, D., Slovic, P., Gastil, J., & Cohen, G. (2009). Cultural cognition of the risks and benefits of nanotechnology. Nature Nanotechnology, 4, 87–91.
- Kahan, D. M., Jenkins-Smith, H., & Braman, D. (2011). Cultural cognition of scientific consensus. Journal of Risk Research, 14, 147–174.
- Kahan, D. M., Peters, E., Dawson, E., & Slovic, P. (2013). Motivated numeracy and enlightened self-government. Cultural Cognition Project Working Paper No. 116.
- Kahan, D. M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L. L., Braman, D., & Mandel, G. (2012). The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change, 2, 732–735.
- Kahneman, D. (2003). Maps of bounded rationality: Psychology for behavioral economics. American Economic Review, 93, 1449–1475.
- Keil, F. C. (2003). Folkscience: Coarse interpretations of a complex reality. Trends in Cognitive Sciences, 7(8), 368–373.
- Koch, W. (2013, November 13). Americans back greenhouse gas cuts from power plants. USA Today. Retrieved from http://www.usatoday.com/story/news/nation/2013/11/13/americans-greenhouse-gas-cuts-power-plants/3508001/
- Kohut, A. (2010, June 13). Views on climate change: What the polls show. New York Times, p. A22. Retrieved from http://www.nytimes.com/2010/06/14/opinion/l14climate.html?_r=0
- Krosnick, J. A., Malhotra, N., & Mittal, U. (2014). Public misunderstanding of political facts: How question wording affected estimates of partisan differences in birtherism. Public Opinion Quarterly, 78, 147–165.
- Lawson, A. E., & Worsnop, W. A. (2006). Learning about evolution and rejecting a belief in special creation: Effects of reflective reasoning skill, prior knowledge, prior belief and religious commitment. Journal of Research in Science Teaching, 29, 143–166.
- Lewandowsky, S., Gignac, G. E., & Vaughan, S. (2012). The pivotal role of perceived scientific consensus in acceptance of science. Nature Climate Change, 3, 399–404.
- Lichter, S. R. (2008). Climate scientists agree on warming, disagree on dangers, and don't trust the media's coverage of climate change. Statistical Assessment Service, George Mason University.
- Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making, 21, 37–44.
- Loewenstein, G. F., Weber, E. U., Hsee, C. K., & Welch, N. (2001). Risk as feelings. Psychological Bulletin, 127, 267–287.
- Marx, S. M., Weber, E. U., Orlove, B. S., Leiserowitz, A., Krantz, D. H., Roncoli, C., & Phillips, J. (2007). Communication and mental processes: Experiential and analytic processing of uncertain climate information. Global Environmental Change, 17, 47–58.
- McCright, A. M., & Dunlap, R. E. (2012). Bringing ideology in: The conservative white male effect on worry about environmental problems in the USA. Journal of Risk Research, 16(2), 1–16.
- McCright, A. M., Dunlap, R. E., & Xiao, C. (2014). The impacts of temperature anomalies and political orientation on perceived winter warming. Nature Climate Change, 4, 1077–1081.
- National Oceanic and Atmospheric Administration. (2009). Climate literacy: The essential principles of climate science. Retrieved from http://cpo.noaa.gov/OutreachandEducation/ClimateLiteracy.aspx
- National Oceanic and Atmospheric Administration. (2014). Frequently asked questions about the arctic. Retrieved from http://www.arctic.noaa.gov/faq.html
- National Science Foundation. (2006). Science and engineering indicators 2006. Washington, DC.
- National Science Foundation. (2014). Science and engineering indicators. Washington, DC.
- Oreskes, N. (2004). The scientific consensus on climate change. Science, 306, 1686.
- Osterlind, S. J. (1998). Constructing test items: Multiple-choice, constructed-response, performance, and other formats. Boston, MA: Kluwer Academic.
-
Osterlind, S. J., &
Everson, H. T. (2009). Differential item functioning. Thousand Oaks, CA: Sage.
10.4135/9781412993913 Google Scholar
- Peters, E., Västfjäll, D., Slovic, P., Mertz, C. K., Mazzocco, K., & Dickert, S. (2006). Numeracy and decision making. Psychological Science, 17, 407–413.
- Pew Research Center for the People & the Press. (2009). Public praises science; scientists fault public, media. Washington, DC: Pew Research Center.
- Pew Research Center for the People & the Press. (2013). Public's knowledge of science and technology. Washington, DC: Pew Research Center.
-
Popper, K. R. (1959). The logic of scientific discovery. New York, NY: Basic Books.
10.1063/1.3060577 Google Scholar
- Powell, J. (2012). Why climate deniers have no scientific credibility—in one pie chart. Retrieved from www.desmogblog.com
- Reynolds, T. W., Bostrom, A., Read, D., & Morgan, M. G. (2010). Now what do people know about global climate change? Survey studies of educated laypeople. Risk Analysis, 30(10), 1520–1538. doi:10.1111/j.1539-6924.2010.01448.x
- Roos, J. M. (2012). Measuring science or religion? A measurement analysis of the National Science Foundation sponsored science literacy scale 2006–2010. Public Understanding of Science.
- Saad, L. (2014, March 18). A steady 57% in U.S. blame humans for global warming. Retrieved from http://www.gallup.com/poll/167972/steady-blame-humans-global-warming.aspx
- Schuman, H. (1998). Interpreting the poll results better. Public Perspective, 1, 87–88.
- Sherman, D. K., & Cohen, G. L. (2006). The psychology of self-defense: Self-affirmation theory. Advances in Experimental Social Psychology (Vol. 38, pp. 183–242). New York, NY: Academic Press.
- Shtulman, A. (2006). Qualitative differences between naïve and scientific theories of evolution. Cognitive Psychology, 52(2), 170–194.
- Silver, D. (2013, June 6). Most Americans say global warming is real, but opinions split on why. Pew Research Center. Retrieved from http://www.pewresearch.org/fact-tank/2013/06/06/most-americans-say-global-warming-is-real-but-opinions-split-on-why/
- Slovic, P., Finucane, M. L., Peters, E., & MacGregor, D. G. (2004). Risk as analysis and risk as feelings: Some thoughts about affect, reason, risk, and rationality. Risk Analysis, 24, 311–322.
- Stanovich, K. E. (2009). What intelligence tests miss: The psychology of rational thought. New Haven, CT: Yale University Press.
- Stanovich, K. E. (2011). Rationality and the reflective mind. New York, NY: Oxford University Press.
- Strong, K. (2010). Polar puzzle: Will ice melting at the North or South Poles cause sea levels to rise? Science Buddies. Retrieved from http://www.sciencebuddies.org/science-fair-projects/project_ideas/OceanSci_p015.shtml
- Tobler, C., Visschers, V. H. M., & Siegrist, M. (2012). Addressing climate change: Determinants of consumers' willingness to act and to support policy measures. Journal of Environmental Psychology, 32, 197–207.
- University of Queensland. (2013). Study shows scientists agree humans cause global-warming. Retrieved from http://www.uq.edu.au/news/article/2013/05/study-shows-scientists-agree-humans-cause-global-warming
- Watson, W. E., Kumar, K., & Michelson, L. K. (1993). Cultural diversity's impact on interaction process and performance: Comparing homogeneous and diverse task groups. Academy of Management Journal, 36, 590–602.
- Watts, D. J. (2011). Everything is obvious: How common sense fails us. New York, NY: Atlantic Books.
- Weber, E. (2006). Experience-based and description-based perceptions of long-term risk: Why global warming does not scare us (yet). Climatic Change, 77, 103–120.
- Weber, E. U., Blais, A.-R., & Betz, N. E. (2002). A domain-specific risk-attitude scale: Measuring risk perceptions and risk behaviors. Journal of Behavioral Decision Making, 15, 263–290
- Weber, E. U., & Stern, P. C. (2011). Public understanding of climate change in the United States. American Psychologist, 66, 315–328. doi:10.1037/a0023253
- Weller, J., Dieckmann, N. F., Tusler, M., Mertz, C. K., Burns, W., & Peters, E. (2013). Development and testing of an abbreviated numeracy scale: A Rasch analysis approach. Journal of Behavioral Decision Making, 26, 198–212.
- Yetton, P. W., & Bottger, P. C. (1982). Individual versus group problem solving: An empirical test of a best-member strategy. Organizational Behavior and Human Performance, 29(3), 307–321. doi:10.1016/0030-5073(82)90248-3
-
Zaller, J. R. (1992). The nature and origins of mass opinion. Cambridge, UK: Cambridge University Press.
10.1017/CBO9780511818691 Google Scholar




