Differentiating theory from evidence in determining confidence in an assessment finding

The Guidance Notes for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties describes a process for consistently evaluating and communicating levels of certainty in findings. The process begins with an assessment of the scientific evidence and agreement supporting a finding, where evidence is defined as including mechanistic understanding, theory, data, models, and expert judgment. The appropriateness of categorizing theory as one line of evidence varies by scientific discipline; for the natural and social sciences, developing theory and collecting data are different steps in the scientific method. Further, decision-makers often find it valuable for scientists to differentiate situations where a theory is generally agreed but for which supporting data are limited, from situations where empirical data lack an explanatory theory. The paper describes the approach used by the International Agency for Research on Cancer (IARC) for assessing the relative robustness of a theory separately from the strength and quality of its supporting evidence, and then developing consensus statements of whether an agent is a human carcinogenic. Although the IARC and IPCC processes are very similar, the IARC process also differs by combining theory, evidence, and agreement as equal partners in a limited set of standardized categories of confidence. Incorporating aspects of the IARC approach into the IPCC guidance could improve the evaluation and communication of theory, evidence, and agreement in future versions of the uncertainty guidance.