Articles | Volume 9, issue 10
Biogeosciences, 9, 3857–3874, 2012
Biogeosciences, 9, 3857–3874, 2012

Reviews and syntheses 09 Oct 2012

Reviews and syntheses | 09 Oct 2012

A framework for benchmarking land models

Y. Q. Luo1, J. T. Randerson2, G. Abramowitz3, C. Bacour4, E. Blyth5, N. Carvalhais6,7, P. Ciais8, D. Dalmonech6, J. B. Fisher9, R. Fisher10, P. Friedlingstein11, K. Hibbard12, F. Hoffman13, D. Huntzinger14, C. D. Jones15, C. Koven16, D. Lawrence10, D. J. Li1, M. Mahecha6, S. L. Niu1, R. Norby13, S. L. Piao17, X. Qi1, P. Peylin8, I. C. Prentice18, W. Riley16, M. Reichstein6, C. Schwalm14, Y. P. Wang19, J. Y. Xia1, S. Zaehle6, and X. H. Zhou20 Y. Q. Luo et al.
  • 1Department of Microbiology and Plant Biology, University of Oklahoma, Norman, OK 73019, USA
  • 2Department of Earth System Science, University of California, Irvine, CA 92697, USA
  • 3Climate Change Research Centre, University of New South Wales, Sydney, Australia
  • 4Laboratory of Climate Sciences and the Environment, Joint Unit of CEA-CNRS, Gif-sur-Yvette, France
  • 5Centre for Ecology and Hydrology, Wallingford, Oxfordshire, OX10 8BB, UK
  • 6Max-Planck-Institute for Biogeochemistry, Jena, Germany
  • 7Departamento de Ciências e Engenharia do Ambiente, DCEA, Universidade Nova de Lisboa, 2829-516 Caparica, Portugal
  • 8Laboratoire des Sciences du Climat et de l'Environnement, CEA-CNRS-UVSQ, CE Orme des Merisiers, 91191 Gif sur Yvette, France
  • 9Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109 USA
  • 10National Centre for Atmospheric Research, Boulder, CO, USA
  • 11College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter EX4 4QF, UK
  • 12Pacific Northwest National Laboratory, Richland, WA, USA
  • 13Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA
  • 14School of Earth Science and Environmental Sustainability, Northern Arizona University, Flagstaff, AZ 86011, USA
  • 15Met Office Hadley Centre, FitzRoy Road, Exeter, EX1 3PB, UK
  • 16Earth Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA
  • 17Department of Ecology, Peking University, Beijing 100871, China
  • 18Department of Biological Sciences, Macquarie University, NSW 2109 Sydney, Australia
  • 19CSIRO Marine and Atmospheric Research PMB #1and Centre for Australian Weather and Climate Research, Aspendale, Victoria 3195, Australia
  • 20Research Institute of the Changing Global Environment, Fudan University, 220 Handan Road, Shanghai 200433, China

Abstract. Land models, which have been developed by the modeling community in the past few decades to predict future states of ecosystems and climate, have to be critically evaluated for their performance skills of simulating ecosystem responses and feedback to climate change. Benchmarking is an emerging procedure to measure performance of models against a set of defined standards. This paper proposes a benchmarking framework for evaluation of land model performances and, meanwhile, highlights major challenges at this infant stage of benchmark analysis. The framework includes (1) targeted aspects of model performance to be evaluated, (2) a set of benchmarks as defined references to test model performance, (3) metrics to measure and compare performance skills among models so as to identify model strengths and deficiencies, and (4) model improvement. Land models are required to simulate exchange of water, energy, carbon and sometimes other trace gases between the atmosphere and land surface, and should be evaluated for their simulations of biophysical processes, biogeochemical cycles, and vegetation dynamics in response to climate change across broad temporal and spatial scales. Thus, one major challenge is to select and define a limited number of benchmarks to effectively evaluate land model performance. The second challenge is to develop metrics of measuring mismatches between models and benchmarks. The metrics may include (1) a priori thresholds of acceptable model performance and (2) a scoring system to combine data–model mismatches for various processes at different temporal and spatial scales. The benchmark analyses should identify clues of weak model performance to guide future development, thus enabling improved predictions of future states of ecosystems and climate. The near-future research effort should be on development of a set of widely acceptable benchmarks that can be used to objectively, effectively, and reliably evaluate fundamental properties of land models to improve their prediction performance skills.

Final-revised paper