The Data Assimilation Linked Ecosystem Carbon (DALEC) model suite includes 16 related intermediate-complexity models of the terrestrial carbon cycle. Each model variant tracks the state and dynamics of both live and dead carbon pools, their interactions, and responses to meteorology and disturbance such as fire or biomass removals. From an initial DALEC model (Williams et al., 2005), we produced alternate structures that either aimed to reduce complexity by focusing on core variables/processes and removing others or aimed to increase complexity by including hypothesized missing carbon pools or improving on over-simplified processes.

Accordingly, the DALEC suite spans a range of model structures (i.e., number of carbon pools, carbon pool connectivity) and process representations (component sub-models of varying complexity) related to different simulations of photosynthesis, plant respiration, decomposition, and water cycle feedbacks. These representations are listed in Table 1 and described in further detail in Appendix A. To facilitate disentanglement of the impacts of specific alternate process representations, the different sub-models can be related to a common baseline structure of the carbon cycle (Fig. 2a). Specific variants of this general structure for the least and most detailed models in this analysis are presented in Fig. 2b–c, while additional diagrams for the remaining models are shown in Appendix B (Figs. B1–B7). Across models, carbon enters the system via gross primary productivity (GPP), which is allocated to autotrophic respiration (Ra) and non-canopy live tissues based on fixed fractions. Canopy growth and mortality is determined by a phenology sub-model which is sensitive to day of year (sub-model scheme CDEA), environmental factors (GSI), or a combination of environmental factors and estimated net canopy carbon export (NCCE). Mortality of wood and fine roots follows continuous turnover based on first-order kinetics. Decomposition of dead organic matter and associated heterotrophic respiration (Rh) follows first-order kinetics with an exponential temperature sensitivity (and, in models C2–C5, a linear soil moisture sensitivity).

COMPLEX uses information from six globally distributed eddy covariance sites participating in FLUXNET (Pastorello et al., 2020) (Table 2). Our site selection procedure aimed to maximize biogeographical spread and diversity of natural ecosystems while fulfilling specific data requirements. These constraints collectively yielded a series of site selection criteria that are described in detail in Appendix C. As an example, the sites must not be dominated by the C4 photosynthetic pathway, nor arable agriculture nor intensively grazed grassland. Additionally, we required that the range of time series observations to be used for model calibration and validation spanned at least a decade. Data collated at each site are described below (see Sect. 2.3).

We used the CARDAMOM model–data fusion system (Bloom and Williams, 2015; Bloom et al., 2016) to parameterize the DALEC model suite with available observations of the carbon cycle. Specifically, we employed Bayesian inference to retrieve time-invariant, site-specific, optimized parameters and initial conditions for a given DALEC model (y) as informed by observations (O), where p(y|O)∝p(y)⋅p(O|y). Here, p(y|O) is the posterior parameter probability distribution, p(y) is the prior parameter probability distribution, and p(O|y) is proportional to the likelihood of parameters y given observations O.

For each model, p(y) is derived as the product of (i) the prior probability density functions for each model parameter and (ii) ecological and dynamical constraints (EDCs, i.e., functional constraints). EDCs are simple mathematical functions that impose conditions on inter-relationships between model parameters based on known ecological theory. They are used to inform parameter prior information with broader ecological knowledge and tend to reduce bias and equifinality (Bloom and Williams, 2015). One example of an EDC in CARDAMOM is the imposed constraint that litter turnover times are faster than soil organic matter turnover times (e.g., Gaudinski et al., 2000). In this analysis, each model includes some or all of the EDCs documented in Bloom et al. (2016).

The likelihood pO|y is derived as a function of the mismatch between observations O and the model realization M corresponding to y, such that pO|y∝exp⁡-12∑n=1NOn-Mnσn2, where σn is the error for the nth observation. This formulation requires no assumptions on the normality of prior or posterior parameter distributions and is robust to missing data. In our analysis, monthly-averaged eddy covariance NEE measurements from FLUXNET, monthly-averaged leaf area index (LAI) estimates from the Copernicus Global Land Service (Verger et al., 2014; Fuster et al., 2020), and in situ wood stock surveys were made available for ingestion into the model (see Appendix C). NEE uncertainty was assumed to be 0.58 gC m-2 d-1 based on estimates of random errors in eddy covariance measurements from Hill et al. (2012). A time-varying uncertainty estimate was included with the Copernicus LAI product, and site-specific, locally derived biomass uncertainties were provided by the site PI or drawn from relevant publications when necessary. Model drivers included monthly average site meteorology (air temperature, shortwave radiation, atmospheric CO2 concentration, vapor pressure deficit, precipitation, and wind speed). Here models were run at the monthly time step.

To sample the distribution p(y|O) (namely the product of pO|y and py), we used an adaptive proposal Metropolis–Hastings Markov chain Monte Carlo (MCMC) approach (Haario et al., 2001). We performed 108 iterations for each of four chains, which were checked for convergence using the Gelman–Rubin criterion (<1.2). A subset of 100 samples of y was selected from the latter half of each chain for our analysis. For additional details on the implementation of this algorithm within CARDAMOM, see Bloom and Williams (2015).

We performed a factorial experiment such that each of the 16 structurally distinct carbon cycle models was run within CARDAMOM under all possible combinations of sites, observational and functional constraints, and assumptions on data uncertainties. These scenarios represented differing degrees of certainty with which parameter distributions were determined. Specifically, we considered (a) six sites; (b) six options for assimilated data, including one for which no data were ingested into the model; (c) four options for the magnitude of error assumed on the assimilated datasets (represented by scalar multipliers on the prescribed nominal uncertainties); and (d) two options for EDC state (either present or absent) (Table 3). In total, this factorial approach yielded 4032 unique model runs (16 models × 6 sites × 21 data scenarios × 2 EDC states). Using a high number of factorial model runs both added robustness to our interpretation and allowed for consideration of each factor's influence across a range of background conditions.

Figure 3 shows examples of three model analyses at the FR-LBr site, highlighting the range in NEE prediction performance across different model structures and data scenarios. Each model run contains a calibration period (the first 5 years of the site record; shown in white) during which optimized parameters were derived and a forecast period (the remaining years of the record, which always spanned at least 5 years because no site contained fewer than 10 years of data; shown in gray) during which fluxes and pools were predicted with the optimally parameterized model. In the scenario presented, model S2 is highly constrained by multiple datasets (Fig. 3a). By contrast, model C2 is moderately constrained (Fig. 3b), and model G4 is poorly constrained (Fig. 3c), which is evident by comparing the relative uncertainty of the NEE forecasts (blue shading) for each model. To highlight the effectiveness of the assimilation system, corresponding time series based only on prior parameter distributions are presented in Fig. S1.

Accounting for prediction uncertainty – as well as data uncertainty (red shading) – is a key goal of our model skill evaluation approach. Forecast skill for each model run was computed by comparing predictions and observations drawn strictly from the forecast period, using the histogram intersection algorithm (see Sect. 2.5.1). The complexity of each run was quantified based on its effective complexity (see Sect. 2.5.2).

Effective complexity – defined as the number of principal components for which 95 % of the variance in the posterior parameter space is explained (see Sect. 2.5.2) – is primarily determined by model structure (Fig. 4a, inset). Specifically, over all runs included in the experiment, effective complexity varies far more between different models than between the other tested factors (assimilated data, observational error scalar, site, and EDC presence or absence). This link to model structure provides insight into the metric's interpretability and justifies its use as a measure of model complexity.

While predominantly determined by the choice of model, effective complexity also varies according to the degree to which parameters are constrained (Fig. 4a). It therefore captures the inter-relationship between model structure and parameterization. Within a given model structure, each of the experimentally varied factors yields a range of distinct complexities that follows a predictable pattern: effective complexity is higher for runs with weaker constraints on parameters than it is for runs with stronger constraints on parameters. This is easily interpretable in the case of assimilated data, which is the dominant within-model control on effective complexity (Fig. 4b). Runs for which no observations are ingested into the model have consistently higher effective complexities than runs for which NEE, LAI, and biomass observations are all ingested (compare yellow and purple circles in Fig. 4b), since the observational constraints reduce the possible model solution space. Similar behavior is also observed across the different error scalars tested in the experiment (larger observational error assumptions correspond to higher effective complexities (Fig. S2)) and between the presence versus absence of EDCs (the absence of non-observational realism constraints yields higher effective complexities (Fig. S3)). Conceptually, this pattern can be understood in the following way. Parameters in a given model's high-complexity runs were sampled from wider posterior distributions (due to weak or absent constraints) than in its low-complexity runs. This implies greater variance between parameter sets selected in high-complexity runs – and thus more distinct dimensions of variability in the posterior parameter space – than in low-complexity runs for the same model.

Across all runs performed in the experiment, the hypothesis that an intermediate-complexity carbon cycle model can outperform a low- or high-complexity model is confirmed, both when NEE is predicted (Fig. 5a) and when LAI is predicted (Fig. S4a). Runs on both extremes of the complexity axis perform poorly, due to overfitting in the low-complexity case (parameters are over-determined, leading to accurate predictions in the training period but poor ones in forecast) and underfitting in the high-complexity case (parameters are under-determined, yielding poor predictions in both training and forecast). Figure 3a and 3c demonstrate this contrasting behavior at the FR-LBr site.

When runs for which no data were assimilated – that is, runs with the least informed parameters – are withheld from the analysis, increasing complexity no longer degrades skill (Fig. 5b). More specifically, the relationship between effective complexity and skill increases monotonically when all runs have some baseline constraint on parameters. This result also holds regardless of which variable is predicted (Fig. S4b) as well as when the number of runs within each complexity bin is standardized via bootstrapping (Fig. S5). The decline in performance attributable to the most extreme effective complexity scenarios is also preserved across RMSE and R2 metrics (not shown; further comparison between different metrics is beyond the scope of this paper). This finding implies that increasing complexity by introducing suitable data-constrained parameters can improve performance, but that doing so by adding unconstrained dimensions can degrade it. That is, the processes and parameters introduced in the most detailed models (such as G1–G4) can lead to improvements in predictive skill over simpler models only when they are sufficiently well-characterized (i.e., adequately informed by data). Importantly, larger observational uncertainty assumptions reduce the effectiveness of assimilated data at constraining parameters in high-complexity models. The monotonically increasing relationship between complexity and skill is strongest when observational error is assumed to be relatively small (Fig. S6).

Assimilated data determine the shape of the overall complexity–skill relationship in COMPLEX. Not only does the presence of any assimilated observations control the response of skill to increasing complexity, but the specific choice of assimilated observations also matters. In particular, assimilating monthly NEE observations improves both NEE (Fig. 6a–c) and LAI predictions (Fig. S7a–c) by complex models over simple models: note the positive/increasing trends between complexity and skill in these cases. However, such improvements in predictive performance are not consistently observed across the complexity axis when other data, but not NEE, are ingested. The ingestion of LAI data and biomass estimates yields a small positive trend (Fig. 6d) – although this relationship is clearly weaker than when NEE is also assimilated (Fig. 6a) – and simple models informed only by LAI perform just as well as complex models when predicting NEE. Indeed, these runs show a constant skill level across the complexity axis (Fig. 6e). When predicting LAI, though, complex models outperform simple models with only the assimilation of LAI (Fig. S7e). All such combinations contrast with the case in which no data are assimilated: forecast skill for those runs declines with complexity, regardless of target variable (Figs. 6f, S7f).

Recall that the magnitude of skill – the degree of overlap between model predictions and observations (see Sect. 2.5.1) – reflects the ability of the model to capture the data along with its uncertainty. Particularly in scenarios corresponding to low effective complexities, models tend to overfit when NEE is assimilated (as demonstrated in Fig. 3a). Overfitting is a key factor causing the discrepancy in performance between low-complexity runs that do (e.g., Fig. 6c) and do not assimilate NEE (e.g., Fig. 6e).

Regardless of which data are assimilated, site-specific characteristics also introduce additional variability into the form of the relationship between effective complexity and skill (Fig. 7). To better understand and isolate site-specific dynamics, here we only interpret runs for which at least one data type is assimilated. Most sites show high-complexity performance optima, consistent with Fig. 5b. However, several are characterized by a threshold effect for which performance increases significantly once a certain effective complexity is attained and remains stagnant thereafter (e.g., a low-complexity threshold around 10 for FI-Hyy and FR-Pue). This “diminishing returns” effect suggests that the performance benefit of added structural detail has the potential to stabilize for all but the simplest models. The two tropical sites included in our analysis demonstrate additional unique dynamics. GF-Guy is the only site for which the performance of the most complex models appears to slightly degrade, even when all observations including NEE are assimilated, and no threshold is apparent at AU-How. Overall, the site analysis demonstrates the large variability in model performance across space, including between sites sharing biome classifications (e.g., FI-Hyy and FR-LBr) or broadly similar climate types (e.g., GF-Guy and AU-How).

We defined a concept of effective complexity that is linked to model structure and number of parameters as well as to the information content of calibration data (Fig. 4). This metric can inform future studies seeking to investigate the role of model complexity by providing a simple and comparable quantification of parameter posteriors. Conventional complexity measures (e.g., counts of observable model attributes) can serve as reasonable approximations of the more nuanced definition specific to ensemble methods that we present here. Still, effective complexity is rarely identical to the number of model parameters: it is generally lower. Correlations between model parameters can and do occur whether the model is poorly or well-constrained (Keenan et al., 2013) and whether it is simple or complex, implying that all carbon cycle models have “constrainable” dimensions. Importantly, though, none of the high-parameter models in our experiment have so much redundancy that their average effective complexity across runs is equivalent to that of any low-parameter model (Fig. 4). Whether this is also true for large-scale TBMs remains an open question.

Overall, the behavior of the effective complexity metric highlights that the best-performing analyses (i.e., runs with the highest forecast skill) in the COMPLEX maximize model structural breadth and minimize parametric uncertainty. Models built with high numbers of processes but without effective parameter constraints (i.e., runs that maximize structural breadth but do not attempt to minimize parametric uncertainty) are not sufficient to optimize performance (Fig. 5). Additionally, models of the carbon cycle can overfit if they are calibrated in too narrow a subset of conditions and underfit if they are improperly parameterized and therefore biased, as shown in Fig. 3.

The main factors controlling the observed complexity–skill relationship are (a) whether, and which, data are assimilated into the model and (b) the geographical location at which the analysis is undertaken. One way to interpret the role of data in the relationship is explicit: models with the ability to assimilate monthly observations of NEE, which uniquely represent the integrated behavior of terrestrial carbon cycling and its internal dynamics, are more likely to experience gains in skill with increased complexity than those that cannot. This result is consistent with the prominent role of NEE observations in reducing model projection uncertainty identified by Keenan et al. (2013). The effects of LAI and biomass observations in COMPLEX are somewhat more nuanced. All models in the DALEC suite are able to extract information from the LAI data and produce reasonably skilled NEE predictions (Fig. 6e), though such data do not improve the skill of complex models over simple ones. The ingestion of LAI data most directly constrains specific features relating to growth or carbon allocation, potentially informing the seasonality of NEE. Finally, the impacts of biomass observations on forecast skill were relatively muted in our experiments. Given that biomass data are particularly useful for informing the carbon cycling of slow pools (Williams et al., 2005), the relatively short calibration (5 years) and forecast periods (≥5 years) tested here, along with the temporal sparsity of these data in COMPLEX (i.e., a few measurements per site instead of continuous time series for LAI or NEE), may have obscured their utility.

Several recent TBM efforts have sought to enable the assimilation of eddy covariance or remote sensing observations (e.g., Bacour et al., 2015; Raoult et al., 2016; Schürmann et al., 2016; Peylin et al., 2016; MacBean et al., 2018; Norton et al., 2019) as well as measurements of functional traits (e.g., LeBauer et al., 2013). Our results underscore the value of such efforts to reduce parameter uncertainty, despite the fact that the computational costs associated with data assimilation are relatively high (e.g., MacBean et al., 2016). Increased use of emulators may help reduce this computational cost (Fer et al., 2018).

Given the demonstrated value of data constraints and the specification of their uncertainty (Fig. S6), the need to characterize and quantify this uncertainty (Keenan et al., 2011) remains particularly critical for model–data fusion studies. In this analysis, NEE uncertainty was assumed to remain constant both in time (i.e., for all observations regardless of season or year) and in space (i.e., across sites), which likely over-generalizes the specifications of individual sensors and the possibility of systematic or increasing biases. These assumptions become even more important to account for when assimilating global datasets, for which retrieval accuracy can vary across land cover types or with atmospheric conditions such as clouds or snow (e.g., Fang et al., 2013). One benefit of the Copernicus LAI product used here is its explicit, spatially variable quantification of uncertainty, which is still relatively rare for remote sensing datasets. Though the robustness of these uncertainties has been challenged with independent observations in some locations (e.g., Zhao et al., 2020), this approach represents a level of detail well-suited to the coupling of data to large-scale or global models.

The observed variability in the complexity–skill relationship across sites (Fig. 7) suggests that predictability itself is spatially heterogeneous. Further, it implies that the benefit to model performance accrued by the addition of a given process should not be expected to affect all locations uniformly, even when site-specific parameter uncertainty is minimized through calibration or optimization. Models not tuned locally likely smooth this spatial variability in predictability drastically (van Bodegom et al., 2012; Berzaghi et al., 2020), and thus model development and calibration must include locations spanning a wide range of vegetation, climate, soil characteristics, and disturbance histories.

Overall, our results suggest that the benefits of increased model complexity (e.g., gains in skill attributable to the introduction of specific processes or to additional detail applied to existing mechanisms) are attainable only when parameters are sufficiently well characterized. Here, this benefit is achieved when high complexity is balanced by data-assisted parameter optimization (in particular, when NEE observations are assimilated). More broadly, the relationship between complexity and skill is dynamic and extends beyond model structural choices. As a result, it is difficult to quantify whether model parameters corresponding to any specific model implementation – including outside the DALEC suite – are adequately informed such that increased model complexity is beneficial to performance. To assist in this endeavor, we present the following recommendations for model development and evaluation.

Assimilate well-characterized, repeat-observation datasets to constrain model parameters at the scale of model application.

Finally, while beyond the scope of this study, future work will investigate the linkage between specific processes or process representations (e.g., the inclusion or exclusion of water cycling) and predictive performance to better parse ecological controls on the complexity–skill relationship.

This analysis tested a spectrum of structurally distinct representations of the carbon cycle based on the intermediate-complexity ecosystem model DALEC, which allowed for coupling with the CARDAMOM model–data fusion system in a computationally tractable manner. Because our findings are not explicitly linked to the roles of specific processes or model features, however, their implications extend beyond the use of DALEC-like models to a wide variety of ecological models, including TBMs.

Traditional (based on plant functional type, or PFT) parameter determination in TBMs is far from random. It is informed by data – for example, by hypotheses or generalizations derived from prior literature (e.g., Oleson et al., 2010; Lawrence et al., 2011) or by model calibration at specific locations (e.g., Williams et al., 1997) – and therefore endowed with ecological knowledge. Accordingly, TBM parameters are likely more informed than the least constrained parameters retrieved in our analysis, which were freely sampled from wide uniform distributions and caused the high-complexity decline in performance (Fig. 5). However, while this may be true locally, the common assumption on uniformity of parameters within PFTs casts doubt on their precision across the regional or global scales at which TBMs typically make predictions (van Bodegom et al., 2012). Indeed, using a suite of global TBMs participating in the Multi-scale Synthesis and Terrestrial Model Intercomparison Project (MsTMIP; Huntzinger et al., 2013), Schwalm et al. (2019) showed that increases in model performance were more often linked to the omission rather than inclusion of various processes, suggesting a tradeoff between complexity and skill similar to that observed here. This conclusion calls into question the conventional paradigm that greater complexity significantly and consistently improves skill across current TBMs.

Earth observation (EO) is one key approach that can provide the high-spatial- and high-temporal-resolution data on carbon cycling needed for more localized calibrations (Exbrayat et al., 2019). In COMPLEX, we used Copernicus LAI data, though there are also opportunities to ingest biomass maps from space lidar or radar, estimates of photosynthesis from solar-induced fluorescence (SIF), and satellite-based atmospheric inversions of regional NEE, among others, in future studies. If supplied with appropriate error estimates, these datasets can over time provide powerful constraints for high-resolution carbon cycle analyses with TBMs or DALEC-like models. A key research goal is to determine the appropriate model complexity for maximizing the information content of these EO data for robust forecasts and analyses.

Alternative methodologies for deriving ecosystem parameters outside the realm of PFTs are also becoming increasingly common (van Bodegom et al., 2012; Bloom et al., 2016; Exbrayat et al., 2018; Berzaghi et al., 2020; Fisher and Koven, 2020) and may represent a way forward in addressing the tradeoff between structural and parametric uncertainty. Recent work has focused on upscaling in situ trait data (e.g., from the TRY database; Kattge et al., 2020) to yield spatially variable maps of key ecosystem parameters, using modeled relationships with climate or canopy properties (often referred to as environmental filtering relationships, since the environment “filters” the possible distribution of parameters at a given location; e.g., Verheijen et al., 2013; van Bodegom et al., 2014; Butler et al., 2017), leaf economics (Sakschewski et al., 2015), or optimality theory (e.g., Smith et al., 2019). Other studies have investigated how TBM parameters optimized at eddy covariance sites covary with climate (e.g., Peaucelle et al., 2019; Wu et al., 2020b). These efforts are not without their challenges, however. The spatial coverage of in situ trait data as well as eddy covariance sites is sparse relative to the large diversity of ecosystem behavior (Schimel et al., 2015), and such datasets also comprise a non-representative sample of species and disturbance histories (Sandel et al., 2015). These biases may limit the representativeness of the modeled relationships. Taking a different approach, a small subset of models has also been developed to operate altogether independently from the paradigm of PFTs (e.g., using trait-based approaches; Scheiter et al., 2013; Pavlick et al., 2013; Fyllas et al., 2014). Our results imply that these and future developments to improve the flexibility of model parameters will play critical roles in enabling the trend of increasing model complexity and may be a more fruitful avenue towards reducing the uncertainty of TBM prediction than model structural changes and additions.