the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Variability and Uncertainty in Flux-Site Scale Net Ecosystem Exchange Simulations Based on Machine Learning and Remote Sensing: A Systematic Evaluation
- 1State Key Laboratory of Desert and Oasis Ecology, Xinjiang Institute of Ecology and Geography, Chinese Academy of Sciences, Urumqi, Xinjiang, 830011, China
- 2University of Chinese Academy of Sciences, 19 (A) Yuquan Road, Beijing, 100049, China
- 3Research Centre for Ecology and Environment of Central Asia, Chinese Academy of Sciences, Urumqi, China
- 4Department of Geography, Ghent University, Ghent 9000, Belgium
- 5Sino-Belgian Joint Laboratory of Geo-Information, Ghent, Belgium and Urumqi, China
- 6Department of Computer Vision & Remote Sensing, Technische Universität Berlin, 10587 Berlin, Germany
- 1State Key Laboratory of Desert and Oasis Ecology, Xinjiang Institute of Ecology and Geography, Chinese Academy of Sciences, Urumqi, Xinjiang, 830011, China
- 2University of Chinese Academy of Sciences, 19 (A) Yuquan Road, Beijing, 100049, China
- 3Research Centre for Ecology and Environment of Central Asia, Chinese Academy of Sciences, Urumqi, China
- 4Department of Geography, Ghent University, Ghent 9000, Belgium
- 5Sino-Belgian Joint Laboratory of Geo-Information, Ghent, Belgium and Urumqi, China
- 6Department of Computer Vision & Remote Sensing, Technische Universität Berlin, 10587 Berlin, Germany
Abstract. Net ecosystem exchange (NEE) is an important indicator of carbon cycling in terrestrial ecosystems. Many previous studies have combined flux observations, meteorological, biophysical, and ancillary predictors using machine learning to simulate the site-scale NEE. However, systematic evaluation of the performance of such models is limited. Therefore, we performed a meta-analysis of these NEE simulations. Total 40 such studies and 178 model records were included. The impacts of various features throughout the modeling process on the accuracy of the model were evaluated. Random Forests and Support Vector Machines performed better than other algorithms. Models with larger time scales have lower average R-squared, especially when the time scale exceeds the monthly scale. Half-hourly models (average R-squared = 0.73) were significantly more accurate than daily models (average R-squared = 0.5). There are significant differences in the predictors used and their impacts on model accuracy for different plant functional types (PFT). Studies at continental and global scales (average R-squared = 0.37) with multiple PFTs, more sites, and a large span of years correspond to lower R-squared than studies at local (average R-squared = 0.69) and regional scales (average R-squared = 0.7). Also, the site-scale NEE predictions need more focus on the internal heterogeneity of the NEE dataset and the matching of the training set and validation set. The results of this study may also be applicable to the prediction of other carbon fluxes such as methane.
- Preprint
(2942 KB) -
Supplement
(273 KB) - BibTeX
- EndNote
Haiyang Shi et al.
Status: final response (author comments only)
-
RC1: 'Comment on bg-2022-46', Anonymous Referee #1, 06 Apr 2022
In this manuscript, the impacts of features such as the machine learning algorithm, the temporal scales of the observed flux data, and the PFT of the flux sites on the accuracy of the model were evaluated by the authors. The results of this study can provide some general guidance for the selection of feature factors during future NEE simulations. This manuscript logic is clear and well arranged, in terms of criteria for article selection, the choice of analysis methods, and the uncertainties in this analysis of the article. While, there are some problems need to be revised before this manuscript can be published, following is the detailed advices.
L34, the 2 of CO2 should be subscripted;
L63, "soil temperature(Ta)" in parentheses should be “Ts”;
L68, It can be expressed as "in models that include multiple PFTs" without writing the full name of the PFT;
L189-191, please rewrite the sentences. I guess the author wants to express that MLR is weaker than ANN, SVM, and RF because MLR did not divide the training and validations sets. The logic of this sentence is confusing because of the inappropriate use of the words “Unexpectedly” and “not worse than”;
L230, please add references to support "the lag of precipitation and NDVI/EVI in effect on NEE";
L431-434, the reference title formatting is inconsistent with others;
Figure 3, there is no scale bar and north arrow;
The 2 in R2 needs to be superscripted in all figures throughout the manuscript, e.g. Figure 5, Figure 6, Figure 7, Figure 8, etc.
-
AC1: 'Reply on RC1', Haiyang Shi, 07 May 2022
The comment was uploaded in the form of a supplement: https://bg.copernicus.org/preprints/bg-2022-46/bg-2022-46-AC1-supplement.pdf
-
AC1: 'Reply on RC1', Haiyang Shi, 07 May 2022
-
RC2: 'Comment on bg-2022-46', Anonymous Referee #2, 09 Apr 2022
This study implemented a meta-analysis of current NEE prediction studies. Overall, the topic is interesting and the methodology is innovative, as few researchers in past studies have used R2 or other accuracy metrics to compare models of different studies. Although the number of available models is not large, some of the findings of this study have adding-values and implications at the cross-study (different focus, data, models) level. This manuscript is of interest to BG readers (especially researchers using machine learning to predict NEE). The following issues should be clarified before acceptance.
Main comments:
- The authors have already mentioned the inconsistency between the area of the flux footprint and the area extracted from remote sensing data (e.g. 2x2 km). So, could the authors extract this information from the literature and further analyze this effect? I believe this analysis will be interesting.
- The discussion section is not in-depth enough. The authors should adequately compare the differences between some conclusions in previous studies and the findings of this manuscript.
Other comments:
- In Table 1, GPP is also used as a keyword in the literature collection? Clarify.
- In Table 2, evapotranspiration (ET) is also used as a predictor. Is ET here the latent heat observed by the flux station? Clarify.
- In Figure 8, the categories should be reordered.
- The area observed by the flux station should be larger than 100 x 100 m (usually a few hundred meters).
-
AC2: 'Reply on RC2', Haiyang Shi, 07 May 2022
Response to referee comments
Referee #2
This study implemented a meta-analysis of current NEE prediction studies. Overall, the topic is interesting and the methodology is innovative, as few researchers in past studies have used R2 or other accuracy metrics to compare models of different studies. Although the number of available models is not large, some of the findings of this study have adding-values and implications at the cross-study (different focus, data, models) level. This manuscript is of interest to BG readers (especially researchers using machine learning to predict NEE). The following issues should be clarified before acceptance.
Response: We would like to thank the reviewer for the positive comments and the time invested to review our manuscript. The revised manuscript will follow the reviewer’s recommendations.
Main comments:
The authors have already mentioned the inconsistency between the area of the flux footprint and the area extracted from remote sensing data (e.g. 2x2 km). So, could the authors extract this information from the literature and further analyze this effect? I believe this analysis will be interesting.
Response: Thank you for the insightful comments. Indeed the scale of the explanatory variables affects how well they match the scale of the flux observations. We will consider discussing this issue more in-depth or extracting this information from the literature and further evaluate this effect in various PFTs.
The discussion section is not in-depth enough. The authors should adequately compare the differences between some conclusions in previous studies and the findings of this manuscript.
Response: Thank you for the insightful comments. We will further improve the discussion section of this manuscript by incorporating/comparing findings from previous literature (e.g., the study of uncertainty in modeling practices in some local studies).
Other comments:
In Table 1, GPP is also used as a keyword in the literature collection? Clarify.
Response: Our inclusion of the keyword GPP was to ensure that as much of the literature as possible was included because NEE was predicted along with GPP in some literature.
In Table 2, evapotranspiration (ET) is also used as a predictor. Is ET here the latent heat observed by the flux station? Clarify.
Response: We would modify it to 'evapotranspiration (ET) as the latent heat observed by the flux station'.
In Figure 8, the categories should be reordered.
Response: We will modify the order.
The area observed by the flux station should be larger than 100 x 100 m (usually a few hundred meters).
Response: We will modify it to ‘a few hundred meters'. The observation extent of the flux footprint is influenced by many factors such as wind speed and therefore varies within a few hundred meters.
-
RC3: 'Comment on bg-2022-46', Anonymous Referee #3, 09 Apr 2022
This manuscript conducts a meta-analysis to explore the uncertainty of simulating NEE using comparing machine learning techniques, time-scale and spatial-scale changes, and input variables. This is an important topic to solve the difference between observed and predicted NEE. However, this manuscript doesn’t clarify the objectives and detail of data processing. Oversimple descriptions in the Methods section makes readers confusing. Additionally, the usage of too many speculative explanations in the discussion section is hard to draw universal conclusions. This manuscript doesn’t clarify the motivation of the work, especially in the advantages and potential of ML. In summary, the paper needs to be substantially revised, and some parts need further elaboration.
L32, This sentence hardly reflects the scientific value of this paper.
L40-L43, The advantages and the current situation applied to ML need to be further reviewed, which is beneficial for readers to understand the purpose of introducing ML in this paper.
L45, The sentence, “a synthesis evaluation is …limited”, needs to be further explained otherwise it is hardly understood.
L49-50, need references, preferably with 2 examples
L52-54, There is a logical gap between this sentence and the previous statements.
L88-93, The uncertainty caused by spatio-temporal heterogeneity cannot be confused with the volume of data sets. Because large-data volume does not equate to higher heterogeneity. Big data provides more opportunities to build balanced-training data. This section may need to be rewritten.
L107-108, need references
L116, "Other Features" needs to be clarified. The purpose of this manuscript may be to explore: the uncertainty of NEE evaluation results caused by ML techniques, spatio-temporal resolution remote sensing data, and verification methods according to the introduction?
L144, An oversimplified description of the workflow, please give an overview and detailed sub-steps of data processing and simulation. It is hard to know the objectives of each analysis for readers.
L150, Abbreviations in the figure need to be clarified
L178, Need scale bar and north arrow in figure 3
L198, It is difficult to find the differences among algorithms using simple comparisons in figure 5a, and needs more statistically testing. Additionally, this figure confuses me. Why are MLR, RF, SVM, and ANN separately compared? Please provide explanations. Why is PLSR with high R2 removed? Finally, there are also some problems with the image. The caption does not explain the details of the box. Does the line in the box represent the mean or the median?
L205, Avoid using the word “significant” without statistically testing
L206-210, It is hard to read the trend in Figure 6. Recommend adding a line chart to demonstrate the decreasing trend.
L212, There are no details of the boxplot. Are all models incorporated into the time-scales comparison, or only RF, SVM, and ANN? Please add the details of data processing.
L223, Also, use these words carefully without statistically testing.
L263, Need to reorder the y-axis text in figure 8. Furthermore, a serious question is whether the comparison analysis of these variables keeps other variables constant? If not, conclusions based on comparisons of R2 may not hold water.
L299, Lacking the in-depth discussion of the uncertainty of NEE prediction resulting from time-scale change.
L308, There are too many speculative parts and insufficient supporting materials in section 4.1 of discussed.
L321-323, The discussion of model accuracy difference caused by satellites needs careful. This sentence needs further support. Are you implying that the time scale compensates for the uncertainty caused by the spatial scale?
L326-330, This sentence is too long
L330-332, The time-scale discussion containing spatial-scale matching will confuse readers.
L349, Does "coarse-resolution" here note spatial resolution or temporal resolution?
-
AC3: 'Reply on RC3', Haiyang Shi, 07 May 2022
The comment was uploaded in the form of a supplement: https://bg.copernicus.org/preprints/bg-2022-46/bg-2022-46-AC3-supplement.pdf
-
AC3: 'Reply on RC3', Haiyang Shi, 07 May 2022
Haiyang Shi et al.
Haiyang Shi et al.
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 374 | 95 | 22 | 491 | 32 | 8 | 11 |
- HTML: 374
- PDF: 95
- XML: 22
- Total: 491
- Supplement: 32
- BibTeX: 8
- EndNote: 11
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1