Reviews and syntheses: parameter identification in marine planktonic ecosystem modelling

Schartau, Markus; Wallhead, Philip; Hemmings, John; Löptien, Ulrike; Kriest, Iris; Krishna, Shubham; Ward, Ben A.; Slawig, Thomas; Oschlies, Andreas

doi:https://doi.org/10.5194/bg-14-1647-2017

Articles | Volume 14, issue 6

https://doi.org/10.5194/bg-14-1647-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

Special issue:

Data assimilation in carbon/biogeochemical cycles: consistent...

https://doi.org/10.5194/bg-14-1647-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 14, issue 6

Reviews and syntheses

|

29 Mar 2017

Reviews and syntheses |

| 29 Mar 2017

Reviews and syntheses: parameter identification in marine planktonic ecosystem modelling

Markus Schartau, Philip Wallhead, John Hemmings, Ulrike Löptien, Iris Kriest, Shubham Krishna, Ben A. Ward, Thomas Slawig, and Andreas Oschlies

Download

Final revised paper (published on 29 Mar 2017)
Preprint (discussion started on 20 Jun 2016)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review comments for the manuscript: ``Reviews and syntheses: Parameter identification in marine planktonic ecosystem modelling (bg-2016-242)'' by Markus et al.', Anonymous Referee #1, 28 Jul 2016
- AC1: 'We thank Referee #1 for taking the time to read our manuscript and to provide helpful comments.', Markus Schartau, 29 Oct 2016
RC2: 'Review of "Reviews and syntheses: Parameter identification in marine planktonic ecosystem modelling" by M. Schartau et al.', Anonymous Referee #2, 04 Aug 2016
- AC2: 'Concerns raised by Referee \#2 were respected and we introduced considerable changes to our manuscript.', Markus Schartau, 29 Oct 2016
RC3: 'Review of "Reviews and syntheses: Parameter identification in marine planktonic ecosystem modelling"', Anonymous Referee #3, 05 Aug 2016
- AC3: 'Basic aspects of sequential DA approaches are now explicitly described. We thank Referee #3 for reviewing our manuscript.', Markus Schartau, 31 Oct 2016
RC4: 'Review of "Reviews and syntheses: Parameter identification in marine planktonic ecosystem modelling"', Anonymous Referee #4, 08 Aug 2016
- AC4: 'According to Referee's comments we changed parts of the manuscript's structure, which has improved the readability.', Markus Schartau, 31 Oct 2016

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (08 Nov 2016) by Marko Scholze

AR by Markus Schartau on behalf of the Authors (13 Dec 2016)

ED: Referee Nomination & Report Request started (14 Dec 2016) by Marko Scholze

RR by Anonymous Referee #2 (27 Jan 2017)

Suggestions for revision or reasons for rejection

Review of revised version of "Reviews and syntheses: Parameter identification in marine planktonic ecosystem modelling" by M. Schartau et al.

The authors performed a very good work in revising the manuscript. The big issue of the neglected sequential data assimilation schemes has been widely resolved. Also, the manuscript appears more balanced with regard to presenting examples of the authors' own work and those by others.

I still see some minor issues, which should be resolved before I can recommend the publication of the manuscript. Unfortunately, the manuscript still shows a clear preference against sequential data assimilation methods. The major disadvantage of the manuscript seems to lie in the fact that the group of authors does not include any scientist who applies sequential data assimilation. To this end, the apparent goal of being a fully comprehensive review and synthesis is not reached. This, however should not prevent the publication, because the manuscript is otherwise a very comprehensive, but also extremely long, review.

Main comments:

Abstract, last sentence:
As I already commented in my first review, the last sentence of the abstract - the recommendation to balance level of sophistication of the model and the data assimilation treatment - is not a clear result of the study. The authors only responded that from literature review they had the 'general impression' and hence 'felt' that it would be helpful to report their impression to the readers. For me, this is pretty surprising response, because, as the statement itself, the response is not clearly based on scientific insight but rather represents an opinion. On the other hand, Section 6.2 does in fact discuss the model complexity in a scientific way. Thus, I can still only recommend to reformulate the last sentence in a way that it is not just a statement of opinion (which is not suitable for an abstract of a scientific paper), but shows that it is a result of literature review and studies on model complexity.

In the new section 2.2.2 on sequential methods, the authors state that sequential data assimilation approached can be expedient in cases that assimilating all data at different times is 'computationally impractical'. This statement is misleading, because sequential schemes can be used (and actually have been used) also in cases where a variational scheme could still be applied (There are various cases in the literature). Even more, a sequential scheme can also have a 4D-component, when a smoother method is applied. Rather, it depends on the scientific question whether a sequential data assimilation method results in the required result and the manuscript itself provides examples for these cases.
In this respect also the statement that sequential data assimilation method '...break the large integration problem ... into a number of smaller problems...' (page 6, lines 69-70) is incorrect. The sequential methods reformulate the data assimilation problem into a sequential form. This, however, does not necessarily 'break' it. Furthermore, the sequential methods are not necessarily a 'sequential approximation' (page 6, line 74). Please revise the first paragraph of Section 2.2.2 accordingly.

Page 7, lines 49-53:
Here the (Extended) Kalman filter is discussed. Actually, these filters are not relevant for large-scale systems like in ocean biogeochemical modeling, because of model nonlinearities and because of the high-dimension of the models, which makes it impossible to store the full state error covariance matrix. This is know to most researchers for more than a decade and accordinly studies use variants of the ensemble Kalman filter, which can cope with with high-dimension and partly with the nonlinearities. I recommend to remove the lines discussing the (extended) Kalman filter.

Page 7, lines 74-81:
The text mentiones that there are 'powerful mathematical tools' for variational DA and that the adjoint methods are 'extremely efficient'. Here, the authors should be more specific as the statements are too superficial. What mathematical tools are meant; why are they 'powerful'? Further, what is 'extremely efficient' and compared to which methods is this the case? Actually, given the importance of the adjoint method, I would recommend that the authors state here in the main text, what the adjoint method actually is.

Sections 2.3 and 2.4:
These sections actually describe further aspects of the theoretical data assimilation background. I recommend to change them into sub-subsections of Section 2.2. Section 2.3 could also be merged with Section 2.2.3, because the cost function discussed here is only relevant for variational methods, which are discussed in Section 2.2.3. When Section 2.4 is changed in to sub-sub-section of 2.2 also its first sentence 'We close this section' would be reasonable since 2.2 is concerned with DA methods.

Last lines of Section 3.2 and Section 3.5:
Section 3.5 seems to repeat some aspects on model formulations that account for acclimation dynamics which are already mentioned in Section 3.2. I recommend to focus 3.2 clearly on the aspect of limitation to avoid the redundancy.

Page 12, line 96:
The texts cites '(Simon and Bertino, 2012, Fig. 1)'. I can only guess that 'Fig. 1' does actually refer to Figure 1 of the manuscript and not Fig. 1 of Simon and Bertino (2012). This guess is based on the fact that I didn't find any other place where the manuscript refers to Fig. 1. Actually, in the current form, the figure is nothing more than a pure illustration because the only connection with the main text is that it refers to the figure. As for the other figures, please describe in the text what is the particular result shown in Fig. 1.

Page 13, lines 14-16:
It is stated 'Neglected correlation may result in parameter estimates that are less efficient... and more strongly correlated'. Please provide a reference for this statement.

Page 13, lines 67-68:
It is stated 'To our knowledge no application has yet incorporated prior correlations between parameters'. This statement does actually ignore that ensemble-based sequential DA schemes naturally include correlations between parameters, if the model dynamics yield them.

page 16, lines 17-19:
'The determination of parameter uncertainties has many facets, getting to the core of discussions between Bayesian and frequentists approaches...'. This statement cannot be understood unless the reader already knows what it actually means. One needs to read the full section 5 to get the idea which facets are meant and what the Bayesian and frequentist approaches are. Please revise the text so that readers don't need to speculate what the authors actually mean to say.

Section 5 in general:
Unfortunately, the authors missed to include uncertainty estimates from ensembles methods. Please also discuss it for completeness.

Section 6.2:
The section contains the statements:
'the appropriate degree of model complexity in any given situation is both one of the most important, and one of the least well defined' (p 20, lines 49-50)
'there exists a fundamental trade-off between simplicity and complexity' (p20, l54-55)
'the extra degrees of freedom can lead to the introduction of compensatory errors at the assimilation site' (p20, l67-68)
'an extra flexibility may lead to very different model solutions with only small variations in the assimilated data' (p20, l69-71)
All these claims appear to be results from scientific studies rather than the authors' opinion. Accordingly supporting references are required to make the claims valid for a scientific paper.

page 20, line 75-76:
What makes the review by Johnson and Omland (2004) 'useful'? Why is cross-validation 'most practical' and perhaps 'most general'? The particular expressions are opinions of the authors. It would be preferable for a scientific paper if the authors focus on facts.

page 21, line 1:
I recommend to start a new paragraph at 'A perhaps more intuitive....'

page 21, lines 68-69:
It is stated 'Among models with a similar score, the simplest should be favoured'. Here, I again recommend to rephrase the statement to be scientific, which excludes 'favours'. Scientifically, the optimal model choice seems to be that one with the least parameters and minimum score or score within a certain threshold from the minimum.

Section 7.1.4:
Given the fact that Section 7.1.3 already discussed studies considering time- or space-varying parameters, the first sentence of the section reads quite odd. Please rephrase it.

Section 8:
This sections appears to be overly detailed compared to the treatment of others aspects in the manuscript. At the same time it is too short to really understand details. E.g. from reading the text, I could not really understand the meaning of he 'alignment operator' (pages 25/26). How can the emulator equal the model (page 26, line 2)? this seems to ignore the presence of the alignment operator matrices A_li. However, given the small number of references in Section 8, its length is not consistent with the current relevance of the methods. Thus, I recommend to shorten the section to a concise overview of the dynamic and statistical emulators. One clear possibility for shortening is also the example in Section 8.2 (page 26, right column), which is too detailed and incomplete at once. While for the model itself it is referred to a publication, the manuscript lists explicit parameter values, which is of no use without knowing the model equations. Further, the example is only concerned with a 0D case, which appears to be trivial in particular as the authors intent to discuss emulators in the context of high-dimensional models. The application in 3D cases of higher dimension, where the method could be most useful, appears to be extremely difficult.

Section 9:
The authors stated in their response that they shortened this section. However, this doesn't seem true as in the original version the Section spread over about 4.5 pages and now (in the document version 'author_response_version1', it's again 4.5 pages. Particularly long is subsection 9.3. While its title suggests that the section discusses 'Parameters relevant for global ocean BGC modelling', the section does almost exclusively discuss the parameter 'b' of the power law for particle flux. Here also an example from Kriest et al. (2016) is included, which is described in quite some detail. This again seems to be too detailed, even more as the conclusion appears to be that the value of 'b' is 'well identifiable' (page 39, line 76) (similarly in line 62 for a different case). To this end, the main result appears to be that the value of 'b' can be determined but is specific for each model. I wonder, why so much space is used to describe this result.

Page 31, line 17:
Please add a reference to the assessment report of the IPCC.

Page 31, lines 94-95:
It is stated: 'In BGC models the conservation of mass can be essential, in particular for large-scale or global ocean simulation'. Actually, the conservation requirement is not resulting from a model being large-scale or global, but it results from the scientific question to be considered. In the current form, the text implies that all data assimilation applications of sequential methods with large-scale or global models are wrong because they don't conserve the mass. This is certainly not true and the authors contradict their own statements in Section 2.2. Please reformulate the statement.

Page 32, right column, lines 2-3:
I already recommended in my first review to mentioned here what the dynamics and statistical emulators are. Unfortunately, the authors just replied that they don't see a need for this, because they defined these emulators in Sections 8.1 and 8.2. To this end, I like to remind the authors about the fact that this is the summary section. Usually, one doesn't expect that readers will read the whole paper but many readers will focus on the introduction and summary (which are already quite long in this manuscript). Thus, it would just help readers if the authors would add one or two sentences shortly mentioning that a dynamical emulator is a computationally cheap approximation of the model operator, while a statistical emulator simulates the output from inputs in a statistical way based on a prior training with independent input/output sets. This should be possible in a very short way so that the overall length of the manuscript is not significantly changed.

Fig. 6:
It's written 'Geographic extent of the two sub-domains'. I can only guess that the colors in the plot distinguish the two sub-domains. Unfortunately, this is never described.

Typos:

page 13, l43: 'biased'
page 14, l11: 'that the'
page 21, l55: 'degrees'
page 23, l108: 'problem, i.e.'
page 28, l79: '(Sect. 2.2.3)' or '(Sect. (2.2.3))'
page29, l61: 'accomplished'

Hide

ED: Publish subject to minor revisions (Editor review) (27 Jan 2017) by Marko Scholze

AR by Markus Schartau on behalf of the Authors (17 Feb 2017) Manuscript

ED: Publish as is (21 Feb 2017) by Marko Scholze

AR by Markus Schartau on behalf of the Authors (27 Feb 2017)

Short summary

Plankton models have become an integral part in marine ecosystem and biogeochemical research. These models differ in complexity and in their number of parameters. How values are assigned to parameters is essential. An overview of major methodologies of parameter estimation is provided. Aspects of parameter identification in the literature are diverse. Individual findings could be better synthesized if notation and expertise of the different scientific communities would be reasonably merged.