The basic formulation of SVR is similar to that of linear regression as described by : f(x)=〈w,x〉+b with b∈R, where b is an intercept and 〈⋅,⋅〉 denotes the dot product of the weights (w) and x, the training data. The weights and intercept are found by solving the cost function: minimize12||w||2  subject toyi-〈w,xi〉-b≤ϵ〈w,xi〉+b-yi≤ϵ.

In this form, w is minimized according to the target values (yi) to a precision of ϵ – i.e. there is no room for error greater than ϵ. However, with the majority of problems, meeting these constraints is not possible if data are noisy or ϵ is set to be small. The inclusion of slack variables (ξi,ξi*) relaxes the constraints and the problem is now formulated as follows: minimize12||w||2+C∑i=1n(ξi+ξi*)subject toyi-〈w,xi〉-b≤ϵ+ξi〈w,xi〉+b-yi≤ϵ+ξi*ξi,ξi*≥0. Here C is a parameter that adjusts for the amount of error that the minimization allows. The slack variable |ξ| is only counted towards the cost if the point lies outside the margin (|ξ|≥ϵ). The points on or outside the margins are called support vectors and are used to construct the hypothesis function, h(x). This is shown in Fig. a where a linear SVR is fitted to noisy data produced from a cubic spline. The optimization problem shown in Eq. () is solved in its dual formulation (see , for the full description). Importantly, solving the dual formulation allows for efficient kernelization of SVR.

Kernelization describes the process that maps the proxy variables (x) onto a higher dimensional feature space. In this study we used a Gaussian kernel (or radial basis function – RBF), which allows for potentially infinite complexity determined by the number of support vectors . The RBF kernel introduces an additional hyper-parameter (γ) that defines the width of the Gaussian. Selection of the SVR hyper-parameters (ϵ, C, γ) is done using a two-stage exhaustive grid search approach with cross-validation. We use K-fold cross-validation, where the data are divided into eight equal “folds” (k=8). Seven of the folds are used to train the model, while the remaining fold is used for validation. This is done iteratively until each excluded fold has been used to test the results.

Decision trees form the basic building block of a random forest (RF), with the average of n decision trees taken as the ensemble estimate (Fig. b). The basic idea of a decision tree is to iteratively partition data into boxes using simple rules that minimize the error at each split (referred to as a node) – these boxes would become hypercubes in higher dimensional problems. This is described by the basic formulation as described in :

Start at the root node.

This experiment used the synthetic dataset to test the influence of including or excluding transformed coordinates (time, latitude and longitude) as proxies of ΔpCO2. There are four major results from the experiment results. Firstly, the RMSE estimates are smaller when coordinates are included as proxies for both in- and out-of-sample subsets (Table ). Secondly, RFR achieves marginally better out-of-sample RMSE than SVR (5.76 and 6.19 µatm, respectively) when trained with coordinates. Third, both RFR and SVR have comparable out-of-sample RMSE estimates (7.46 µatm) for ΔpCO2 estimates trained with and without coordinate proxies. Lastly, the ensemble mean of SVR and RFR has lower out-of-sample RMSE estimates than the individual estimates for implementations with and without coordinate proxies, though these gains are marginal (Table ).

These points can also be gleaned from RMSE maps (Fig. ). Both RFR and SVR errors are low; however, the RFR outperforms the SVR marginally for the open-ocean regions. Errors in coastal regions remain high for each of the experiments and methods (Fig. a, b, d, e), such as in the Argentine Sea, the Agulhas retroflection, and the marginal ice zone. The ensemble mean of the estimates achieves a balance between the two methods with low and moderate RMSE scores in the open-ocean and coastal regions. Lastly, the distributions of the errors for RFR and SVR without coordinate proxies (and thus the ensemble mean) are similar.

The time series (Fig. ) show that including coordinate variables plays an important role in achieving accurate phasing of the seasonal cycle. When coordinates are not included as proxies, the phasing shifts earlier for both methods. There is also an improvement of estimates over time, where the first 2 years (1998 and 1999) have worse estimates for both SVR and RFR (Fig. ). This does not seem to be linked to the number of observations, but could be due to the distribution. The ensemble ΔpCO2 in the 1998 to 1999 period is closer to BIOPERIANT05 output as the over- and underestimates of RFR and SVR, respectively, compensate for each other.

A second experiment is performed to assess the inaccuracies that arise due to the spatial and temporal sampling biases in the SOCAT v3 dataset. Training locations are chosen at random and uniformly in time and space. This eliminates any summer–winter biases as well as clustering of cruise tracks in certain regions (such as the Argentine sea). Note that coordinates are included as proxies of ΔpCO2 with the random sampling regime.

Firstly, the results show that the biases in SOCAT v3 do contribute to out-of-sample errors, as the random sampling regime achieved lower RMSE scores than any of the other experiments (4.88 and 4.94 µatm for RFR and SVR, respectively, as in Table ). However, RFR is marginally less susceptible to sampling biases than SVR, as the relative improvement for the latter is larger (with differences of 0.88 and 1.70 µatm, respectively). The spatial distributions of RMSE for the random sampling implementations (Fig. c, f) show that errors in coastal regions remain high (> 12 µatm) with uniform sampling. Lastly, there is an improvement in the estimates from 1998 to 2000, particularly with the random sampling for the SVR (Fig. ), suggesting that the method is more susceptible to the temporal bias than RFR (if coordinates are included as proxies).

Based on the results, the SOM-FFN method (by ) proves to be an elegant implementation of neural network methods that is able to estimate SOCAT ΔpCO2 (in-sample estimate) better than the RFR and SVR methods (with respective RMSE estimates of 14.84, 16.45 and 24.40 µatm). Here we assess these differences and try to identify the possible reasons for the differences.

One of the largest differences in the methods' ability to fit the training data is in the SAZ where the RFR and SVR score poorly in comparison to SOM-FFN, particularly from 2000 to 2006 (Fig. a). This is during a period where the number of observations are still relatively low in the SOCAT v3 database (Fig. a). This may then be due to an increase in the complexity of ΔpCO2 measurements in the SAZ from SOCAT v2 to v3 from 1998 through 2006, which makes the data more challenging to fit accurately. This is exemplified in the maps of RMSE (Fig. g–i), where coastal regions typically have larger error estimates. A comparison of SOCAT v2 and v3 for this period shows that the increase in the number of observations occurs primarily in the Argentine Sea, thus confirming this hypothesis (Fig. a). The comparison of SOCAT v2 and v3 RMSE results for RFR and SVR confirm this (Table ), where there is a marked improvement when using the older dataset. Importantly, this shows that increasing the number of measurements does not necessarily improve the in-sample error estimates, but may yield a more accurate out-of-sample estimate; however, this is difficult to test with limited data.

Despite the improvement in performance when testing against SOCAT v2, SVR and RFR still have poorer performance than the SOM-FFN approach. We attribute this in part to the SOM-FFN's ability to reduce the large RMSE contributions observed in the other two methods. This notion is supported by the smaller difference between RMSE and MAE, especially in the SAZ (Table ). The SOM-FFN achieves this by increasing the flexibility of the algorithm by having multiple regression models that can each be optimized for data with a particular length scale of variability. This allows the SOM-FFN approach to adapt to short scales of variability in dynamic regions such as the Argentine Sea and the coastal Antarctic (Fig. i).

In comparison, this implementation of SVR, which is theoretically similar to an artificial neural network, only has one length scale for the entire domain . This becomes apparent in the AZ, where many of the observations are in the more biogeochemically dynamic coastal Antarctic, where melting sea ice results in short decorrelation length scales . The SVR has much larger RMSE scores in the AZ than the RFR or SOM-FFN (35.69, 23.49 and 21.32 µatm, respectively). This suggests that implementing the SVR approach without an initial clustering or regionalization step will not yield good results.

By comparison, the RFR approach is more adept at fitting various length scales of variability, accounting for both the higher and lower variability in the AZ and PFZ, respectively (with SOCAT v3 standard deviations of 54.65 and 20.01, respectively). The high r2 scores achieved by RFR in the AZ and PFZ (0.81 and 0.71, respectively) highlight the flexibility in the method (Table ). This is due to the differences in the underlying mathematics of the methods. Decision trees, which are the building blocks of RFR, separate data at each decision node with a discrete boundary . Conversely, ANNs and SVRs often use Gaussian functions in the cost function, resulting in smoother approximations . This makes decision trees prone to overfitting, but the ensemble implementation of random forests eliminates this to a large extent.

One of the largest differences in ΔpCO2 is the weaker sink estimated by the SOM-FFN method in the SAZ (Fig. ). This difference can be traced to the eastern Atlantic SAZ, where the SOM-FFN has higher estimates of ΔpCO2 (Fig. e shown by the red oval and the differences between the methods in Fig. a, b). A comparison between the SVR and RFR trained with SOCAT v2 and v3 further eliminates the use of different training datasets as primary sources of difference, where methodology is a higher-order driver of difference (Fig. ). The lack of this feature in the eastern Atlantic sector of the Southern Ocean in SVR and RFR estimates suggests that this is a function of the initial clustering step in the SOM-FFN. The clustering process separates pCO2 observations into clusters that are not restricted in time and space . This allows the SOM-FFN to “transfer knowledge” from a remote location (even outside the Southern Hemisphere) if proxies are similar to the Southern Ocean. This knowledge transfer assumes that the relationship between pCO2 and the measured proxies is globally consistent. Moreover, there is the assumption that all pCO2 variability (within a cluster) can captured by the measured proxies. This assumption is not made when using coordinates or regional subsets as locations are isolated, but there is then the potential loss of knowledge from remote locations. This question will be addressed further in the discussion on the use of coordinate variables as proxies of ΔpCO2.

Another difference between ΔpCO2 estimates is the tendency for SVR to overestimate ΔpCO2 relative to the RFR and SOM-FFN approaches, particularly in the PFZ and AZ where winter data are sparse (Fig. b, c). We attribute this to the SVR's sensitivity to outliers, determined by the fact that the cost function penalizes outliers heavily (Eq. ). In context of the SOCAT v3 dataset, the algorithm may treat the sparse winter data as outliers. This is due to the fact that sparse winter measurements of ΔpCO2 are positive, while the abundant summer measurements are negative (). This may then be a positive realization of a methodological attribute that is typically considered a weakness.

Conversely, RFR winter estimates of ΔpCO2 are often lower than the SOM-FFN and SVR estimates, again in the AZ and PFZ (Figs. g–i and b, c). This may be due to the method's resilience against outliers, which could be due to two attributes . Firstly, outliers are less likely to dominate the feature space with the use of bootstrap aggregation as these points will be sampled less frequently. Secondly, individual decision trees regress values by using the average of samples in a terminal node (or leaf), where the minimum number of samples per terminal node is set by the user. This second attribute means that estimates will never be outside the bounds of the minimum and maximum of the training dataset, thus leading to conservative estimates (as shown in Fig. b). The differences between the methods shown in Fig.  could be a good case for an ensemble approach, where the strengths of one model compensate for the weakness of another. This is assessed in the synthetic data and will be discussed further.

These attributes may also be the reason for the differences in the magnitudes of the autumn peak in ΔpCO2 in the SAZ and PFZ. Mechanistically this peak could be attributed to a sharp increase in cooling leading into winter, resulting in increased solubility of CO2 and thus a sharp reduction of ΔpCO2 . Deeper mixing of the water column shortly thereafter would entrain CO2-rich waters, thus increasing ΔpCO2 . However, the trend for this peak to shrink in the SAZ and PFZ for all methods suggests that this may an artefact that is specific to the SOCAT dataset.

This topic has to some extent already been mentioned in the discussion of the observational data, in which the cases for and against the inclusion of coordinate variables as proxies for ΔpCO2 were put forward. If coordinates are not included there is the benefit of potential information transfer from remote parts of the domain, but this assumes that the satellite observable proxies (and assimilative model output) constrain ΔpCO2 in a globally consistent way. If coordinates are included the information transfer is lost and the assumption is made that the proxy variables are not able to constrain ΔpCO2 in a globally consistent manner.

The results of this experiment show that coordinates improve estimates of ΔpCO2 with better RMSE scores for both SVR and RFR (Table ). We are thus in favour of the second hypothesis that the available proxies cannot sufficiently constrain ΔpCO2 without coordinates. A two-step clustering approach, such as SOM-FFN, may be able to achieve comparable results without coordinates, but this would have to be tested with that specific method. However, this may also lead to trends in the data that may be artefacts of remote knowledge transfer, as potentially seen in the observational data (Fig. f).

An important outcome of this experiment is that the inclusion of coordinates improves the seasonal phasing of the methods (Fig. a, b). It is critical for the empirical methods to correctly estimate the phasing of ΔpCO2 as the seasonal cycle phasing may be a useful indicator of anthropogenically driven changes to the marine carbonate system.

One of the assumptions in these synthetic data experiments is that the models are, to some extent, representative of the variability in the observed ocean. However, the BIOPERIANT05 output does not achieve this, with a standard deviation of 19.80 µatm for synthetic SOCAT v3 data compared to 38.20 µatm of the gridded SOCAT v3 observations (according to Southern Ocean as defined by ). This could be a cause for concern. However, we believe that this creates an even a stronger case for the use of coordinates as proxy variables. The increased variability in the observations could be due to processes that deterministic models cannot yet constrain due to our lack of understanding of the marine carbonate system .

The lack of winter pCO2 data is a problem throughout the mid-latitude and high latitude oceans, but is particularly severe in the Southern Ocean , and the impact of the lack of data in the Southern Ocean is not known. Moreover, the efficacy of various methods to fill this large temporal gap is unknown . Here we show that there is a considerable impact in this synthetic data environment, but the effect of the sampling bias is perhaps smaller than we would have anticipated. Both methods are able to estimate the spatial distribution and the seasonal cycle of ΔpCO2 with relative accuracy (Figs.  and and Table ). This could be due to two factors.

Firstly, winter data are less variable than summer data and require less sampling. Mechanistically, this is a likely scenario. In summer ΔpCO2 is spatio-temporally heterogeneous in the Southern Ocean due to the uptake of CO2 by phytoplankton . The drivers of phytoplankton are complex due to the co-limitation of light and iron (as a micronutrient) in the Southern Ocean . This complexity would require more sampling: perhaps additional proxies or increased spatial resolution to capture the variability of ΔpCO2. Conversely, processes driving winter ΔpCO2, namely the interaction of mixing and buoyancy, act on larger scales, potentially leading to less spatio-temporal heterogeneity. However, the lack of observations means that we simply cannot know with certainty. This makes a strong case for autonomous sampling platforms to the Southern Ocean's winter sampling gap. The SOCCOM float project may soon yield such measurements with pH-derived estimates of pCO2 .

Secondly, the model used to generate the synthetic data may not be representative of the Southern Ocean. This has been discussed in the previous section, but here, rather than increasing our confidence, it diminishes our confidence in the result. Studies have shown that process models are not able to accurately represent the seasonal cycle of CO2 in the Southern Ocean . Moreover, ΔpCO2 is often driven by processes that are not representative of observations .

The most likely scenario is likely a combination of these two factors, where winter data are in fact less variable than summer data, but the error is larger than the experiment shows due to incomplete knowledge of the processes that describe pCO2 in the process models.

The synthetic data also allow us to compare the two methods relative to each other, in the context of the SOCAT v3 data. The data show that the RFR method performs better than the SVR (trained with coordinates as proxies) with respective out-of-sample RMSEs of 5.76 and 6.19 µatm (Table ). However, it is the average of these two methods (ensemble mean) that achieves the lowest RMSE (5.36 µatm), albeit marginal. The time series in Fig. c shows that the improvement may come from the period 1998 to 2000, when RFR is plagued by underestimation of the sink strength, while SVR overestimates the sink strength. This supports the notion that the strengths and weaknesses of these two methods compliment each other. Moreover, it supports the merit of multiple approaches and further development of empirical methods for the estimation of ΔpCO2.