An important manifestation of climate change is widespread alteration of the composition of the energy balance at the Earth's surface (Trenberth et al., 2009; Wild et al., 2013). Given the importance of being able to predict the consequences of climate change, both measurement and modelling of the components of surface energy balance attract significant attention from a broad range of related scientific disciplines (Stephens et al., 2012). Two such disciplines are hydrology and meteorology, which share a common interest in resolving the balance between sensible, H, and latent, λE, heat fluxes over a broad range of spatial and temporal scales (Anderson et al., 2012).

Net available energy, Φ, is a core variable used to predict the magnitude of H and λE given that it is defined as the sum of these two fluxes (Wright et al., 1992; Migletta et al., 2009; Anderson et al., 2012),

Φ=λE+H. The utility of this definition arises from being able to also specify Φ as the difference between the net broadband radiation, RN, and the rate of heat accumulation, G, below the plain across which RN is specified, Φ=RN-G. Given that RN is routinely measured using net radiometers, this affords an opportunity to specify Φ and hence either H or λE. For example, in modelling studies λE is invariably specified as a function of Φ using the ubiquitous equations such as those of Penman (1948) for open water or Monteith (1965) for land surfaces (Mu et al., 2011; Mallick et al., 2014a). Despite being the rate of change of heat stock in terrestrial environments, G is often interpreted as the “ground heat flux”, and attempts to measure this using heat flux plates are commonplace (Mayocchi and Bristow, 1995; Sauer and Horton, 2005; Heitman et al., 2010). These measurements prove somewhat less reliable than RN due to greater spatial heterogeneity in ground heat uptake (Gao et al., 1998; Tittebrand and Berger, 2009; Verhoef et al., 2012) allied to the fact that significant heat capacity resides in other elements of the land surface (Ochsner et al., 2007). As a result, G proves problematic in surface energy balance studies and is either ignored (Foken et al., 2006; Foken, 2008) or treated somewhat superficially (Choudhury, 1987), despite being significant under a broad range of conditions (Santanello and Friedl, 2003; Ochsner et al., 2007). Large-scale estimates of G are useful in the context of regional and global evapotranspiration modelling and for verification of regional and global circulation models (Kergoat et al., 2011).

The arrival of satellite retrievals for many of the components of RN has opened up opportunities to develop large-scale estimates of this variable and hence λE (Batra et al., 2006; Mu et al., 2007, Anderson et al., 2012). For example, retrievals for the components of RN have been available through the International Satellite Cloud Climatology Project (ISCCP) (Pinker and Laszlo, 1992; Stephens et al., 2012), the Earth Radiation Budget Experiment (ERBE) (Priestley et al., 2011), and the Clouds and the Earth's Radiant Energy System (CERES) (Mlynczak et al., 2011; Chen et al., 2013) onboard NASA's Earth Observing System (EOS) and Tropical Rainfall Measuring Mission (TRMM) satellites (Wielicki et al., 1998). Several studies have reported the estimation of RN using a combination of MODIS (MODerate Resolution Imaging Spectroradiometer) atmospheric and land products over the USA, China and India (Cai et al., 2007; Mallick et al., 2009; Bisht and Bras, 2010, 2011) or NOAA-14 (National Oceanic and Atmospheric Administration) data over the Tibetan Plateau (Ma et al., 2002).

Unfortunately, in the absence of direct observations of G at spatial scales and coverage of satellite RN, retrievals for Φ have had to rely on the parameterisation of G using surface temperature, albedo and vegetation index information (Bastiaanssen et al., 1998; Batra et al., 2006) or by assigning some fixed proportion of RN (Choudhury, 1987; Humes et al., 1994) in satellite-based surface energy balance models (Mecikalski et al., 1999; Anderson et al., 2012). However studies by Murray and Verhoef (2007), Hsieh et al. (2009) and recently Verhoef et al. (2012) also demonstrated that G is, by definition, a highly dynamic quantity, and that the ratio G / RN can range anywhere from 0.05 to 0.50 depending on the time of day, soil moisture and thermal properties, and vegetation density. Therefore, methods that are able to provide defensible estimates of G in conjunction with RN would clearly be of great benefit to this area for determining Φ directly from satellite data and without relying unduly on any offline calibration. In this paper we present a method for retrieving RN and Φ based on exploring both satellite radiance data and day–night surface temperature difference. The approach is necessarily simple in order to avoid over-reliance on models in the pre-processing and to reflect the fact that the focus of this work is the production of satellite estimates of monthly midday (13:30; all times listed are in local time, LT) Φ for use in a simple Bowen ratio (Bowen, 1926) λE specification framework as detailed in a companion paper by Mallick et al. (2014b) (we refer to this as M2 hereafter). Taking advantage of the extensive network of terrestrial eddy covariance tower sites (Baldocchi et al., 2001) which record direct measurements of RN, H and λE, we use these measurements to derive independent non-radiative estimates of Φ in order to critically evaluate our satellite estimates of this quantity.

The method we present here for estimating RN contrasts with more sophisticated model-based approaches which attempt to accommodate the complexity of atmospheric radiative transfer explicitly (e.g. Fouquart and Bonnel, 1980; Mlawer et al., 1997; Bisht and Bras, 2010, 2011; Hou et al., 2014). There are several reasons for adopting this stance. Firstly, the estimates of RN need to be in agreement with the simple dynamic energy balance used to accommodate G when specifying Φ. Secondly, we believe it to be important that the complexity of the methods used here is commensurate with those methods used in the simple Bowen ratio approach as described in M2. Related to this, we have tried to restrict the approach to largely using only AIRS (Atmospheric InfraRed Sounder) data which provide the satellite soundings required for the Bowen ratio estimates. This single-platform approach is adopted to ensure the estimates do not suffer unduly from blending different data sources. Finally, as is the case with this method, complex radiative transfer approaches are also prone to the effects of uncertainty (Betts et al., 1993; Morcrette, 2002; Seidel et al., 2010), and therefore the parsimony implicit in the methods used here may be seen as advantageous.

Table 2 shows the results from the sensitivity analysis. For G we see the importance of the longwave specification and in particular εC. The standard deviation of the estimate of G from the ensemble is 18 W m-2, giving approximate 95 % confidence detection limits of ±36 W m-2 on the estimates. RN is sensitive to all components of the radiation balance calculation as expected (Table 2). The standard deviation of the estimate of RN from the ensemble is 40 W m-2, giving approximate 95 % detection limits of ±80 W m-2 on the estimates (Table 2). Not surprisingly, the sensitivity results for Φ mirror those of RN, albeit with a marginally higher ensemble standard deviation of 44 W m-2 (Table 2).

The locations of the 30 terrestrial evaluation sites are marked in Fig. 1. Figure 5 shows annual average, global satellite scenes for 13:30 LT RN, c, G and Φ for the year 2003. Missing data in the images are mainly due to missing data in the AIRS soundings at high latitudes or over the mountain belts, where it is difficult to profile air temperature and relative humidity reliably. In addition, persistent cloudy conditions also prevent reliable retrieval and hence are rejected, although these will be less evident in the monthly or annual average data.

Figure 5a shows the global distribution of RN, which generally decreases with latitude, as expected. RN also decreases over land due to the generally higher albedo, resulting in reduced absorption of the net shortwave radiation (Giambelluca et al., 1997; Gao and Wu, 2014) or relatively higher surface temperature increasing the net longwave component, especially over the drier regions (Liang et al., 1998; Trenberth, 2011). As a result the magnitude of RN was around 200–300 W m-2 over the dry desert regions, whereas the oceanic values of RN were 450–700 W m-2.

Figure 5b shows the global distribution of c (surface heat capacity). The oceanic values of 4 to 8 MJ m-2 K-1 are equivalent to 1 to 2 m of seawater, which appears reasonable on the daily time step to which they relate (Stramma et al., 1986; Schwartz, 2007). These oceanic values are somewhat noisy due to the small day–night temperature differences observed for the oceans giving a relatively poor signal-to-noise ratio. However, behind this noise the pattern of oceanic c appears relatively uniform as one might expect. Over land c varies between 0.05 and 0.5 MJ m-2 K-1 with wetter tropical and high-latitude areas showing significantly higher values than the drier, less vegetated areas, as expected. The soil equivalent depth of this heat capacity is approximately 0.01 m, which again appears reasonable for a daily time step (Li and Islam, 1999), although in heavily vegetated areas c is obviously comprised of a more complex aggregation.

Figure 5c shows the global distribution of G. These are the 13:30 LT values, hence their being net positive as an annual average. Between 20∘ north and south, G is approximately 10 to 20 % of RN, and this rises to more than 40 % above 50∘ north and south (Hsieh et al., 2009). Given that this opposes the pattern of RN, one would conclude that there are either some deficiencies in the way G is specified here or that RN partitions into latent heat far more effectively than surface heating in these warm wet environments (Liu et al., 2005). Again, terrestrial values are lower than their oceanic equivalents, mainly due to the lower heat capacity as well as reduced RN as discussed above. This also highlights the role of the vegetation layer in preventing ground heating (Baker and Baker, 2002; Bounoua et al., 2010). The Sahara appears particularly prominent in this scene, with high rates of midday heat accumulation, which appears to be associated with a combination of moderate net radiation and relatively high heat capacity. The heterogeneity in this region appears to be related to the pattern of bare darker rock.

Figure 5d shows the global distribution of Φ, which follows a similar pattern to RN as expected, although the pattern of G shown in Fig. 5c dictates that the north–south gradients in Φ are somewhat stronger than those of RN. Before discussing these results, we consider their evaluation. In the first step, we validated the new method of Φ retrieval at representative FLUXNET sites using ground observations of the surface radiation components (TS, RN and G) as input before directly evaluating the satellite-based retrievals. Tower-scale evaluation of daily midday (13:30 LT) Φ is illustrated in Fig. 6a, b, c and d for four broad biome categories, which shows a modest correlation [r= 0.91 (±0.03) to 0.98 (±0.04)]

All uncertainties are expressed as ± 1 standard deviation unless otherwise stated.

Figure 7a shows the pooled evaluation of satellite RN, which produced an overall correlation of r= 0.88 (±0.03). Assuming both tower and satellite observations are linearly related through some “true” value, then the pooled values are co-related by RN(satellite) = 0.75(±0.02)RN(tower) + 23.37(±8.20), i.e. a small but significant underestimation in RN(satellite) relative to RN(tower). The RMSD between the two was 98 W m-2. The biome-specific statistics for RN are given in Table 3, which reveals correlations ranging between 0.65 (EBF) and 0.96 (ENF), RMSD ranging between 74 (GRA) and 127 W m-2 (EBF), and regression statistics ranging between 0.58 (±0.08) and 0.87 (±0.04) for the gain and -32.40 (±23.73) and 107.45 (±39.93) for the offset.

Figure 7b shows the evaluation for Φ, which produced pooled statistics of r= 0.87 (±0.03) and an RMSD of 72 W m-2, and the regression between the satellite-predicted and tower-observed Φ produced a regression line of Φ(satellite) = 0.90(±0.03)Φ(tower)-2.43 (±8.19). The biome-specific statistics for Φ are also given in Table 3 showing correlations ranging from 0.70 (EBF) to 0.95 (ENF), RMSD ranging between 62 (GRA & SAV) and 88 W m-2 (EBF) and regression coefficients ranging between 0.66 (±0.08) and 1.01 (±0.05) and between -65.25 (±27.07) and 108.71 (±32.10) for the gain and offset, respectively.

Figure 8 shows a sample of monthly time series for Φ for both the satellite and the towers. The sites were selected to represent the biome classes considered here, as well as ones for which complete annual data sets for 2003 were available. These results show that the satellite estimates generally track the trends in the tower data and hence that the pooled statistics are not masking the within site variability. Again, the site-wise comparative statistics for these data are given in Table 3.

For RN the statistics relating the satellite and tower data are different with to results of the following authors: Bisht et al. (2005), who obtained 74 W m-2 RMSD when evaluating MODIS Terra geophysical land products over the Southern Great Plains of the USA (our RMSD in grassland is only comparable here, while other biomes show larger error); Jacobs et al. (2004), who obtained a 14–46 W m-2 RMSD (12.2 % relative RMSD) when determining hourly RN using GOES (Geostationary Operational Environmental Satellite) data over wetlands in southern Florida; Cai et al. (2007), who obtained 13.7 % error when evaluating MODIS Terra–Aqua data over China; Bisht and Bras (2010), who obtained 39–51 W m-2 RMSD over the central USA using MODIS Terra atmospheric data at 5–10 km spatial resolution; and Hwang et al. (2013) and Hou et al. (2014), who reported RMSD in instantaneous and daily RN to be 58–142 and 37–40 W m-2 over Southeast Asia and China, respectively, using MODIS Terra data products. Stackhouse et al. (2000) evaluated the International Satellite Cloud Climatology Project (ISCCP) data and found errors in the range of 10 to 15 W m-2 in monthly average shortwave and longwave radiative fluxes. Other studies reported 33–60 W m-2 RMSD in daily RN using 5 km MODIS Terra optical and thermal data (a comprehensive list of relevant studies is given in Table 4). It is important to emphasise that, in the present study, the RMSD is being impacted in two ways: due to spatial scale mismatch and due to the time integration. When these errors are compounded in the derivation of RN and compared with tower data, an RMSD of the order of 98 W m-2 appears reasonable considering the coarse spatial resolution of the AIRS data (1∘ × 1∘).

There have been very few attempts to retrieve satellite estimates of Φ and compare these with ground truth data, although the statistics from our attempt appear to be parallel to the results of Stisen et al. (2008), who studied a single grassland site in the Senegal River basin using moderate (high) spatio-temporal resolution (5 km spatial resolution, 15 min temporal resolution) MSG (Meteosat Second Generation) geostationary satellite data and obtained a correlation of r= 0.71 and an RMSD of 43 W m-2 in comparison to the surface measurements. While estimating evapotranspiration over Indian agroecosystems, Bhattacharya et al. (2010) obtained an RMSD of 56 W m-2 for noontime Φ using 8 km resolution Indian geostationary satellite data. In another study with MODIS Aqua data over semi-arid agroecosystems in India, Bhattacharya et al. (2011) reported an RMSD of 34 W m-2 in daily average Φ, which was associated with a significant tendency to underestimate Φ. As was common to all these studies, the ground heat flux was either modelled as an empirical approximation employing remotely sensed surface variables (albedo, vegetation index and TS) or as a fixed fraction of RN. Murray and Verhoef (2007) argued that these empirical approaches do not generalise well. In particular, prescribing G as a fixed fraction of RN overlooks the role played by the thermal inertia of the land surface (Santanello and Friedl, 2003), leading to an underestimation of G in the morning and overestimation during the afternoon (Gentine et al., 2007). The retrieval of G proposed here using day–night surface temperature information attempts to account for this thermal inertia effect, and the results appear to support this approach especially when considering the scale mismatch between the tower and satellite observations.

As seen in Fig. 7a, there is a systematic underestimation of RN relative to the tower values which exceeds the typical accuracy of net radiometer measurements of 20 W m-2 quoted by Foken (2008). We examined this underestimation in more detail wherever possible by evaluating three of the individual radiation components of RN (RS↓, RL↓ and RL↑). All tower sites provided measurements of RS↓ (but not RS↑). Figure 9a shows RS↓ is systematically underestimated at the satellite scale, with RS↓(satellite) = 0.70(±0.02)RS↓(tower) + 68(±12.24), which accounts for the mismatch of RN(satellite) ≈ 0.75RN(tower). Before attempting to account for the various reasons for this underestimation, it is important to realise that, unlike the IR components, the shortwave components are all-sky retrievals, i.e. like the tower data they do not omit cloudy-sky conditions. As a result, any bias in the shortwave is not as a result of biased sampling when comprising the monthly average. Furthermore, the omission of non-clear-sky data would tend to lead to RS↓(satellite) > RS↓(tower).

Clearly, the retrieval of atmospheric shortwave transmissivity (τA) using cloud cover fraction is the principal reason for RS↓(satellite) < RS↓(tower) (Fig. 9a). The sensitivity analysis presented in Table 2 also indicates the significant sensitivity of RN and Φ to the cloud cover fraction and atmospheric transmissivity. This shows that the method presented in the manuscript to estimate RS↓ needs further improvements. If we assume τA to be the principal reason for RS↓(satellite) < RS↓(tower), then a global value of 0.75 would be, on average, too low (Gueymard, 2003). A recent study of Longman et al. (2012) for the Mauna Loa Observatory (MLO) demonstrated the clear-sky τA could go up to 0.90. Given the relatively well defined relationship between RS↓(satellite) and RS↓(tower) seen in Fig. 3c, one would imagine that a more sophisticated dynamic representation of τA would offer substantial improvements in RS↓(satellite). Retrieval of τA including other atmospheric (e.g. cloud optical depth, aerosol optical depth, total precipitable water etc.) and surface (for example, single scattering albedo) variables in addition to the cloud cover fraction would offer a potential possibility of refining the RS↓ estimates (Chen et al., 2014; Longman et al., 2012; Kim and Hogue, 2008). The exo-atmospheric shortwave radiation frequently interacts with the clouds, aerosols and water vapour during the transmission towards the Earth's surface. This interaction is wavelength-dependent over the entire shortwave spectrum (Chen et al., 2014; Kim and Hogue, 2008; Gueymard, 2003), and therefore a spectrally resolved τA scheme will be valuable to accurately determine RS↓. Recalibration of τA using the tower data is also a possibility, although we have avoided this given that the AIRS cloud cover fraction and scale mismatch between the satellite and tower could also be involved in the observed bias. For example, the diffuse fraction of RS↓(tower) can become enriched by surface-reflected solar radiation, particularly in undulating terrain (Dubayah and Loechel, 1997; Sultan et al., 2014). Nonlinear scaling effects of surface albedo (Oliphant et al., 2003; Salomon et al., 2006) can also be included in this because surface albedo interacts nonlinearly with surface characteristics such as surface wetness and land surface temperature (Ryu et al., 2008) or the leaf area index (Hammerle et al., 2008). Although, the RMSD of instantaneous RS↓ obtained in the present study (110 W m-2) is different to other studies where RS↓ retrieval was based on either using parametric (radiative transfer) models or look-up tables derived from high-spatial-resolution MODIS data, it is worth comparing it with the statistics of some of those studies. Table 4 summarises the characteristics and associated errors of some of the recent RS↓ estimation studies, and shows an RMSD of 36–89 W m-2 at flux tower footprint, 54–137 W m-2 at 5 km spatial resolution and 77–158 W m-2 at 1∘ spatial resolution for the instantaneous RS↓ estimates and 20–39 W m-2 for the daily RS↓ (and net shortwave, RNS) estimates. Considering the simplicity of the current approach and the large spatial scale of the AIRS data, an RMSD of the order of 110 W m-2 appears reasonable.

To probe the specification of RN further, we investigated the individual longwave radiation components in relation to measures of theses fluxes available for a limited subset (14) of tower sites where the longwave radiative flux components were directly measured by pyrgeometers. From Fig. 9b and c it appears that there is quite good agreement between the satellite and tower data for both RL↓ and RL↑ and that any mismatch is insufficient to explain the discrepancy in RN. This is somewhat surprising for two reasons. Firstly, unlike the shortwave component, RL(tower) is all-sky whilst RL(satellite) is only from clear-sky conditions where IR retrieval is possible. As a result, one would anticipate very significant differences in the monthly average values of the longwave components. However, it is difficult to predict the effect of this biased sampling on RNL(satellite) given that cloud interacts with both RL↓ and RL↑ in complex ways. Secondly, one would anticipate significant scaling effects from the T4 nonlinearity in Eq. (6), which can result in a disproportionate contribution of warmer elements within the system to both RL↓ and RL↑ (Kustas and Norman, 2000; Lakshmi and Zehrfuhs, 2002, Corbari et al., 2010). The fact that these effects are not seen to any significant degree could point to compensating errors in the analysis but does not distract from the central message of the importance of the bias in the shortwave when accounting for RN(satellite) < RN(tower).

Figure 7b and Table 3 show that Φ(satellite) ≈ 0.90Φ(tower), suggesting a slight compensation for the underspecification of RS↓ through the underspecification of G from the satellite data. However, this evaluation assumes the energy balance to be closed in the tower data (i.e. RN-G =λE+H), which typically is not the case, with λE+H often falling short of RN-G by 20 % (Wilson et al., 2002). Because the causes of this energy imbalance remain controversial (see Foken, 2008, for a review), it is difficult to estimate how much the tower values of λE+H are actually biased and hence the extent to which this bias affects our evaluation. Stoy et al. (2013) recently found a systematic relationship of the surface energy balance closure with landscape heterogeneity over 173 FLUXNET tower sites and reported an energy imbalance of 9–30 % over diverse biomes. In another study, Amiro et al. (2009) found relatively better fulfilment of energy balance closure by averaging data over longer periods. The monthly averages of (AIRS overpass time) 13:30 LT surface energy balance closure of the 30 sites used here (Table 5) show an average energy imbalance of ∼ 20 % (ranging from 8 to 34 %). The errors in the tower data are also believed to be associated with different footprint characteristics for the different instruments used (Lin et al., 2008). For example, RN observations typically have a footprint size of ∼ 10 m2, whilst air properties (e.g. air temperature, humidity) have footprint sizes of > 1 km2. By way of illustration, if, in the worst case, the entire energy imbalance were to be attributed exclusively to λE+H (i.e. RN-G are quantified correctly), then the true midday λE+H could be some 20 % greater (Wilson et al., 2002). As a result, the present bias seen in Table 3 would change to Φ(satellite) ≈ 0.72Φ(tower) , again suggesting RS↓ to be the main source of bias in the satellite retrievals for both RN and Φ.

We have only evaluated the satellite retrievals using data from terrestrial sites, and clearly it would be worthwhile repeating this for the ocean retrievals if possible. We have held back on this evaluation here because of the lack of an extensive network of instantaneous latent and sensible heat flux or radiative flux data over the oceans, although we note that the SEAFLUX project within the Global Energy and Water Experiment (GEWEX) initiative should give rise to such a database in the near future. From the terrestrial evaluation we would argue that the methodology employed here shows promise for specifying both noontime RN and Φ, although the results suggest the need for improvements, particularly in the specification of RS↓. More detailed studies evaluating the representativeness of each tower site footprint in relation to the 1∘ scale within which it is situated could prove useful in this regard as would methods for cloud-proofing the satellite retrievals under persistent cloudy-sky conditions. Similarly, evaluation under extreme conditions (e.g. high altitudes and latitudes) is also required.

We have demonstrated a novel retrieval for midday (13:30 LT) surface net available energy (Φ) by blending the monthly atmospheric and land surface variables of AIRS and MODIS sensors. We have attempted to structure the method such that the Φ retrieval does not depend on any offline calibration. We performed a two-step evaluation of the retrieved values for Φ at some representative surface radiation measurement sites and also over 30 FLUXNET sites from all over the globe.

Current Φ estimates performed well when compared against high spatio-temporal in situ measurements and coarse-spatial-resolution satellite retrievals. Consistent underestimation of RN was noted due to the underestimation of shortwave radiation, and in tropical latitudes the RN (and Φ) agreement was relatively weaker. Combination of high-frequency cloud dynamics and relatively low seasonal variability of RS↓ makes it difficult to accurately model both Φ and RN. One of the key challenges in modelling RN (and Φ) in the tropics is to account for fast-changing cloud cover fraction and cloud optical properties throughout the day.

With the availability of high-spatial-resolution (1–5 km) MODIS day–night optical and thermal data, our present approach could be extended to derive high-spatial-resolution Φ estimates at the global scale. This could be achieved by estimating MODIS-based day–night RN and combining day–night RN with day–night TS observations. At the same time, the current methodology could also be used on high-temporal-frequency observations of geostationary satellites (e.g. GOES and METEOSAT). Once surface heat uptake and heat capacity (through Eq. 10) has been estimated, hourly G and Φ could be determined from hourly RN and TS observations of geostationary satellites by assuming conservation of heat capacity over a particular day. Operational generation of satellite-based Φ product would be a valuable resource for a variety of investigations, such as estimating latent and sensible heat, evaluation of Earth system model outputs and quantifying the land–atmosphere coupling strength. Given that we have resorted to the minimal amount of calibration in deriving Φ, it would appear sensible to adopt a similar philosophy in developing satellite-based schemes for latent and sensible heat fluxes as proposed in M2.

In addition to opportunities to specify large-scale surface heat and water vapour fluxes, the heat capacity estimates made here clearly carry information on variations in terrestrial properties such as surface moisture storage, and we envisage that studies to develop this concept further could prove fruitful, particularly because of the emergence of satellite microwave data against which the results could be compared.