Sensitivity Analysis of the GOTM Model

posted in: Lake model blog | 0
  1. Introduction

Sensitivity analysis is used in modelling to identify which parameters and forcing data variables that a model is sensitive to (Bueche & Vetter, 2014; Pianosi et al., 2016). This procedure is beneficial because it: 1) helps in the understanding of how the model works and 2) aids in the interpretation of results (Pannell, 1997). Prior to the use of an environmental model it is critical that a parameter sensitivity analysis, and model calibration and validation are carried out (Jørgensen, 1995; Refsgaard et al., 2007). For example, Bueche & Vetter (2014) carried out a sensitivity analysis on the meteorological input data on the lake physical model DYRESM and were able to identify errors in the modelled water temperature and hypothesize why the errors occurred based on the sensitivity analysis. Understanding the sensitivity of model outputs to changes in model parameters can inform the selection of a range of parameters for use in calibration to improve the model performance.

Carrying out model sensitivity analysis, calibration and validation are key to demonstrating the value of an applied model. Sensitivity analysis quantifies the effect that the key ecological parameters in the model setup have on the output data (Schladow & Hamilton, 1997). Investigating the effects of gradual changes in the forcing meteorological data also allows investigation of changes in, for example, changes in local weather. This can give insight into a lake’s resilience and its ability to respond to these changes (Bueche & Vetter, 2014). Patterns seen within the sensitivity analysis can be used to show which types of lakes exhibit similar behaviours, for example Bruce et al. (2018) reported that lakes with high transparency had deeper thermocline depths. Analysing model sensitivity to changes in meteorological forcing data allows the identification of in-lake variations because of meteorological changes and also it gives an insight to the possible source of error within the model as a result of uncertainties in meteorological datasets (Bueche & Vetter, 2014).  Most importantly sensitivity analysis of model parameters helps to understand the changes that will result from changes in model parameters during calibration and to identify which parameters should be included. In addition, some model parameters are included to allow for differences between measured meteorological data and actual meteorological conditions on the lake. For example, wind speed is usually not measured directly on the lake, and data might come from a meteorological station on the lake shore or a nearby airport, so a scaling parameter is included in the model to allow for differences between the sites. In this blog we report on a sensitivity analysis undertaken for the GOTM physical model(Burchard et al., 2006), the physical model in use in PROGNOS, for PROGNOS site Lough Feeagh in Ireland.


2.0 Methods:

For the sensitivity analysis, the focus was on the ability of the model to accurately simulate the measured temperature profiles at different times of the annual cycle for Feeagh. Two differing separate sensitivity analyses were carried out: 1. on the parameters within the model and 2. on the effect of changes in each of the meteorological forcing data variables.

2.1 Model Parameters

The model was first calibrated for the period 2009-2012 including a 1-year warm-up period. Then each of the parameters were adjusted ±12.5% and ±25% and the model was simulated for the year 2014. Each input was changed individually while keeping all the other variables constant (Table 1). The simulations were compared using one example month from each season; January (winter), April (spring), July (summer) and October (autumn), calculating an average thermal profile and then calculating the difference throughout the water column compared to the original model run (Figure 1). This allowed identification of where in the water column the effect was seen and the impact it had on the lake at different times of the year.

Table 1 Adjustments of model parameters for sensitivity analysis. Calibrate column refers to the values of the parameters after the model has been calibrated.

    Calibrated + ++
shf_factor Adjust surface heat flux x 0.75 x 0.875 1.053 x 1.125 x 1.25
wind_factor Adjust wind speed x 0.75 x 0.875 0.938 x 1.125 x 1.25
swr_factor Adjust shortwave radiation x 0.75 x 0.875 1.259 x 1.125 x 1.25
k_min Minimum turbulent kinetic energy x 0.75 x 0.875 2.57e-06 x 1.125 x 1.25
A Extinction for non-visible fraction of light x 0.75 x 0.875 0.999 x 1.125 x 1.25
g1 e-folding depth for non-visible fraction x 0.75 x 0.875 0.603 x 1.125 x 1.25
g2 e-folding depth for visible fraction x 0.75 x 0.875 3.417 x 1.125 x 1.25



Average temperature profiles for each of the four months. b. Heat map of the temperature profiles for Lough Feeagh for 2014 with the months used for the average temperature profile highlighted in the relevant colours.


2.2 Meteorological inputs

An analysis of the model’s sensitivity to changes meteorological forcing data was carried with a stepwise modification of each meteorological parameter: air temperature, dewpoint temperature, wind speed, precipitation, incoming short-wave radiation, mean sea level pressure, and cloud cover. Cloud cover is used as a proxy for long-wave radiation. Air temperature, dewpoint temperature, mean sea level pressure and river water temperature were adjusted by adding 0.5 X standard deviation and 1 X standard deviation to the measured values to represent a shift in the values while keeping the same distribution (Table 1). Where values which included zeros and were highly variable, they were changed by using a multiplication factor to adjust the range of the data (e.g. wind speed, precipitation, shortwave radiation and cloud cover). The method for comparison was as in 2.1.

Table 2 AT, air temperature; DT, dewpoint temperature; WS, wind speed; PC, precipitation; SWR, short wave radiation; CC, cloud cover; MSLP, mean sea level pressure. The number of symbols is representing the intensity of the variation (+: increase, –: decrease).

Parameter Variations of parameter
  + ++
AT (°C) -4.35 -2.175 +2.175 +4.35
DT (°C) -4.26 -2.13 +2.13 +4.26
WS (m s-1) x 0.8 x 0.9 x 1.1 x 1.2
PC (mm) x 0.95 x 0.98 x 1.02 x 1.05
SWR (W m-2) x 0.9 x 0.95 x 1.05 x 1.10
CC x 0.95 x 0.98 x 1.02 x 1.05
MSLP (mbar) -13 -6.5 +6.5 +13
WT (°C) -4 -2 +2 +4


3.0 Results

3.1 Model parameters

For the month of January the shf_factor, surface heat flux factor, was the factor that had the largest impact on the modelled water temperature profile for Feeagh, as measured by the mean error over all depths (Table 3). A decrease in shf_factor led to an increase in water temperature and vice versa with a range of -0.27 to 0.4°C. The effect was consistent throughout the water column. Changes in the range of values for both swr_factor and wind_factor resulted in a minor change in simulated water temperature. In contrast, changes to k_min, the minimum turbulent kinetic energy, had no effect on the simulated water temperature profiles in January.

During April and in contrast to January, both the swr_factor and wind_factor had a strong influence on the simulated water temperature profile when they were adjusted over the selected ranges (Table 3). For the swr_factor, the effect was largely towards the surface (0-25m) and in the lowest depths (35-44m) areas. Increases in the wind_factor resulted in a mean error that was larger in the bottom layers (0.8-1.4°C); when the wind factor used in simulations was lower (0.75, 0.875), there was larger mean error in the middle of the water column (-1.4 to -0.8°C). The shf_factor had a slight effect on the model performance throughout the water column (-0.2 to 0.3°C) and k_min slightly increased the error in the lower depths (± 0.2°C) (Table 3).

In the month of July, when the lake was stratified, changes to k_min and wind_factor increased the error in the lower half of the lake. The wind_factor had a very strong effect: when it was scaled down with factors of 0.750 and 0.875, the error ranged from -2.1 to -0.9°C for the depth range of 10-44m (Figure 2). When the wind_factor was scaled up by 1.125 and 1.25, the errors ranged from 0.5 to 2.1°C but at lower depths of 15-44m (Figure 2). While changes to shf_factor had a much stronger effect towards the surface ±0.9°C at 0-10m and the error decreased largely further down the water column. There was a similar effect for the swr_factor, where the error was ±1.1°C from 0-12m but then decreased to 0 at a depth of 40m.

Error in modelled water temperature throughout the water column for the monthly averaged temperature profile (January, April, July and October) for changes to the wind factor (wind_factor).


For simulations for October, the error associated with changes to the wind_factor was greatest below 20m, with a range from -1.6 to 0.9°C (Table 3). The magnitude of this error was larger for a decrease in wind in October (Table 4.1). For k_min, the major changes in simulated temperatures were below 30m. Reduced k_min values resulted in lower temperatures (-0.9 to -0.4°C) and the opposite was seen for higher k_min values (0.3 to 0.6°C). For swr_factor there was a smaller change in magnitude of error than in July of ±0.5°C, while for the shf_factor there was a stronger effect seen to a deeper level of around 30m of ±0.9°C and a smaller error at 30-44m of ±0.3°C (Table 3).

Table 3 Mean error between modelled and observed lake profile temperature for each of the different changes to parameters. (The symbols correspond to the changes in the parameters in Table 1).

  Mean Error (°C)
  January   April
Parameters + ++   + ++
shf_factor 0.40 0.18 -0.15 -0.27   0.26 0.12 -0.10 -0.19
swr_factor -0.08 -0.04 0.04 0.08   -0.42 -0.21 0.21 0.41
wind_factor 0.09 0.04 -0.03 -0.06   -0.46 -0.27 0.37 0.52
k_min 0.00 0.00 0.00 0.00   -0.05 -0.02 0.02 0.05
  July   October
  + ++   + ++
shf_factor 0.45 0.20 -0.17 -0.32   0.71 0.32 -0.28 -0.52
swr_factor -0.63 -0.30 0.29 0.55   -0.47 -0.23 0.21 0.41
wind_factor -1.37 -0.77 0.71 1.01   -0.68 -0.31 0.14 0.10
k_min -0.33 -0.16 0.15 0.28   -0.28 -0.13 0.10 0.18


3.2 Meteorological variables

For simulations for the month of January, air and dewpoint temperature, cloud cover, and inflow temperatures were the input variables which affected simulated water column temperatures, while wind speed, precipitation and short-wave radiation had little to no effect (Table 4, January).  This effect was observed throughout the water column.  The mean error for simulations in which air temperature was varied ranged from -2.64 to 2.72°C (Figure 3). In contrast, during the other time periods all seven variables influenced the water temperature profile except for mean sea level pressure and precipitation. Changes to mean sea level pressure and precipitation had little to no effect on model performance throughout the year and throughout the water column (Table 4).

Error in modelled water temperature throughout the water column for the monthly averaged temperature profile for changes to the air temperature and dewpoint temperature.


Table 4. Mean error in water temperature (°C) for the full water column profile (average for 50 depths between 0-45 m) for changes in air temperature and dewpoint temperature (AT+DT), wind speed (WS), precipitation (PC), short wave radiation (SWR), cloud cover (CC), mean sea level pressure (MSLP) and river inflow temperature (WT). The number of symbols represents the intensity of the variation and corresponds to Table 3.5 in the Materials and Methods section.

  Mean Error (°C)
  January   April
Parameters + ++   + ++
AT + DT -2.64 -1.33 1.35 2.72   -2.73 -1.21 1.31 2.72
WS 0.07 0.03 -0.03 -0.05   -0.39 -0.23 0.29 0.48
PC 0.00 0.00 0.00 0.00   0.00 0.00 0.00 0.00
SWR -0.03 -0.02 0.02 0.03   -0.16 -0.08 0.08 0.17
CC -0.11 -0.04 -0.81 -0.80   -0.12 -0.05 -0.92 -0.90
MSLP 0.00 0.00 0.00 0.00   -0.01 -0.01 0.01 0.02
WT -1.43 -0.71 0.69 1.36   -0.93 -0.48 0.46 0.90
  July   October
  + ++   + ++
AT + DT -2.46 -1.06 0.99 2.11   -3.22 -1.54 1.38 2.70
WS -1.16 -0.63 0.58 0.90   -0.54 -0.25 0.13 0.11
PC 0.00 0.00 0.00 0.00   0.00 0.00 0.00 0.00
SWR -0.24 -0.12 0.12 0.23   -0.18 -0.09 0.09 0.17
CC -0.10 -0.04 -0.78 -0.76   -0.11 -0.05 -0.85 -0.83
MSLP -0.04 -0.02 0.02 0.03   -0.01 0.00 0.01 0.01
WT -0.58 -0.30 0.30 0.60   -0.44 -0.23 0.22 0.43


4.0 Discussion

4.1 Model parameters

It is necessary to include parameters in the model that 1. account for the inaccuracies in the measured meteorological data but 2. also are governing parameters in the model which are not measured and are not possible to measure such as minimum turbulent kinetic energy required for mixing. Optimisation of such parameters allows the user to apply values which are crucial in accurately simulating the lake temperature profile. Quantifying the effect that changes to each parameter has on model output provides information on how the model reacts to differing parameters.

In GOTM, wind_factor is a scaling factor that either increases or decreases the wind speed. Within lakes, wind speed affects the amount of turbulent kinetic energy that is generated within the lake which facilitates mixing within the water column. For this analysis, this parameter was found to have the largest effect on the simulated temperature profile in magnitude, and on the whole temperature profile and during the different stages of the year. The swr_factor is a scaling factor for the amount of incoming shortwave radiation to the lake. This influences the amount of heat energy that gets absorbed by the water. It was found to have a strong impact on the upper layers of the water column.

The parameter k_min is used in calculating the distribution of heat to the lower levels in the lake. Calibrating the k_min parameter within the model can help the model to achieve a better fit to observed water temperatures particularly in the lower depths of the water column.

4.2 Meteorological variables

In the current analysis, air temperature and wind speed were the meteorological parameters which had the strongest effect on the model simulations.  Similar results have been reported from other studies (Hadley et al., 2014; Magee & Wu, 2017). Air temperature/dewpoint temperature affects the upper mixed layer most because temperature is a strong driver of surface water temperature. Cloud cover influences the amount of solar radiation the lake receives during the night.  This is likely why when cloud cover was increased in the simulations, there was a reduction in simulated water temperature. The effect of increasing cloud cover gave similar results to decreasing shortwave radiation. Increases of 2% and 5% had a large impact on the simulated temperatures (temperature decreases of ~-1°C) which highlights the importance of ensuring accurate measurement of cloud cover.

A sensitivity analysis is a useful tool to further understand the dynamics which the lake is sensitive to. It can help to explain the behaviour seen within the model as well by understanding its response to changes in meteorological values. The value of the model parameters is shown especially how they can be used to calibrate the model to achieve a more accurate water temperature simulation.


5.0 References

Burchard, H., Bolding, K., Kühn, W., Meister, A., Neumann, T., & Umlauf, L. (2006). Description of a flexible and extendable physical-biogeochemical model system for the water column. Journal of Marine Systems, 61, 180–211.

Hadley, K. R., Paterson, A. M., Stainsby, E. A., Michelutti, N., Yao, H., Rusak, J. A., … Smol, J. P. (2014). Climate warming alters thermal stability but not stratification phenology in a small north-temperate lake. Hydrological Processes, 28(26), 6309–6319.

Magee, M. R., & Wu, C. H. (2017). Response of water temperatures and stratification to changing climate in three lakes with different morphometry. Hydrology and Earth System Sciences, 21(12), 6253–6274.