To optimize data applicability and smooth the step from data collection to data utilization a new QA/QC procedure has been developed for the experimental lake facility in Lemming. The workflow is intended to be executed automatically on a daily basis to enable e.g. day-to-day estimates of primary production. The procedure has been developed as a python workflow with a suite of sub-routines. Firstly, data (currently encompassing the observed variables: dissolved oxygen, water temperature and pH) is flagged according to sensor specific ranges. Afterward, flags are set according to information from a digital field log, where technicians during their routinely maintenance schedules on site, records information on the operational state of the experimental facility. Data may for instance be flagged for periods where calibration of sensors is carried out. Subsequently, flags are set for data with odd frequency (i.e. outside the 30 min sampling strategy).
For data passing through the prior flagging steps, data is then flagged according to a two-step approach. In step 1 the value of each recorded observation is evaluated against the standard deviation bounds of a centered, rolling 24 to 48 data-point-window where bounds are stretched by a factor to adjust the degree of conservatism in the flagging. For observations flagged in step 1, a second step then evaluates whether the observed change from the prior un-flagged value (in rate pr. 30 min) to the present flagged value exceeds the maximum rate observed within a 20 data-point-rolling-window again stretched by a factor to adjust the degree of conservatism. All values flagged in step 1, which are not exceeding the adjusted max rate evaluation in step 2 is then un-flagged and passes positively the QA-QC routine.
Example of QA/QC: