Water-quality sensors are exposed to changing environments and extreme weather conditions and thus are prone to errors, including failure. These technical errors make data unreliable and untrustworthy and affect performance of any subsequent data analysis.
ACEMS researchers, led by Priyanga Dilini Talagala, have proposed a feature based procedure, named oddwater, for detecting technical outliers in water-quality data derived from in situ sensors. The procedure incorporates expert domain knowledge and statistical theory to identify outliers due to technical malfunction of the sensors. The oddwater procedure was evaluated using two data sets obtained from in situ sensors in rivers flowing into the Great Barrier Reef lagoon. We showed that our oddwater procedure, with carefully
selected data transformation methods derived from data features, can greatly assist in increasing the performance of a range of existing outlier detection algorithms.
The oddwater algorithm is expected to expand into space and time so that it can deal with the spatiotemporal correlation structure along branching river networks. This will in turn provide a fundamental step-change in scientific understanding of the spatiotemporal dynamics of water quality in rivers and their networks.
The work is based on the collaborative research project carried out with the Queensland University of Technology, Monash University and the Queensland Department of Environment and Science, Great Barrier Reef Catchment Loads Monitoring Program.
- The proposed framework is implemented in the open source R package oddwater: https://github.com/pridiltal/oddwater
- The article was published by Water Resources Research Journal and it can be found at https://doi.org/10.1029/2019WR024906
