WARREDOC International Winter School on Data Rich Hydrology 2019

The "Data Rich Hydrology" Winter School 2019 took place in beautiful Colombella, Perugia (Italy). It was jointly organised by the Water Resources Research and Documentation Center (WARREDOC) and UNESCO World Water Assessment Program (WWAP). The WARREDOC was established at the Università per Stranieri di Perugia (UNISTRAPG) since 1985 - developing research, advanced training and scientific communication in the field of water, environment and disaster risk management.

http://warredoc.unistrapg.it/en/events/2019-winter-school/

The days were organised into a serious of lectures and lab sessions and food and accommodation were provided all at the location of the Villa Colombella, that was an extraordinary experience. We were completely immersed in this place students and lecturers altogether.

The program encompassed mainly lectures of absolute high scientific standard and well presented by the experienced and well-known lecturers.

- The Era of Data Rich Hydrology, 1st keynote lecture, by Prof. Rafael L. Bras

Prof. Bras is one of the forefathers of hydrology (Google Scholar). He gave us a history lesson of conceptual, numerical and later computational hydrology and modelling of catchments. He concluded with the outlook of what we as young hydrologists should keep striving towards to improve understanding and modelling of the hydrological cycle.

- The WWDR and SDG 6 Synthesis Report, 2nd keynote lecture, by Prof. Stefan Uhlenbrook, also head of UNESCO WWAP

- Remote sensing and data assimilation in hydrology by Prof. Fabio Castelli

- Hydrologic modelling in a data rich world by Prof. Prof Riccardo Rigon

- Citizen science and big data in hydrology by Prof. Fernando Nardi

- Beyond traditional extreme value theory: lessons learned from rainfall and hurricane intensity by Prof. Marco Marani

More topics got covered by further renowned professors, researchers and practitioners in hydrological and hydraulic modelling:

- Groundwater hydrology and hydrological process mechanics
- The water-food-energy nexus
- Modelling scaling properties of precipitation fields
- Hydrologic measurements and novel observation technologies
- Drones in Hydrology (lecture & hands on)
- Hydrological risk assessment: Return period and probability of failure
- Advances in the space-time analysis of rainfall extremes
- Data poor vs. data rich cases for flood hazard (lecture & hands on)
- Distributed Data quality and urban flood modelling uncertainty
- Stream flow measurements: ground and satellite observations
- Remote sensing data and tools to foster inland water monitoring and flood modeling

I also had the pleasure to get interviewed by research fellow and PhD student Francisco Pena, who does a radio show on Disaster Risk Reduction. We had a great chat about our ideas and views on the topics and lectures during this Winter School on Hydrology and did some brainstorming:

http://www.radiophonica.com/podcast/13941 (link to the radio show)

If you like to check out Francisco's pages: https://www.linkedin.com/in/franciscope%C3%B1a/ (LinkedIn) and and https://twitter.com/FebronioPena (Twitter)

Interoperable exchange of groundwater data with OGC GroundWaterML2

WaterML2 has become a well-known synonym for internationally standardised hydrological data exchange, in particular for government agencies and research institutes across North America, Europe, Australia and New Zealand. Technically, WaterML2 is becoming a suite of standards actively promoted and endorsed by the World Meteorological Organisation (WMO), more details: http://www.whycos.org/wordpress/?page_id=929)

- WaterML 2.0: Part 1 - Time series of Observations

- WaterML 2.0: Part 2 - Ratings, Gaugings and Sections

- WaterML 2.0: Part 3 - Surface Hydrology Features (aka HY_Features)

- WaterML 2.0: Part 4 - aka GroundWaterML 2 (GWML2) Data Exchange for Groundwater Features (including wells, springs, borelogs and well constructions)

Now there is a scientific publication that explains the GWML2 standard, its development and application in hydrogeology in detail:

"GWML2 is an international standard for the online exchange of groundwater data that addresses the problem of data heterogeneity. This problem makes groundwater data hard to find and use because the data are diversely structured and fragmented into numerous data silos. Overcoming data heterogeneity requires a common data format; however, until the development of GWML2, an appropriate international standard has been lacking. GWML2 represents key hydrogeological entities such as aquifers and water wells, as well as related measurements and groundwater flows. It is developed and tested by an international consortium of groundwater data providers from North America, Europe, and Australasia, and facilitates many forms of data exchange, information representation, and the development of online web portals and tools."

Brodaric, B., Boisvert, E., Chery, L. et al. (2018) Enabling global exchange of groundwater data: GroundWaterML2 (GWML2). Hydrogeology Journal. https://doi.org/10.1007/s10040-018-1747-9

Related links and information:

https://link.springer.com/article/10.1007%2Fs10040-018-1747-9

https://www.researchgate.net/publication/323914313_Enabling_global_exchange_of_groundwater_data_GroundWaterML2_GWML2


WaterML2 Part 4: GroundWaterML2 (GWML2) http://www.opengeospatial.org/standards/gwml2

Groundwater SWG http://www.opengeospatial.org/projects/groups/groundwaterswg

OGC GroundWaterML 2 – GW2IE FINAL REPORT https://portal.opengeospatial.org/files/?artifact_id=64688

Hydrological Society Conference - Web-based real-time processing of environmental measurements

This case study was presented as poster abstract.

Kmoch, A., White, P. A., & Klug, H. (2015). Sensor Observation Service and web-based real-time Processing of environmental Measurements in the Upper Rangitaiki Catchment (Poster). In The NZ Hydrological Society Conference 2015, 26th November, in Hamilton, New Zealand

Abstract:

Environmental assessments naturally depend on field observations and technological advancements. , such as tTelemetry, allow the automated collection, transmission and processing of these measurements. However, modelling of natural processes is typically a complex challenge and involves applying expertise of scientists as well as a host of data preparation steps (White, 2006, White et al., 2003).
In addition, automation of model execution with the most recent observation data is dependent on the integration of the data collection, storage and processing elements (Klug and Kmoch, 2014). This paper demonstrates a system that integrates a Sensor Observation Service (SOS) that includinges field observations and internet-based environmental data with a rainfall recharge model that allows near-real time calculation of rainfall recharge in the Upper Rangitaiki catchment, Bay of Plenty region.
The SOS specification is an Open Geospatial Consortium (OGC) standard for the open and standardised integration of environmental sensors into an internet-based environmental data infrastructure (Klug and Kmoch, 2015, Klug, Kmoch, Reichel, 2015).

Figure 1. Process of data flow from field site sensors, to SOS data service to a simulation model process

Results:

• We showed that it is possible to link the collected data directly to a simple rainfall recharge model (Figure 1)
• The low cost sensor and circuit board instrumentation collects data and forwards them to the field computer in 10-minute intervals via robust, low power, ZigBee wireless protocol
• The field computer running a standard Linux operating system, transfers observation data in 10 minute intervals via a 3G mobile data connection to an online SOS server.
• From the service the observations are available in a standardised open format.
• A website can access the raw data from the SOS server and plotted data points within 5-10 minutes of field measurement
• A rainfall recharge model runs with the latest data points from the online SOS server.

References:

Klug, H., & Kmoch, A. (2014). A SMART groundwater portal: An OGC web services orchestration framework for hydrology to improve data access and visualisation in New Zealand. Computers & Geosciences, 69(0), 78–86. http://dx.doi.org/10.1016/j.cageo.2014.04.016

Klug, H., Kmoch, A. (2015). Operationalizing environmental indicators for real time multi-purpose decision making and action support. Ecological Modelling, 295, 66-74. http://dx.doi.org/10.1016/j.ecolmodel.2014.04.009.

White, P. A. (2006). Some Future Directions in Hydrology. Journal of Hydrology (NZ), 45(2), 63–68.

White, P. A., Hong, Y.-S., Murray, D. L., Scott, D. M., & Thorpe, H. R. (2003). Evaluation of regional models of rainfall recharge to groundwater by comparison with lysimeter measurements, Canterbury, New Zealand. Journal of Hydrology (NZ), 42(1), 39–64.

Klug, H., Kmoch, A., & Reichel, S. (2015). Adjusting the Frequency of Automated Phosphorus Measurements to Environmental Conditions. GI_Forum 2015 - Journal for Geographic Information Science - Geospatial Minds for Society, 1, 590–599. http://doi.org/10.1553/giscience2015s590

AGILE 2016 - SensorWeb Semantics on MQTT for responsive Rainfall Recharge Modelling

Integrating Wireless Sensor Networks (WSNs) and spatial data web services is becoming common in ecological applications. However, WSNs were developed in application domains with different sensor and user types, and often with their own low-level metadata semantics, data format and communication protocols. The sensor web enablement initiative (SWE) within the Open Geospatial Consortium (OGC) has released a set of open standards for interoperable interface specifications and (meta) data encodings for the real time integration of sensors and sensor networks into a web services architecture.
Such XML-based web services exhibit disadvantages in terms of payload and connectivity in low-bandwidth low energy unreliable networks, such as remote 3G uplinks. Monitoring stations deliver frequent measurements in real-time, but dynamic implementation of measurement frequencies, adapted to certain environmental conditions, are rarely implemented. We describe a responsive integrated hydrological monitoring prototype to calculate rainfall recharge for water management purposes.
When rainfall is observed, a threshold event triggers a reconfiguration task for the soil moisture sensors, using asynchronous, push-based communication implemented with an MQTT queue. A Sensor Planning Service commits that request via MQTT into the wireless sensor network, and updates the measurement frequency of the target sensors to gain higher resolution for the vertical soil water infiltration.
The system integrates a Sensor Observation Service (SOS) including field observations and internet-based environmental data with a rainfall recharge model that allows near-real time calculation of rainfall recharge in the Upper Rangitaiki catchment, Bay of Plenty region in New Zealand.

Figure 1: Setup and location of the sensor field site, central North Island, New Zealand

The prototype site comprises a main station conducting comprehensive measurements of meteorological, hydrological and pedological parameters. For the wireless data transmission within the local site installation XBee-PRO modules from the Digi Company  ZigBee IEEE 802.15.4 protocol are used. The main station receives continuous sensor measurements from the attached sensor units, and acts as the gateway to the online SOS and SPS services by providing the communication channel from the local sensor network to the web-enabled data management infrastructure.

The field site has been established in the Upper Rangitaiki catchment (Figure 1) and comprises a field computer (Raspberry Pi) with a direct internet link (GPRS/3G) and a sensor board (Waspmote) that has 12 typical meteorological, hydrological and pedological sensors attached (i.e., wind speed, wind direction, rainfall, 1x groundwater probe, 5x temperature and 3x soil moisture). The Raspberry Pi and Waspmote can be monitored and reprogrammed from an online server.

Figure 2: Raw sensor series visualized in a website from a SOS query.

The site setup allows scaling up to a multitude of low cost, low energy sensor stations throughout the catchment, with only one field computer that serves as data logger for backup. The observations were available in a standardized open format. The website accessed the raw data from the SOS server and plotted data points within 5-10 minutes of field measurement. This website was easily accessible via browsers and smartphones (Figure 2).

The paper was was presented at the 19th AGILE International Conference on Geographic Information Science, 15th of June, in  Helsinki, Finlkand.

Kmoch, A., Klug, H., White, P., & Reichel, S. (2016). SensorWeb Semantics on MQTT for responsive Rainfall Recharge Modelling. In 19th AGILE International Conference on Geographic Information Science. Helsinki.

ResearchGate milestone and GSoC finale ahead

What an exciting start of this week:

Google is re-organising itself into Alphabet, Google Summer of Code 2015 announced 'Pencils down' date in two weeks and ResearchGate informed me about reaching 200 publication downloads.

Well, being in the 3rd year of my PhD the number of publications, views and citations might not be outstanding but it is continuous progress along the early researcher path and a form of acknowledgement.

The GSoC project work with GeoTreliis and Azavea is also great opportunity to get more involved with Big Data technologies like Apache Spark and Cassandra and cloud technologies. The support from the project team, notably Rob and Chris, but also vibe on the GeoTrellis Gitter channel is fantastic.

Although it's time to wrap up GSoC in the next weeks, it is also the starting point of getting these new software development insights applied to my research in the SMART aquifer characterisation (SAC) programme, where I develop tools and web platforms for the SMART data portal. If all goes well, the SMART data portal will be overhauled end of this year and my PhD completed early next year :-)

IAH 2013, Perth, Australia

This presentation provides a great overview of my PhD research.
(presented at IAH 2013, Perth, Australia)

Starting to play with spatial-temporal data

Thinking  the big picture is a different thing to actually implement it :-) Well, who doesn't know that. Having some hydrological time-series (groundwater levels) in place in a sensor observation service (SOS), the hydrogeological all-in-one-wonder-portal is going to get a glance of the next level :-p
The last weeks I started to play around with the R environment for statistical computing and visualization (The R Project). 52°North developed a neat R toolkit to access and digest SOS time-series - sos4R.

It is well documented and pretty easy to connect to a SOS server, and query observations. So for the fun of it and to demonstrate the general feasibilty, I quickly queried the groundwater levels of the Horowhenua area in New Zealand, where I got some sample data (courtesy by the regional council).

With the R sos4R, fields and akima packages from the CRAN R packages archive I (quite coarsely) interpolated the groundwater surfaces for the years 1991-2009 and put the images together as an animated gif (meters above mean sea level over time).

Discussion

I am aware of the total uselessness of this particualr way presenting :-) No years, the scale changes slightly, and the exact spatial extent and north orientiation are not reliable :-p

Nevertheless, for just playing around, this was a motivating simple first shot to easily visualise changes over time.

I would like to play around with the gstat and spacetime R package, integrate a more sophisticated script as a 52°North WPS process and have those things happening automagically in the interwebz.