Making environmental research articles discoverable with OGC catalogue service

I am very happy to announce that our new article "Enhancing Location-related Hydrogeological Knowledge" has been published in the ISPRS International Jounral of Geo-information.

Kmoch, A.; Uuemaa, E.; Klug, H.; Cameron, S.G. (2018) Enhancing Location-Related Hydrogeological Knowledge. ISPRS Int. J. Geo-Inf. , 7, 132, http://www.mdpi.com/2220-9964/7/4/132

In a joint study by the University of Tartu (Estonia), the Institute of Geological and Nuclear Sciences (New Zealand) and the University of Salzburg (Austria) more than 5,800 scientific articles from three environmental research journals were digitised and analysed. In addition, a geographical search method was developed to identify the location of a studied area.

We use Georeferencing of scientific journal articles and text-mining in order to provide spatial search capabilities for environmental research. Journal articles are made discoverable through spatial queries. We propose that journal publishers should provide these capabilities on their platforms. This would allow everyone to search for journal articles for their desired regions of interest and it would really well complement existing search functionalities with keywords etc.

The press release from Tartu University explains really nicely how our results enable geographic search for scientific papers through text mining and geocoding; and how to better find environmental research articles via location.

http://researchinestonia.eu/2018/04/17/enabling-geographic-search-for-scientific-papers-through-text-mining-and-geocoding-how-to-better-find-environmental-research-articles-via-location/

Estonian version of the press release: https://novaator.err.ee/821096/kuidas-otsida-teadustoid-geograafilise-piirkonna-jargi


(This article belongs to the Special Issue Place-Based Research in GIScience and Geoinformatics)
MDPI OpenAccess: http://www.mdpi.com/2220-9964/7/4/132

ResearchGate: https://www.researchgate.net/publication/323990934_Enhancing_Location-Related_Hydrogeological_Knowledge

Hydrological Society Conference - Web-based real-time processing of environmental measurements

This case study was presented as poster abstract.

Kmoch, A., White, P. A., & Klug, H. (2015). Sensor Observation Service and web-based real-time Processing of environmental Measurements in the Upper Rangitaiki Catchment (Poster). In The NZ Hydrological Society Conference 2015, 26th November, in Hamilton, New Zealand

Abstract:

Environmental assessments naturally depend on field observations and technological advancements. , such as tTelemetry, allow the automated collection, transmission and processing of these measurements. However, modelling of natural processes is typically a complex challenge and involves applying expertise of scientists as well as a host of data preparation steps (White, 2006, White et al., 2003).
In addition, automation of model execution with the most recent observation data is dependent on the integration of the data collection, storage and processing elements (Klug and Kmoch, 2014). This paper demonstrates a system that integrates a Sensor Observation Service (SOS) that includinges field observations and internet-based environmental data with a rainfall recharge model that allows near-real time calculation of rainfall recharge in the Upper Rangitaiki catchment, Bay of Plenty region.
The SOS specification is an Open Geospatial Consortium (OGC) standard for the open and standardised integration of environmental sensors into an internet-based environmental data infrastructure (Klug and Kmoch, 2015, Klug, Kmoch, Reichel, 2015).

Figure 1. Process of data flow from field site sensors, to SOS data service to a simulation model process

Results:

• We showed that it is possible to link the collected data directly to a simple rainfall recharge model (Figure 1)
• The low cost sensor and circuit board instrumentation collects data and forwards them to the field computer in 10-minute intervals via robust, low power, ZigBee wireless protocol
• The field computer running a standard Linux operating system, transfers observation data in 10 minute intervals via a 3G mobile data connection to an online SOS server.
• From the service the observations are available in a standardised open format.
• A website can access the raw data from the SOS server and plotted data points within 5-10 minutes of field measurement
• A rainfall recharge model runs with the latest data points from the online SOS server.

References:

Klug, H., & Kmoch, A. (2014). A SMART groundwater portal: An OGC web services orchestration framework for hydrology to improve data access and visualisation in New Zealand. Computers & Geosciences, 69(0), 78–86. http://dx.doi.org/10.1016/j.cageo.2014.04.016

Klug, H., Kmoch, A. (2015). Operationalizing environmental indicators for real time multi-purpose decision making and action support. Ecological Modelling, 295, 66-74. http://dx.doi.org/10.1016/j.ecolmodel.2014.04.009.

White, P. A. (2006). Some Future Directions in Hydrology. Journal of Hydrology (NZ), 45(2), 63–68.

White, P. A., Hong, Y.-S., Murray, D. L., Scott, D. M., & Thorpe, H. R. (2003). Evaluation of regional models of rainfall recharge to groundwater by comparison with lysimeter measurements, Canterbury, New Zealand. Journal of Hydrology (NZ), 42(1), 39–64.

Klug, H., Kmoch, A., & Reichel, S. (2015). Adjusting the Frequency of Automated Phosphorus Measurements to Environmental Conditions. GI_Forum 2015 - Journal for Geographic Information Science - Geospatial Minds for Society, 1, 590–599. http://doi.org/10.1553/giscience2015s590

AGILE 2016 - SensorWeb Semantics on MQTT for responsive Rainfall Recharge Modelling

Integrating Wireless Sensor Networks (WSNs) and spatial data web services is becoming common in ecological applications. However, WSNs were developed in application domains with different sensor and user types, and often with their own low-level metadata semantics, data format and communication protocols. The sensor web enablement initiative (SWE) within the Open Geospatial Consortium (OGC) has released a set of open standards for interoperable interface specifications and (meta) data encodings for the real time integration of sensors and sensor networks into a web services architecture.
Such XML-based web services exhibit disadvantages in terms of payload and connectivity in low-bandwidth low energy unreliable networks, such as remote 3G uplinks. Monitoring stations deliver frequent measurements in real-time, but dynamic implementation of measurement frequencies, adapted to certain environmental conditions, are rarely implemented. We describe a responsive integrated hydrological monitoring prototype to calculate rainfall recharge for water management purposes.
When rainfall is observed, a threshold event triggers a reconfiguration task for the soil moisture sensors, using asynchronous, push-based communication implemented with an MQTT queue. A Sensor Planning Service commits that request via MQTT into the wireless sensor network, and updates the measurement frequency of the target sensors to gain higher resolution for the vertical soil water infiltration.
The system integrates a Sensor Observation Service (SOS) including field observations and internet-based environmental data with a rainfall recharge model that allows near-real time calculation of rainfall recharge in the Upper Rangitaiki catchment, Bay of Plenty region in New Zealand.

Figure 1: Setup and location of the sensor field site, central North Island, New Zealand

The prototype site comprises a main station conducting comprehensive measurements of meteorological, hydrological and pedological parameters. For the wireless data transmission within the local site installation XBee-PRO modules from the Digi Company  ZigBee IEEE 802.15.4 protocol are used. The main station receives continuous sensor measurements from the attached sensor units, and acts as the gateway to the online SOS and SPS services by providing the communication channel from the local sensor network to the web-enabled data management infrastructure.

The field site has been established in the Upper Rangitaiki catchment (Figure 1) and comprises a field computer (Raspberry Pi) with a direct internet link (GPRS/3G) and a sensor board (Waspmote) that has 12 typical meteorological, hydrological and pedological sensors attached (i.e., wind speed, wind direction, rainfall, 1x groundwater probe, 5x temperature and 3x soil moisture). The Raspberry Pi and Waspmote can be monitored and reprogrammed from an online server.

Figure 2: Raw sensor series visualized in a website from a SOS query.

The site setup allows scaling up to a multitude of low cost, low energy sensor stations throughout the catchment, with only one field computer that serves as data logger for backup. The observations were available in a standardized open format. The website accessed the raw data from the SOS server and plotted data points within 5-10 minutes of field measurement. This website was easily accessible via browsers and smartphones (Figure 2).

The paper was was presented at the 19th AGILE International Conference on Geographic Information Science, 15th of June, in  Helsinki, Finlkand.

Kmoch, A., Klug, H., White, P., & Reichel, S. (2016). SensorWeb Semantics on MQTT for responsive Rainfall Recharge Modelling. In 19th AGILE International Conference on Geographic Information Science. Helsinki.

SMART Groundwater Portal Dev going full "Cloud"

What a concise quote from Erik Dietrich, founder of DaedTech LLC:

Software developers demand the ability to work effectively from anywhere.  They have attained a coolness factor, and demand for them is so high that there is no need for them to guard their source code like squirrels preparing for winter.  GitHub is a good idea because it effectively captured what software developers really want and offered it to them pretty flawlessly.  GitHub is a zeitgeist that is taking over the world precisely because software developers are taking over the world and software developers really like GitHub. (source)

Although, I work at a research institute which is one half a commercial consultancy with IP to protect, on-going international research collaborations, and governmental research funding require us to be flexible, open and accessible.  I work as a research scientist/analyst programmer in a science department, not in the IT or applications department, and thus, IT infrastructure interaction in commercial entities is "challenging" - for the scientists as well as for the IT folks.

In our current project we embraced the Zeitgeist now, too. For our geodata portal development and deployment processes, we adopted following paradigm:

Google Cloud Platform, Compute and Container Engine with Kubernetes as the our computational platform.

Google Drive, Google Docs and Sheets for assets, functional and implementation specifications development, user stories and use cases.

GitHub as our distributed version control system, which allows us to collaborate, yet, keep contributions transparent and easily and publicly traceable.

Trello eventually serves as our workflow board, for sprints, keeping links of specs, repos and other soft information together. I believe, that we could have done everything in GitHub, but Google Docs and Trello provide gently mechanisms to also invite non-technical folks to contribute. And actually, what we really want is this, right?

Dreamteam FOSSGIS NZ eResearch 2013 Ignite Talk

In July 2013, NeSI, the New Zealand E-Science Infrastructure, organised lightning talks in Ignite format (20 slides, 15 seconds each) at the eResearch NZ conference in Christchurch.

I had the chance to present a little piece on the dream team combination of FOSSGIS (free and open source software for Geographic Information Systems) and OGC (Open Geospatial Consortium) standards, and how they are great enablers for Science and Education. Stumbled over this presi in my archives and thought I could share it:


A rough transcript :-)

1. Hi, I’m Alex and I’m a Geoinformatics PhD student at AUT University and I’m working in a New Zealand groundwater research project with GNS Science in Taupo.

2. Today I’d like to tell you the story of my journey. I think it is a great story and I want to share it with you - and being in my first year I needed to get my head around stuff.

3. So my part in the so called SMART aquifer characterisation project is about a groundwater web-portal for New Zealand. I need to collect, analyse, mash-up, visualise and share again a lot of different groundwater related data sets – and everything shall happen in the web

4. Little did I know before …The diversity of available content, formats and technologies is quite overwhelming – So I had to figure out where to start.

5. So my story is basically about discovery –a step by step exploration – to build something for the better of humanity (well, definitely for New Zealand)

6. And what helped me in this rather iterative and incremental process? Open Source software, especially Free and Open Source Software for Geoinformation Systems (FOSSGIS)

7. It worked for me for two reasons: If it didn’t perform, I could have a look inside and fiddle around with it to make it fit my demands – well, or I’ll just try something else, you know, freedom of choice.

8. And if you even want to give back to the community, you often can propose and contribute enhancements to make such software better – however all things optional

9. Alright, second thing, data sharing - Let’s have a look what’s happening out there in the vast expanses(expansiveness) of the internet.

10. So there is the Open Geospatial Consortium, the OGC, is an international consortium of 482 companies, government agencies and universities participating in an open consensus-based process to develop publicly available standards that "geo-enable" the Web, and thus fostering interoperability. Interoperability is key.

11. They create specifications for data formats and web services and so forth, to interact, integrate and communicate with each other. You can download the full specifications for free and again, you could also participate.

12. So now, does that OGC stuff work with open source? Bamm, there’s a huge open source software ecosystem supporting OGC standards –  go to osgeo.org or 52north.org and you will likely find everything you need to start.

13. And that’s actually really high quality software, They are often even the reference implementations for particular OGC standards - and you still can do with it whatever you want.

14. Ok, I don’t want to pull out what’s happening internationally, you know, Aussie, North America or Europe … Well, I learned New Zealand is just really awesome, too

15. So NZ has a Geospatial Office that published the NZ Geospatial strategy in 2007 which actually says “USE OGC STANDARDS” and make environmental and other spatial datasets available.

16. And boom, agencies, research institutes and even Kiwi-based commercial companies open up massive NZ datasets to the public – accessible through OGC interfaces and open source – OPEN STANDARDS work

17. So data and technology was available. I could start with small steps, be open, be agile and flexible. I really felt enabled to discover, grow and share in return.

18. So how can you facilitate that wealth of knowledge? I heard in New Zealand there is a GIS Masters programme available, is it covering OGC and Open Source software? I don’t know actually, but I’ve been asked to co-author an “Open GIS” module in another international GIS Masters (I could make it open access perhaps)

19. Well, having OGC and FOSSGIS at hand – you can conduct as well as support top-notch science and research, and also foster education of kids at school, students at uni even citizens at home with the same set of tools … communicate science and knowledge about this country and its beautiful nature that deserves protection and sustainable development

20. So when do you start? Thanks everyone, acknowledgements to GNS, AUT, eResearch NZ, MBIE/MSI

Environment Southland Information Management Conference

Regional councils and government agencies are increasingly under pressure to resolve data questions and discover how best to acquire, manage, collate, analyse, report and disseminate data, while managing  the associated costs. Steering organisations through these complex issues requires a solid understanding of what technologies are available and the information demands of the future *(source).

I had the great opportunity to speak at the Environment Southland Information Management Conference in Invercargill. It was a great event, well organised and very informative. I believe I could contribute my part to the line up and fill a few more gaps in the whole picture.
This was not a business as usual conference, it was obvious that the speakers took it serious to cater their presentations to the needs of the stakeholders. And with 70 attendees from regional and central government, as well as visitors from research and industry.

It was great to see the emerging patterns around NZ and similar approaches to a holistic, comprehensive and modern data strategy. If you are interested, this is a link to programme, and please see below for my slides. Watch the talk on Youtube.

A Spatial Data Infrastructure Approach for the Characterization of New Zealand's Groundwater Systems

I was very happy when I got informed that our latest research article was published in "Transactions in GIS". While being embedded in our New Zealand SMART Aquifer Characterisation programme, it was a great joint effort which results went beyond SMART.

Kmoch, A., Klug, H., Ritchie, A. B. H., Schmidt, J. and White, P. A. (2015), A Spatial Data Infrastructure Approach for the Characterization of New Zealand's Groundwater Systems. Transactions in GIS. doi: 10.1111/tgis.12171

It explains technical and methodological approaches of web groundwater data infrastructure and uses NIWA's CLIDB and GNS's NGMP as examples and describes the long way from stakeholder interaction, workshops and meetings towards an NZ data standard for the National Environmental Monitoring Standards (NEMS) framework - the Environmental Observation Data Profile (EODP) as data access and transfer blueprint for NEMS.

Wiley Online Library "Transactions in GIS" Journal

Alexander Kmoch on ResearchGate.

The technical work with colleagues, collaborators and partners from New Zealand research institutes (CRIs) like GNS, NIWA, Landcare Research, New Zealand regional councils like Horizons and Waikato (WRC), Hawke's Bay (HBRC) and and Bay of Plenty (BOPRC) regional councils which culminated in a draft OGC profile and is being incorporated into national standards in New Zealand, GitHub EODP and are based on the work over the last two years including a great review paper on OGC standards for groundwater,  customising the 52°North SOS server to interchangeably encode WaterML2 time-series data along O&M2 through a Google Summer of Code (GSoC) programme, and then customising the SOS server database access to demonstrate it as an adaptor on legacy databases of national significance, in specific an "NGMP/GGW-SOS" and a "CLIDB-SOS" demo through a NZ eResearch programme.

An approach towards a unified hydro-climate metadata search across New Zealand

The SMART project was presenting at the joint Water Symposium 2014 of the New Zealand Hydrological Society, New Zealand Freshwater Sciences Society and the IPENZ Rivers Group.  The Symposium was held at the Marlborough Convention Centre, 24 - 28 November 2014. The theme was “Integration: ‘The Final Frontier’ ~ Whakakotahi te amine rohenga" - integration recognises the continuity of hydrological processes in space and time.In that context we talked about our approach towards a unified hydro-climate metada search across New Zealand.

Acquisition of relevant and useful data for hydrological, meteorological or hydrogeological assessments and research projects in New Zealand’s regions is primarily based on personal communication and reliance on the knowledge of domain specialists. To support the task of identification and subsequently retrieval of available datasets for an area of interest, a new unified method for metadata and data search in New Zealand with a focus on hydro-climate and geo-scientific datasets is presented. The method is implemented within an online accessible search website which searches the many distributed New Zealand based data providers and web portals of scientific and governmental organisations.

The research work undertaken comprises of two parts. First, the publicly accessible abstracts from the website of the Journal of Hydrology New Zealand (NZHS 2014) are analysed. The title, date, authors and abstract text were indexed and where possible a spatial context was derived. Additional keywords were selected for a thesaurus based on occurrence of domain specific terms in the abstract text. Subsequently, metadata records in the New Zealand metadata standard format are created and provided online. Secondly, the web search algorithm distributes the search to a selection of pre-registered geo- and hydro-climate data portals like “LINZ Data Service” (LINZ 2014), “Geodata.govt.nz” (NZGO 2014), “NIWA DC” (NIWA 2014), “Landcare LRIS”(Landcare 2014), “Koordinates” (Koordinates 2014) and “data.govt.nz” (NZ_DIA 2014). The search query can be enriched with related terms from a thesaurus or glossary and spatial context with place names or coordinates. Results are ranked based on spatial and semantic correlations with New Zealand place names register and hydrological terms from the developed glossary.

The slides of the presentation are now publicly accessible. Please click here to view the presentation slides,

IAH 2013, Perth, Australia

This presentation provides a great overview of my PhD research.
(presented at IAH 2013, Perth, Australia)

Software used in NZ and NZ SDI

Software used in NZ and NZ SDI

Following table could be compiled from the "Groundwater workshop" at the NZ Hydrological Society Conference in Dec. 2011 and several more meetings with Regional Councils, e.g. Hawke's Bay (HBRC), Waikato (EW) and Manuwatu-Wanganui (Horizons).

TIDEDA™	DOS/Windows-based database and reporting application for hydrology related time series data, uses special file formats, prequel to Hilltop, now maintained be NIWA http://www.niwa.co.nz/software/tideda-time-dependent-data
Hilltop Data Tamer™	Windows-based database and reporting application suite for hydrology related time series data, uses special file formats, server system which provides REST-style XML access, can import HydroTel™ data, OGC SOS 2.0 interface planned http://www.hilltop.co.nz/
Hydstra™	Database and reporting application for hydrologic time series data, Australian-based company was acquired by German-based company Kisters around 2003 that promotes Wiski and provides data migration path http://www.kisters.com.au/english/html/au/homepage.html
Kisters Wiski™	Full-fledged data management and reporting system for hydrological data and time series, Kisters is working on WaterML2.0 and took part in the OGC surface water interoperability experiment http://www.kisters.net/wiski.html Kisters OGC news http://kiwis.kisters.de/KiWIS/
HydroTel™	Telemetry / Sensor system from New Zealand based iQuest company (Hamilton), which was acquired by Kisters in 2007 http://www.iquest.co.nz/telemetry-systems-monitoring.php Kisters and HydroTel interoperability tests http://www.iquest.co.nz/environmental-monitoring-blog/index.php/2011/10/05/kisters-using-new-ogc-standards
Oracle™ and Spatial/Locator™	Multiple regional councils and agencies use Oracle database with and without its spatial extensions and implemented different, independent data models to store hydro(geo)logical data, bore, wells, springs etc., e.g. NGMP/GGW, EW bore database

The establishment of a national spatial data infrastructure in NZ is gaining momentum, further info at www.geospatial.govt.nz


Land Information New Zealand (LINZ) is a New Zealand government department responsible for land titles, geodetic and cadastral survey systems, topographic information, hydrographic information, managing Crown property and a variety of other functions. LINZ’s purpose is to:

• maintain and build confidence in property rights in land and geographic information
• encourage land information markets to develop and mature

LINZ established the New Zealand Geospatial Office (NZGO) that is the coordinating body for implementing the New Zealand Geospatial Strategy. Activities carried out by the NZGO include:

• promoting the discovery, access and interoperability of geospatial information, particularly through facilitation of a national Spatial Data Infrastructure (SDI)
• advising government on policy for geospatial information
• facilitating communication between members of the geospatial community

NZGO adheres to the widely accepted OGC standards and established them as national standards. They developed a SDI cookbook, to which organisations and institutions that want to share geo-spatial information shall comply to. Find out more...