The SMART project was presenting at the joint Water Symposium 2014 of the New Zealand Hydrological Society, New Zealand Freshwater Sciences Society and the IPENZ Rivers Group. The Symposium was held at the Marlborough Convention Centre, 24 - 28 November 2014. The theme was “Integration: ‘The Final Frontier’ ~ Whakakotahi te amine rohenga" - integration recognises the continuity of hydrological processes in space and time.In that context we talked about our approach towards a unified hydro-climate metada search across New Zealand.
Acquisition of relevant and useful data for hydrological, meteorological or hydrogeological assessments and research projects in New Zealand’s regions is primarily based on personal communication and reliance on the knowledge of domain specialists. To support the task of identification and subsequently retrieval of available datasets for an area of interest, a new unified method for metadata and data search in New Zealand with a focus on hydro-climate and geo-scientific datasets is presented. The method is implemented within an online accessible search website which searches the many distributed New Zealand based data providers and web portals of scientific and governmental organisations.
The research work undertaken comprises of two parts. First, the publicly accessible abstracts from the website of the Journal of Hydrology New Zealand (NZHS 2014) are analysed. The title, date, authors and abstract text were indexed and where possible a spatial context was derived. Additional keywords were selected for a thesaurus based on occurrence of domain specific terms in the abstract text. Subsequently, metadata records in the New Zealand metadata standard format are created and provided online. Secondly, the web search algorithm distributes the search to a selection of pre-registered geo- and hydro-climate data portals like “LINZ Data Service” (LINZ 2014), “Geodata.govt.nz” (NZGO 2014), “NIWA DC” (NIWA 2014), “Landcare LRIS”(Landcare 2014), “Koordinates” (Koordinates 2014) and “data.govt.nz” (NZ_DIA 2014). The search query can be enriched with related terms from a thesaurus or glossary and spatial context with place names or coordinates. Results are ranked based on spatial and semantic correlations with New Zealand place names register and hydrological terms from the developed glossary.
The slides of the presentation are now publicly accessible. Please click here to view the presentation slides,
Thursday, 23 April 2015
Thursday, 26 March 2015
ZOO-Project WPS Java-API and JGrasstools Java Hydrological Toolbox
The ZOO-Project (http://zoo-project.org/) is a solid Open Geospatial Consortium (OGC) Web Processing Service (WPS - http://www.opengeospatial.org/standards/wps) standard server implementation with an open flexible API that works well with many different programming languages. The Java bindings have never been tested in advanced configurations and complex data types, and to date only implement the minimum necessary interfaces. The JGrasstools project is a modular processing library and its highly annotated nature makes it possible to adapt quite easily to other toolboxes. JGrasstools contains a wide variety of powerful and efficient GIS, hydrology and geomorphological tools and processes, that can be exposed to and used by other libraries and toolkits. One example has been the adaptation to the Geotools Process API. The JGrasstools project, as well as other java based projects (as JTS, Sextante or even Geotools) would benefit greatly from the possibility to be used within a web-enabled WPS execution environment, as well as being integrated with the open standards suite of the OGC. Some time ago Moovida tried integrating the JGrasstools libraries with the ZOO-Project Java binding to expose them as native WPS processes. This would allow them to work inside the ZOO-Project and serve its modules under the WPS standard.
Some Background
The ZOO-Project WPS implementation is a flexible, modular high performance HTTP CGI implementation. ZOO-Kernel is a powerful server-side C Kernel which makes it possible to manage and chain Web services, by loading dynamic libraries and handling them as on-demand Web services. The ZOO Kernel is written in C language, and supports several common programming languages in order to connect to numerous libraries and models (http://zoo-project.org/trac/wiki/ZooWebSite/ZooKernel). The generic ZOO API is basically accessible for every possible programming and web scripting language that can be run under the CGI interface. Main API implementations, the ZOO services, are available for C/C++, Python, JavaScript, PHP, Fortran and Java. Some API bindings are more advanced and complete and make the full ZOO-API (http://zoo-project.org/trac/wiki/ZooWebSite/ZOOAPI/Classes#ZOOAPIClasses) accessible to the ZOO service in the particular programming language (e.g. C, Python or JavaScript). In comparison the the Java API binding only exposes the minimum functionality to run from the ZOO Kernel.
JGrasstools (http://moovida.github.io/jgrasstools/) is a powerful GIS toolkits with functionality reaching from standard geoprocessing algorithms to advanced processing features used in hydrology and geomorphology. JGrasstools is based on a Maven (http://maven.apache.org/) build process, which takes care of dependency resolution and creating the succinct jar packages with the compiled classes. Maven is a defacto standard for managing (source and dependencies) building and deploying (jar packaging, resources, copying, publishing, archiving and installing) Java-based software projects. JGrasstools is also used as a toolbox in the uDig desktop GIS software (http://udig.refractions.net/). If JGrasstools could be exposed via a open standards and interfaces, web-based processing and execution environment (like ZOO-Project provides) it can be widely used in WebGIS deployments and large scale cloud based processing chains.
The next level
Andrea from Moovida said he didn't have enough time to continue developing this idea. He developed a generator which would programmatically scan through the annotated JGrasstools modules and generate respective ZooJavaWps classes per JGrasstools module/method and the corresponding ZOO-Project .zcfg config file. The only struggle I had was getting the CLASSPATH properly set up, as the ZOO-Project is basically an HHTP CGI application which will start a JVM per request. When I picked up on this in preparation for a GSoC proposal, I found that there were a few little botches with the parameter mapping from ZOO Java API into the very nicely annotated JGrasstools methods. So I took one example generated (WPS-ified) JGrasstool process and adjusted the parameter mapping and got it running with ZOO-Project. Additionally I adjusted the JGrasstools Maven config files to download the necessary dependencies in the target folders to copy them collectively in the ZOO-Project Java CLASSPATH.Unfortunately I also didn't have time to drive this further still. However, it is just soooo close really :-) Alternatively a 52North WPS implementation based on the super practical JGrasstools annotations and Andrea's generator would also be relatively straightforward.
Tuesday, 10 February 2015
EODP - An OGC-based Environmental Observation Data Profile
A group around Landcare, NIWA, GNS, HillTop Software and Horizons Regional council worked the last 2 years to develop an interoperable web-based data access and sharing standard. Working group formed as results from Hydrological Society Annual Conference Data Access Workshops. Subsequent meetings and workshops at Horizons regional Council office in Palmerston North even featured visits and input from Dr David Maidment as well as from delegates from BOM and Kisters, Australia. Further workshops were held at GNS in Wairakei and NIWA offices in Auckland and Wellington.
The NEMS-endorsed Environmental Observation Data Profile (EODP) is based on proven open standards of the Open Geospatial Consortium (OGC) and comprises a profile of WFS that allows the discovery of timeseries data associated with monitoring stations. This forms part of a larger project to enhance the interoperability of Environmental Observations throughout New Zealand.
Organisations such as NIWA, Landcare, GNS and Regional councils maintain a range of monitoring stations making environmental observations. These include Climate, Hydrology, Air Quality, Soils, Water Quality, and Marine data.
We wish to make this data available for research, analysis and reporting through open data protocols such as the Sensor Observation Service (SOS), Web Feature Service (WFS), Web Coverage Service (WCS) and Web Mapping Service (WMS). We also want to publish the existance of such data sets and services through online metadata catalogues (CSW).
In order to make use such data services users must first be able to discover their existence and then accurately select the subset of data that is relevant to their needs. The EOPD is focussed on this selection process in essence addressing the Where, What and When questions.
WFS provides a rich filtering capability, but typical WFS sources use a flat property/value schema that makes it difficult to express the many to one relationship of the timeseries measurements taken at a station. The EODP is an Application Schema for WFS that encodes the station metadata as a nesting of SF_SpatialSamplingFeature and OM_Observations using the language of Observations and Measurements to describe the characteristics of the timeseries.
The profile is extended by the use of an external Vocabulary service used to dereference measurement identifiers into familiar timeseries names such as rainfall or temperature.
Alistair Ritchie from Landcare Research did the main formal write-up and published the documentation on GitHub: https://github.com/EODP-NZ/eodp-dev
A implementation testbed is currently underway in joint collaboration with NIWA on the NGMP database at GNS and the Climate Database (CLIDB) at NIWA.
http://portal.smart-project.info/sos-smart/service?service=SOS&version=2.0.0&request=GetDataAvailability&observedProperty=http://vocab.smart-project.info/ngmp/phenomenon/1679
http://portal.smart-project.info/gs-smart/wfs?request=GetFeature&service=WFS&typename=sams:SF_SpatialSamplingFeature
The NEMS-endorsed Environmental Observation Data Profile (EODP) is based on proven open standards of the Open Geospatial Consortium (OGC) and comprises a profile of WFS that allows the discovery of timeseries data associated with monitoring stations. This forms part of a larger project to enhance the interoperability of Environmental Observations throughout New Zealand.
Organisations such as NIWA, Landcare, GNS and Regional councils maintain a range of monitoring stations making environmental observations. These include Climate, Hydrology, Air Quality, Soils, Water Quality, and Marine data.
We wish to make this data available for research, analysis and reporting through open data protocols such as the Sensor Observation Service (SOS), Web Feature Service (WFS), Web Coverage Service (WCS) and Web Mapping Service (WMS). We also want to publish the existance of such data sets and services through online metadata catalogues (CSW).
In order to make use such data services users must first be able to discover their existence and then accurately select the subset of data that is relevant to their needs. The EOPD is focussed on this selection process in essence addressing the Where, What and When questions.
WFS provides a rich filtering capability, but typical WFS sources use a flat property/value schema that makes it difficult to express the many to one relationship of the timeseries measurements taken at a station. The EODP is an Application Schema for WFS that encodes the station metadata as a nesting of SF_SpatialSamplingFeature and OM_Observations using the language of Observations and Measurements to describe the characteristics of the timeseries.
The profile is extended by the use of an external Vocabulary service used to dereference measurement identifiers into familiar timeseries names such as rainfall or temperature.
Alistair Ritchie from Landcare Research did the main formal write-up and published the documentation on GitHub: https://github.com/EODP-NZ/eodp-dev
A implementation testbed is currently underway in joint collaboration with NIWA on the NGMP database at GNS and the Climate Database (CLIDB) at NIWA.
http://portal.smart-project.info/sos-smart/service?service=SOS&version=2.0.0&request=GetDataAvailability&observedProperty=http://vocab.smart-project.info/ngmp/phenomenon/1679
http://portal.smart-project.info/gs-smart/wfs?request=GetFeature&service=WFS&typename=sams:SF_SpatialSamplingFeature
Monday, 24 November 2014
Web-based 3D Data Visualisation for Hydrogeology
When I was in Salzburg/Austria last year at the GI_Forum 2014 conference, I had the chance to present some of my recent experiments with the mapping of OGC interoperable geo-data to X3D interoperable open web 3D scenes and visualisation in the browser. Back in New Zealand later that year I had to present this work to the New Zealand Hydrological Society, too, of course :-)
Characterisation of a hydrogeological setting is a multi-faceted complex task. The assessment of usefulness and quality of relevant data is a major challenge. Statistical analysis and visual exploration of the datasets demand practical support by computer applications. Although a variety of software for this purpose is freely available nowadays, they require a good understanding of the technology or programming language for application in complex hydrogeological settings. Thus, integrated proprietary software products are often used to analyse and particularly provide high-quality visualisation of the system. However, these software tools are typically desktop programs with a strict licensing scheme and a limited extensibility and lack of interoperability with other applications.
We present an open and free to use web-based (platform independent) framework to enable retrieval, exploration and visualisation of hydro-climate time series data as well as three-dimensional geological information via a web browser. How distributed data and processing services can be linked to prepare an on-demand 3D visualisation of geological and hydrological data is demonstrated. A flexible toolbox design enables extensibility via open standards.
The method developed is applied to a case study area presented (s. figure), which is the Horowhenua district in the Manawatu-Wanganui region. Available datasets of 3D geology, hydrology and hydrogeology are combined and serve as example data for demonstrating the framework.
The slides of the presentations are now made accessible here (click here).
The full ISI-indexed conference paper is available here (click here).
Characterisation of a hydrogeological setting is a multi-faceted complex task. The assessment of usefulness and quality of relevant data is a major challenge. Statistical analysis and visual exploration of the datasets demand practical support by computer applications. Although a variety of software for this purpose is freely available nowadays, they require a good understanding of the technology or programming language for application in complex hydrogeological settings. Thus, integrated proprietary software products are often used to analyse and particularly provide high-quality visualisation of the system. However, these software tools are typically desktop programs with a strict licensing scheme and a limited extensibility and lack of interoperability with other applications.
We present an open and free to use web-based (platform independent) framework to enable retrieval, exploration and visualisation of hydro-climate time series data as well as three-dimensional geological information via a web browser. How distributed data and processing services can be linked to prepare an on-demand 3D visualisation of geological and hydrological data is demonstrated. A flexible toolbox design enables extensibility via open standards.
The method developed is applied to a case study area presented (s. figure), which is the Horowhenua district in the Manawatu-Wanganui region. Available datasets of 3D geology, hydrology and hydrogeology are combined and serve as example data for demonstrating the framework.
The slides of the presentations are now made accessible here (click here).
The full ISI-indexed conference paper is available here (click here).
Monday, 14 April 2014
2014 NASA International Space Apps Challenge - TETRIS GroupSpace Apps Challenge 2014 - TETRIS Space Suit HUD demo with a Raspberry
As part of the 2014 NASA International Space Apps Challenge I joined an Auckland AUT University Textile Lab group to support their space suit concept idea with some cool wireless video and sensor streaming based on Raspberry Pi.
TETRIS (Terran Expeditionary Technologically Radical Instrumental Suit) is designing a suit, aimed for use by astronauts in space, which will make the astronaut’s work easier and safer to perform. It achieves this by embedding many of the new technologies being released in the recent years into the suit itself in order to: - monitor the astronaut’s life signs and condition via biometric sensors, such as a galvanic sensor for emotions, pulse sensor for the heart rate - allow simple integrated communication via a shoulder mounted camera and microphone - display biometric information to user via projected HUD - take simple electrical readings (when testing/diagnosing equipment) via embedded measurement tools (threaded conductive fabric in gloves) These hardware devices all interface with a RaspberryPi, being used as a central control unit in our design, as well the transceiver for all wireless communications between the astronaut and their remote operations control room.
Link to the 2014 Space Apps project website
The Raspberry Pi video camera live capture was streamed into the remote browser with an overlay of the HUD mockup.
Sources here https://github.com/TetrisGroup/spaceapps
TETRIS (Terran Expeditionary Technologically Radical Instrumental Suit) is designing a suit, aimed for use by astronauts in space, which will make the astronaut’s work easier and safer to perform. It achieves this by embedding many of the new technologies being released in the recent years into the suit itself in order to: - monitor the astronaut’s life signs and condition via biometric sensors, such as a galvanic sensor for emotions, pulse sensor for the heart rate - allow simple integrated communication via a shoulder mounted camera and microphone - display biometric information to user via projected HUD - take simple electrical readings (when testing/diagnosing equipment) via embedded measurement tools (threaded conductive fabric in gloves) These hardware devices all interface with a RaspberryPi, being used as a central control unit in our design, as well the transceiver for all wireless communications between the astronaut and their remote operations control room.
Link to the 2014 Space Apps project website
The Raspberry Pi video camera live capture was streamed into the remote browser with an overlay of the HUD mockup.
Sources here https://github.com/TetrisGroup/spaceapps
Sunday, 15 September 2013
IAH 2013, Perth, Australia
This presentation provides a great overview of my PhD research.
(presented at IAH 2013, Perth, Australia)
Thursday, 15 August 2013
PLACE 2013 - Maori GIS Conference
I had the great opportunity to present my PhD work at the 2013 Maori GIS conference in Auckland. In order to provide a seamless spatial and multi-purpose view of collected groundwater related datasets, the SMART project joins forces to establish a valuable basis for groundwater analysis and decision support tools (Kmoch et al., 2012, Klug et al., 2011). One of the project’s objectives is to build a web-based data and knowledge portal and attached three-dimensional web visualisation tool according to OGC and ISO compliant standards (OGC, 2012).
Iwi/Māori have a well-recognised relationship with the natural environment which spans many centuries and is the result of interaction and adaptation with native flora and fauna of Aotearoa/New Zealand. Integral to this relationship is water which sustains life and is a taonga (treasure) with significant cultural and physical dimension. This is reflected through the on-going desire of many iwi/Māori groups to have a role in the way water is managed in New Zealand to ensure its sustainable utilisation moving forward (Kawharu, 2002). The development of scientific research tools and models that incorporate mātauranga Māori (Māori knowledge) and te reo Māori (the Māori language) are also beneficial to iwi/Māori resource policymakers, planners and decision makers.
GNS has been collecting and compiling Māori terms on hydrology, geology and geothermal phenomena to, amongst other things, explore the contribution that traditional indigenous knowledge can make to the research outcomes and model development by identifying the cultural significance of groundwater and the associated cultural links with surface water (Tipa and Tierney, 2003, Boast, 1991). The potential benefits of creating research tools that utilise te reo Maori and mātauranga Maori (within government institutional settings) includes generating increased uptake in and familiarity with te reo Māori and exposure of te reo Māori as a minority language to broader audiences. Further research could identify gaps in the dual knowledge systems (either the western scientific knowledge paradigm, or mātauranga Māori) that could be explored as an outcome of this combined research (De Bres, 2008).
To support te reo Māori and mātauranga Māori within the SMART portal web mapping and catalogue application, we evaluate a multi-language concept to incorporate semantic web methodologies to map and connect English and Māori terms and descriptions of presented natural phenomena as well as metadata and descriptive text within the application (Lutz et al., 2009). Beside a language template system for in-application-navigation use, a vocabulary web service is demonstrated to access content the thesauri, classification schemes, taxonomies, metadata and other types of controlled vocabulary and to document, link and merge concepts/terms to be with other spatial and non-spatial data (Antoine Isaac and Ed Summers, 2008). Further intentions include also a possibility to upload and geolocate orally passed on knowledge in the regard of ancient place information.
Evaluation of the lexicon’s effectiveness will be measured in part by its ability to be applied successfully to the SMART portal web mapping and cataloguing application.
References:
ANTOINE ISAAC, V. U. A. & ED SUMMERS, L. O. C. 2008. SKOS Simple Knowledge Organization System Primer [Online]. Available: http://www.w3.org/TR/2008/WD-skos-primer-20080221/.
BOAST, R. P. 1991. The legal framework for geothermal resources : a historical study : a report to the Waitangi Tribunal. Wai 153. Wellington: Waitangi Tribunal, 1991.
DE BRES, J. 2008. Planning for tolerability: promoting positive attitudes and behaviours towards the Māori Language among non-Māori New Zealanders. Ph. D (Linguistics). Wellington: Victoria University of Wellington.
KAWHARU, M. 2002. Whenua : managing our resources, Auckland: Reed, 2002.
KLUG, H., DAUGHNEY, C., VERHAGEN, F., WESTERHOFF, R. & WARD, N. D. 2011. Freshwater resources management: Starting SMART characterization of New Zealand’s aquifers. Earthzine. http://www.earthzine.org/2011/12/13/freshwater-resources-management-starting-smart-characterization-of-new-zealands-aquifers/.
KMOCH, A., KLUG, H. & WHITE, P. 2012. Freshwater resources management: first steps towards the characterisation of New Zealand's aquifers. GI_Forum 2012: Geovisualization, Society and Learning, 2012 2012 Salzburg. Car, A., Griesebner, G., Strobl, J., 376-385.
LUTZ, M., J.SPRADO, E.KLIEN, C.SCHUBERT & CHRIST, I. 2009. Overcoming semantic heterogeneity in spatial data infrastructures. Computers &Geosciences, 35, 739-752.
OGC 2012. OGC Standards. http://www.opengeospatial.org/standards: Open Geospatial Consortium.
TIPA, G. & TIERNEY, L. 2003. A Cultural health index for streams and waterways : indicators for recognising and expressing Māori values. New Zealand. Ministry for the Environment.
WHITE, P. A. 2006. Some Future Directions in Hydrology. Journal of Hydrology (NZ), 45, 63-68
Iwi/Māori have a well-recognised relationship with the natural environment which spans many centuries and is the result of interaction and adaptation with native flora and fauna of Aotearoa/New Zealand. Integral to this relationship is water which sustains life and is a taonga (treasure) with significant cultural and physical dimension. This is reflected through the on-going desire of many iwi/Māori groups to have a role in the way water is managed in New Zealand to ensure its sustainable utilisation moving forward (Kawharu, 2002). The development of scientific research tools and models that incorporate mātauranga Māori (Māori knowledge) and te reo Māori (the Māori language) are also beneficial to iwi/Māori resource policymakers, planners and decision makers.
GNS has been collecting and compiling Māori terms on hydrology, geology and geothermal phenomena to, amongst other things, explore the contribution that traditional indigenous knowledge can make to the research outcomes and model development by identifying the cultural significance of groundwater and the associated cultural links with surface water (Tipa and Tierney, 2003, Boast, 1991). The potential benefits of creating research tools that utilise te reo Maori and mātauranga Maori (within government institutional settings) includes generating increased uptake in and familiarity with te reo Māori and exposure of te reo Māori as a minority language to broader audiences. Further research could identify gaps in the dual knowledge systems (either the western scientific knowledge paradigm, or mātauranga Māori) that could be explored as an outcome of this combined research (De Bres, 2008).
To support te reo Māori and mātauranga Māori within the SMART portal web mapping and catalogue application, we evaluate a multi-language concept to incorporate semantic web methodologies to map and connect English and Māori terms and descriptions of presented natural phenomena as well as metadata and descriptive text within the application (Lutz et al., 2009). Beside a language template system for in-application-navigation use, a vocabulary web service is demonstrated to access content the thesauri, classification schemes, taxonomies, metadata and other types of controlled vocabulary and to document, link and merge concepts/terms to be with other spatial and non-spatial data (Antoine Isaac and Ed Summers, 2008). Further intentions include also a possibility to upload and geolocate orally passed on knowledge in the regard of ancient place information.
Evaluation of the lexicon’s effectiveness will be measured in part by its ability to be applied successfully to the SMART portal web mapping and cataloguing application.
References:
ANTOINE ISAAC, V. U. A. & ED SUMMERS, L. O. C. 2008. SKOS Simple Knowledge Organization System Primer [Online]. Available: http://www.w3.org/TR/2008/WD-skos-primer-20080221/.
BOAST, R. P. 1991. The legal framework for geothermal resources : a historical study : a report to the Waitangi Tribunal. Wai 153. Wellington: Waitangi Tribunal, 1991.
DE BRES, J. 2008. Planning for tolerability: promoting positive attitudes and behaviours towards the Māori Language among non-Māori New Zealanders. Ph. D (Linguistics). Wellington: Victoria University of Wellington.
KAWHARU, M. 2002. Whenua : managing our resources, Auckland: Reed, 2002.
KLUG, H., DAUGHNEY, C., VERHAGEN, F., WESTERHOFF, R. & WARD, N. D. 2011. Freshwater resources management: Starting SMART characterization of New Zealand’s aquifers. Earthzine. http://www.earthzine.org/2011/12/13/freshwater-resources-management-starting-smart-characterization-of-new-zealands-aquifers/.
KMOCH, A., KLUG, H. & WHITE, P. 2012. Freshwater resources management: first steps towards the characterisation of New Zealand's aquifers. GI_Forum 2012: Geovisualization, Society and Learning, 2012 2012 Salzburg. Car, A., Griesebner, G., Strobl, J., 376-385.
LUTZ, M., J.SPRADO, E.KLIEN, C.SCHUBERT & CHRIST, I. 2009. Overcoming semantic heterogeneity in spatial data infrastructures. Computers &Geosciences, 35, 739-752.
OGC 2012. OGC Standards. http://www.opengeospatial.org/standards: Open Geospatial Consortium.
TIPA, G. & TIERNEY, L. 2003. A Cultural health index for streams and waterways : indicators for recognising and expressing Māori values. New Zealand. Ministry for the Environment.
WHITE, P. A. 2006. Some Future Directions in Hydrology. Journal of Hydrology (NZ), 45, 63-68
Subscribe to:
Posts (Atom)