Friday 8 December 2017
Thursday 30 November 2017
2017 fieldtrip to soontaga fluxnet tower
The Soontaga station is an EDI Fluxtower installation of the Department of Geography at the University of Tartu, Estonia. We visited one nice winter day. Later I built some scripts for the field researchers and physical geographers to check and export the data.
Source code: https://github.com/LandscapeGeoinformatics/fluxnet_daily_png
Tuesday 14 November 2017
A geeky intro into Fortran
On the 9th of November I organised another local coding meetup. This time we got an introduction Introduction to Fortran and Forth in order to understand some of the origins of programming.
Our resident CompSci member Benson prepared and presented the materials and gave us a glance onto these two arcane, almost ancient programming languages. Yet Fortran is still very much alive and used in a variety of high-performance computing (HPC) scenarios.
You can use command line and a simple text editor (vi, emacs, nano, gedit, atom, notepad++, kate). For the faint-hearted Atom ( https://atom.io/ ) has most of the features we needed for this session, is open and cross platform.
Links to Fortran language standard and tutorial:
While Benson prepared a preconfigured VM for us, you can try out Fortran with a Gnu Fortran GCC installed by yourself, and maybe even Eclipse Fortran IDE:
Links to Forth language standard and tutorial:
The materials are also collated on my GitHub account again. Happy coding :-)
Our resident CompSci member Benson prepared and presented the materials and gave us a glance onto these two arcane, almost ancient programming languages. Yet Fortran is still very much alive and used in a variety of high-performance computing (HPC) scenarios.
You can use command line and a simple text editor (vi, emacs, nano, gedit, atom, notepad++, kate). For the faint-hearted Atom ( https://atom.io/ ) has most of the features we needed for this session, is open and cross platform.
Links to Fortran language standard and tutorial:
- Standards committee: https://wg5-fortran.org/
- Tutorial: https://en.wikibooks.org/wiki/Fortran_77_Tutorial
While Benson prepared a preconfigured VM for us, you can try out Fortran with a Gnu Fortran GCC installed by yourself, and maybe even Eclipse Fortran IDE:
Links to Forth language standard and tutorial:
- Standard: https://forth-standard.org/
- Tutorial: http://www.complang.tuwien.ac.at/forth/gforth/Docs-html/
- Example simple interpreter: https://github.com/howerj/libforth
- Example interpreter for educational purposes: https://github.com/sayon/forthress
The materials are also collated on my GitHub account again. Happy coding :-)
Friday 10 November 2017
Copernicus info session November 2017 in Tallinn, Estonia
On the 6th of November 2017, several Geoinformatics staff member of the Department of Geography and our international MSc in Geography/Geoinformatics students of the University of Tartu jointly attended the Copernicus Info Session in Tallinn.
The event was held in Tallinn in the conference rooms of the Building of Ministries as the eleventh in a series of national Copernicus events of the European Commission. We had several international visitors as well national estonian officials. For example we had the chance to meet with members of the Estonian Landboard, ESTCube and other innovative satellite data processing companies, and even the Estonian National Library.
One really interesting presentation was summarising the Sentinel mission really well:
After the event about the European Copernicus programme finished we used the "field trip" to visit the Tallinn old town, and we we did a nice walk in the event.
Online references to the event:
http://www.copernicus.eu/events/copernicus-training-and-information-session-estonia
and
http://copernicus.eu/infosession-estonia
Tuesday 17 October 2017
A Python Pandas Coding Meetup
On October, 12th, I organised a local meetup event at the Department of Geography. We had around 15 attendees, both from Tartu industry and university students.
In this meetup we played with the Jupyter Notebook (http://jupyter.org/) and did some data wrangling with Python, Pandas (http://pandas.pydata.org/) and co., some data analysis and visualisation.
In this meetup we played with the Jupyter Notebook (http://jupyter.org/) and did some data wrangling with Python, Pandas (http://pandas.pydata.org/) and co., some data analysis and visualisation.
I presented on how we can use Miniconda, an encapsulated versatile virtual python environment installer, that works under the hood of the big Anaconda python distribution. Miniconda is basically a mini version of Anaconda that includes only the conda package manager and its dependencies. We used a Python version 3.4 because some of the libraries/tools we wanted to work with supposedly only are supported up to Python 3.4. Here it becomes obvious how practical virtual environments can be. They help you to keep various Python versions around without messing up your system.
After we had set-up our local Python working environment, I introduced the Jupyter Notebook. The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.
I prepared a variety of resources that we worked through in the meetup. I shared the collated tutorials and course materials on my GitHub account, The repository contains a curated collection of Jupyter Notebooks of introductory materials about programming in Python in general, the Pandas data manipulation and visualisation toolkit, and the Geopandas geospatial library.
This repository aims at different degrees of experience with the Python programming language and Python data manipultation and visualisation libraries. Each folder contains a README.md (for the the online repository) and README.notebook.md (readme as readable Notebook) file that list the contents and some details for each of the 3 big modules here:
- 01 - Introduction to Python and the Jupyter Notebook
- 02 - Introduction to the Pandas library
- 03 - Introduction to the Geopandas library
In order to get started, download the full Meetup-Repositiry from https://github.com/allixender/meetup-notes/archive/master.zip and extract it to a LOCAL folder. Go back into the console/commandline prompt and change into that folder. There you should then start the Jupyter notebook.
Friday 29 September 2017
A few Impressions of the ESTGIS conference of the Estonian Society for Geoinformatics
On the 29. September 2017 geographers and GIS specialists of the Estonian Landboard, Estonian GIS companies, and Estonian Universities - especially from University of Tartu Department of Geography and environmental Technologies, Tartu University of Life Sciences and Tallinn Technical University - joined at the annual gathering of the Estonian Society of Geoinformatics (EstGIS). Our Head of Chair of Geoinformatics at Dept. of Geography, Tartu, Prof Tönu Oja, is also the chairman of the boards of EstGIS.
 |
| The conference took place in the Vooremaa district North of Tartu. |
The 2-day conference took place in a small village north of Tartu in small family-run holiday park. Embedded in idyllic country-side the place had accommodation, sauna of course, and a large conferencing room.
 |
| "Think Spatially" says the big flyer outside, welcoming the guests. |
 |
| Beautifully embedded in the landscape. |
The conference talks were mixed in English and Estonian. There were a few international guests and a few students, but mainly local (Estonian) GIS practitioners.
More details about the article: https://allixender.blogspot.com/2018/04/making-environmental-research-articles.html
Friday 18 August 2017
PhD Dissertation publicly accssible
My dissertation to achieve the degree 'Doctor of Philosophy' at the Auckland University of Technology (AUT) has the title:
A Context-based Groundwater Data Infrastructure
Online access via the AUT Online Library: http://hdl.handle.net/10292/10740
Abstract
Groundwater bodies are among the most important and valuable natural resources available, but at the same time they are also the least understood. To better understand the hydrological state of the environment and groundwater dynamics, data sets and measurements need to be made available and accessible to scientists, planners, and stakeholders to allow for proper decision making support. Information exchange via the internet has become faster, but at the same time data sets remain scattered both in location and formats. Present research in hydrogeology and freshwater resources management can be significantly supported and accelerated by relating, reusing and combining existing data sets, models and simulations in a streamlined, computer-aided and networked fashion and yield more new and reproducible insights.
In this thesis Design Science Research, Grounded Theory and Case Studies are applied in order to design a spatial data infrastructure that addresses the full data life cycle in the context of hydrogeology in New Zealand. This 'Hydrogeology Infrastructure' design was successfully implemented and evaluated via a networked and open standards-based prototype. Formerly disconnected and distributed data sets may now, for the first time, be used for hydrogeological data analysis, visualisation and modelling within one data portal.
My Supervisor(s)
Many thanks go out again to my supervisors who supported me continuously.
Assoc Prof Jacqueline Whalley (AUT, New Zealand); Assoc Prof Hermann Klug (Z_GIS, Salzburg University, Austria); Prof Philip Sallis (AUT, New Zealand)
The interested reader can find the publication also via ResearchGate:
https://www.researchgate.net/publication/319164590_A_Context-based_Groundwater_Data_Infrastructure
A Context-based Groundwater Data Infrastructure
Online access via the AUT Online Library: http://hdl.handle.net/10292/10740
Abstract
Groundwater bodies are among the most important and valuable natural resources available, but at the same time they are also the least understood. To better understand the hydrological state of the environment and groundwater dynamics, data sets and measurements need to be made available and accessible to scientists, planners, and stakeholders to allow for proper decision making support. Information exchange via the internet has become faster, but at the same time data sets remain scattered both in location and formats. Present research in hydrogeology and freshwater resources management can be significantly supported and accelerated by relating, reusing and combining existing data sets, models and simulations in a streamlined, computer-aided and networked fashion and yield more new and reproducible insights.
In this thesis Design Science Research, Grounded Theory and Case Studies are applied in order to design a spatial data infrastructure that addresses the full data life cycle in the context of hydrogeology in New Zealand. This 'Hydrogeology Infrastructure' design was successfully implemented and evaluated via a networked and open standards-based prototype. Formerly disconnected and distributed data sets may now, for the first time, be used for hydrogeological data analysis, visualisation and modelling within one data portal.
My Supervisor(s)
Many thanks go out again to my supervisors who supported me continuously.
Assoc Prof Jacqueline Whalley (AUT, New Zealand); Assoc Prof Hermann Klug (Z_GIS, Salzburg University, Austria); Prof Philip Sallis (AUT, New Zealand)
The interested reader can find the publication also via ResearchGate:
https://www.researchgate.net/publication/319164590_A_Context-based_Groundwater_Data_Infrastructure
A Context-based #GroundwaterData Infrastructure #Hydrogeology #GIScience # SpatialDatahttps://t.co/QMpSKTpsp2
— Scholarly Commons (@AUT_SC) 17 August 2017
Wednesday 16 August 2017
2017 LA Conference on diffuse pollution
It was an awkward conference - not knowing anyone, but also, really experiencing America first hand, in LA. Presenting went well, with barely connecting to the audience, a comparatively anonymous dinner.
UCLA has a pompous campus, it was nice weather, dry and warm. Comparing it to other conferences like EGU, or ANZ Hydrological conferences, Hydrogeology Association and Digital Earth summits, that conference was a strange experience.
Saturday 22 July 2017
OGC OWS GeoJSON
We are implementing OWC GeoJSON in the SAC groundwater portal as part of the SAC research program. In the latest OGC OWS Context GeoJSON Encoding Standard 14-055r2 we came across a few ambiguities that we couldn't solve for ourselves immediately.
I am summarising those subsequently here. The examples in the OGC Schemas repository are mostly still based on the former OWS-10 testbed results. Hut the OWC working group announced to update the official samples.
specReference: <xz>.properties.links.profiles requires the Specification Reference (requirements class) identifying that this is an OWC Context document and its version. Table 1 defines the data type as an Array of links profiles (as defined in Table 10 - data type "links") where one element SHALL have the href value “http://www.opengis.net/spec/owc-geojson/1.0/req/core”
But the multiplicity defines: One (mandatory) and the GeoJSON example 7.1.1.1 shows another different picture, which is only an array of String.
We implemented links.profiles as an array of links as objects of data type links from 7.1.10, table 10 with href, type, lang, title and length. And one mandatory link object element is required that has the specified core profile url.
Is that the desired behaviour?
Secondly, we found a quirk with the "<>.properties.links.via" description.
In 7.1.2 Class OWC:Resource "contextMetadata" is an Array of links as defined in Table 10 (Zero or more), and the example shows a JSON array. all good so far. However in OWC:Resource it is "resourceMetadata" also a "<>.properties.links.via" but explicitly call "Link object" (whereas links.data, links.previews and links.alternatives are Array of Link objects). But then in the column multiplicity it states "Zero or more" as with all the other link object arrays? But the example explicitly shows a single link object under the path "<>.properties.links.via":
This is confusing. What is the intended behaviour? We chose an array of link objects to mimic the other link object arrays general behaviour. Is that appropriate?
Finally, there seems to be an exceptionally different behaviour for "7.1.1.9 creator". In OWC:Context there is only mentioned the path: <xz>.properties.creator
with a presumably wrong and irrelevant example:
"creator" : "ACME CSW Server”,
Because the abstract 7.1.7 DataType OWC:Creator is later typed into specific:
- owc:CreatorApplication - <xz>.properties.generator
(as defined in Table 8)
- owc:CreatorDisplay - <xz>.properties.display
(as defined in Table 9)
But the actual issue that I mean is that in 7.1.7 DataType OWC:Creator specifies at first a generic extension (without a JSON path, because it is abstract) - then extension for <xz>.properties.display.* in Table 9 - Definitions of owc:CreatorDisplay elements is explicitly described BUT COMPLETELY ABSENT for <xz>.properties.generator.* in Table 8 - Definitions of owc:Creator/OWC:CreatorApplication elements.
That seems either accidental or the reason is not obvious. We chose the freedom to allow for an extension in OWC:CreatorApplication the same way the extension is declared for ALL other data types.
If I can be of any help to improve the standard or provide material or examples, please don't hesitate to contact me. I would be happy to join an OWC Context working group if possible.
- Our GitHub repository: https://github.com/ZGIS/smart-owc-geojson
- OpenHub Open Source statistics: https://www.openhub.net/p/smart-owc-geojson
- Bintray Maven/Ivy artefacts download: https://bintray.com/allixender/ivy2/smart-owc-geojson
Looking forward to support the use of OGC OWC Context for data exchange in the geospatial web.
Alex
I am summarising those subsequently here. The examples in the OGC Schemas repository are mostly still based on the former OWS-10 testbed results. Hut the OWC working group announced to update the official samples.
7.1.1 Class OWC:Context:
specReference: <xz>.properties.links.profiles requires the Specification Reference (requirements class) identifying that this is an OWC Context document and its version. Table 1 defines the data type as an Array of links profiles (as defined in Table 10 - data type "links") where one element SHALL have the href value “http://www.opengis.net/spec/owc-geojson/1.0/req/core”
But the multiplicity defines: One (mandatory) and the GeoJSON example 7.1.1.1 shows another different picture, which is only an array of String.
"links" : {
"profiles" : [
"http://www.opengis.net/spec/owc-geojson/1.0/req/core"
], ...
},
We implemented links.profiles as an array of links as objects of data type links from 7.1.10, table 10 with href, type, lang, title and length. And one mandatory link object element is required that has the specified core profile url.
Is that the desired behaviour?
Secondly, we found a quirk with the "<>.properties.links.via" description.
In 7.1.2 Class OWC:Resource "contextMetadata" is an Array of links as defined in Table 10 (Zero or more), and the example shows a JSON array. all good so far. However in OWC:Resource it is "resourceMetadata" also a "<>.properties.links.via" but explicitly call "Link object" (whereas links.data, links.previews and links.alternatives are Array of Link objects). But then in the column multiplicity it states "Zero or more" as with all the other link object arrays? But the example explicitly shows a single link object under the path "<>.properties.links.via":
"via" : {
"href" : "http://www.acme.com/products/algal20090123090856.xml",
"type" : "application/xml",
"length" : "435",
"title" : "XML metadata file for the entry 2009-01-23 09:08:56"
}
This is confusing. What is the intended behaviour? We chose an array of link objects to mimic the other link object arrays general behaviour. Is that appropriate?
Finally, there seems to be an exceptionally different behaviour for "7.1.1.9 creator". In OWC:Context there is only mentioned the path: <xz>.properties.creator
with a presumably wrong and irrelevant example:
"creator" : "ACME CSW Server”,
Because the abstract 7.1.7 DataType OWC:Creator is later typed into specific:
- owc:CreatorApplication - <xz>.properties.generator
(as defined in Table 8)
- owc:CreatorDisplay - <xz>.properties.display
(as defined in Table 9)
But the actual issue that I mean is that in 7.1.7 DataType OWC:Creator specifies at first a generic extension (without a JSON path, because it is abstract) - then extension for <xz>.properties.display.* in Table 9 - Definitions of owc:CreatorDisplay elements is explicitly described BUT COMPLETELY ABSENT for <xz>.properties.generator.* in Table 8 - Definitions of owc:Creator/OWC:CreatorApplication elements.
That seems either accidental or the reason is not obvious. We chose the freedom to allow for an extension in OWC:CreatorApplication the same way the extension is declared for ALL other data types.
If I can be of any help to improve the standard or provide material or examples, please don't hesitate to contact me. I would be happy to join an OWC Context working group if possible.
- Our GitHub repository: https://github.com/ZGIS/smart-owc-geojson
- OpenHub Open Source statistics: https://www.openhub.net/p/smart-owc-geojson
- Bintray Maven/Ivy artefacts download: https://bintray.com/allixender/ivy2/smart-owc-geojson
Looking forward to support the use of OGC OWC Context for data exchange in the geospatial web.
Alex
Subscribe to:
Posts (Atom)