The Australian Geoscience Data Cube - foundations and lessons learned
-
Altmetric Citations
Lewis, Adam; Oliver, Simon; Lymburner, Leo; Evans, Benjamin; Wyborn, Lesley; Mueller, Norman; Roberts, Dale; Trenham, Claire; Sixsmith, J; Wu, Wenjun; Tan, K P
Description
The Australian Geoscience Data Cube (AGDC) aims to realise the full potential of Earth observation data holdings by addressing the Big Data challenges of volume, velocity, and variety that otherwise limit the usefulness of Earth observation data. There have been several iterations and AGDC version 2 is a major advance on previous work. The foundations and core components of the data cube are: (1) data preparation, including geometric and spectral radiometric corrections to Earth observation...[Show more]
dc.contributor.author | Lewis, Adam | |
---|---|---|
dc.contributor.author | Oliver, Simon | |
dc.contributor.author | Lymburner, Leo | |
dc.contributor.author | Evans, Benjamin | |
dc.contributor.author | Wyborn, Lesley | |
dc.contributor.author | Mueller, Norman | |
dc.contributor.author | Roberts, Dale | |
dc.contributor.author | Trenham, Claire | |
dc.contributor.author | Sixsmith, J | |
dc.contributor.author | Wu, Wenjun | |
dc.contributor.author | Tan, K P | |
dc.date.accessioned | 2019-12-09T00:33:23Z | |
dc.date.available | 2019-12-09T00:33:23Z | |
dc.identifier.issn | 0034-4257 | |
dc.identifier.uri | http://hdl.handle.net/1885/188486 | |
dc.description.abstract | The Australian Geoscience Data Cube (AGDC) aims to realise the full potential of Earth observation data holdings by addressing the Big Data challenges of volume, velocity, and variety that otherwise limit the usefulness of Earth observation data. There have been several iterations and AGDC version 2 is a major advance on previous work. The foundations and core components of the data cube are: (1) data preparation, including geometric and spectral radiometric corrections to Earth observation data to produce standardised surface reflectance measurements that support time-series analysis, and collection management systems which track the provenance each data cube product and formalise re-processing decisions; (2) the software environment used to manage and interact with the data, including a minimal relational model that uses ‘not-only-SQL’ to simplify the process of adding new datasets to the data cube, or to simply ‘reference’ external datasets; and (3) the supporting, integrated, high performance computing - high performance data environment (HPC-HPD) provided by the Australian National Computational Infrastructure which supports both large scale analysis within the NCI, and direct access to data using standards-based web services. A growing number of exemplars demonstrate that the data cube approach allows analysts to extract rich new information from Earth observation time series, including through new methods that draw on the full spatial and temporal coverage of the Earth observation archives. To enable easy-uptake of the AGDC, and to facilitate future cooperative development, our code is developed under an open-source, Apache License, Version 2.0. This open-source approach is enabling other organisations, including the Committee on Earth Observing Satellites (CEOS), to explore the use of similar data cubes in developing countries. | |
dc.description.sponsorship | This work funded by the Australian Government through Geoscience Australia and the NCI. Funding for the supporting HPC-HPD infrastructure at NCI came from the Australian Government Department of Education, through the National Collaborative Research Infrastructure (NCRIS) and Education Investment Fund (EIF) Super Science Initiatives via the NCI, Research Data Storage Infrastructure (RDSI) and Research Data Services (RDS) projects (particularly the A1.1 National Earth Systems Science Data Services Domain). | |
dc.format.mimetype | application/pdf | |
dc.language.iso | en_AU | |
dc.publisher | Elsevier | |
dc.rights | © 2017 Published by Elsevier Inc. | |
dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ | |
dc.source | Remote Sensing of Environment | |
dc.subject | Landsat | |
dc.subject | Time-series | |
dc.subject | Big data | |
dc.subject | Data cube | |
dc.subject | High performance computing | |
dc.subject | High performance data | |
dc.subject | Collection management | |
dc.subject | Geometric correction | |
dc.subject | Pixel quality | |
dc.subject | Australian Geoscience Data Cube | |
dc.title | The Australian Geoscience Data Cube - foundations and lessons learned | |
dc.type | Journal article | |
local.description.notes | Imported from ARIES | |
local.identifier.citationvolume | 202 | |
dcterms.dateAccepted | 2017-03-12 | |
dc.date.issued | 2017-04-12 | |
local.identifier.absfor | 050206 - Environmental Monitoring | |
local.identifier.absfor | 080306 - Open Software | |
local.identifier.absfor | 010401 - Applied Statistics | |
local.identifier.ariespublication | u1027566xPUB57 | |
local.publisher.url | https://www.sciencedirect.com/ | |
local.type.status | Submitted Version | |
local.contributor.affiliation | Lewis, Adam, Geoscience Australia | |
local.contributor.affiliation | Oliver, Simon, Geoscience Australia | |
local.contributor.affiliation | Lymburner, Leo, Geoscience Australia | |
local.contributor.affiliation | Evans, Benjamin, Administrative Division, ANU | |
local.contributor.affiliation | Wyborn, Lesley, Administrative Division, ANU | |
local.contributor.affiliation | Mueller, Norman, Geoscience Australia | |
local.contributor.affiliation | Roberts, Dale, College of Business and Economics, ANU | |
local.contributor.affiliation | Trenham, Claire, Administrative Division, ANU | |
local.contributor.affiliation | Sixsmith, J, Geosciences Australia | |
local.contributor.affiliation | Wu, Wenjun, Geoscience Australia | |
local.contributor.affiliation | Tan, K P, Geoscience Australia | |
local.bibliographicCitation.startpage | 276 | |
local.bibliographicCitation.lastpage | 292 | |
local.identifier.doi | 10.1016/j.rse.2017.03.015 | |
dc.date.updated | 2019-07-28T08:17:19Z | |
local.identifier.scopusID | 2-s2.0-85017413041 | |
dcterms.accessRights | Open Access | |
dc.provenance | This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). | |
dc.rights.license | Creative Commons Attribution 4.0 International (CC BY 4.0) | |
Collections | ANU Research Publications |
Download
File | Description | Size | Format | Image |
---|---|---|---|---|
1-s2.0-S0034425717301086-main.pdf | 3.51 MB | Adobe PDF |
This item is licensed under a Creative Commons License
Updated: 17 November 2022/ Responsible Officer: University Librarian/ Page Contact: Library Systems & Web Coordinator