The integrated delivery of large-scale data mining

Abstract

Data Mining draws on many technologies to deliver novel and actionable discoveries from very large collections of data. The Australian Government’s Cooperative Research Centre for Advanced Computational Systems (ACSys) is a link between industry and research focusing on the deployment of high performance computers for data mining. We present an overview of the work of the ACSys Data Mining projects where the use of large-scale, high performance computers plays a key role. We highlight the use of large-scale computing within three complimentary areas: the development of parallel algorithms for data analysis, the deployment of virtual environments for data mining, and issues in data management for data mining. We also introduce the Data Miner’s Arcade which provides simple abstractions to integrate these components providing high performance data access for a variety of data mining tools communicating through XML. Show more