Data wars over data stores: challenges in medical data linkage

Abstract

A primary concern of the medical e-research community is the availability of suitable data sets for their analysis requirements. The quantity and dubious quality of data present significant barriers to the application of many automated analysis technologies, including data mining, to the medical and health domain. Publicly available data is frequently poorly coded, incomplete, out-of-date or simply not applicable to the analysis or algorithm being applied. Work has been done to overcome these issues through the application of data linking processes but further complications have been encountered resulting in slow progress.

The use of locally held medical data is difficult enough due to its structural complexity and non-standardised language, however linking data from disparate electronic sources adds the challenges of privacy, security, semantic compatibility, provenance, and governance, each with its own inherent issues. A focal requirement is a mechanism for the sharing of medical and health data across multiple sites which incorporates careful management of the semantics and limitations of the data sets whilst maintaining functional relevance for the end user. Our paper addresses this requirement by exploring recent conceptual modeling and data evaluation methodologies that facilitate effective data linking whilst ensuring the semantics of the data are maintained and the individual needs of the end user are met. Show more