Text Simplification Using Natural Language Processing and Machine Learning for Better Language Understandability

Abstract

Natural language understanding is fundamental to knowledge acquisition in today's information society. However, natural language is often ambiguous, with frequent occurrences of complex terms, acronyms, and abbreviations that require translation from complex to simpler text for better understanding. Moreover, due to the explosive growth of information, many texts we come across are relatively complex, requiring adequate knowledge of the context for accurate understanding. For example, complexity and ambiguity are particularly prevalent in knowledge-rich medical and scientific domains, that heavily use domain-specific terminologies. Thus, understanding natural language in medical or scientific domains can be challenging for most of the general population.

This PhD thesis in computer science explored methods for improving the understandability of complex language using Natural Language Processing and Machine Learning (ML, including, but not limited to, Deep Learning (DL)), focusing on general and domain-specific text. More specifically, it addressed the individual sub-fields of complex word identification (CWI), acronym identification, lexical substitution, and acronym disambiguation and their combination into a modular lexical simplification system. It also explored the applicability of text editing models for text simplification.

First, this thesis focused on CWI, the initial step in improving language understandability and explored rule-based and ML/DL methods. Under CWI, the thesis investigated general CWI, medical CWI, and acronym identification. Results on CWI and acronym identification indicated that Transformer-based language models gave superior performance. For the medical CWI, impact of the word and character-level information was explored. Results indicated that a Convolutional Neural Network trained on both word and character embeddings achieved the best F score of 93.9%.

The second focus area was lexical substitution for simplification. The recent methods relying on lexical resources and contextual embeddings do not comprehensively analyse their contribution for lexical substitution. Thus, the thesis investigated the impact of lexical resources, contextual word, and sentence embeddings. Moreover, it proposed a solution called CILex which is now state-of-the-art on the widely used LS07 and CoInCo datasets, with P@1 scores of 55.96% and 57.25% for lexical substitution.

Thirdly, the thesis focused on acronym disambiguation, which has received significant attention, specifically from the scientific and medical communities. The thesis investigated contrastive-learning-based methods, intending to effectively learn better representations to distinguish between acronym expansions. The main outcome was proposing an extended version of triplet networks called m-networks that gave a comparable performance with previous methods but only needing about 12% of their number of parameters.

For an improved understanding of complex language, it is essential to bring these three focus areas together. Thus, the thesis conducted a user study to identify essential components needed for improved comprehension. It built on these evaluation results by proposing a modular lexical simplification framework combining the above components. Finally, based on the proposed framework, the thesis developed an end-to-end demonstration system for lexical simplification.

Text editing models have gained increased attention from monolingual translation problems. Thus, the thesis explored the applicability of text editing models and the effect of error identification for text simplification. Results indicated 100% error sample suppression improved the performance.

This research is expected to be valuable for better understandability of complex text, reducing the cognitive burden of people with comprehension difficulties. To assure this influence and impact of this research, the codebase and software tools are available open source. Show more