ArXiv Preprint
Current approaches for clinical information extraction are inefficient in
terms of computational costs and memory consumption, hindering their
application to process large-scale electronic health records (EHRs). We propose
an efficient end-to-end model, the Joint-NER-RE-Fourier (JNRF), to jointly
learn the tasks of named entity recognition and relation extraction for
documents of variable length. The architecture uses positional encoding and
unitary batch sizes to process variable length documents and uses a
weight-shared Fourier network layer for low-complexity token mixing. Finally,
we reach the theoretical computational complexity lower bound for relation
extraction using a selective pooling strategy and distance-aware attention
weights with trainable polynomial distance functions. We evaluated the JNRF
architecture using the 2018 N2C2 ADE benchmark to jointly extract
medication-related entities and relations in variable-length EHR summaries.
JNRF outperforms rolling window BERT with selective pooling by 0.42%, while
being twice as fast to train. Compared to state-of-the-art BiLSTM-CRF
architectures on the N2C2 ADE benchmark, results show that the proposed
approach trains 22 times faster and reduces GPU memory consumption by 1.75
folds, with a reasonable performance tradeoff of 90%, without the use of
external tools, hand-crafted rules or post-processing. Given the significant
carbon footprint of deep learning models and the current energy crises, these
methods could support efficient and cleaner information extraction in EHRs and
other types of large-scale document databases.
Anthony Yazdani, Dimitrios Proios, Hossein Rouhizadeh, Douglas Teodoro
2023-02-08
