Abstract

Objective: To develop a system to extract follow-up
information from radiology reports. The method may be
used as a component in a system which automatically
generates follow-up information in a timely fashion.
Methods: A novel method of combining an LSP (labeled
sequential pattern) classifier with a CRF (conditional
random field) recognizer was devised. The LSP classifier
filters out irrelevant sentences, while the CRF recognizer
extracts follow-up and time phrases from candidate
sentences presented by the LSP classifier.
Measurements: The standard performance metrics of
precision (P), recall (R), and F measure (F) in the exact
and inexact matching settings were used for evaluation.
Results: Four experiments conducted using 20 000
radiology reports showed that the CRF recognizer
achieved high performance without time-consuming
feature engineering and that the LSP classifier further
improved the performance of the CRF recognizer. The
performance of the current system is P¼0.90, R¼0.86,
F¼0.88 in the exact matching setting and P¼0.98,
R¼0.93, F¼0.95 in the inexact matching setting.
Conclusion: The experiments demonstrate that the
system performs far better than a baseline rule-based
system and is worth considering for deployment trials in
an alert generation system. The LSP classifier
successfully compensated for the inherent weakness of
CRF, that is, its inability to use global information.