Kumar
of Technology Raipur, Raipur, Chhattisgarh, India
Avinash Kumar School of Computer Engineering, KIIT DU, Bhubaneswar, India
Ujjwal Maulik Department of Computer Science and Engineering, Jadavpur University, Jadavpur, Kolkata, India
Neha Miglani Department of Computer Engineering, National Institute of Technology, Kurukshetra, India
Pragatika Mishra Gandhi Institute for Technology, Bhubaneswar, India
Shubham Mittal Delhi Technological University, Delhi, India
Jayraj Mulani Department of Computer Science and Engineering, Institute of Technology Nirma University, Ahmedabad, India
Jigna Patel Department of Computer Science and Engineering, Institute ofTechnology Nirma University, Ahmedabad, India
Jitali Patel Department of Computer Science and Engineering, Institute of Technology Nirma University, Ahmedabad, India
Chittaranjan Pradhan School of Computer Engineering, KIIT DU, Bhubaneswar, India
U Reshma Arnekt Solutions Pvt Ltd., Magarpatta City, Pune, Maharashtra, India
Jayita Saha Computer Science and Engineering, Jadavpur University, Kolkata, India
Sohail Saif Department of Computer Science and Engineering, Maulana Abul Kalam Azad University of Technology, Kolkata, West Bengal, India
E Sandeep Kumar Department of Telecommunication Engineering, M.S. Ramaiah Institute of Technology, Bengaluru, India
Sobhangi Sarkar School of Computer Engineering, KIIT DU, Bhubaneswar, India
Pappu Satya Jayadev Department of Electrical Engineering, IIT Madras, Chennai, India
Sagnik Sen Department of Computer Science and Engineering, Jadavpur University, Jadavpur, Kolkata, India
Minakshi Sharma Department of Computer Engineering, National Institute of Technology, Kurukshetra, India
Gayatri Shinde Department of Computer Engineering, VESIT, Mumbai, India
K P Soman Amrita School of Engineering, Center for Computational Engineering and Networking (CEN), Amrita Vishwa Vidyapeetham, Coimbatore, India
Vignesh Subramanian Department of Computer Engineering, VESIT, Mumbai, India
G Swapna Amrita School of Engineering, Center for Computational Engineering and Networking (CEN), Amrita Vishwa Vidyapeetham, Coimbatore, India
Xinhui Tu School of Computer Science, Central China Normal University, Wuhan, China
Kalpan Tumdi Department of Computer Science and Engineering, Institute of Technology Nirma University, Ahmedabad, India
R Vinayakumar Amrita School of Engineering, Center for ComputationalEngineering and Networking (CEN), Amrita Vishwa Vidyapeetham, Coimbatore,India
Runjie Zhu Information Retrieval and Knowledge Management Research Lab,Department of Electrical Engineering and Computer Science, York University,Toronto, Canada
ADC Analog-to-digital converter
ADHD Attention-deficit hyperactivity disorder
ADNI Alzheimer’s disease neuroimaging initiative
AFLC Adaptive fuzzy leader clustering algorithm
AMBE Absolute mean brightness error
ANFIS Adaptive neuro-fuzzy inference system
Anti-CPP Anti-cyclic citrullinated peptide
BERT Bidirectional Encoder Representations From Transformer
BETA Blackbox Explanations Using Transparent Approximations Bi-LSTM Bidirectional long short-term memory
Bio-NER Biomedical named entity recognition
BMESO B-Begin M-Middle E-End S-Single O-Outside
BoW Bag of words xix
BRCA1 Breast cancer gene type 1
CAMDM Computer-aided medical decision making
CBIR Content-based image retrieval
CBoW Continuous bag of words
CDBN Convolutional deep belief networks
CDC Centers For Disease Control And Prevention
CDP Code On Dental Procedures And Nomenclature
CDSS Clinical decision support system
CGMS Continuous glucose monitoring system
CLAHE Contrast-limited adaptive histogram equalization
CRPS Continuous ranked probability score
DCNNs Deep convolutional neural networks
DRMM Deep relevance matching model
ELMO Embeddings from language models
ESRD End-stage renal disease
FDA Food And Drug Administration
FHE Fuzzy logic-based histogram equalization
FITBIR The Federal Interagency Traumatic Brain Injury Research
GBDT Gradient boosting decision trees
GRAM Graph-based attention model
HCPCS Healthcare Common Procedure Coding System
HCUP Healthcare Cost And Utilization Project
HPI History of patient illness
HRV Heart rate variability i2b2 Informatics For Integrating Biology and The Bedside
ICD International Classification of Diseases
ICD9 International Classification of Diseases 9
KPCA Kernel principal component analysis
LIDC Lung Image Database Consortium Dataset
LSTM RNN Long short-term Memory RNN
LSTM Long short-term memory
MCEMJ Medical Concept Embeddings From Medical Journals
MDF Markov decision process medGAN Medical Generative Adversarial Network
MEMM Maximum entropy Markov model
MICCAI Medical image computing and computer-assisted intervention MIDAS The Multimedia Medical Archiving System
MILA Montreal Institute For Learning Algorithms
MIMIC Medical Information Mart For Intensive Care
NIHCC National Institute of Health Clinical Centre
NLM National Library of Medicine
OASIS Open Access Series of Imaging Studies
OGTT Oral glucose tolerance test
PINN Pairwise input neural network
POMDP Partially observed Markov decision process
PSNR Peak signal–noise ratio
QSAR Quantitative structure−activity Relationship
RCNNs Region convolutional neural networks
RMSE Recursive mean separate histogram equalization
RMSProp Root mean square propagation
RP LIME Random pick local interpretable model
RSNA Radiological Society of North America
SBE Surrounding-based embedding feature
SEER Survival Epidemiology And End Results Program
SHMS Smart healthcare monitoring system
SIFT Scale-invariant feature transform
SiRNA Small interfering ribonucleic acid
SP LIME Selective pick local interpretable model
SPECT Single-photon emission computed tomography
SPPMI Shifted positive pointwise mutual information
SRL-RNN Supervised reinforcement learning with recurrent neural network SSIM Structural similarity index mean
STARE Structured analysis of the retina
TCIA The Cancer Imaging Archive
TP True positive t-SNE T-distributed stochastic neighbor embedding
UCI University of California, Irvine
UMLS Unified medical language system
USF University of Southern California
VEGF Vascular endothelial growth factor
VHL Von-Hippel–Lindau Illness
VIA Visual and image analysis
WBAN Wireless body area network
WBCD Wisconsin Breast Cancer Dataset
H B Barathi Ganesh, U Reshma, K P Soman and M Anand Kumar
Natural Language Understanding (NLU) is crucial for developing clinical text-based applications, primarily achieved through Vector Space Models and Sequential Modelling This study emphasizes sequential modelling techniques, specifically Named Entity Recognition (NER) and Part of Speech Tagging (POS), achieving an impressive F1 score of 93.8% on the i2b2 clinical corpus and 97.29% on the GENIA corpus The paper discusses the effectiveness of feature fusion by combining word, feature, and character embeddings for these tasks Additionally, we introduce MedNLU, a framework designed for sequential modelling that excels in POS tagging, chunking, and entity recognition in clinical texts MedNLU integrates a Convolutional Neural Network, Conditional Random Fields, and a Bi-directional Long-Short Term Memory network for enhanced performance.
Medical fields generate digital data in the form of clinical reports—structured/semi- structured data, raw data and the amount of data consumers/patients generate in
Amrita School of Engineering, Center for Computational Engineering and Networking (CEN), Amrita Vishwa Vidyapeetham, Coimbatore, India e-mail: barathiganesh.hb@gmail.com
K P Soman e-mail: kp_soman@amrita.edu.com
Arnekt Solutions Pvt Ltd., Pentagon P-3, Magarpatta City, Pune, Maharashtra, India e-mail: reshma.u@arnekt.com
Department of Information Technology, National Institute of Technology Karnataka, Surathkal, India e-mail: m_anandkumar@nitk.edu.in © Springer Nature Switzerland AG 2020
S Dash et al (eds.), Deep Learning Techniques for Biomedical and Health Informatics,
Studies in Big Data 68, https://doi.org/10.1007/978-3-030-33966-1_1
In the realm of social media, the vast amount of data generated makes it impossible for any one person to fully grasp all the information This is where Natural Language Processing (NLP), a key subfield of Artificial Intelligence, becomes essential MedNLU is a specialized framework designed to tackle the complexities of interpreting hidden insights within digital health data It serves as a crucial building block for various healthcare applications that rely on effective natural language processing and understanding.
The MedNLU encompasses various subfields of Artificial Intelligence, including Natural Language Processing (NLP), Conventional Machine Learning, and Deep Learning It processes healthcare texts and documents, generating outputs such as tokenized text, chunked text, parsed text, recognized entities, and Part of Speech (POS) tags related to the medical content.
Leveraging entities and POS tags enables the construction of a knowledge base, essential for database management and conversational systems The MedNLU framework's components are illustrated in Fig 1.
Electronic Health Records (EHRs) are essential for generating health documents, encompassing comprehensive patient information such as family history, initial complaints, diagnoses, treatments, prescriptions, lab results, visit records, billing data, demographics, progress notes, vital signs, medical histories, immunizations, allergies, and radiology images This extensive data ensures that clinicians and physicians have immediate access to nearly all pertinent details about a patient.
The MedNLU framework illustrates the transformative impact of Natural Language Processing (NLP) in healthcare, enabling systems to function independently without the need for human document verification Hospitals globally are increasingly adopting NLP technologies in their daily operations, enhancing efficiency and accuracy in medical documentation.
Extracting information is essential for the development of applications such as decision support systems, adverse drug reaction identification, pharmacovigilance, and effective management of pharmacokinetics This process also aids in patient cohort identification and the development and maintenance of electronic medical records (EMR) Key information is primarily obtained through natural language understanding (NLU) tasks, including Named Entity Recognition (NER), Part of Speech (POS) tagging, and chunking.
The medical domain has traditionally relied on Natural Language Understanding (NLU) using rule-based methodologies, which involve hand-coded rules to extract valuable information from medical data These rules were tailored to the specific structure of each document, allowing for effective knowledge extraction However, the introduction of algorithm-driven models has significantly reduced the burden of manual data encoding Recently, researchers have begun applying Deep Learning techniques to healthcare data for tasks such as Named Entity Recognition (NER) and relation extraction, marking a shift towards more advanced analytical methods.
The MedNLU framework enhances sequential modeling tasks by integrating word, feature, and character embeddings, utilizing a combination of Convolutional Neural Networks (CNN), Long-Short Term Memory (LSTM) networks, and Conditional Random Fields (CRF) This innovative approach enables the framework to effectively perform Named Entity Recognition, Part of Speech Tagging, Parsing, and Chunking on clinical texts Notably, the experiment demonstrates that the sequence model architecture achieves state-of-the-art performance even without a domain-specific word embedding model, relying instead on word embeddings derived from general English text.
The effective computation of dense word matrices and the integration of downstream models on word2vec with various architectures have significantly influenced research in leveraging Big Data within the healthcare domain Consequently, text classification tasks, including sentiment analysis, text summarization, information extraction, and information retrieval, are increasingly utilizing word embeddings This trend is particularly pronounced in healthcare and bioinformatics, where specific challenges such as relation extraction, named entity recognition, drug-disease interactions, medical synonym extraction, and chemical-disease relations are receiving heightened focus Both closed-set small corpora and large general corpora, like Google News, are being employed in these efforts.
Wikipedia is frequently utilized for training embedding models; however, these models are not directly applicable to clinical texts, which contain a higher frequency of specialized terminology and do not adhere to standard grammatical structures.
Various methodologies were employed to evaluate word embedding models after computing word vectors, including context predicting and context counting, which assess the relationship between data and correlation issues for lexical semantic tasks The counter predicting model is preferred over the count-based model due to its superior performance Landauer Thomas utilized Latent Semantic Analysis for indirect knowledge acquisition from text, analyzing similarities through local co-occurrence Additionally, Turney applied unsupervised vectors for classification in analogy tasks, and many researchers have since sought to refine this unsupervised learning approach for text applications In the bioinformatics field, Pakhomo conducted assessments of word embeddings.
Clinical text analysis faces significant challenges due to HIPAA restrictions, resulting in limited resources for clinical Part-of-Speech (POS) tagging Notably, Pestian et al reported a POS annotation of 390,000 pediatric sequences at Cincinnati Children’s Medical Centre, achieving 91.5% accuracy with a tagger enhanced by a Special Lexicon However, both the tagger and corpus remain unavailable To address the limitations of clinical text annotation, Liu et al developed sampling methods to co-train a POS tagger alongside the Wall Street Journal (WSJ) corpus Their evaluation of one sampling method on pathology reports revealed a remarkable 84% reduction in training data, while still achieving an impressive accuracy of 92.7%.
The Mayo Clinic in Rochester, Minnesota, developed the MED corpus, which consists of 100,650 POS-tagged tokens derived from 273 clinical notes, addressing the lack of accessible annotated corpora for research This initiative achieved an impressive accuracy of 93.6% when annotations were combined with GENIA and other POS-tagged corpora Additionally, despite the scarcity of clinical text corpora, the Mayo Clinic created cTAKES, a comprehensive biomedical NLP package that serves as a pre-trained, reusable tagging model.
Traditional Named Entity Recognition (NER) methods primarily relied on dictionary and rule-based approaches, necessitating domain expertise for effective rule detection Initially, researchers focused on conventional machine learning techniques or a combination of these with rule-based methods Various supervised and semi-supervised machine learning algorithms have been applied to NER tasks, emphasizing domain-specific attributes and specialized text features Hybrid models that integrate Conditional Random Fields (CRF) with Support Vector Machines (SVM) and various pattern matching rules have demonstrated improved results Additionally, incorporating pre-processing techniques such as annotation and true casing with CRF-based NER has enhanced concept extraction performance The top-performing models in the i2b2 challenge utilized CRF and semi-Markov Hidden Markov Models (HMM), achieving an impressive F-score of 0.85.