COVID-19 Smart Chatbot Prototype for Patient Monitoring

Our contributions in this work are as follows:

• •

  First, we developed a natural language processing (NLP) model using text data from the existing scientific research literature on COVID-19 and related diseases. This model performs named entity recognition (NER) to extract the names of symptoms, drugs, other terminology associated with the disease. These entities then form relationships and construct a knowledge graph for our chatbot to query.

• •

  Subsequently, we extended the pipeline to have the ability to extract patient information from conversational input, hold follow-up responses using the extracted data insights, and track the progression of patient symptoms and build a personalized profile.

Due to legal reasons, the chatbot cannot prescribe medicines and offer medical advice but instead directs the user or patient to a few associated sentences (or perhaps a summary) from a piece of literature in the dataset, and a link to an official guideline, after which they can follow up themselves with a doctor.

The COVID‑19 pandemic is an ongoing pandemic of the novel coronavirus disease 2019 (COVID‑19). First identified in Wuhan, China, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the virus responsible for the COVID-19. It has since been declared a Public Health Emergency of International Concern in January of 2020 and a pandemic in March of 2020 [3]. As of March 9, 2021, more than 118 million cases have been reported in 214 countries and territories, resulting in more than 2.5 million deaths[4].

The COVID-19 virus and pandemic have stimulated an upsurge in research, data sharing, and technological and medical innovation as the scientific world searches for solutions to combat and mitigate the disease. Further, these resources are used to track its spread and analyze the unfamiliar symptoms and effects the virus may have. While there are innumerable scientists from every country tackling this issue, the countermeasures for and long-term effects of COVID-19 will only be revealed with time.

As mentioned, during the pandemic, many patients are opting not to go to the hospital and, thus, healthcare providers lack the opportunity to know their health conditions as quickly as possible. Doctors are using alternative strategies to check on their patients and limit hospital visits [5]. However, existing techniques like surveys or telemedicine have limitations [6, 7]. The former can only focus on known risk factors while failing to collect other preexisting and life-threatening factors[8]. Telemedicine conducted by nurse practitioners has a bandwidth issue as individual nurses can only see a limited number of patients during the day[7].

Our smart chatbot has a symptom tracking functionality allowing for a more comprehensive diagnosis when the patient has a chance to meet with a doctor. We use machine learning to help patients through intelligent conversations and collect information from potential patients, which is particularly valuable as hospital visits become more and more infrequent. Ultimately, this smart chatbot can potentially benefit the population at large, especially the elderly, who might have less inclination toward utilizing internet-based search engines to research their symptoms and risks.

Along with the essential features and functionalities of our chatbot, we also provide options for “virtual appointment” interactively and regularly. This allows the chatbot to occasionally check in on self-isolating or under-investigation patients for infection, monitor symptoms, and health conditions, and build a personalized patient profile, which will help healthcare providers make decisions based on evidence. A demo of our chatbot is shown in Figure 2.

Penn Medicine, Google, and Verily worked in collaboration on a prior chatbot for answering questions about COVID-19 and doing basics assessment of symptoms[9]. The application is targeted towards susceptible populations rather than confirmed COVID-19 patients and focused on a simple yes/no and categorical questions rather than performing NLU-based analysis on a patient’s input, such as descriptions of their symptoms. The user experience was measured directly by asking for feedback after giving answers. Another chatbot developed by researchers at the University of California, San Francisco [10] was designed for a screening of health system employees only and is not designed for use by the public.

In this work, we explored NER models specifically trained with a medical corpus using Med7[11], which is a clinical NER with seven different categories: drug, form, route, frequency, dosage, strength, and duration (see Figure 7(b)). Some widely-used medical NER models rely on Bidirectional Encoder Representations from Transformers (BERT)[12]. Medical models based on BERT are pre-trained with additional medical text corpora to capture domain-specific proper nouns and terms. BioBERT[13], ClinicalBERT[14], and scispaCy[15] are two examples of such models that we explored in addition to Med7.

In order to track patients’ symptoms and activity over time, we constructed a knowledge graph by combining information about the patients gathered by the chatbot along with information from literature extracted using NER models. We store the information from a patient in a knowledge graph (an example is shown in Figure 3). The graph represents multiple different features, such as diagnosis, treatment, and lab results, as nodes. The edges in this particular graph represent the decision-making process of the physician. Typically, knowledge graphs are formed with directed weighted edges, where the weights express the existence of a semantic linkage between two entities[16].

The Rasa Stack is a commonly used NLP library and tool specifically designed to aid the creation of chatbots. Rasa NLU is the approach that we used for intent classification and entity extraction from the patient input [17]. The Rasa NLU system has three components – intent classification, entity extraction, and hyperparameter optimization.

To showcase an example of the Rasa pipeline, we can formulate a query question, such as, “What is the dosage per day for my magnesium hydroxide prescription?” With the Rasa stack, the sentence will be tokenized, and an intent-entity structure will be returned. An example is shown in Figure 4), where the intent was classified as a ”find_dosage” intent, and several entities are extracted, including information such as the duration and medicine/drug type.

For extracting information from scientific literature related to COVID-19, we focused on CORD-19[18] (COVID-19 Open Research Dataset), which is an open-source, publicly available dataset of scholarly articles related to COVID-19, SARS-CoV-2, and other coronaviruses for use by the global research community. It is organized and updated by the Allen Institute daily, and as of September 11, 2020, CORD-19 has collected 253,454 publications. Supplementary datasets found on the COVID-19 Data Index (https://www.covid19dataindex.org/), such as the Symptoms Checker, were used to query the full text and help provide associated terms and medical information to use as a chatbot response.

In order to implement the NLP model to perform NER for drugs, symptoms, and other terms from the scholarly articles, we transformed the text from the CORD-19 dataset (titles, abstracts, and full-text JSONs from the collection of JSON files that contain full-text parse of a subset of CORD-19 papers) into a database, and then into a usable format (String) in Python.

We created a word cloud of the most frequent words within the titles of articles directly related to COVID-19 (Figure 5). From the word cloud, we identified words that show up more frequently than others, giving us insight into what words we would want to be included as nodes in our knowledge graph and whether or not they were captured as entities from NER.

Using the spaCy matcher library and a predefined list of generic symptoms from scientific literature, we did a basic exploration of the articles in the dataset. We counted the number of articles that reference certain symptoms related to viruses in the texts that have a reference to COVID-19. For example, the frequencies of the different symptoms appearing in literature are shown in Figure 6(a). Of the most frequently mentioned symptoms, we can see that fever, cough, and diarrhea are the top three. We recorded how the frequency of the word “fever” increased over time as more papers on COVID-19 started to be published around March 2020 6(b). Although symptoms like fever (Fig. 6(b)), cough, and diarrhea have different frequencies, which is consistent with Fig. 6(a), they show a similar trend.

We can also see that there are papers about these symptoms before the start of the pandemic. We considered papers that are related to ’covid-19’, ’coronavirus’, ’cov-2’, ’sars-cov-2’, ’sars-cov’, ’hcov’, and ’2019-ncov’. The papers before the pandemic examine the known coronaviruses and SARS cases. While fever, cough, and diarrhea are also symptoms of SARS, there are new symptoms that we discovered from this pandemic, namely loss of smell (anosmia) and loss of taste (ageusia).

To understand the CORD-19 dataset, the specific feature engineering techniques and methods that we incorporated were tokenization, lemmatization, and, lastly, named-entity recognition (NER). We first tokenized the raw literature text data from our CORD-19 dataset for both the abstracts and full text. We removed stop words and other punctuation as the initial preprocessing steps and also performed lemmatization in order to extract root forms of words for more accurate tagging. An overview of these processes is shown in Figure 7(a).

After performing NER on the abstracts and full text in the dataset, sentence-by-sentence associations between the symptoms were employed to build a knowledge graph. Building the knowledge graph is one of the major components of our proposed method. In addition to the information from NER, we incorporated natural language understanding (NLU) to extract features from chats using the chatbot to update the knowledge graph. The example of a knowledge graph is shown in Figure 8(a).

For this work, we used machine learning-based NLP techniques for interacting with patients to collect data and extract meaningful and useful information from it. Regular virtual appointments with the chatbot can help patients who are self-isolating or under-investigation for infection keep in touch with their healthcare provider. The chatbot can monitor symptoms and health conditions, and build a personalized patient profile, which allows providers to make decisions based on evidence.

The smartbot is trained using text data from the existing scientific research literature on COVID-19 and related diseases to learn the names of symptoms, drugs, and other terminology associated with the disease. Using this trained model, the smartbot is able to create a conversational environment to interact with the patients, answer questions from patients, and conduct in-depth exploration for concerning conditions. Our end-to-end chatbot is built on top of state-of-the-art framework Rasa [19].

With the patient information retrieved from smart chatbot conversations, we built a personalized profile for each patient, recording symptom development and drug usage in a time series. Beyond the profile, we constructed a graph for each patient, one sub-graph for each conversation. The patient acts as the central node, with neighboring symptoms nodes (from the patient profile). We then map each sub-graph to the knowledge graph built from NER results in chronological order to get a patient-specific trajectory or a set of exterior symptom/condition nodes from the original knowledge graph that we can predict for the patient. In Figure 8(b), the patient subgraph is in bold while the translucent nodes along the fringe of the patient’s subgraph form the patient’s trajectory over time.

Since training NER models to fit COVID-specific categories require annotation, which is a collection of pre-tagged texts, it is infeasible to start from scratch using the full literature text. However, pre-trained, medical-related NER models, such as Med7 [11], work well as an alternative. Med7 is a clinical NER that classifies tokens into seven categories which are sufficient for our use cases. In Figure 7(b), when a patient provides a query asking for the duration that corresponds to Magnesium Hydroxide prescription (assuming he/she follows the conditions in which this sentence derives from) the trained knowledge graph can be searched to find the correct duration (5 days).

In the case that Med7 does not provide us sufficient data or features to answer basic queries, other medical NER models were explored. In our chatbot prototype, we combined Med7 and scispaCy [15] to improve the entities extracted and model performance.

As we conducted NER on the dataset, we constructed a knowledge graph. Each named entity that we detect becomes a node in our Knowledge Graph. Rather than using edges to show a simple relationship or a physician’s decision, we weighted the edges to represent the significance of the relationship between two entities. These edge weights can be assigned by counting the number of times two named entities appear in the sentence by calculating conditional probabilities, especially for conditions and symptoms. This proves useful when making predictions or suggestions to the patients. If a patient has certain underlying conditions as well as symptoms, we use the edge weights to try and predict the likelihood that they have the virus. An entirely different method for assigning edge weights is to have a semantic meaning encoded into the edge instead. In the process of performing NER, we apply speech tagging to recognize the subject, object, and root verb of each sentence. The subject and object become nodes in the Knowledge Graph, and the root verb is used as a semantic descriptor on edge. This way, the sentence “A headache is a symptom of COVID-19” would make an edge from node “headache” to node “COVID-19” with the descriptor “symptom” in the Knowledge Graph.

For building our chatbot, we used the Rasa NLU framework, which also has built-in connectors with Facebook Messenger, Slack, Google Hangouts, Telegram, and several other popular social media applications[19]. Rasa met our requirements most readily because of its built-in entity extraction feature. We embedded pre-trained named entity recognition models, including those we used in Section 3.4 into the chatbot. This way, the chatbot is able to recognize entities never seen in training data. The language model we chose is the ”en_ner_bc5cdr_md” model, which is trained on a medical corpus, from scispaCy [15] as the built-in entity extractor and trained our Rasa model to recognize symptoms (marked as ”DISEASE”) and drugs (marked as ”CHEMICAL” from patient input and record it as shown in Fig.9(a). The bot message is on the left and the patient (”You”) is on the right. To make the conversation smooth, we also have basic conversation functions, including greeting, farewell, confirmation, and comforting (see Figure 2).

Once we have collected a large number of patient trajectories, we can predict one patient’s future trajectory if the current trajectory is similar to some existing trajectory on file. In other words, Bob in Figure 8(b) acts as a ”hidden” node, and through time, if other patients have experienced a similar set of exterior symptoms/nodes, then we can predict that Bob may have symptoms that develop from time $G_{t}$ to $G_{t+1}$, such as ”Low-normal procalcitonin”.

We then implemented a feature to recognize symptoms and drugs and respond with a corresponding ”Symptom recorded” or ”Drug recorded” message. First, any entity of ”DISEASE” and ”CHEMICAL” extracted by the scispaCy model is recognized as symptoms and drugs respectively. Additionally, we added descriptions of symptoms frequently related to COVID-19 in oral language as training examples so that the DIETClassifier[20] in Rasa can recognize them in causal communication. Given the patient profile database built as described in Section 4.2, we would also be able to raise personalized questions based on patient history. As shown in Figure 2(b), if the latest state for one patient included the symptoms of nose, coughing, and headache, our chatbot would ask, ”You mentioned in our last conversation that you have symptoms of nose, coughing, headache. How do you feel about them now?”

Based on the input from the patient, we can extract entities including symptoms, drug usage, etc., to build a personalized profile for the patient. One example is shown in Fig. 9(b). We match our patient’s responses, symptoms, conditions, and current state to nodes in the existing knowledge graph from Phase 1. Then, each patient’s state becomes a subgraph of the larger knowledge graph. Ultimately, we continue to build on top of this knowledge graph and formulate equivalent responses to be used on patients with similar symptoms or conditions.