Is this true? Confirming validity of Past Medical History data
This is Part 2 in a series trying to understand if the data in a medical record is true. Part 1 reviews some problems with Past Medical History data, and Part 2 here offers some high level solutions to ascribe confidence to issues in the Past Medical Record.
This is just a proposal of some considerations, it is not meant to be fully comprehensive, nor prescriptive. Though, I do think all medical records should implement Levels 1 through 3 immediately.
Listen to this post on YouTube below or your favourite podcast app under: Gregory Schmidt
1. Why is displaying the ‘certainty’ or ‘truth’ of information in medical records important?
Obviously using real information will improve patient care (both quality of, and speed it can be delivered). But How?
Ideally, the electronic medical record user interface would display the level of confidence beside facts in the record. For instance, items in a patient’s Past Medical History list, or active medication list would each have an associated ‘level of confidence’ beside the particular issue or medication. If the confidence was low - the user interface would alert the clinician to this fact. [Yes, I realize, in the future the EHR shouldn’t even show the user low confidence information…but we’ll get there …slowly]
The benefit of displaying the confidence level beside each fact is that it greatly improves the clinician’s ability to review a patient’s chart, and improves the quality of the care given. Without this information, as we saw in the previous post, each new clinician would have to manually fact check the record.
Let’s look at a hypothetical 5 level scale of increasing complexity to help ascribe the truthfulness of an issue listed in the Past Medical History. As one moves from Level 1 to Level 5 the system becomes more automated, and relies more on aggregate data analysis techniques and machine learning and less on straightforward simple rules based approaches.
Level 1: Manual Linkage
I have been talking about manual linkage of data to issues in the Past Medical History for years. However, have yet to see a system that does this remotely well. In general the systems that I have seen incorporate a ‘free text box’ where people can write what they want beside that Past Medical History issue. This free text box often has horrible version control (ie. some silly med student can write over what an experienced physician wrote), and the box provides no linking to primary level material.
The way it should work, is that when item is added to the patient’ list of Past Medical Issues, as much as possible, that item should be linked to the primary source material that supports the claim being made. This material should be viewable as a ‘snippet’ and ‘summary’ of the related issue, and then the user can click on this to view the original documentation themselves.
For instance, if Chronic Obstructive Lung Disease was an issue listed, the gold standard for diagnosis is spirometry / pulmonary function tests. Therefore, the clinician who adds COPD to the Past Medical History should link directly link this to the patient’s lung tests that first diagnosed this. Ideally, if new information was added, these new tests would be linked to the issue.
Furthermore, if all the data that is available to make this ‘diagnosis’ is a note with patient symptoms, or a chest x-ray that is a bit hyper-inflated, then this should be linked. And then the clinician in the future reviewing the chart, will be able to see…ok…the prior person said the patient has COPD… it seems there is no spirometry to support this… I will not consider my confidence in this diagnosis ‘high’.
As mentioned, this user interface is easy to build into an electronic medical records. However, it can also be use with paper records. Lawrence Weeds as part of his Problem Orientated Medical Record, has advocated since the 1970s that the medical chart must have on the front page an itemized list of all the patient’s Past Medical History and ‘Active Issues’. It still boggles my mind that I have never seen this in clinical practice. (Though I have tried to institute something like this when looking after complex ICU patients for several weeks at a time). In theory, after this cover page, could be a documentation support index that outlines for each Issue what primary data supports the item in the Past Medical History.
Another quick example: If the patient had a “left cerebral ischemic stroke in 2013” link to that CT Brain report, as well as the documentation around that admission. Under “no residual deficits at 6 months, full recovery” link to the physiotherapy note indicating so.
Problems with manual linkage
The great benefit of manual linkage is if the EHR was designed properly, you could start using this today, and it would dramatically improve the quality of medical documentation and care and save considerable long term time.
However, there are multiple problems with manual linkage. A few that come to mind are:
The process is manual, and relies on the good will of the clinician documenting that past medical issue to make the linkage to the source document.
There is variability in the accuracy of items added to the Past Medical History. A medical student may enter that a patient has venous stasis (pooling of fluid in their legs), while missing the fact the patient actually has congestive heart failure.
There is variability what clinicians consider is appropriate linkage. Each clinician may link to different type of supporting documentation. If the patient was admitted through the Medicine ward, the clinician may enter the patient had “6 seizures of unknown etiology in 2016” and link to an EEG report. However, if the patient was admitted by neurology, the neurologist may link that issue to the three sets of spinal fluid results, MRI brain scans, various blood samples people have never heard of, genetic and autoimmune tests, and multiple consultant notes. All supporting the “unknown etiology”.
The linkage process is static in time and the data can become outdated. If the patient’s condition around that item in the Past Medical History changes, those changes are not reflected in their Past Medical History list. For instance, if the patient’s lung function drops from FEV1 of 65% to 35%, unless someone links the new lung function tests to ‘COPD’, the Past Medical History will continue to read “COPD - FEV1 65%”.
The item in the Past Medical History may no longer be correct. A patient may have listed “Pneumonia Sept 2018” and “Pneumonia Oct 2018”. Each may link to a chest x-ray with an infiltrate. However, when the patient is diagnosed with lung cancer in November 2018 - nobody will go back and actually update the Past Medical List and update the two prior issues (that were in fact incorrect diagnosis), and merge them into the new diagnosis of Lung Cancer - with the first diagnostic result to link being the September 2018 chest x-ray.
This technique is prospective. It does not auto-link all the patient’s past issues in their old charts, making it time consuming and not useful from data mining perspective.
Error of Omission
Another key downside with the manual linkage technique is that it does not solve errors of omission. As mentioned at the start, the problem with data in the medical record is both assuming that data inside of the record is true (when in fact it is not). But the other major error is seeing the absence of a reported fact, as truth of its absence.
Unless one manually enters (and as proposed above, links) issues in a patient’s Past Medical History to their source material, clinicians may not know the patient in fact has the condition.
Still unable to validate data’s truthfulness
But worst of all, the computer still cannot judge the veracity and truthfulness of an item in the Past Medical History simply based on manual user linkage. It makes it easier for the clinician to see what the previous clinicians were thinking - and this is a huge step forward - but it does not solve the problem we set out to solve.
Level 2: Guided Linkage
Guided linkage builds upon manual linkage, by proposing common source documents that should be linked for each issue.
For instance, if you enter COPD, in Guided Linkage the system would automatically propose the types of documents to link & actually show you the very documents.
This system requires a basic rule based set of associations of which documents should be associated with the most common diagnosis. We don’t need to create a system for all 100,000 items in the ICD-10 system, but starting with the most common items in the Past Medical History is a good place. I think a team of half a dozen or a dozen bright internets could easily create the guided linkage patterns required for over 80% of their clinical cases in a weekend retreat.
Level 3: Data Capsules
One of the issues with simple ‘linkage’ for past documentation to medical issue is that doing so may get very ‘busy’ and ‘complicated’.
This is another example of where the concept of ‘data capsules’ can help. I’ve written about this before (and will again soon). In short, a data capsule is a self contained unit that holds all the important data relevant to a Medical Issue.
For instance, the COPD data capsule would have the relevant medications, test, labs, imaging, notes, and clinicians related to this condition.
The capsule functions as
(a) a way to display all this related data at once
(b) an automated way to ‘pull’ and ‘import’ data that may be elsewhere into the record. For instance, the COPD data capsule may automatically ‘pull’ into it any chest x-rays, even if done for other purposes.
Similar to Level 2: Guided Linkage, the data capsule logic can be done using a straightforward rule based approach designed for the most common issues. Again, I really think this could be build with a small team of internists over a weekend retreat.
Level 4: Automated linkage and automated data encapsulation
In the examples above, the linkage patterns, and the addition of data to data capsules was dependent on the user ‘agreeing’ that the linkage is correct, or that the data should be associated with that data capsule.
However, ideally, the data linkage algorithms and data capsule schemas should be robust enough that they automatically aggregate this data whenever a clinician enters an issue or diagnosis.
This means the system could also retrospectively link data together, and in real-time link data together, to ‘propose’ to the clinician that a patient may in fact have a particular diagnosis. However, in this model, the clinician still has to ‘accept’ that the proposed diagnosis and issue for the Past Medical History is correct and makes sense.
Level 5: Automated diagnosis & automated data validation
The highest level of data verification would rely on techniques more sophisticated than the rule based Issue Manual and Data Capsules proposed above.
This would ideally consider multiple variables and factors in the patient’s chart to ascribe a diagnosis and corresponding level of certainty. (The example in Part 1 of the article shows how many considerations are needed even for a ‘very straightforward’ diagnosis like HIV or Diabetes).
The system would do this in real time - both on historical data and as new data is added.
Ultimately it would ensure that the medical record data is always up today. This is where we need to get to, it will just take time. I suspect that multiple different vendors and research groups will come up with different and better ways to do this type of Past Medical History problem list composition. Ideally, it should be able to plug-and-play into an electronic health record. Perhaps these types of call requests could be within the Clinical Reasoning part of FHIR.
How to visualize a data’s level of truth?
OK, let us pretend we have implemented some of the above solutions, and we can determine the level of truth corresponding to an item in the Past Medical History. How should this level of certainty be displayed in the medical record?
I’m not certain at this time the optimal strategy. However, my suspicion is that to start medical records should flag information that it believes is incorrect. It is easier to make a mistake here. As opposed to making a mistake when indicating information that the medical record believes is ‘correct’.
So, some visual flag for information that is incorrect.
Likely information that is of reasonable certainty should be in a ‘neutral’ style.
Information that has an extremely high level of certainty, perhaps needs a ‘verified’ stamp or unique style.
Ideally, clinicians will be able to interact with the user interface, and correct information the computer has mischaracterized. These corrections can be validated by a third party, and if accepted, the algorithms used to flag data will improve through continuous user feedback.
