Health Information Technology Evaluation Handbook

From Meaningful Use to Meaningful Outcomes

Vitaly Herasevich, MD, PhD, MSc, Brian W. Pickering, MD, MSc

• Publisher: Productivity Press
• Year: 2021
• ISBN:
  9781000478044
• Edition: 2
• Topic: Médecine
• Subtopic: Santé publique, administration et soins

Governments and clinical providers are investing billions of dollars in health information technologies (HIT). This is being done with the expectation that HIT adoption will translate into healthier patients experiencing better care at lower cost. In the initial push to roll out HIT, the reliability of these claims was often not substantiated by systematic evaluation and testing. As the first wave of widespread adoption of HIT comes to an end and the next wave begins, it is more important than ever that stakeholders evaluate the results of their investment, evaluate their success (or failure), and make decisions about future directions.

Structured evaluations of a project's impact are an essential element of the justification for investment in HIT. A systematic approach to evaluation and testing should allow for comparison between different HIT interventions with the goal of identifying and promoting those which improve clinical care or other outcomes of interest. The question of the day is no longer "why perform evaluations, " but "how to perform evaluations."

This updated book provides an easy-to-read reference outlining the basic concepts, theory, and methods required to perform a systematic evaluation of HIT. Chapters cover key domains of HIT evaluation: study structure and design, measurement fundamentals, results analysis, communicating results, guidelines development, and reference standards. Updated case studies and examples are included demonstrating the successes or failures of these investments. The authors also include new initiatives put in place by the government and discuss how they are being adopted and used by health systems.

Chapter 1

The Foundation and Pragmatics of HIT Evaluation

DOI: 10.4324/9781003042969-1

1.1 Need for Evaluation

The introduction of new, disruptive technologies and innovations may bring significant benefits for patients including enhanced quality of life and more efficient care. Innovations in clinical practice give healthcare providers the opportunity to improve the effectiveness and quality of the care they provide and to apply safer treatment while at the same time improving their job satisfaction. On a national level—and in line with government objectives—technological innovation may improve the quality of care with better outcomes at reduced costs.

A recent analysis of Healthgrades’ hospital performance database on mortality and complication rates shows that electronic medical records (EMRs) do have a positive impact on care [1]. These large-scale, national-level analyses, however, are prone to use less scientifically sound methodologies and are not able to answer many other specific questions that are important for other stakeholders and users of healthcare information technology (HIT or health IT). What is usability in this particular module? What impact on time does that application have? What are the consequences of data misinterpretation? How does system downtime impact clinical procedures?

In 2015, a systematic review and meta-analysis was published dealing with the impact of the EMR on mortality, length of stay, and cost in the hospital and intensive care unit (one of the most data-dense clinical environments). This analysis shows that electronic interventions did not have a substantial effect on the metrics reviewed. One of the reasons was the small number of studies that could be systematically aggregated due to the heterogeneity of study populations, interventions, and endpoints [2].

Current evaluation mostly targets the national level but grossly omits data at the provider and patient levels. Many technological innovations seem to offer potential benefits to patients, clinicians, and healthcare in general, but their diffusion can be delayed or omitted due to the absence of evidence regarding specific benefits. A significant number of technologies, however, have not turned out to be important advances. Also, some innovative technologies have the potential to do more harm than have a positive impact.

The field of health technology assessment (HTA) originated in the 1970s when a growing array of costly medical equipment concerned taxpayers and health insurers regarding their ability and willingness to fund their use [3].

Ultimately, HTA sought decision-making evidence. In 1970, HTA started synthesizing the available evidence—efficacy and cost-effectiveness of technological interventions—compiling it in a format that was helpful for health policy-makers, mostly at a national level. The field became more mature over the years and spread from the United States to Europe. HTA became more popular in Europe in the 1980s, and increasing effort was devoted to more effectively disseminate information to influence clinicians and local administrators. Many developed countries have national programs to evaluate health technologies and guide coverage and pricing decisions like The National Institute for Health and Care Excellence (NICE) in the UK; however in 2021, the United States still does not have a national HTA program.

Historical Essay

The tobacco smoke enema was a medical procedure that was widely used by Western medical practitioners at the end of the eighteenth century as a treatment for drowsiness, respiratory failure, headaches, stomach cramps, colds, and others conditions.

In 1774, two London-based physicians, William Hawes and Thomas Cogan, formed The Institution for Affording Immediate Relief to Persons Apparently Dead from Drowning. Their practice quickly spread, reaching its peak in the early nineteenth century.

The tobacco smoke enema procedure declined after 1811, when English scientist Ben Brodie discovered nicotine’s toxicity to the cardiac system using an animal model [4].

As technologies and science progress, a tremendous amount of information technology (IT) is being added every year to the HIT space. Recent government incentives combined with technological progress have given additional reasons for clinicians to implement and adopt HIT.

Like medical technologies in the 1970s, however, we have little evidence regarding the specific risks and benefits of HIT. We are in a situation much like the beginning of the nineteenth century, when a single statement, “our software can save lives,” is believed and trusted. New medications cannot be introduced to the market without rigorous clinical evaluation (even outside regulatory bodies). While commercial IT products undergo technical validation and testing before being delivered to customers, unfortunately, this testing has nothing to do with clinically oriented metrics such as mortality, complications, medical errors, length of hospitalization, and so on.

Direct users of HIT have the following specific questions that are not answered by technical or policy-oriented evaluation:

1. Is the HIT (software) safe to use on my patients?
2. Does the HIT do what it claims and is designed to do?
3. Is the end result of this technology useful for patients and users?
4. Can this HIT be usefully applied in my specific practice, and what is the cost-benefit of usage?

1.2 HIT: Why Should We Worry About It?

Technological advances combined with government incentives and requirements are providing solid reasons for individual physicians and hospitals to move toward implementing HIT.

The value of the quick-moving information that is provided for decision-making, however, is not often supported by evidence. An informed decision regarding the use of new technology is based on solid scientific evidence, not elevated speech. The regulations and requirements for the rapid deployment of software products, however, have a net limiting effect on the ability to evaluate the practical utility of HIT.

Most of the time, when people talk about HIT, they are talking about EMRs. An EMR is defined by the National Alliance for Health Information Technology (NAHIT) as “an electronic record of health-related information on an individual that can be created, gathered, managed, and consulted by authorized clinicians and staff within one health care organization.” In contrast, an electronic health record (EHR) is defined as “an electronic record of health-related information on an individual that conforms to nationally recognized interoperability standards and that can be created, managed, and consulted by authorized clinicians and staff across more than one healthcare organization.”

Very often, the terms EMR and EHR are used interchangeably. The principal difference is the interoperability of an EHR.

The ultimate goal of HIT adoption is a health information exchange (HIE), a national infrastructure to provide a network where health information can be exchanged among hospital and physician offices using EMRs. This structure does not exist and is still under development.

The resources for successful EMR deployment and utilization do exist, but many questions arise when considering the end-users of such technology.

Do EMRs deliver on their promises? What are the intended and unintended consequences of EMR adoption for healthcare providers?

In a 2015 article published in the Journal of Medical Economics, health-care providers shared opinions about whether EMRs have delivered on promises such as increased efficiency, better time management, and faster charting. The vast majority of clinicians expressed disappointment with EMRs [5].

Physicians are key stakeholders in the adoption of EMR technologies. Their satisfaction and support of their hospitals’ EMR efforts are critical to ensuring EMRs become a positive influence on the hospital environment. Figure 1.1 presents common problems leading to EMR dissatisfaction, such as information overload, lack of information, and time spent on data entry. As key stakeholders in the successful deployment and utilization of EMRs, physician feedback is essential.

Figure 1.1 Visualization of physiological parameters within EMR (information overload), (Bottom Left) visualization of laboratory parameters in spreadsheet-like EMR (lack of information), and (Right) printed-on-paper pre-anesthesia evaluation electronic form (EMR data entry).

Healthcare administrators should continuously evaluate the benefits of EMRs for physicians and their patients regardless of the level of EMR adoption in their particular organizations.

Furthermore, decision-makers need a more comprehensive approach to set HIT priorities and obtain maximum benefits from limited resources, and they must be able to do this without compromising the ethical and social values underpinning their health systems [6].

Historical Essay

In the late 1960s, Dr. Larry Weed introduced the concept of the problem-oriented medical record instead of recording diagnoses and their treatment. Multiple parallel efforts then started to develop EMRs. In the mid-1960s, Lockheed created an EMR system that later became Eclipsys (now called Allscripts). This system included computerized physician order entry (CPOE). In 1968, the computer-stored ambulatory record (COSTAR) was developed by Massachusetts General Hospital and Harvard. This system had a modular design separated into accounting and clinical elements. COSTAR was created using the MUMPS programming language, which 40 years later dominates modern EMR suites and financial applications. In 1972, the Regenstrief Institute developed their system, which featured an integration of electronic clinical data from their laboratories and pharmacies. In the 1980s, systems moved from inpatient to outpatient settings. This change was made based on the need for simple billing functions to justify the return on investment for EMRs. Over time, those functions became more dominant than clinical functions. Through the 1990s and 2000s, technical platforms and core systems did not change much. This left clinical functionality far behind the current requirements for safety, efficacy, effectiveness, and usability.

Definitions

Technology is a broad concept that deals with the use and knowledge of tools and crafts and how such use affects people’s ability to control and adapt to the social and physical environment.

• Technology assessment is a scientific, interactive, and communicative process that aims to contribute to the formation of public and political opinions on societal aspects of science and technology. Technology assessment was (and is) an extremely broad field.
• Health technology is defined as the drugs, devices, and medical and surgical procedures used in the prevention, diagnosis, treatment, and rehabilitation of disease.
• Assessment is a process of measuring existing technology against claimed goals and objectives.
• Health technology assessment is a multidisciplinary activity that systematically examines the safety, clinical efficacy and effectiveness, cost, cost-effectiveness, organizational implications, social consequences, and legal and ethical considerations of the application of health technology—usually ...