GRAND JURY

Ventura County Grand Jury 2014 - 2015 Final Report Ventura County Electronic Health Record Implementation Risks June 11, 2015 This page intentional/y blank Ventura County 2014 – 2015 Grand Jury Final Report Ventura County Electronic Health Record Implementation Risks Note: see “Glossary” for definitions Summary The 2014-2015 Ventura County Grand Jury (Grand Jury) received several public complaints listing examples of potential hospital patient care issues associated with the recent Electronic Health Record (EHR) system implementation by the Ventura County Health Care Agency (VCHCA). Because of the potential seriousness of patient care issues, the Grand Jury decided to investigate the complaints. After several interviews, the Grand Jury decided it was prudent to expand the scope of its investigation to examine the processes that were used for the EHR project requirements specification and risk management. The Grand Jury performed this investigation principally by interviewing various Ventura County (County) staff and management and studying documents that they provided. Additional research was done by studying documents involved in the purchase of the Cerner EHR system and by searching the Internet. The Grand Jury reached the following significant findings:  After “Go-Live” (July 1, 2013) a significant level of concern was raised by clinical staff to VCHCA Information Technology (IT) regarding potential impacts of observed EHR-related risks on patient well-being.  Systemic deficiencies existed in the process used by VCHCA to develop and vet the adequacy of the EHR project requirements specification.  There was no evidence that project requirements were formally specified, which precluded generating a complete and quantifiable test plan to verify overall EHR quality throughout the Implementation stage.  There was no effective independent review of the EHR project before the release of the Request for Proposal (RFP), before contract signing, nor continuing periodically during the course of the project.  The lack of an effective Risk Management Plan resulted in a significant impact on project quality and cost.  EHR project execution was directed solely by the EHR vendor (Cerner) using its Cerner Event Driven Methodology to the exclusion of other important VCHCA-specific considerations. The EHR Implementation had significant undiscovered problems at Go-Live.  By failing to have quantitative data to predict impacts on the Go-Live date, project management was unable to convince VCHCA administration to support the project staffing levels and ordering dates of materials necessary to deliver an operationally acceptable product. Ventura County EHR Implementation Risks 1 Ventura County 2014 – 2015 Grand Jury Final Report  VCHCA research and ITC status reports both indicated a shortage of personnel assigned to the EHR project. However, VCHCA and ITC failed to take the necessary and timely corrective action.  VCHCA failed to develop a project plan to reflect VCHCA staffing hours and resources necessary to integrate with the Cerner production schedule. The Grand Jury recommends the Ventura County Board of Supervisors (BOS) take the following significant actions:  Direct the VCHCA to establish a policy to charter Independent Review Boards composed of project-applicable SMEs to review all of its capital projects.  Direct the VCHCA to establish a policy that all capital projects sponsored by VCHCA create and periodically update a Risk Management Plan to identify project risks and their associated impacts, to propose mitigation activities, and to periodically track and publish the status of risks and mitigation efforts.  Direct the VCHCA to establish an Informatics Department with appropriate full-time staffing to satisfy the needs for maintenance and future upgrades of the VCHCA EHR. To be effective in this role, the Informatics Department should report directly to clinical VCHCA management to ensure that patient care is always given proper clinical concern and priority. Background The federal American Recovery and Reinvestment Act (ARRA) was signed into law in 2009. Along with this law Congress passed the Health Information Technology for Economic and Clinical Health (HITECH) Act. HITECH made available up to $25.9 billion nationwide to help encourage and partially subsidize the healthcare community’s transition to EHR systems. Healthcare providers could realize financial rewards (for early adoption) and incur penalties (for late adoption) of EHR technology. [Ref-01, Ref-02] HITECH established “Meaningful Use” criteria to enable grading conformance to requirements. Meaningful Use criteria state that “providers need to show they’re using certified EHR technology in ways that can be measured significantly in quality and in quantity.” After its initial adoption, HITECH was revised to contain new criteria that required one integrated EHR system encompassing outpatient care, inpatient care, and federally mandated Stage 1 “Meaningful Use” criteria. Examples are “use of electronic prescribing” and “electronic exchange of health information” to improve the quality of healthcare. Federal penalties were intended to apply to all hospitals, patient care facilities, and eligible providers that did not convert from paper and nonintegrated EHR recordkeeping to an integrated EHR by the end of 2014. Additionally, those that failed to achieve the scheduled Stage 1 standards by the end of December 2014, irrespective of the loss of any Meaningful Use incentives, were subject to federal penalties. [Ref-02, Ref-03] 2 Ventura County EHR System Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report On November 17, 2010, VCHCA responded to the HITECH rules and incentives by issuing Request for Proposal (RFP) package #5565 to procure an EHR system. In June 2011, VCHCA selected Cerner Corporation of Kansas City, Missouri (Cerner), as its EHR supplier. Cerner’s EHR system was qualified by the Centers for Medicare and Medicaid (CMS) as satisfying Stage 1 Meaningful Use requirements. [Ref-04] Over a four-year period beginning July 1, 2013, VCHCA expects to earn $20 million in scheduled Meaningful Use incentive payments from the federal government and avoid Medicare penalties for noncompliance. In response to several public complaints, the Grand Jury investigated VCHCA’s implementation of the EHR with emphasis on the issues presented in the complaints. Methodology The Grand Jury conducted this investigation by:  Interviewing numerous individuals involved in the implementation and management of the VCHCA EHR  Reviewing documents supplied by interviewees  Researching and reviewing BOS meeting minutes and attachments related to VCHCA’s adoption and implementation of the EHR  Reviewing VCHCA and BOS responses to the 2013-2014 Grand Jury “Final Report on Healthcare Records Processes and Procedures”  Reviewing documents relating to risk management on the Forms and Policies (F&P) page of the County Intranet website  Researching documents relating to best practices for developing project requirements specifications and conducting risk management assessments Facts FA-01. In response to the 2013-2014 Grand Jury report “Healthcare Records Processes and Procedures,” VCHCA stated that in 2011 it used its most senior Information Technology (IT) manager to lead the EHR project during the RFP process, contract negotiations, and planning the details of the project Kick-Off. Subsequently VCHCA contracted a full-time project manager who reported to the VCHCA director and had full authority for all aspects of the project related to VCHCA staff and resources planning and coordination. He also had full authority to manage the Cerner contract for VCHCA. The project manager was required to adhere to the Work Plan documented in the contract with Cerner. All decisions related to tasks and timelines were to be jointly made by both VCHCA and Cerner project management teams. [Ref-05] FA-02. VCHCA did not completely document its system requirements nor was there evidence of a review of the requirements by an independent EHR Ventura County EHR Implementation Risks 3 Ventura County 2014 – 2015 Grand Jury Final Report review team or an independent EHR Subject Matter Expert (SME) consultant. The Grand Jury could not find the system specification document that the contract required. [Ref-04] FA-03. VCHCA did not develop a comprehensive Risk Management plan that would have identified significant project risks and associated mitigation strategies. [Ref-06, Ref-08] FA-04. The International Organization for Standardization (ISO) Standard “ISO 31000:2009” webpage provides links to principles and generic guidelines on risk management using techniques explained in ISO 31000:2009. It can be applied to any type of risk, whatever its nature, whether having positive or negative consequences. [Ref-07] FA-05. Prior to negotiating the contract with Cerner, VCHCA determined the number of simultaneous users expected on the Cerner EHR by using statistics from the existing legacy Meditech EHR as a model. The analysis resulted in an estimate of 600 simultaneous users, but did not take into account other hospital and clinic staff who were either not using Meditech or who were using “paper” patient treatment records at the time the estimates were made. [Ref-05, Ref-09] FA-06. The Cerner EHR offers a semi-customizable computerized framework for managing client healthcare records electronically. Cerner provides a suite of digital applications or “Solutions” that are intended to streamline administration, reduce costs, retain and display patient records, prepare reports with EHR statistics, and enhance patient care. FA-07. The Cerner EHR selected by VCHCA is not turnkey—the Solutions in the Cerner framework had to be customized for VCHCA. This customization process implements elements such as VCHCA’s specific work environment, procedures, and parameters (e.g., policies, practices, preferences, workflows, equipment, medications, billing procedures). This process was performed jointly by teams of Cerner staff and VCHCA SMEs assigned to become “Informatics” professionals (i.e., staff who are familiar with its workflow and the EHR). Selected doctors, nurses, pharmacists, lab technicians, other clinicians, and patient billing staff were designated by VCHCA to serve in the Analyst and SME positions in the customization endeavor and were guided by Cerner “Clinical Strategist” staff. FA-08. As part of the EHR contract negotiations, VCHCA chose to implement 56 Cerner Solutions. According to reports attributed to Cerner, 56 Solutions were more than any other hospital had ever attempted to implement and activate simultaneously. Many Cerner Solutions roughly correspond to rolespecific software applications used by hospital and clinic staff to view and update patient records (e.g., a doctor entering orders in a Solution “Order Window,” or a nurse acknowledging drug administration in a Solution “Chart Window”). Other Solutions implement miscellaneous functions such as producing reports and executing special processing. [Ref-09] 4 Ventura County EHR System Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report FA-09. VCHCA provided Cerner with several parameters that would directly influence the design of the computer server farm infrastructure:  600 simultaneous users  56 Cerner Solution applications  2 hospitals  40 clinics  Ventura, California-hosted server farm [Ref-05, Ref-09] FA-10. Cerner would not agree to any requirement on window update time in the contract. Most clinical staff users consider any update time exceeding 2 to 3 seconds unacceptable because it affects concentration and degrades productivity. [Ref-04] FA-11. After agreeing on terms and conditions, Cerner and the County negotiated Cerner Contract No. 6433 for development of the VCHCA EHR project. [Ref04, Ref-10] FA-12. The contract required Cerner to develop a “Work Plan” that would describe mutual expectations and work to be performed by Cerner and VCHCA during the EHR delivery. The Cerner Work Plan was supposed to contain detailed information, including but not limited to schedule, tasks, estimates, durations, deliverables, critical events, task dependencies, resource assignments, specifications, and payment schedules. No provision of the Cerner EHR contract limited VCHCA to exclusively use the Cerner Work Plan for managing VCHCA labor and/or material. [Ref-04] FA-13. The Work Plan Cerner delivered during the course of the project was documented in a Microsoft Project file. This file was described by VCHCA as reflecting the Cerner “Event Driven” Project Management Methodology. Cerner Event Driven Project files contain only Cerner-owned tasks, with scheduling and manpower loading. They do not contain any VCHCA labor hours. The key event in the schedule was the project Go-Live milestone of July 1, 2013. VCHCA’s project manager was expected to ensure that VCHCA maintain this schedule in order to qualify for the financial incentives of HITECH Stage 1 Meaningful Use. [Ref-03, Ref-04] Upon examining the Cerner Microsoft Project file for “Implementation” Phase 1 of the EHR project—spanning the time period from “contract signing” (October 2011) through “end of maintenance” (October 2013)— the Grand Jury observed that:  Cerner did not “populate” the project file with any VCHCA labor tasks or hours.  VCHCA did not augment the project file with its own staff resources and tasks. Ventura County EHR Implementation Risks 5 Ventura County 2014 – 2015 Grand Jury  Final Report VCHCA did not create any independent project plan for the VCHCA staffing resources and tasks. FA-14. As indicated in VCHCA’s response to the 2013-2014 Grand Jury report “Healthcare Records Processes and Procedures,” VCHCA EHR Implementation resources were coordinated to link up with key-event dates specified in Cerner’s Event Driven Project plan (e.g., Project KickOff, four trips to Kansas City, Unit & Integration Test, and Go-Live deadline). The Cerner project file provided a list of milestones and expected dates that VCHCA had to meet to achieve the established Go-Live date. [Ref-05] FA-15. Throughout the Implementation period, VCHCA management and staff status reporting was handled at weekly or as-needed staff meetings. Project management coordination between Cerner and VCHCA was handled at weekly or as-needed teleconferences or in-person meetings. FA-16. All County IT projects must receive the approval of the ITC for large projects and purchases. The ITC requires the project sponsor to complete the Automation Project Assessment Questionnaire (APAQ). This document outlines the project description, scope, objectives, risk assessment, outcome, and measurements for success. The APAQ form can be found on the County Forms & Policy (F&P) Intranet website. FA-17. The October 3, 2011 APAQ for the Cerner EHR project presented to the ITC identified three goals, one measurement for success, and a minimal risk assessment.  Goal 1: To replace VCHCA’s clinical record system with a single system that complies with the HITECH provision of ARRA  Goal 2: To automate and integrate the patient accounting and supply chain management with the new clinical record system  Goal 3: To automate and integrate billing and claim management for leveraging information across the County  Measurement: The single measure of this project’s success would be achieving its first “attestation” in accordance with federal requirements under the “Stage 1 Meaningful Use” criteria by September 1, 2013.  Risk assessment: Risk would be limited to the loss of federal reimbursement allocations and the issuance of fines if the project was not started by January 1, 2012, and completed by September 1, 2013. FA-18. On October 4, 2011, the BOS approved $32 million to acquire the Cerner EHR system. This appropriation did not include provisions for computer hardware, staffing, or medical hardware, which were to be determined at a later time and funded out of VCHCA’s operating budget. [Ref-10] FA-19. As indicated in VCHCA’s response to the 2013-2014 Grand Jury report “Healthcare Records Processes and Procedures,” Cerner was responsible for system design and acquisition of the Ventura-based server farm and 6 Ventura County EHR System Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report some Cerner-specific end-user hardware. VCHCA was responsible for generic end-user hardware such as workstations, laptops, barcode scanners, and printers. [Ref-05] FA-20. Cerner performed the overall EHR system design based on VCHCA’s parameters (i.e., 600 simultaneous users, 56 Solutions, 2 hospitals, 40 clinics, and a Ventura-hosted server farm). [Ref-05, Ref-09] FA-21. EHR Project Kick-Off for VCHCA was originally planned by Cerner for month three of the contract timeline (January 2012) but did not happen until May 2012 (month seven)—a four-month schedule slip. FA-22. During the EHR Implementation Phase 1, Cerner was responsible for the EHR “Build” process that customized the basic Cerner framework to implement the specific VCHCA workflow. Cerner Clinical Strategists—working in conjunction with VCHCA-selected analysts, SME staff who were familiar with the workflow (operating as Informatics personnel in training), and VCHCA-hired contractors—were primarily engaged with Implementation tasks. Implementation involved describing, modifying, and improving the hospital and clinic workflows using formatted Cerner Design Control Worksheets (DCWs). Implementation also involved reviewing interim functionality and testing for correct Build functionality and accuracy. FA-23. VCHCA had three methods to get the staff needed for the Implementation: 1) share existing VCHCA staff; 2) share personnel from County IT staff; and 3) hire outside contractors. FA-24. In December 2011 VCHCA recognized the need to hire contract staff to help meet staffing requirements. In July 2012 (month nine of the original timeline) the BOS approved hiring requested contract staff. [Ref-12] FA-25. The HITECH Act, by requiring a short compliance period and offering enticing grant subsidies, reduced the pool of available qualified Informatics personnel. VCHCA and all other hospital and clinic institutions and EHR providers (including Cerner) were competing nationwide to secure needed Informatics staff. This situation complicated VCHCA’s ability to recruit and hire qualified EHR talent. FA-26. VCHCA did not provide sufficient analyst and SME staff to meet scheduled key dates:  Many other comparable-size Cerner customer institutions employ over 50 Informatics support staff.  Cerner’s original estimate for VCHCA’s labor for Implementation was 31.5 experienced Full-Time Equivalent (FTE) staff (analysts and SMEs). [Ref-04]  The APAQ for the EHR stated that approximately 30 dedicated clinical analysts would be needed. [Ref-11] Ventura County EHR Implementation Risks 7 Ventura County 2014 – 2015 Grand Jury  Final Report VCHCA provided on average 24 FTE staff to support Phase 1 Implementation: o 14 full-time VCHCA staff (=14 FTE) o 22 part-time “borrowed” VCHCA staff (=5 FTE) o 5 full-time contractors (=5 FTE)  VCHCA management and staff did not have prior hands-on experience with Cerner system Solutions Implementation, Build, or Maintenance.  Limiting staffing to less than Cerner-recommended and IT-requested levels helped VCHCA hold down costs. It also delayed efforts to identify and fix EHR quality issues (e.g., “bugs”) until after Go-Live. FA-27. Since Go-Live VCHCA has had a chronic shortage of available Report Writers. Report Writers are skilled in the extraction of data from the EHR database and producing formatted reports and statistics needed by various healthcare departments. VCHCA staff found that many of the stock Cerner reports did not produce the statistics or data needed by requesting departments. In March 2015 VCHCA had a backlog of over 35 reports. A typical report takes four to eight weeks to create. FA-28. Throughout the EHR Implementation in 2012 and 2013, the required ITC quarterly Project Status Reports indicated the following concerns (without quantitative supporting backup):  The project experienced delays with the design of a few modules due to lack of personnel allocations. Additional staff would have been needed to make up the lost time.  Delays in approval for additional staff impacted the ability to meet milestones for the design phase. FA-29. Neither ITC nor VCHCA took corrective action regarding the risks resulting from staff shortages and the related consequences as documented in the quarterly ITC reports. FA-30. VCHCA did not perform simulated or actual load testing before Go-Live. Testing could possibly have exposed storage capacity limitations, response time problems, and other limitations in the EHR system. FA-31. Beginning at Go-Live on July 1, 2013, and for several weeks thereafter, much of the staff had difficulty logging into the EHR system to access patient records. To overcome this situation VCHCA had to rapidly purchase and install an additional 600 Citrix licenses and triple the number of servers in the server farm by July 30, 2013. VCHCA acknowledged this situation was a direct result of underestimating the number of simultaneous users at 600. [Ref-05, Ref-09] FA-32. After adding the 600 Citrix licenses and tripling the servers in July 2013, a new problem became apparent and lingered until VCHCA abandoned its Ventura server farm and switched to Cerner Remote Hosting (RHO) in April 8 Ventura County EHR System Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report 2015. The new problem was that the “Order Entry” window response time, initially several minutes, was intolerable for most users. One of the causes was system design limitations in the server farm (e.g., the Storage Area Network [SAN] did not have enough ports) due to VCHCA’s underestimating the number of simultaneous users. FA-33. Both VCHCA’s and Cerner’s system administrators managed to speed up response time slightly while the EHR was still hosted in Ventura by adjusting system software parameters. However, they were never able to get response time to acceptable levels. VCHCA decided not to pursue further hardware upgrades to the server farm in Ventura. Instead servers and server support were switched to Kansas City by purchasing Cerner’s RHO option. FA-34. For six months after Go-Live, there were occasional planned and unplanned downtimes when the EHR network would be unavailable. During such intervals clinical staff had to temporarily revert to paper recordkeeping and then enter the paper information into the EHR when it came back online. FA-35. To protect against an outage of the EHR, Cerner has the capability to periodically backup patient records (e.g., medication prescribed/ administered, lab results) “locally” in the hospitals, independent of the central EHR server farm. These backup “724 systems” are read-only to be used for retrieval of recent patient records during a system outage. At GoLive, these 724 systems had not been configured and activated. After the Go-Live date, over a period of several months, thirty 724 systems were deployed by IT at strategic locations throughout the hospitals. FA-36. The Wi-Fi network at the Ventura County Medical Center was not adequately assessed and tested before Go-Live. The network experienced intermittent problems beginning at Go-Live and for several months thereafter. This condition interfered with staff productivity and led to frustration. FA-37. VCHCA personnel discovered that the standard Cerner-formatted prescription label did not contain all the content/dosage information that the compounding pharmacist and administering nurse needed. This deficiency and many other issues considered high priority by hospital staff were duly reported to the Help Desk and to management as patient care issues. The Pharmacy label format issue was not resolved for nine months. FA-38. Before hardware was ordered, focus groups were used to gauge end-user hardware preferences. At these sessions, selected staff got to view and touch a variety of end-user equipment, but the equipment was not tested in a live environment as it would be used in the hospital. Users did not have an opportunity to evaluate the hardware as it would be used in their normal work environment. For example, tablets were selected as a choice for nurses. But after Go-Live, nurses tried to use them for charting but found they were inappropriate for a variety of reasons (e.g., the charting area was too small with the current Cerner Solutions; the pop-up on-screen Ventura County EHR Implementation Risks 9 Ventura County 2014 – 2015 Grand Jury Final Report keyboard covered valuable chart area; battery life was only a couple of hours). The tablets had to be replaced with alternative hardware. In addition, the laptops with built-in scanners were focus group selected, but in practice with the EHR system they were impractical to use and had to be replaced. [Ref-09] FA-39. The purchase requisitions for end-user hardware needed to support the EHR Go-Live event were forwarded to VCHCA administration in December 2012 by the VCHCA IT organization. But the end-user hardware was not ordered until May-June 2013. Thus a significant amount of equipment was unavailable to be properly configured and in place for staff to use for check-out and refresher training in their work environment before Go-Live. FA-40. Due to inadequate planning, a significant number of workstations and tablets had to be ordered after Go-Live. FA-41. At Go-Live, many of the computer printer assignments were incorrectly configured by IT technicians. Printouts were directed to out-of-area printers that potentially exposed critical data until the default destination printer was located and the printout picked up by the requester. It took many weeks to get all associated printer problems fixed. FA-42. There were EHR Implementation related concerns regarding potential risks due to a variety of factors. Issues of concern included: 10  Due to the frequent early EHR instability, staff had to temporarily administer medical care without access to recent patient records; they had to fall back to handwritten paper recordkeeping; and then, retroactively, update the EHR when it became accessible again.  Saturation of EHR login capacity led to frequent staff login failure attempts, a condition that went unresolved during the first several weeks after Go-Live.  Frequent crashes of the EHR during first 6 months after Go-Live  Incomplete/inadequate/inconsistent data entry windows, order sentences, and pick-list choices used by physicians, nurses, pharmacists, and other healthcare staff to select from in the various Solution charts  Sluggish response times for users launching/updating Solution window displays  Printer queues (particularly label printers used by the Pharmacy and Labs) frequently stalled and stopped printing labels. Pharmacy staff had to resort to handwriting labels—usually for several hours. On third shift or weekends, IT support was not readily available to fix the blockage. The handwritten labels used to work around EHR outages precluded the automatic checks normally performed by the EHR when verifying correct medication/patient administration. This situation was not resolved for over nine months after Go-Live when IT reconfigured printer servers in the server farm. Ventura County EHR System Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report FA-43. While there are no reported incidents of harm to patients because of EHR problems, there are documented occasions that potentially could have put patients in danger if alert clinical staff had not taken corrective actions with workarounds. FA-44. During the EHR Implementation, the communication paths within VCHCA’s organizational structure became ambiguous. IT problems involving patient care tended to be reported to IT personnel and may not have reached clinical management. FA-45. Over 50,000 hours of EHR user training was provided over a four-month period to 3,000+ clinicians, ancillary, and support VCHCA staff. Typical training involved 12 to 16 hours of instruction and hands-on training spread over multiple sessions in nine dedicated training rooms. [Ref-05, Ref-09] FA-46. The user training did not include competency testing before Go-Live. It was also noted that training did not satisfactorily address learning retention losses with timely hands-on refresher courses using an EHR domain and more robust training materials. Nor did it adequately stress the importance of accuracy using discipline-specific examples of correct vs. incorrect situations (e.g., data entry accuracy). FA-47. Immediately after EHR Go-Live, many of the VCHCA staff were not comfortable using the system in spite of the training opportunities that had been provided and the availability of experts to help. Many users were confused and frustrated—a situation that was compounded by unplanned system downtime, slow window response time, and frequent failure of login attempts. FA-48. Many factors contributed to patient billing problems associated with the EHR:  Some users did not consistently enter data correctly into Solution windows, which ultimately led to downstream uncollectable patient billing.  Beginning with Go-Live, much of the patient information used for billing by the EHR was not accurate. Many bills produced from the EHR were rejected by the “Scrubber” checking process and simply set aside to be looked at later for diagnosis and correction.  By second quarter 2014, the backlog of unresolved billing produced by the EHR was 9 to 10 months behind, due to rejected claims having incorrect/inconsistent/missing data on patient billing.  After a deep-dive analysis by VCHCA, the rejected claims were found to be due to a variety of problems, most notable being data entry issues such as: o Ineffectual training Ventura County EHR Implementation Risks 11 Ventura County 2014 – 2015 Grand Jury Final Report o Lack of attention by staff entering patient and treatment data into the EHR o Lack of proper supervisory oversight FA-49. Diagnosis of the VCHCA’s billing issues was initially compounded by a variety of problems including:  A backlog of growing rejected billing  A lack of adequate staffing resources to correct the problems  Cerner “canned” report writing applications that produced inadequate visibility into the billing problems  A shortage of skilled staff to quickly develop new and more detailed diagnostic reports from the Cerner database FA-50. Several insurance reimbursement entities such as Medi-Cal, Gold Coast, and private insurance companies limit the length of time allowed between patient treatments or discharge from the hospital until a medical provider submits accurate billing. Following the EHR Go-Live event, a significant portion of VCHCA billing claims had not been corrected within the time limit and were denied payment. As of March 2015, this potentially uncollectable amount may have exceeded millions of dollars. The VCHCA was reported to be in the process of trying to reduce this collections deficit. FA-51. VCHCA successfully met the Stage 1 Meaningful Use requirements:  Both hospitals completed a successful 90-day attestation cycle on September 30, 2013.  148 eligible providers (99%) completed individual 90-day cycles by December 1, 2013. [Ref-09] Findings FI-01. The Grand Jury found that after Go-Live a significant level of concern was raised by clinical staff to IT regarding potential impacts of observed EHRrelated risks on patient well-being. (FA-27, FA-37, FA-42, FA-43, FA-44) FI-02. The Grand Jury found systemic deficiencies in the process used by VCHCA to develop and vet the adequacy of the EHR project requirements specification. For example: 12  The “number of simultaneous users” specification was clearly developed using an inadequate analysis strategy, and the specification reasonableness was not validated by appropriate independent EHR SMEs.  A performance requirement for a maximum window update time was not developed. VCHCA failed to develop a mutually agreeable Ventura County EHR System Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report specification with Cerner in the contract, as part of an EHR acceptance requirement.  VCHCA did not have an effective mechanism to gauge the comprehensiveness and quality of the EHR implementation and its test development process.  VCHCA did not specify the minimum required FTE staffing level that IT/Informatics management and an independent EHR SME agreed was both necessary and sufficient to fully accomplish the goals of the project. Without this staffing it was not possible to conduct rigorous testing in the time period specified by the Cerner Event Driven Project file. (FA-02, FA-05, FA-09, FA-10, FA-17, FA-26) FI-03. The Grand Jury found no evidence that project requirements were formally specified, which precluded generating a complete and quantifiable test plan to verify overall EHR quality throughout the Implementation stage. The actual project was guided primarily by untestable goals to meet the federal Stage 1 attestation. (FA-02, FA-17, FA-22, FA-41, FA-42) FI-04. The Grand Jury found no effective independent review of the EHR project before the release of the RFP, before contract signing, nor continuing periodically during the course of the project. Such an independent review would include SMEs from outside the VCHCA who have EHR Implementation experience and also clinical staff with experience in the VCHCA. (FA-02, FA-05, FA-17, FA-28, FA-29) FI-05. The Grand Jury found that the lack of an effective Risk Management Plan resulted in significant impact on project quality and cost. Developing and maintaining such a plan would have exposed potential problems and triggered mitigations that could have avoided or lessened the undesirable consequences. For example, training did not satisfactorily address learning retention losses with timely hands-on refresher courses using an EHR domain and more robust training materials. Nor did it adequately stress the importance of accuracy using discipline-specific examples of correct vs. incorrect situations (e.g., data entry accuracy). (FA-03, FA-10, FA-17, FA27, FA-39, FA-46) FI-06. The Grand Jury found that EHR project execution was directed solely by the Cerner Event Driven Methodology and key events and dates in the Cerner Microsoft Project file—to the exclusion of other important VCHCAspecific considerations. The EHR Implementation had significant undiscovered problems at Go-Live caused by issues such as: the inflexible July 1, 2013 Go-Live date; the 14-month integration schedule; the lack of slack in the schedule; and the lack of documented testable requirements before proceeding to the next stage. As a consequence, waiting to address residual quality issues (e.g., software bugs) until after Go-Live made patient care more challenging in the interim. However, due to alert staff, Ventura County EHR Implementation Risks 13 Ventura County 2014 – 2015 Grand Jury temporary workarounds were developed to standards. (FA-12, FA-13, FA-22, FA-26, FA-43) Final Report maintain patient care FI-07. The Grand Jury found that, by failing to have quantitative data to predict impacts on the Go-Live date, project management was unable to convince VCHCA administration to support the project staffing levels and ordering dates of materials necessary to deliver an operationally acceptable product. EHR project management did not utilize industry-accepted best practices project management techniques (e.g., PMI) for project planning and quantitative reporting of VCHCA labor and material schedules, nor for status against those schedules. (FA-12, FA-13, FA-14, FA-15, FA-24, FA26, FA-28, FA-29, FA-31, FA-32, FA-33, FA-36, FA-37, FA-38, FA-39, FA40, FA-41, FA-42, FA-48, FA-49) FI-08. The Grand Jury found that VCHCA research and ITC status reports both indicated a shortage of personnel assigned to the EHR project. However, VCHCA and ITC failed to take the necessary and timely corrective action. (FA-26, FA-28, FA-29) FI-09. The Grand Jury found that VCHCA failed to develop a project plan to reflect VCHCA staffing hours and resources necessary to integrate with the Cerner production schedule. (FA-13) Recommendations R-01. The Grand Jury recommends that the Board of Supervisors direct the VCHCA to establish a policy to charter Independent Review Boards composed of project-applicable SMEs to review all of its capital projects. In particular these Boards should review adequacy and accuracy of technical specifications in RFPs and proposed contracts. They should periodically review all capital projects sponsored by VCHCA for project risks and adequacy of mitigation efforts. (FI-02, FI-03, FI-04, FI-05, FI-06, FI-07) R-02. The Grand Jury recommends that the Board of Supervisors direct the VCHCA to establish a policy that all capital projects sponsored by VCHCA create and periodically update a Risk Management Plan (e.g., utilizing ISO guidelines) to identify project risks and their associated impacts, to propose mitigation activities, and to periodically track and publish the status of risks and mitigation efforts. (FI-04, FI-05) R-03. The Grand Jury recommends that the Board of Supervisors direct the VCHCA to establish a policy that all capital projects sponsored by VCHCA utilize industry-accepted best practices project management tools (e.g., PMI) for project planning and quantitative status reporting of progress against the plan for both labor and material. (FI-06, FI-07, FI-09) R-04. The Grand Jury recommends that the Board of Supervisors direct the VCHCA to establish an Informatics Department with appropriate full-time staffing to satisfy the needs for maintenance and future upgrades of the VCHCA EHR. To be effective in this role, the Informatics Department should 14 Ventura County EHR System Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report report directly to clinical VCHCA management to ensure that patient care is always given proper clinical concern and priority. (FI-02, FI-03) R-05. The Grand Jury recommends that, for any future capital projects of the VCHCA, the Board of Supervisors assign to the ITC the responsibility and authority to: regularly monitor achievement of stated project goals; ensure compliance with the approved project process; enforce utilization of quantitative data to measure project progress; identify problems; and assure that prompt corrective action is taken. (FI-03, FI-04, FI-05, FI-07, FI-08, FI-09) Responses Responses required from: Ventura County Board of Supervisors (FI-02, FI-03, FI-04, FI-05, FI-06, FI-07, FI-08, FI-09) (R-01, R-02, R-03, R-04, R-05) Responses requested from: County of Ventura, County Executive Officer (FI-02, FI-03, FI-04, FI-05, FI-06, FI-07, FI-08, FI-09) (R-01, R-02, R-03, R-04, R-05) Ventura County Health Care Agency (FI-02, FI-03, FI-04, FI-05, FI-06, FI-07, FI-08, FI-09) (R-01, R-02, R-03, R-04) Commendations The Grand Jury commends the many VCHCA employees who, through dedication and hard work, were able to meet federal requirements and manage patient care, while overcoming inadequate project requirements specification and risk management. References Ref-01. Wikipedia, The Free Encyclopedia “American Recovery and Reinvestment Act of 2009,” http://en.wikipedia.org/wiki/American_Recovery_and_Reinvestment_Act _of_2009 (accessed May 30, 2015). Ref-02. Wikipedia, The Free Encyclopedia, “Health Information Technology for Economic and Clinical Health Act,” http://en.wikipedia.org/wiki/Health_Information_Technology_for_Econo mic_and_Clinical_Health_Act (accessed May 30, 2015). Ref-03. Centers for Medicare and & Medicaid Services, “2014 Definition Stage 1 of Meaningful Use” webpage, http://www.cms.gov/Regulations-andGuidance/Legislation/EHRIncentivePrograms/Meaningful_Use.html (accessed May 30, 2015). Ref-04. Cerner Corporation, Contract 6433, Ventura County Board of Supervisors Meeting, October 4, 2011, Addendum Agenda Item #31, Attachment “Approval of, and Authorization for the Purchasing 15 Ventura County EHR Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report CernerAgreement.pdf” (PDF number 2), http://bosagenda.countyofventura.org/sirepub/agdocs.aspx?doctype=ag enda&itemid=42875 (accessed May 30, 2015). Ref-05. Fisher, Barry. Ventura County Healthcare Agency, May, 29, 2014, “Response to 2013-14 Grand Jury Report: Healthcare Records Processes and Procedures,” http://vcportal.ventura.org/GDJ/docs/reports/201314/Healthcare_Records-Response-HCA.pdf (accessed March 30, 2015). Ref-06. Wikipedia, The Free Encyclopedia, “Risk Management” webpage, http://en.wikipedia.org/wiki/Risk_management#Enterprise_risk_manag ement (accessed May 30, 2015). Ref-07. International Organization of Standardization, “ISO 31000: 2009 Risk management – Principles and guidelines” webpage, http://www.iso.org/iso/catalogue_detail.htm?csnumber=43170 (accessed May 30, 2015). Ref-08. The Public Risk Management Association, “A structured approach to Enterprise Risk Management (ERM) and the requirements of ISO 31000” webpage, https://www.theirm.org/media/886062/ISO3100_doc.pdf (accessed May 30, 2015). Ref-09. Fisher, Barry; Theobald, Terry. Ventura County Healthcare Agency, “CERNER UPDATE: March 2014,” Attachment to Agenda Item #29, Ventura County Board of Supervisors Meeting, March 18, 2014, http://ventura.granicus.com/MediaPlayer.php?view_id=67&clip_id=353 0 (accessed March 30, 2015). Ref-10. BOS approval of $32 million to purchase Cerner EHR, Attachment “Board Letter.pdf” (PDF number 1), http://bosagenda.countyofventura.org/sirepub/agdocs.aspx?doctype=ag enda&itemid=42875 (accessed May 30, 2015). Ref-11. Herzog, David. Ventura County Healthcare Agency, “County of Ventura, Automation Project Assessment Questionnaire, Project: Electronic Health Record, October 3, 2011,” County of Ventura Intranet website, http://vcportal.ventura.org/committees/itc/docs/274.pdf (accessed May 30, 2015). Ref-12. Ventura County Board of Supervisors (BOS) letter, July 24, 2012, Agenda Item Number 38, “Approval of and Authorization for the Health Care Agency (HCA) to Expend a Total of $5,748,500 for the Implementation of the Electronic Health Record (EHR) Project for Staffing…,” http://ventura.granicus.com/DocumentViewer.php?file=ventura_f1970cf b9c0ffdc20f1c6683e7be2571.pdf&view=1 (accessed May 30, 2015). 16 Ventura County EHR System Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report Glossary TERM DEFINITION 724 computer A specially equipped local workstation that copies several hours of patient records into a protected area Analyst A person with detailed knowledge about design, build, implementation, and maintenance of specific EHR Solutions APAQ ITC Automation Project Assessment Questionnaire; a document required before requesting Board of Supervisors funding of an IT project [Ref-11] ARRA American Recovery and Reinvestment Act of 2009; includes funding for HITECH and Meaningful Use [Ref-01] Attestation The act of certifying a formal document which states that something is true, correct, or real; in an EHR context, it is affirmation that the installed system meets federal Meaningful Use requirements Best Practices Project management methodologies and work products promoted by industry recognized standards organizations (e.g., PMI, ISO) BOS Board of Supervisors, Ventura County Bug Software defect in Implementation of EHR Build process Cerner’s part of EHR customization (coding EHR with VCHCA specific workflows) Cerner The Cerner Corporation of Kansas City, MO Citrix Front end server (i.e., for user verification) CMS Centers for Medicare and Medicaid [Ref-03] Contract employee Non-VCHCA employee with special skills and knowledge hired to fill a temporary expertise need County County of Ventura EHR Electronic Health Record EHR domain Active Online EHR system Eligible professional Independent medical professional such as a doctor who attests to meaningful use and Ventura County EHR Implementation Risks 17 Ventura County 2014 – 2015 Grand Jury Final Report TERM DEFINITION receives less than 10% of compensation in form of hospital-based billing End user hardware Computer workstations, laptops, barcode scanners, and printers used for patient data entry and viewing by hospital and clinic staff Event Driven Methodology A project planning methodology where work is determined by the customer’s response Project F&P Forms and Policy; a Ventura County Intranet website Focus group Staff sharing common interests asked to advise on a particular topic FTE Full Time Equivalent; percentage of labor hours worked relative to a full time employee Go-Live Start of online deployment of VCHCA EHR (July 1, 2013) Grand Jury 2014-2015 Ventura County Grand Jury HITECH Health Information Technology for Economic and Clinical Health Act (2009); funds the allocation of monies for EHR Meaningful Use under ARRA [Ref-02] Implementation process VCHCA’s part of EHR customization (clinical workflow definition and test of fidelity of Cerner’s Build process) Informatics Specially skilled staff with expertise to understand clinical workflows and interactions, and modify parameters in the EHR Intranet An organization’s internal network, available to the general public ISO International Organization for Standardization IT Information Technology ITC County Information Technology Committee Kansas City, MO Cerner headquarters, training venue, and location of RHO server farm Kick-Off Start of Cerner (May 2012) Legacy Of or pertaining to old or outdated computer hardware, software, or data that, while still 18 EHR project at not VCHCA Ventura County EHR System Implementation Risks Ventura County 2014 – 2015 Grand Jury Final Report TERM DEFINITION functional, may not work well with up-to-date systems Meaningful Use Sets of criteria with deadlines established by HITECH & CMS intended to improve patient care by healthcare providers; includes possible financial incentives and penalties to coerce compliance; involves coordination of care, improved safety, patient engagement, and improved population health. Meditech A legacy EHR previously in use at VCHCA (prior to HITECH) Microsoft Project A project management software program that is designed to assist a project manager in developing a plan, assigning resources to tasks, tracking progress, managing the budget, and analyzing workloads Milestone An identifiable stage of a project Order Entry Window Pop-up window in various Cerner Solutions; allows healthcare professionals to enter orders (reported to be very slow) Patient care issue Concern possibly affecting patient care Pick-list A multiple-choice list of items in a menu from which a user can make a selection using a computer mouse or keypad device; similar to a drop-down menu PMI Program Management Institute; an organization for promotion, standardization, and improvement of project management professionals Project file Microsoft Project management file Quality Measure of success in achieving an EHR free of bugs that could compromise patient care or hospital productivity Report writer Skilled professional who can produce custom filtered reports and statistics from the EHR database Response time Time for a window to open or update; measured from keystroke or mouse click to update (2 seconds is considered marginally acceptable) Ventura County EHR Implementation Risks 19 Ventura County 2014 – 2015 Grand Jury Final Report TERM DEFINITION RFP Request For Proposal RHO Cerner Remote Hosting Option (EHR servers located in Kansas City, MO) Risk Management Plan A list of plausible risks to a project’s success and plan to mitigate each risk SAN Storage Area Network; multi-ported central database storage unit Scope A project’s defined objectives Scrubber Process that checks patient billing for accuracy Server farm Array of host computers for the EHR SME Subject Matter Expert; has expert knowledge about a specific workflow, process, or instrumentation Solution A function-specific Cerner software application used by hospital and clinic staff to view and update patient records, and to interface equipment into the EHR system Stage 1 1st Stage Meaningful Use Criteria [Ref-03] Success Project success is measured by product completeness, quality, timeliness, budget compliance, and degree of customer satisfaction. VCHCA Ventura County Health Care Agency WBS Work Breakdown Structure; a project broken down to a system of manageable tasks that can be arranged to show dependencies and resources Wi-Fi Short-range wireless communication protocol Work Plan Document describing mutual expectations and work to be performed by a customer and client during the project Workflow Description and sequencing of tasks and work products 20 Ventura County EHR System Implementation Risks