North Carolina’s Multi-Payer Medicaid System Test Management Services

Public Knowledge®’s Test Management services were instrumental in assisting the North Carolina Department of Health and Human Services (DHHS) successfully implement a new Medicaid Management Information System (MMIS). Public Knowledge® helped the agency receive CMS certification ahead of schedule.

The Situation

The Medicaid Management Information legacy system was built 35 years ago on technology too inflexible to the need for ongoing modifications and did not support the State’s vision for the first public multi-payer system in the US. At same the same time, the MMIS program had become more complex due to the need to conform to the regulatory framework of the Affordable Care Act (ACA).

To address these issues, DHHS initiated the NC Tracks project to replace its aging MMIS and the related Reporting & Analytics (R&A) system. To accomplish these goals, DHHS established contracts with systems integration firms experienced in the implementation of the MMIS and R&A systems. Development work began in early 2009 and was completed in January 2016.

Due to the complexity and overall technical risk inherent in information technology projects of this magnitude. The DHHS determined that outside independent test management support was necessary to assist the State in its obligations to thoroughly test the Medicaid software throughout the project’s life and help ensure a system that met federal certification requirements.

The Solution

Public Knowledge® delivered an approach to staff each development track with a dedicated Test Lead, and an overall Test Manager to coordinate all the team’s activities with DHHS and the systems integration vendors. It was determined that full-time, on-site support was required, so the Public Knowledge® Test Management Team was co-located with the development teams at the client’s work location in Raleigh, NC.

The Public Knowledge® Team provided multiple test management support services including the review and assessment of vendor-led test planning deliverables and test artifacts produced by the vendors for the two sub-projects. Public Knowledge® monitored vendor-led testing through the analysis of actual test results to determine readiness for production of each release. Public Knowledge® reviewed and refined defect management procedures to ensure proper defect classification and turnaround.

The State-led testing was managed by Public Knowledge® through coordination of the Medicaid agency participants, determination of testcases and data scenarios for User Acceptance Test (UAT), and final evaluation of the system for live production status. Public Knowledge® Test Management Services included:

• Planning, executing, and reporting of system/user testing, involving both internal and outsourced development
• Coordinating and training of State-controlled resources and schedule for testing
• Providing expertise in the chosen testing software, Silk Central Test Manager; providing custom reports and analysis of testing velocity
• Interfacing with System Integrators to assist and oversee its testing effort to ensure the success of the State’s testing effort. Interface with DHHS and non-DHHS organizations and individuals, as necessary, to enhance the testing program.
• Focusing efforts on a value-added fashion that achieved the best return on investment and overlaps the developers’ efforts only where prudent and necessary
• Participating in project risk management programs to ensure that testing was used as an appropriate identification and management tool; and verification of non-system elements of the overall solution, such as manual processes, as directed by the State

Public Knowledge®’s role was to act as a testing coach, expert, and quality partner. Throughout the Test Management effort, Public Knowledge® facilitated the work of the NC Tracks Project by developing constructive relationships with the State and with the System Integration vendors, with the goal of ensuring comprehensive test coverage to maximize software quality. For example, Public Knowledge® uncovered an issue with the scope, coverage, and the organizing model for the MMIS vendor’s automated test suite. This finding enabled the State Team Subject Matter Experts to clarify the Medicaid coverage ‘holes’ present within the System Integrator’s regression testing approach and correct the gaps before they caused significant delays and rework.

 The Outcome

Public Knowledge®’s efforts were a significant factor in the agency receiving CMS certification of the MMIS on the first attempt, ahead of schedule. Joe Cooper CIO of NC DHHS.

Public Knowledge®’s Test Management services were instrumental in assisting DHHS in a successful implementation. Test Plans, Scenarios and Scripts were consistently submitted on time and were of a high quality. The State testers were trained on proper procedures to execute tests, record the results, and document defects. Public Knowledge® was also effective in tracking defect resolution and conducting effective regression testing.

Of particular importance was the work the Public Knowledge® team conducted to help make valid go/no go implementation decisions. Joe Cooper, former CIO of DHHS, commented, “Public Knowledge® was a valuable partner in assisting the agency in deploying a high-quality system that was comprehensively tested. Their team’s combination of testing expertise and MMIS knowledge gave me and our user community confidence that the system would help us deliver more timely and accurate results to our recipients and providers. Public Knowledge®’s efforts were a significant factor in the agency receiving CMS certification of the MMIS on the first attempt, ahead of schedule.”

Ultimately, the project was completed successfully with a deployment of cutting-edge technology. The quality of NC Tracks met the technical requirements, and the business needs of DHHS were fulfilled. Public Knowledge®’s strong local team, staffed with certified test management professionals and Medicaid subject matter experts inspired confidence in the deployment readiness of this MMIS implementation.

In its first year, NC Tracks processed 200 million claims and paid $10.3 billion to healthcare providers. Providers can now access multiple systems through a single sign-on portal for a near real-time view of claims that are being processed more efficiently, and effectively lowering the administrative burden, and cost, of delivering quality health care to eligible families in North Carolina.