Framework for an Impactful Continuous Quality Improvement Process

Introduction

The environment of child welfare organizations is constantly changing, yet the objective remains constant: the well-being of children and promoting family resiliency. Agencies must have optimized Continuous Quality Improvement systems and implement CQI processes for improved outcomes and sustainable changes.

The Children’s Bureau of the Administration for Children and Families (ACF) issued an Information Memorandum in 2012 that acknowledged overall improvements in the ability of states to meet the basic requirements of a Quality Assurance (QA) system based on findings from the first two rounds of the Child and Family Services Reviews (CFSR) and corresponding program improvement planning activities. The Children’s Bureau further indicated, however, that state QA systems needed to be extended and refined, emphasizing the importance of child welfare agencies’ having strong Continuous Quality Improvement systems to both achieve and maintain positive outcomes for children and families.

Often, there is a lack of clarity regarding the differences between Quality Assurance and Continuous Quality Improvement. The table below (adapted from Alan Dever’s 2004 book, Public Health Practice and Continuous Quality Improvement) outlines the major differences between QA and CQI:

Public Knowledge® understands the subtle-but-important differences between a QA and a CQI system, and we firmly believe that for agencies to create and sustain changes that positively impact children and families, a robust and well-functioning Continuous Quality Improvement system is paramount.

The Children’s Bureau defines CQI in its 2012 Information Memorandum as “the complete process of identifying, describing, and analyzing strengths and problems and then testing, implementing, learning from, and revising solutions. “It suggests there are five components that are essential to a well-functioning CQI system. The Center for States, as part of the Child Welfare Capacity Building Collaborative and which is funded by the Children’s Bureau, further highlights the essential role that CQI plays in child welfare by offering seven key sub-domains of a high functioning CQI system, which fall under the two broader domains of Culture and Climate and Functional CQI Components.

5 Essential Components of a Functioning CQI System CB IM (2012)

1. Administrative Structure.
2. Quality Data Collection.
3. Method for Ongoing Case Reviews.
4. Process for Analysis and Dissemination of Quality
   Data.
5. Process for Providing Feedback to Stakeholders and
   Adjusting Programs and Processes as Needed.

7 Sub-Domains of a Functioning CQI System
Center for States – Focused CQI Services (2016)

Domain 1 – Culture and Climate

1. Leadership and Modeling.
2. Staff and Stakeholder Engagement.
3. Communication.

Domain 2 – Functional CQI Components

1. Foundational Administrative Structure to
   Oversee and Implement CQI.
2. Quality Data Collection, Infrastructure,
   Extraction, Analysis and Dissemination.
3. Case Record Review Process.
4. Application of CQI Findings.

In blending these two approaches, Public Knowledge® has found that both structure and culture are critical to developing and maintaining a CQI system that can monitor and promote the agency’s outcomes. Therefore, Public Knowledge® believes three primary activities are critical for its approach to addressing agencies’ CQI needs: investing in a CQI structure, promoting an organizational culture that supports CQI, and aligning the CQI structure and organizational structure with key practices.

Role of Structure and Organizational Culture in Promoting Effective Continuous Quality Improvement

Having a solid Continuous Quality Improvement infrastructure in place will be ineffective if there is no operating agency culture to lead and support it, much in the same way that having leadership and support for CQI is unlikely to be successful if the structural components for CQI are not fully established or working in a sustainable way. Agencies often put tremendous time and money into developing an agency CQI plan, hiring new CQI staff, initiating a case review process, or investing in technology upgrades with the assumption that their agency’s CQI process is officially established and functioning, but then do not put enough thought into how the agency’s broader organizational culture impacts the success and sustainability of the CQI process. Both elements, CQI structure and organizational culture, are vital and also mutually dependent on one another for an agency’s CQI process to be successful.

Investing in a CQI Structure

Having solid structural components in place to support an agency’s Continuous Quality Improvement system is essential to supporting and advancing broader organizational goals and outcomes. It is within the infrastructure of an agency’s CQI system that the key CQI activities take place, such as qualitative case record reviews or CQI team meetings, and where the opportunities to share data and other vital information occur. Public Knowledge® collaborates with child welfare agencies to establish and operationalize the types of core components that will serve as the foundation to a fully functional and sustainable CQI system and also ensure that key CQI activities are operationalized in a consistent manner throughout the entire agency. Such components include the following.

A Continuous Quality Improvement (CQI) Plan

A successful CQI system begins with the establishment of a comprehensive CQI plan, one that is developed with the input of agency staff and stakeholders and is implemented consistently throughout the entire agency. Public Knowledge® assists agencies to develop a CQI plan that aligns with their practice model; describes the full scope of CQI activities, participant roles and responsibilities, and processes for the collection and use of quality data; and details a defined improvement cycle including a feedback loop for communicating CQI results. ledge® White Paper Series 2017

A Problem-Solving or Logic Model

CQI should help agencies answer key questions by identifying root causes and using a problem-solving approach. It is important to map out steps that a child welfare system will take to move from identifying a concern to implementing a change to try to address the concern. Public Knowledge® relies on the tenets of implementation science, including identifying the problem and root causes, developing a theory of change based on an implementation plan, and using data to establish baselines and monitor progress tied to key performance indicators, all of which are managed and monitored through CQI teaming structures.

A Continuous Quality Improvement Teaming Structure

It is important to have a fully functional teaming structure in place to support the active involvement of staff and stakeholders in the agency’s CQI process. An effective teaming structure lays the foundation for establishment of a shared sense of community among participants about key CQI tasks and responsibilities and helps ensure that teams are meeting consistently, using data, and focused on solving problems, and improving outcomes. Essential members of CQI teams include staff representing all levels and program areas of the agency, as well as key stakeholders such as children and families the agency and community providers serve.

A Data Plan

 A comprehensive data plan is a key structural component of an agency’s approach to Continuous Quality Improvement. Data plans serve as an invaluable tool for an agency to have ongoing accurate data about how its system is operating. Such a plan includes formulating data questions around identified areas of agency inquiry; determining which data will be collected and analyzed, as well as potential data resources; preparing reports that provide analysis and insight into whether or not the agency is achieving its goals or desired outcomes; validating the accuracy of the data; and ongoing monitoring and testing of program improvement efforts. Already having a data plan at the onset of the problem-solving cycle also guides the collection of data by establishing early on the specific questions that the data will address and ensures that the correct data are collected to answer these questions.

A Qualitative Case Review Process

Linking information obtained through a qualitative case record review process to an agency’s desired key performance measurements is vital to understanding what is behind an agency’s quantitative data: namely the connection between an agency’s casework practices, the quality of services being provided, and the outcomes an agency wants to achieve. Public Knowledge® believes in the intrinsic value of qualitative data and works collaboratively with agencies to design and implement a qualitative case record review process that strengthens an agency’s ongoing CQI process. Public Knowledge® feels that qualitative case reviews are critical to better understanding how children and families are experiencing the child welfare agency, beyond the aggregate numbers associated with outcome data.

A Reporting and Feedback/CQI Communication Plan

Among the most critical assignments of an effective CQI system is to report findings of all CQI inquiries to relevant staff and stakeholders, and to aid in interpreting the findings of such inquiries, and then relating those findings to areas of concern. Public Knowledge® assists agencies to develop and implement communication plans aimed at effective messaging of CQI activity findings to staff and stakeholders and connecting best practices to improved outcomes.

Promoting an Organizational Culture That Supports Continuous Quality Improvement

Having an organizational culture to support the structural aspects of CQI is essential if CQI is to be effective. This requires:

Committed Leadership

It is vital that agency leadership foster an organizational culture where staff at all levels and stakeholders are inspired to participate in CQI activities. This occurs when the director and other senior leaders speak to and model behaviors that are integral to the effective functioning of CQI systems, which include being data-driven and always in pursuit of root causes when addressing challenges. Leadership can also demonstrate its commitment to CQI by securing the types of resources that support CQI, such as technology, training, and dedicated staffing. Public Knowledge® strives to support agency leaders to create an organizational culture providing clear and consistent messaging to staff and stakeholders that CQI is not an initiative; rather, it is integrated into how the agency does its work every day to improve outcomes for children and families.

Always Data Driven and Evidence-Informed

High functioning CQI systems are driven by using quality data, whether in setting policy, supporting decision making, or monitoring the provision of services to children and families. However, it is important to have a balance so agencies are not inundated with data that, while providing interesting and even useful information, may be too voluminous or overwhelming to be effective. Public Knowledge® works with agencies to reduce the number of reports to align with agency priorities, while improving the accuracy of these reports and their usefulness to agency staff. It is essential that an agency’s organizational culture ensures ready access to timely, concise, and quality data about the children and families it serves, and that there is transparency by posting relevant and appropriate data and reports for public use. Public Knowledge® supports agencies in their efforts to establish and sustain an organizational culture that promotes the access to and the use of quality data for identifying and solving problems.

Active Engagement of Staff and Stakeholders in CQI Activities

A successful CQI system is dependent on the active engagement and participation of agency staff and stakeholders. This requires an organizational culture where participation in CQI activities is a demonstrated high priority throughout the agency, and both staff and stakeholders feel valued and empowered by their participation. Public Knowledge® works with agencies to identify opportunities for staff and stakeholders to participate successfully in the CQI process in ways that support achieving improved outcomes.

Open Communication and Collaboration

Communication is a key catalyst in ensuring a CQI system’s effectiveness. Open communication helps guarantee that staff and stakeholders clearly understand the direction in which the agency is heading and why, and the connection between practice and outcomes. Public Knowledge® works with agencies to develop structured communication processes and activities to help engage and educate participants on such vital structural CQI components as the agency’s CQI plan, teaming structure, data plan, review processes, and logic model.

Environment that is Learning Oriented and Improvement Focused

A successful organizational culture is learning-oriented. Staff and stakeholders are encouraged to ask questions about data, identify bright spots, as well as opportunities for improvement. The overall environment throughout the agency is not punitive, but rather encourages and provides opportunities for staff to be innovative in identifying and addressing problems. Public Knowledge® understands the importance of agencies engaging staff and stakeholders in the process of analyzing data and using that information to inform practice and policy.

Aligning CQI Structure and Organizational Culture with Key Agency Practices

Public Knowledge® believes in the importance of ensuring that an agency’s CQI structure and culture are aligned with the agency’s vision and mission and that CQI also fully supports the implementation of the agency’s core casework practices and services. CQI is most effective when it serves as a vehicle to make certain that all components of a child welfare system have a clear understanding of the agency’s vision and work continuously and collaboratively toward achieving sustainable improvements in practice and outcomes. An agency’s practice model or other core casework practices and desired outcomes should be the basis upon which a successful CQI model is built. This includes the following.

Mission and Values

CQI efforts should identify and communicate areas of systemic strengths and areas needing improvement and support the development of strategies to improve areas of performance.

Practice Model and Organizational Goals

CQI functions should facilitate the ongoing assessment of child welfare performance in relation to the mandates and goals associated with federal or consent decree requirements.

Public Knowledge® assists child welfare agencies to design CQI activities that support and reflect their unique child welfare practice models, and ensure they are integrated into their specific business and leadership structures.

1 Administration for Children and Families. (2012). Information Memorandum. Continuous quality improvement in title IV-B and IV-E programs.
ACYF-CB-IM-12-07. Retrieved from http://www.acf.hhs.gov/programs/cb/resource/im1207

2 https://learn.childwelfare.gov/content/focused-continuous-quality-improvement-services

NOTE: Public Knowledge® was formerly the Center for the Support of Families.