This is an informational report the Commission requested for staff to provide an analysis of the progress and efficacy of the Quality Assessment Program (QAP), as well as measures taken to continually improve its outcomes, and includes a review of costs/expenditures for the past three years, Fiscal Years 2018/2019, 2019/2020, and 2020/2021.

POST was established by the Legislature in 1959 to set minimum selection and training standards for California law enforcement, and facilitates the ongoing development, delivery, and improvement of training for peace officers throughout California to meet legislative mandates or other critical law enforcement training needs. POST staff manages mandated and elective training courses designed to develop and expand the knowledge and skills necessary to improve the quality of police services in California communities. 

Each training course can be certified to several presenters, and courses can be scheduled throughout the year at different locations within the state. 

As of March 31, 2021, there are 4,719 training courses certified to 941 presenters throughout the state.

These courses are certified by means of reviewing a training package that includes an Expanded Course Outline, an Hourly Distribution (schedule), as well as other administrative information, e.g., instructor qualifications and budget. Historically, POST tasked its Regional Law Enforcement Consultants to audit and review certified courses to ensure course quality. Accordingly, the span of control for a small 10-person team of consultants to oversee the quality of several thousand courses is quite significant and demanding. And, due to the ever-increasing complexity of California peace officer training and selection standards, service demands have increased, resulting in an insufficient capacity for full-time POST staff to properly evaluate course delivery.

Many training experts will attest that the best course content may not be effective unless it is followed and taught by competent instructors, who are well-versed in adult learning techniques. In many respects, the quality of instructors and course delivery is equally, if not more important, than certification of the foundational course materials. POST recognized that course and instructor evaluations required a better level of review and moved to develop a process that assessed course quality and instructor performance. 

To address this need, in January 2011, POST initiated the development of a Quality Assessment Program (QAP).

This need was later memorialized in the below POST Strategic Plan (Rev. 7/2015) objectives.

B.8.1: Ensure instructor certification and training requirements are current.

B.8.3: Identify and assess current feedback methods for input on instructor performance.

These directed staff to develop methods for effectively assessing the quality of POST certified training courses and instructors.

The importance of this function and the administrative challenges associated with the tremendous span of control for the full-time consultant team and requisite workload was reiterated as a recommendation in the Organizational Analysis Report conducted by Mission Consulting LLC, in 2018, which resulted in those measures being addressed by staff to improve the efficacy of QAP.

The specific recommendation and concerns from the Mission Consulting LLC report is located in the Bureau-Specific Findings and Recommendations (TPS Section 4, page 47), as follows:

“Dedicate additional resources to quality assessment to ensure courses and instructors continue to meet POST standards"

"The QAP focuses on the instructor and course field assessments. Contracted subject matter experts go into the field, review how the courses are doing, and submit a report back to POST. Incoming requests to conduct the reports depends on the individual requests of TDC and other POST LECs. The program is not proactively advertised across the state, mainly because of the lack of funding to cover this service in all 10 regions, resulting in few evaluations relative to the number of certified courses and instructors. Even when an assessment is conducted, there is no follow-up with TDC Regional Consultants on how the assessment impacts the agency, further diminishing their utility. These assessments provide a service that is the core of POST’s mission – to ensure that the training provided in the field is at a high, professional standard. The QAP program was recently suspended due to POST’s ongoing financial challenges.”

QAP has evolved into a very robust program, where, despite a temporary setback due to the COVID-19 Pandemic, it has evaluated and assessed the efficacy of numerous courses, helped to confirm the content positively impacted the learner’s knowledge and/or performance, thus ensuring a return on investment for the students, agencies, profession, and POST. It has also served to bring attention to instructional deficiencies, identified safety concerns, and been utilized as a tool to help presenters improve their quality of instruction. As part of QAP, POST provides master level instructors to assist presenters in improving the quality of their course content and delivery, thus better serving POST’s law enforcement constituents and therefore the public they serve.

The attachments reflect the number of courses that have been reviewed, as well as the associated costs.

It should be noted that POST has vigorously worked to streamline processes, reduce costs, increase efficacy, and enhance access by means of leveraging technology to conduct remote evaluations, and has recently implemented a process to allow for course reviews to be accessed online so that agencies, training managers, and prospective student can gauge whether or not the training would benefit them. Those efforts are ongoing.

As a government agency, POST must ensure that law enforcement training dollars are used prudently.

Quality measurement is an important factor in the development and delivery of law enforcement training. In law enforcement, the quality of course design and instructional delivery directly impacts how students internalize the information and skills learned, which translates into the application of that knowledge in the field. Training instructors should utilize the most effective teaching techniques and course design to accomplish necessary learning objectives, in a way that results in improved performance of law enforcement officers. By providing a QAP to assess the quality of training delivery, POST identifies areas of improvement and more effectively allocates resources to improve the training process. In this way, law enforcement field performance and decision-making can be improved.

QAP includes administrative oversight, completion of rubric reports, development of course review schedules, and the presentation of an evaluator course. The contract is designed to relieve POST of the administrative burden of reimbursement and management of most evaluations. Evaluators paid through the contract are paid upon completion of each course review and submission of the evaluation report(s) to POST.

QAP reports are reviewed by POST, and staff follow up with the presenter to collaborate on a corrective strategy for improvement, with a requirement for the presenter to address the concerns, if needed. Further, the Training Delivery and Compliance Bureau (TDC), which has the Regional Consultants with the primary point of contact with the field, is involved in distribution and analysis of QAP assessments.

This endeavor, which is currently administered via a contract with South Bay Regional Training Consortium (Community Colleges) is consistent with the legislative direction provided to POST in Penal Code 13503, which relates:

“In carrying out its duties and responsibilities, the commission shall have all of the following powers:”

“(c) To contract with other agencies, public or private, or persons as it deems necessary, for the rendition and affording of those services, facilities, studies, and reports to the commission as will best assist it to carry out its duties and responsibilities.”

“(e) To develop and implement programs to increase the effectiveness of law enforcement and when those programs involve training and education courses to cooperate with and secure the cooperation of state-level officers, agencies, and bodies having jurisdiction over systems of public higher education in continuing the development of college-level training and education programs.”

The total number of courses audited varies depending on the total number of days per course. Evaluators are paid on the days they attend the course. No further compensation will be encumbered for additional hours worked outside of course attendance. Evaluator reports include course validation and instructor rubrics.

Advantages of the QAP:

• Provides Commission and POST staff with information on how monies and resources are being utilized.
• Provides presenters with tools for creating a better student-centered learning environment and considerations for curriculum design or facilitation skills.
• Helps ensure outcomes regarding student performance are better met by course material.
• Provide the presenters, if necessary, with immediate remedies for course improvement with master level assessors.
• Provides presenters with detailed evaluations within 30 days of their course being assessed to implement the observations.
• Provides agencies, training manager, and prospective student with an opportunity to better choose appropriate training courses.
• Ensure the Commission and law enforcement stakeholder realizes a return on investment for monies spent in course delivery.

This report is informational only. No action is requested by the Commission.