POST was established by the Legislature in 1959 to set minimum selection and training standards for California law enforcement and facilitates the ongoing development, delivery, and improvement of training for peace officers throughout California to meet legislative mandates or other critical law enforcement training needs. POST staff manages mandated and elective training courses designed to develop and expand the knowledge and skills necessary to improve the quality of police services in California communities.

Each training course can be certified to several presenters and courses can be scheduled throughout the year at different locations within the state. As of January 31, 2022, there were 4,956 training courses certified to 962 presenters throughout the state.

The courses are certified by means of reviewing a training package which includes an Expanded Course Outline, an Hourly Distribution (schedule), as well as other administrative information, e.g., instructor qualifications and budget. Historically, POST tasked its Regional Law Enforcement Consultants to audit and review certified courses to ensure course quality. Accordingly, the span of control for a small 10-person team of regional consultants to oversee the quality of several thousand courses was quite significant and demanding. Furthermore, due to the ever-increasing specialty and complexity of California peace officer training and selection standards, service demands have increased, resulting in an insufficient capacity for full-time POST staff to properly evaluate course delivery.

Many training experts will attest that the best course content may not be effective unless it is followed and taught by competent instructors who are well-versed in adult learning techniques. In many respects, the quality of instructors and course delivery is equally, if not more important, than certification of the foundational course materials. POST recognized that course and instructor evaluations required a better level of review and moved to develop a process that assessed course quality and instructor performance.

To address this need, in January 2011, POST initiated the Quality Assessment Program (QAP). 

This need was later memorialized in the 2015 POST Strategic Plan (Rev. 7/2015) objectives (Attachment B - 2015 Strategic Plan), as follows:

"A.1.2: Establish a process to ensure POST-developed courses are relevant and contemporary. 

B.8.1: Ensure instructor certification and training requirements are current.

B.8.3: Identify and assess current feedback methods for input on instructor performance."

The 2021 POST Strategic Plan (Rev. 9/2021) re-enforced these objectives (Attachment C – 2021 POST Strategic Plan), as follows:

"1.A.2: Establish a process to ensure POST-developed courses are relevant and contemporary. 

2.I.1: Ensure instructor certification and training requirements in POST regulations are current. 

2.E.5: Assess current course evaluation feedback methods."

Each Strategic Plan directed staff to develop methods for effectively assessing the quality of POST certified training courses.

The importance of this function and the administrative challenges associated with the span of control for the full-time regional consultant team and requisite workload was reiterated as a recommendation in the Organizational Analysis Report conducted by Mission Consulting LLC, in 2018, which resulted in those measures being addressed by staff to improve the efficacy of QAP (Attachment D – Organizational Analysis Report).

The specific recommendation and concerns from the Mission Consulting LLC report is in the Bureau-Specific Findings and Recommendations (TPS Section 4, page 47), as follows:

“Dedicate additional resources to quality assessment to ensure courses and instructors continue to meet POST standards"

"The QAP focuses on the instructor and course field assessments. Contracted subject matter experts go into the field, review how the courses are doing, and submit a report back to POST. Incoming requests to conduct the reports depends on the individual requests of TDC and other POST LECs. The program is not proactively advertised across the state, mainly because of the lack of funding to cover this service in all 10 regions, resulting in few evaluations relative to the number of certified courses and instructors. Even when an assessment is conducted, there is no follow-up with TDC Regional Consultants on how the assessment impacts the agency, further diminishing their utility. These assessments provide a service that is the core of POST’s mission – to ensure that the training provided in the field is at a high, professional standard. The QAP program was recently suspended due to POST’s ongoing financial challenges.”

QAP has evolved into a robust program, where, despite a temporary setback due to the COVID-19 Pandemic, has evaluated and assessed the efficacy of numerous courses, helped to confirm if the content positively impacted the learner’s knowledge and/or performance, and ensured a return on investment for the students, agencies, profession, and POST. QAP has also served to bring attention to instructional deficiencies, identified safety concerns, and has been utilized as a tool to help presenters improve their quality of instruction. The assessment includes recommendations to assist presenters in improving the quality of their course content and delivery, thus better serving POST’s law enforcement constituents and therefore the public they serve.

The attachments to this agenda item reflect the number of courses that have been reviewed, as well as their associated costs.

It should be noted that POST has worked to streamline processes, reduce costs, increase efficacy, and enhance access by means of leveraging technology to conduct remote evaluations, and has recently implemented a process to allow for course reviews to be accessed online so that agencies, training managers, and prospective student can gauge whether or not the training would benefit them. Those efforts are ongoing.

As a government agency, POST must ensure that law enforcement training dollars are used prudently. Quality measurement is an important factor in the development and delivery of law enforcement training. In law enforcement, the quality of course design and instructional delivery directly impacts how students internalize the information and skills learned, which translates into the application of that knowledge in the field. Training instructors should utilize the most effective teaching techniques and course design to accomplish necessary learning objectives, in a way that results in improved performance of law enforcement officers. By providing a QAP to assess the quality of training delivery, POST identifies areas of improvement and more effectively allocates resources to improve the training process. In this way, law enforcement field
performance and decision-making can be improved.

QAP includes administrative oversight, completion of rubric reports, development of course review schedules, and the presentation of an evaluator course. POST utilizes a contract designed to relieve POST of the administrative burden of reimbursement and management of most evaluations. Evaluators paid through the contract are paid upon completion of each course review and submission of the evaluation report(s) to POST.

QAP reports are reviewed by POST, and staff follow up with the presenter to collaborate on a corrective strategy for improvement, with a requirement for the presenter to address concerns, if needed. Further, the Training Delivery and Compliance Bureau (TDC), staffed with ten regional consultants who serve as the primary point of contact for the field, is involved in distribution and analysis of QAP assessments.

This endeavor, which is currently administered via a contract with South Bay Regional Training Consortium (Community Colleges) is consistent with the legislative direction provided to POST in Penal Code 13503, which states:

“In carrying out its duties and responsibilities, the commission shall have all of the following powers:”

“(c) To contract with other agencies, public or private, or persons as it deems necessary, for the rendition and affording of those services, facilities, studies, and reports to the commission as will best assist it to carry out its duties and responsibilities.”

“(e) To develop and implement programs to increase the effectiveness of law enforcement and when those programs involve training and education courses to cooperate with and secure the cooperation of state-level officers, agencies, and bodies having jurisdiction over systems of public higher education in continuing the development of college-level training and education programs.”

The total number of courses audited by QAP varies depending on the total number of days per course. Evaluators are paid on the days they attend the course. No further compensation is encumbered for additional hours worked outside of course attendance. Evaluator reports include course validation and instructor rubrics.

MEASURABLE OUTCOMES:

The QAP process begins with a notification to the presenter of the course from the POST contract coordinator. The coordinator explains to the presenter that an evaluator will be attending the class in its entirety. The QAP evaluator is presented with information about the class, which includes instructor resumes, the expanded course outline (ECO), the hourly distribution schedule, and the course budget. 

When assessing the course, the evaluator uses the TRAINING EVALUATION-COURSE & INSTRUCTOR ASSESSMENT, POST form 2-341 (Attachment E – POST form 2-341).  This form is divided into three sections, Presentation Information, POST Required Documentation, and Instruction and Facilitation. The rubric in Instruction and Facilitation has the ratings of “Unsatisfactory,” “Needs Improvement,” “Meets Standard,” and “Above Standard.” The instructor categories which are assessed address the following categories:

Facilitation Skills
Subject Matter Expertise
Time Management
Professionalism
Learning Resource Management
Classroom Management
Real World Application
Instructional Methods
Learner Validation

From January 1, 2021 to December 31, 2021, QAP evaluated one hundred and fifteen (115) presentations and three hundred thirteen (313) instructors. Three thousand three hundred eighty-five (3,385) hours were dedicated to these evaluations. Out of the 115 presentations, twenty-three (23) of these presentations had at least one “Needs Improvement” rating; three (3) had a rating of “Unsatisfactory.”  

Although POST is no longer offering coaching or mentoring assistance through the QAP, the evaluators do make recommendations on how to improve in all areas assessed.
  
Once the evaluations are submitted to the coordinator, they are sent to POST for disbursement to the course coordinator. Prior to the September 1, 2021 POST Commission meeting, POST had a tracking system in place (Attachment F- Prior Tracking System).  However, after hearing the recommendations from the Commission, POST developed a more robust system of tracking and compliance to remedy sub-par ratings. A POST Law Enforcement Consultant (LEC) from the Training Program Services (TPS) Bureau reviews the evaluations and takes note of any instructor issues, regulatory deficiencies, and safety concerns. After reviewing the evaluation, a letter on official POST letterhead is sent to the course coordinator along with copies of the instructor evaluation forms.  

If there were no issues with the course, the letter simply thanks the presenter for their participation and the course review is marked as completed in the database tracking program (Attachment G- Microsoft TEAMS spreadsheet).  If an evaluation includes a rating of “Needs Improvement” or identifies any minor course issues, the letter to the presenter will ask for a written corrective plan to be submitted to POST. The plan must be submitted in a WORD or PDF document within thirty (30) days of notification.  If there is no response within thirty (30) days, a follow-up reminder letter is sent providing an additional thirty (30) day period as the new deadline. After ninety (90) days without a response, the course is placed into "modification" (i.e. can not be presented) and will remain in modification until POST receives a satisfactory response.

If the result is an “Unsatisfactory” instructor rating, or a regulatory or safety issue, the letter states that the course will be immediately placed into modification until POST is satisfied that the deficiencies are remedied. A follow-up QAP will be scheduled for that course to ensure the quality has improved. The TDC Bureau will be notified and included in the follow up communication.

All courses are tracked in the Microsoft TEAMS List Program. All letters to presenters and their evaluations are attached to the course file and include due dates for “Needs Improvement” notifications. Once the necessary information is provided to POST, the course is marked completed. As of January 31, 2022, there are three (3) presenters being addressed who have yet to respond to POST. Two (2) of these presenters are still within the ninety (90) day response period. One (1) presenter did not respond during the ninety (90) day response period, and as a result that presenter's course was placed into modification and can not presented until corrective action is taken.

Finally, another method POST is using to measure the quality of the course and the instructor(s) is by assessing all-new private presenters and by ensuring courses that have numerous deficiencies are given a follow-up QAP from a different evaluator. If there are still deficiencies with these courses, decertification is used as an option.

ADVANTAGES OF THE QAP:

• Provides the Commission and POST staff with information on how monies and resources are being utilized. 
• Provides presenters with tools for creating a better student-centered learning environment and considerations for curriculum design or facilitation skills.
• Helps ensure outcomes regarding student performance are better met by course material.
• Provide the presenters, if necessary, with immediate remedies for course improvement with master level assessors.
• Provides presenters with detailed evaluations within thirty (30) days of their course being assessed to implement the observations.
• Provides agencies, training manager, and prospective students with an opportunity to better choose appropriate training courses.
• Ensure the Commission and law enforcement stakeholder realizes a return on investment for monies spent in course delivery.