MEETING DATE
2/19/2015 |
BUREAU SUBMITTING THIS REPORT
Computer Services Bureau |
RESEARCHED BY (PRINT NAME)
Mei Yip |
REVIEWED BY (PRINT NAME)
Colin O'Keefe; Bob Holmgren |
REPORT DATE
12/11/2014 |
APPROVED BY
Robert A. Stresak |
DATE APPROVED
01/26/15 |
PURPOSE
Decision Requested |
FINANCIAL IMPACT
Yes
|
| ISSUE: |
| Should the Commission authorize the Executive Director to continue efforts to contract for products and services necessary for the acquisition, configuration, and implementation of software to replace POST’s current automated testing system, in an amount not to exceed the previously-approved amount of $2,868,694? |
| BACKGROUND: |
In January 2010, the State Financial Integrity and State Manager’s Accountability (FISMA) audit reported the Testing Management and Assessment System’s (TMAS) inability to adequately secure and protect tests from inappropriate access. The audit subsequently ranked this issue as the second highest risk to POST.
In June 2010, the Commission directed staff to contract for the completion of an Information Technology Feasibility Study Report (IT FSR). The Commission authorized the Executive Director to contract for FSR authoring assistance because the required effort exceeded POST staff’s resources. Following a competitive bid process, staff contracted with Hubbert Systems Consulting to assist in preparing these analyses. The resulting FSR was approved by the Department of Finance on December 21, 2011.
On June 28, 2012, the Commission approved the request to contract for the development and implementation of a computer-based testing system to replace TMAS. The total cost approved was $2,868,694. Expenditures to date total $89,697. The replacement system is to be named PES (POST Exam System).
In July 2012, POST staff concluded a security analysis under contract with security consulting firm Shooting Star. The analysis evaluated technical and physical security risks inherent in paper-based testing and computer-based testing. The analysis concluded that fully-automated computer-based testing, and the elimination of all paper-based testing, is highly desirable from a security and workload standpoint. In August 2012, POST management made the decision that the PES must include functionality that eliminates paper-based testing,. This would allow presenters of basic training to use testing computers for general-purpose tasks when not being used for testing.
On September 18, 2012, in order to satisfy Government Code (GC) Section 19130, POST staff completed required documentation and advertised the PES project to all State Agency Chief Information Officers (CIO) to inquire about availability of State employees with the requisite expertise to develop the POST Exam System. Government Code Section 19130 dictates that State workforce must be considered before allowing the use of personal services contracts to procure non-State vendors. Personal services contracting may be permissible when the contracting agency clearly demonstrates that the contract results in cost savings to the State pursuant to GC 19130(a), subject to the pre-approval of the State Personnel Board. POST received no response to this announcement and proceeded with vendor solicitation. The project’s exemption from Government Code section 19130 requirements was approved by the DGS Office of Legal Services on January 24, 2013.
On September 5, 2013, POST staff completed the draft functional and technical software requirements for the system procurement.
On May 1, 2014, staff released a Request For Information (RFI) to the public, soliciting feedback and potential interest by vendors in the examination field.
In August 2014, POST staff completed market research to determine potential vendors’ interest in bidding on this project, as well as expected costs. Two vendors showed interest in bidding on the project.
During this period, due to the PES project’s lengthy procurement process, POST staff internally developed and implemented a technical solution known as “flash boot” to mitigate the current TMAS risks detailed in the FISMA audit. This solution has significantly enhanced the technical security of the current TMAS system.
POST staff provided project status updates to the Commission on 5/28/2013 and 8/25/2014. The POST Consortium also received three status reports on this project to date. |
| ANALYSIS: |
Under California State procurement rules, budget and scope of this project mandate its classification as a "reportable" project. Projects with this classification are subject to stringent procurement and project management rules, State control agency processes, and approvals. The Department of Technology (a State control agency) dictates the processes for procurement and project management. POST has experienced significant delays both in this approval process and in determining how to best meet control agency requirements, while at the same time procuring a viable testing product that will be useful to law enforcement statewide.
In July 2013, the IT procurement division of the State Department of General Services (DGS) merged with the Department of Technology. As a result, the Statewide IT Procurement Division was formed. This caused many changes to the State IT procurement processes and the procurement staff assigned to POST changed several times.
In an effort to avoid the problems occurring in the current TMAS system, POST staff wrote detailed specifications for the proposed TMAS replacement system. The State control agencies identified overly-detailed specifications as the key problem causing only two complete responses to the RFI (lower than desired). The market research conducted after the RFI also clarified that the pool of vendors from whom RFI solicitations were requested proved to be off-target. The market research also helped to clarify the scope and complexity of the software solution that POST is seeking. POST is seeking to integrate what is generally considered to be three separate software product areas:
(1) a learning management system,
(2) a test development and validation system, and
(3) a computer-based testing system.
POST staff are currently addressing the issues raised during the RFI process, namely, control agency concerns that the requirements are overly detailed and too complex. Addressing these issues should attract a larger number of bidders to the Request For Proposals (RFP), as required by the control agency. POST staff will also seek to identify other potential bidders within the realm of database development and/or system integration software vendors. With these modifications, the State control agencies have generally agreed that POST’s RFP can be released. Estimated vendor solicitation start date is December 18, 2015.
Due to delays as outlined in this agenda item the original Commission approval to contract has expired. Staff is seeking approval to continue contracting efforts to procure a new testing system for basic course testing.
Expected costs and updated estimated timelines for this project can be found Attachments A and B. |
| RECOMMENDATION: |
| Staff recommends that the Commission authorize the Executive Director to continue efforts to contract for products and services necessary for the acquisition, configuration, implementation, and verification of software to replace POST’s current automated testing system in an amount not to exceed $2,868,694. |
|