HomeMy WebLinkAbout2021-cv-2063-CNS-MEH - City Of Fort Collins V. Open International, Et Al. - 217 - Joint Motion To Unrestrict Record
IN THE UNITED STATES DISTRICT COURT
FOR THE DISTRICT OF COLORADO
Civil Action No. 2021-cv-02063-CNS-SP
CITY OF FORT COLLINS,
Plaintiff and Counter-Defendant,
vs.
OPEN INTERNATIONAL, LLC,
Defendant and Counterclaimant,
and
OPEN INVESTMENTS, LLC,
Defendant.
JOINT MOTION TO UNRESTRICT RECORD
The parties, Open International, LLC and Open Investments, LLC, and the City of Fort
Collins, through their undersigned counsel, request that the Court make available to the public all
briefing and exhibits in this matter currently maintained under Level 1 restriction (ECF Nos. 43,
45, 54, 58, 63, 67, 74, 76–78, 101, 104–106, 116, 119, 123–125, 128–129, 150–158, 173–176,
184, and 185, and all accompanying exhibits except ECF Nos. 128-1, 153-1, and 175-21).
In January of this year, the Court invited a motion to “unrestrict all documents” as the
parties approach trial. See Jan. 25, 2023 Order, ECF No. 190. The final pretrial conference is
1 Concerning ECF Nos. 128-1, 153-1, and 175-2, which are copies of the affirmative expert
report of John Hutchinson, the parties provide the attached, redacted version of the report that
redacts language quoted from a City employee’s performance review. See Ex. 1, Redacted
Hutchinson Affirmative Expert Report.
Case No. 1:21-cv-02063-CNS-SP Document 217 filed 05/09/23 USDC Colorado pg 1 of 3
-2-
now scheduled for July 10, 2023, and trial set to begin on October 23, 2023. The parties
accordingly ask the Court to unrestrict all briefs and exhibits—with the exception of the
Hutchinson report, which the parties request be made available on the public docket with the
minor stipulated redactions reflected in the version attached hereto as Exhibit 1.
Dated: May 8, 2023 Respectfully submitted,
s/ Paul D. Swanson
Paul D. Swanson
Anna C. Van de Stouwe
Alexander D. White
Alexandria E. Pierce
Holland & Hart LLP
555 17th Street, Suite 3200
Denver, Colorado 80202
Telephone: 303-295-8000
adwhite@hollandhart.com
pdswanson@hollandhart.com
acvandestouwe@hollandhart.com
adwhite@hollandhart.com
aepierce@hollandhart.com
Attorneys for Defendants Open International, LLC
and Open Investments, LLC
s/ Maral J. Shoaei
Case Collard
Andrea Ahn Wechter
Maral J. Shoaei
Dorsey & Whitney LLP
1400 Wewatta Street, Suite 400
Denver, CO 80202
Telephone: 303-629-3400
collard.case@dorsey.com
wechter.andrea@dorsey.com
shoaei.maral@dorsey.com
Attorneys for Plaintiff City of Fort Collins
Case No. 1:21-cv-02063-CNS-SP Document 217 filed 05/09/23 USDC Colorado pg 2 of 3
-3-
CERTIFICATE OF SERVICE
I hereby certify that on the 8th day of May, 2023, the foregoing was electronically filed
with the Clerk of Court using the Court’s electronic filing system and that a copy of the
foregoing was sent to all counsel of record via same in compliance with the Federal Rules of
Civil Procedure and the Local Rules of this Court.
s/ Paul D. Swanson
21405818
Case No. 1:21-cv-02063-CNS-SP Document 217 filed 05/09/23 USDC Colorado pg 3 of 3
Exhibit 1
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 1 of
89
Expert Report
Expert Report
John Hutchinson
jrh@tjhconsultancy.com
October 24, 2022
Contents
List of Figures 3
List of Tables 3
1 Introduction 4
1.1 Report structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
2 Biography 5
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
2.2 Degrees, awards and recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
3 CIS Implementation Framework 7
3.1 CIS Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
3.2 Selection, procurement and negotiation . . . . . . . . . . . . . . . . . . . . . . .9
3.2.1 Defining the solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
3.2.2 Vendor evaluation and due diligence . . . . . . . . . . . . . . . . . . . . .15
3.2.3 Vendor selection, contract negotiation and award . . . . . . . . . . . . .17
3.3 Project Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
3.3.1 Project initiation and planning . . . . . . . . . . . . . . . . . . . . . . . . .19
3.3.2 Scope confirmation and solution definition . . . . . . . . . . . . . . . . .20
3.3.3 Solution construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
3.3.4 Report construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
3.3.5 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
3.3.6 Organizational readiness and training . . . . . . . . . . . . . . . . . . . .30
3.3.7 Deployment and go-live . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
3.4 Post-live support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
3.5 Governance and project management . . . . . . . . . . . . . . . . . . . . . . . . .32
3.5.1 Executive leadership and governance . . . . . . . . . . . . . . . . . . . . .32
3.5.2 Solution lifecycle management . . . . . . . . . . . . . . . . . . . . . . . .33
3.5.3 Execution lifecycle management . . . . . . . . . . . . . . . . . . . . . . .34
J
T
H
C
Proprietary and Confidential Page 1 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 2 of
89
Expert Report
4 Project overview and summary of opinions 37
4.1 Overview of the project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37
4.2 Summary of opinions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39
4.2.1 The City’s Broadband business definition was initially rushed then sig-
nificantly delayed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39
4.2.2 The City did not provide adequate staffing . . . . . . . . . . . . . . . . . .40
4.2.3 The City’s governance, oversight and management of the project was
flawed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40
4.2.4 Open and the OSF product performed substantially in accordance with
the agreements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41
5 Opinions 42
5.1 The City’s rushed addition of Broadband to the RFP, failure to timely and clearly
define Broadband services, and failure to timely supply Open with Broadband
business definitions materially impacted the project’s success . . . . . . . . . .42
5.1.1 The late addition of Broadband to the CIS replacement RFP resulted in
insufficiently detailed and incomplete broadband requirements . . . .42
5.1.2 The lack of detailed and timely-delivered broadband definitions and
platforms substantially impacted project execution . . . . . . . . . . . .45
5.1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50
5.2 The city failed to adequately staff the project . . . . . . . . . . . . . . . . . . . . .50
5.2.1 The City failed to meet its testing obligations due in large part to its lack
of adequate resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
5.2.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56
5.3 The City failed to provide effective governance and oversight . . . . . . . . . . .57
5.3.1 The City lacked CIS implementation experience and expertise, and failed
to address that problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57
5.3.2 The City’s broadband business was not ready for a fixed price and scope
implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58
5.3.3 The City provided inadequate project management . . . . . . . . . . . .59
5.3.4 The City’s project governance was not adequate for the needs of the project 62
5.3.5 TheCitywasnotpreparedtoundertakeitsobligationsduringpostgo-live
and hypercare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64
5.3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66
5.4 Open fulfilled its obligations and did not cause the project’s failure . . . . . . .67
5.4.1 Selection, procurement and negotiation . . . . . . . . . . . . . . . . . . .67
5.4.2 Open’s alignment with RFP scope . . . . . . . . . . . . . . . . . . . . . . .69
5.4.3 Open-provided training . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70
5.4.4 Open concessions and investments . . . . . . . . . . . . . . . . . . . . . .71
5.4.5 Compliance with production incident SLAs . . . . . . . . . . . . . . . . .72
5.4.6 Delivery of the Self-service portal and OSF generally . . . . . . . . . . . .72
5.4.7 Open exhibited great flexibility in accommodating project issues . . . .74
5.4.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74
6 Conclusion 75
Appendices 77
J
T
H
C
Proprietary and Confidential Page 2 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 3 of
89
Expert Report
A Professional Resume 77
B Materials considered 80
List of Figures
3.1 Example high level CIS-related process taxonomy . . . . . . . . . . . . . . . . . . .9
3.2 Typical platform operating model choices . . . . . . . . . . . . . . . . . . . . . . . .11
3.3 Impact of hypercare on post go-live performance . . . . . . . . . . . . . . . . . . . .32
3.4 Typical lifecycle of project artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
4.1 Anticipated project timeline as-of project start (August 2018). . . . . . . . . . . .38
5.1 Observations (feature requests) requiring development by month . . . . . . . . . .47
5.2 OASIS Project Managers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61
5.3 Incidents filed against OSF by month . . . . . . . . . . . . . . . . . . . . . . . . . . .65
5.4 Timeline of key events in Open selection process . . . . . . . . . . . . . . . . . . . .68
5.5 Summary of Open and Cogsdale product fit with project objectives . . . . . . . . .70
List of Tables
3.1 RACI table from statement of work . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
3.2 Simplified example of a test case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
J
T
H
C
Proprietary and Confidential Page 3 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 4 of
89
Expert Report
1 Introduction
This report contains the findings of my review of the implementation of Open International’s¶1
Open Smartflex (OSF) Customer Information System (CIS) product carried out by Open Inter-
national and its partners for the City of Fort Collins, Colorado (the OASIS project). Applying my
experience and knowledge from implementing CISs for over twenty years, I conclude that the
implementation of Open’s OSF solution for the City of Fort Collins (the “City”) failed because
of, primarily:
1.A lack of clear, timely, and reliable product, process and requirement details for the
City’s broadband business creation.
2. A lack of adequate staffing resources and expertise by the City.
3.A lack of effective governance and oversight by the City, including a lack of CIS imple-
mentation knowledge and inadequate project preparation.
1.1 Report structure
Section 2 below describes my qualifications and experience in the field of utility CIS imple-¶2
mentations, particularly as they relate to the subject matter of this Expert Report.
Section 3 on page 7 provides a high level overview of a CIS implementation project.¶3
Section 4 on page 37 provides a brief project overview, and introduces four key opinions,¶4
regarding Open, and the OSF product, living up to the obligations agreed-to with the City, and
the City’s shortcomings that contributed to the project’s failure.
Section 5 on page 42 develops the four key opinions from Section 4 by comparing the OSF im-¶5
plementation at the City of Fort Collins with the model described in section 3 and highlighting
areas where the implementation deviated significantly from the model.
Section 6 on page 75 summarizes my findings based on my experience, the reference model¶6
and the project events covered in previous sections.
Appendix A on page 77 contains a copy of my professional resume.¶7
Appendix B on page 80 contains a list of the materials that I considered. Additionally I had¶8
the opportunity to participate in a demonstration / exploration of the OSF system from the
utility perspective, and a separate demonstration of the self-service portal.
J
T
H
C
Proprietary and Confidential Page 4 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 5 of
89
Expert Report
2 Biography
See also my resume included in appendix A on page 77.¶9
2.1 Overview
For the past 24 years I have worked as a manager, executive, project leader, architect and¶10
advisory consultant in the Utilities sector.
For the bulk of that time (approximately 85% since April 1998) I have specialized in the field¶11
of CIS implementation in the global utilities industries. I have worked extensively with clients
in the USA, Canada, Ireland and the United Kingdom, and also with clients in Australia, Hong
Kong, Japan, Singapore, and South Africa.
A key part of my experience has included performing assessments and quality reviews of¶12
both in-flight and completed projects. I have personally been involved in approximately 50
such assessments and reviews during this period, mostly as either the sole consultant, or
leading a small (2-5 person) team performing the assessment. The utilities in these reviews
were divided approximately equally between municipally and investor owned utilities. I also
participated in many similar assessments for other industries prior to starting work in the
utilities industry.
Starting in April 1998, I was a Vice President, then Senior Vice President, at SPL WorldGroup,¶13
a leading developer of multiple CIS products which was subsequently (November 2006)
purchased by Oracle Corporation, one of the largest utility CIS providers in the world. After
purchase,SPL’sproductlineformedthebasisofOracle’sutilityCISproductoffering(including
the current Oracle C2M and CCS products).
While at SPL (from 1998 until leaving at the end of 2003) I led a team of 400 employees in 11¶14
countries delivering CIS implementation and advisory services to upwards of approximately
40 clients, both municipally and investor owned. This included leading and growing a remote
development center with over 100 staff based in Manila, Philippines.
In October 2004, as an independent consultant I began advising IBM on, and co-leading, a¶15
major (successful) CIS consolidation bid for National Grid USA. I joined IBM as a full-time
employee in the role of Associate Partner in April 2005, initially leading the National Grid
project. I remained as an Associate Partner at IBM until July 2017 (12 years), initially in the
North American Energy & Utilities practice and then, from 2013, in their Global Center of
Competence for Oracle (focusing almost entirely on Oracle products in the utilities industry).
While at IBM I played leading (management and / or execution) roles in multiple implementa-¶16
tions, many of which were “first of a kind” in some dimension. This included approximately
18 months as the overall Solution Architect and Management Advisor for the CIS work-stream
assistingin the establishmentofa national waterutility for the country ofIreland (Irish Water).
J
T
H
C
Proprietary and Confidential Page 5 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 6 of
89
Expert Report
I also spent 16 months in the telecommunication industry (2006/7) advising a business /¶17
IT billing and transformation project IBM was leading for Qwest Communications (since
acquired by CenturyLink) based in Denver. At the time Quest had both a well established
traditional land-line business and an emerging broadband and mobile operation.
Since 2018 I have worked (as an independent consultant) with utilities clients in Canada and¶18
the USA in the utilities CIS space, as the lead architect for the Systems Integrator upgrading
Hydro Ottawa’s CIS system, as part of the architect leadership team for the Systems Integrator
implementing Oracle’s cloud-based CIS for a major Canadian utility, and most recently as the
client-side Program Architect for one of the “top 10” US utilities as they embark on a massive,
multi-year major implementation of a new CIS system to replace their 30+ year old aging CIS.
In June 2018, I was contacted by Jeff Valadez of Open who alerted me to the City’s RFP for an¶19
external project manager for the OASIS project. Mr. Valadez was a professional acquaintance
and former colleague of mine from SPL WorldGroup in the 2000–2003 time frame. I reviewed
the RFP and submitted a “no bid” response letter due to the required travel and the proposed
compensation.
2.2 Degrees, awards and recognition
Note: The field of specialized CIS program management is relatively new, small and highly¶20
dynamic, driven partly by the rapid maturation of the associated technology and the man-
ner in which product vendors adopt it. Accordingly there is little or no formal education
/ training / established literature directly linked to the field. Rather, practitioners largely
learn by individual and shared experience, collaboration at conferences and other informal
mechanisms.
•B.Sc. Hons. (2:1) in Computer Science, Queen Mary College, University of London (U.K.)
(Now Queen Mary University, London) 1975.
•Presented a CIS Implementation education session at CS Week 2014 (a leading US CIS
industry conference and trade-show). Most Utilities recognize participation in this
session as continued education for their professional staff.
•Co-author of a US Patent on improving marketing segmentation using coarse grained
interval data from electric meters (awarded 2016).
•IBM “100% Club” inductee (2016). IBM recognition for top performers globally with
respect to goals and objectives
•IBM certifications (various dates): Project Management (various), Design Thinking Prac-
titioner, Energy&UtilitiesInsightsandSolutions(Silver), SellingProfessionCertification
- Level II Expert. IBM awarded these certifications based on a combination of internal
(IBM) training and review of experience and performance by a peer team of experienced
IBM practitioners in the relevant fields.
J
T
H
C
Proprietary and Confidential Page 6 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 7 of
89
Expert Report
3 CIS Implementation Framework
This section provides general background information relating to CISs, their function within¶21
a utility organization and some key characteristics of the CIS selection and implementation
process. It is not a complete survey of the entire subject; it focuses on those aspects of CISs
most directly relevant to the implementation in Fort Collins.
3.1 CIS Overview
In many respects the CIS forms the heart of a utility’s business operation. It typically handles¶22
most or all aspects of the utility’s interactions with its customer base. The core business
processes supported include customer service, billing, and payment & collection services.
In addition the CIS plays a critical role in establishing and provisioning new services for a
customer, and in dealing with issues the customer may experience (such as outages) while a
customer of the utility. The CIS interfaces and interoperates with the majority of key utility
services, departments and systems in order to fully deliver the functions described above. For
example it may integrate with a work management or field dispatch system to handle outages,
or an external collections agency to deal with financially delinquent customer accounts. The
CIS also plays a critical role in supporting regulatory compliance reporting and management,
which is typically a complex and critical component of a utility’s legal obligations.
The CIS is often the primary system that records a customer’s history with the utility, and¶23
keeps track of interactions between the customer and the utility, whether by telephone or,
increasingly, by electronic communication. As such, the CIS touches almost every customer,
employee and dollar of revenue that the utility has; it is considered to be a mission-critical
system for the utility.
Some of the functions a CIS carries out are straightforward, such as billing for a typical¶24
residential customer where the bill is calculated from simple usage and price-per-unit (e.g.
“you used 900KWH of electricity and the price is $0.20 per KWH; you owe $180.00”). Often
the fees charged by a utility are more complex, perhaps including various taxes, additional
fees, interest on unpaid balances etc. The utility may provide several services (electricity, gas,
water) and additional unrelated services which are billed via the CIS (refuse collection, for
example). Some providers, such as the City of Fort Collins, also offer non-traditional services
such as internet, telephone and video.
Increasinglytheratestructureoftheutilityisbecomingmoresophisticated,withratesvarying¶25
based on factors such as time-of-use, or with customers who install (for example) solar panels
having that generation capability being taken into account
1 . Large commercial and industrial
customers have very complicated, often individually negotiated, contracts and rate structures
(for example, sports stadia, shopping malls, college campuses, large heavy industry facilities).
Another dimension of complexity may be driven by multi-location customers such as chains
1 Both of these rate structures are offered by the City (https://www.fcgov.com/utilities/
residential/rates/electric/,https://www.fcgov.com/utilities/business/manage-your-
account/rates/electric/e300/large-commercial-net-metering-rates.)
J
T
H
C
Proprietary and Confidential Page 7 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 8 of
89
Expert Report
of gas stations or food / convenience stores, where the owner of the chain may be responsible
for 10’s or 100’s of individual locations, some subject to different rules and regulations.
AllthistakentogethermakesaCISsystemverycomplex, andalsoverysensitivetothespecific¶26
utility’s needs, requirements and restrictions. This in turn makes replacing a CIS a major,
complex, and lengthy process for any utility.
One goal of a CIS implementation is to minimize the gaps between the requirements of¶27
the utility and the capabilities of the chosen product(s). This reduces the complexity and
risk associated with the implementation, and also contributes to reducing the Total Cost of
Ownership (TCO) of the CIS.
Thus, the CIS is very tightly coupled to the utility’s business processes and business strategy.¶28
In some areas (e.g. collection for non-payment) the business process and regulatory needs
drive the manner in which the CIS is required to operate. In others (e.g. new customer
enrollment) it is more appropriate that the manner in which the CIS operates influences
the business process adopted by the utility, in order to maximize the alignment between
the business process and the CIS capabilities. The successful implementation of a CIS thus
depends heavily on the utility having well understood, and documented, business processes
and requirements that the CIS must support. The utility must also have the willingness and
flexibility to adapt its business processes, where appropriate, to the manner in which the CIS
product operates.
Most utilities replace their CIS infrequently. A CIS may be operational within a utility for 5-10¶29
years; 20 or more years is not unusual. This, coupled with the significant complexity of a CIS,
and the close, symbiotic relationship it has with the utility’s business strategy, processes and
operations, means that knowledge of how best to implement a CIS is often missing within the
utility’s own staff, and so many utilities turn to outside advisors to help guide them through
the process of choosing and implementing a replacement CIS.
Another key reason for soliciting additional external expertise relates to the technology used¶30
to build CIS systems. Early CIS systems were largely built of custom code written in (for
example) COBOL. Each implementation at a different utility required significant amounts of
new or modified code to be written, to implement the specific functions of that utility. Over
time technology changed, and many modern CIS systems rely heavily on configuration and
scripting technologies, and either strongly discourage, or totally prohibit, custom code to be
written. Thus the staff of a utility with a 15-25+ year old CIS system may be very familiar with
the technology in that system, but relatively unfamiliar with the technology used in a modern
CIS system that may be replacing it.
While in no way exhaustive, figure 3.1 on the next page shows a high level list of the business¶31
processes that a typical CIS plays a leading role in supporting for a utility.
J
T
H
C
Proprietary and Confidential Page 8 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 9 of
89
Expert Report
Customer
Management
Credit
Management
Consumption
Management
Customer
Care Billing Payment Collections
Management
Cash
Application Non-Func.
Marketing
Applications
Credit Policy
Development
Manual
Metering
Service
Management
Rate
Schedules
e-Payment
+ACH
Account
Prioritization
Cash
Posting
Look &
Feel
New Customer
Enrollment
Credit / Risk
Assessment
Automated
Metering
Customer
Inquiries
Bill
Calculation
Payment
Centers
Collection
Calls
Unapplied
Payment
Resolution
Architecture
Fit
Contracts
T&C’s
Credit / Risk
Management
Meter Setup &
Management
Account
Adjustments
Bill Prep.
Validation &
Analytics
Field
Collections
Low Income
Assistance
Account
Reconciliation
Contractual
Considerations
Customer
Setup
Pre- / Post-
Credit
Assessment
Revenue
Assurance
Cancel
Rebill
Bill Production
&
Consolidation
Check
Processing
Dunning
Activities
Cash-flow
Forecasting
Maintenance
TCO
Customer
Maintenance
Validation &
Calibration
Speciality
Customer
Assistance
Exception
Processing
Returned
items
Skip
Tracing
Cash
Analytics
Customer
Authentication
Interruption
Verification
Payment
Arrangement
Settlement &
Reconciliation
Payment
Analytics Shut-off
Customer &
Market
Analytics
Usage
Analytics
Customer
Claims Litigation
Dispute
Management Write-off
Figure 3.1: Example high level CIS-related process taxonomy
The sections which follow describe the general process which a utility may follow during the¶32
selection and implementation of a new CIS system.
3.2 Selection, procurement and negotiation
For the purposes of this discussion, the CIS implementation process starts with the utility¶33
selecting a product and a vendor to implement that product2 .
There are a relatively small number of CIS products in the North American marketplace.¶34
Vendors enter and depart the market over time; there is also a core group of 2-3 firms, led for
many years by Oracle and SAP, with substantial (multi-decade) longevity. Additionally, firms
(and products) are often acquired or sold, which can result in essentially the same product
being sold under different names at different times.
ThecapabilitiesofmanyCISproductsoftenlagbehindmarkettrends, sinceavendorhastobe¶35
confidentthatanewcapability, whichrequiresinvestmenttoincludeinitsproducts, isindeed
a lasting market need and not just a passing fad. This means that, for example, relatively
few CIS products in 2017/2018 had well-integrated capabilities to support both traditional
2 It is possible, though exceedingly rare, for a utility to implement a CIS by licensing a product and handling
all of the implementation details itself, without employing the services of an implementation vendor. The vast
majority of utilities will employ the services of a vendor (or more than one vendor) to assist them.
J
T
H
C
Proprietary and Confidential Page 9 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 10
of 89
Expert Report
utility services and internet-based services such as broadband, television and telephone,
since at that time relatively few municipalities had decided to enter the latter market, and
even fewer of those had adopted an integrated CIS approach. Open’s integration of utilities
and internet-based services into its OSF product, proven in other markets, was rare among
CIS product vendors in the North American CIS market.
Similarly, there are a small number of implementation firms in the market, though many¶36
more than there are product vendors. Again, these range from a few large system integrators
with long-standing teams in the field, (e.g. Accenture, IBM, Infosys, PwC) to smaller firms
specializing in particular market or geographic niches. Many product vendors also provide
implementation services, either as stand-alone implementers or as supporting services to
other firms, while some system integrators have the capability to implement several different
CIS products. It is common for product vendors and one or more implementation firms to
form cooperative joint teams to respond to a particular opportunity, leveraging the strengths
of the individual partner firms to provide a more competitive proposition.
A key decision the utility needs to make early in the process is whether or not to separate the¶37
selection of product and implementer. This is a complicated decision; this report focuses on
the option the City of Fort Collins made, which was to look for product recommendations and
implementers in a single procurement. In other words, the City’s ask of the potential vendors
essentially was: “Here are our needs; choose a product and bid on implementing it for us”3 .
The steps in the procurement process usually are relatively straightforward:¶38
1.Theutilityperformsinitialworktodefinethedesiredsolution. Itoftenwillhireanexpert
external consultant to assist in this process. Typically the solution as defined at this
point focuses on “what” the utility wants to accomplish, as opposed to defining “how”
those objectives will be achieved.
2.The utility issues an Request for Proposal (RFP) inviting qualified vendors to submit a
proposal to meet the utility’s needs as set forth in the RFP.
3.The utility evaluates the vendor responses, which typically includes conducting meet-
ing(s) with some or all of the proposing vendors.
4.The utility selects a vendor based on this process, and enters into contract negotiations
to arrive at a mutually agreeable contract.
The total cost to the utility of a CIS replacement represents a significant expenditure often¶39
over multiple years, ranging from several million, to several hundred million dollars (for large,
multi-state utilities with many millions of customers). A CIS replacement project’s duration
similarly represents a significant commitment by the utility. From conception to stabilization,
utilities may spend from two to ten years on their journey, with the actual implementation of
a chosen product taking from about 9 months (rare) to 12-24 months (common) to three or
more years (typically for large, multi-jurisdiction utilities).
3 See CFC_000204–000235.
J
T
H
C
Proprietary and Confidential Page 10 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 11
of 89
Expert Report
3.2.1 Defining the solution
Platform operating model
With the rapid growth and evolution of “cloud computing”, there are several more (often¶40
overlapping) choices available to a utility in terms of the operating model for its CIS. The
businessdriversinfavorof,oragainst,aparticularmodelarecomplex,andtypicallyrelatedto
both internaltechnicalconsiderations(the utility’soverallIT strategy)and external regulatory
considerations (e.g. the manner in which a utility is legally permitted to recoup the cost of
investments from its rate base).
In summary, a utility can choose from a spectrum of options ranging from “On Premise”¶41
hosting, where the CIS is deployed entirely on computing infrastructure owned and oper-
ated by the utility, to “Software-as-a-Service” (SaaS) models where the utility subscribes
to a completely remotely hosted and managed CIS solution. There are also intermediate
models available in the market, including “Infrastructure-as-a-Service” (IaaS), where the
utilitysubscribestobasicfacilities, hardwareandnetworkenvironmentservices, butrunsthe
remaining services itself, and “Platform-as-a-Service” (PaaS), which adds operating system,
middleware and database services to the subscribed portfolio.
This spectrum of options is depicted in figure 3.2.¶42
Facilities
Hardware
Storage
Media
Network
Connectivity
Operating
System
Interfacing
Middleware
Database
Applications
& Packages
Business
Config.
Customer
Data
Facilities
Hardware
Storage
Media
Network
Connectivity
Operating
System
Interfacing
Middleware
Database
Applications
& Packages
Business
Config.
Customer
Data
Facilities
Hardware
Storage
Media
Network
Connectivity
Operating
System
Interfacing
Middleware
Database
Applications
& Packages
Business
Config.
Customer
Data
Facilities
Hardware
Storage
Media
Network
Connectivity
Operating
System
Interfacing
Middleware
Database
Applications
& Packages
Business
Config.
Customer
Data
On
Premise
Infrastructure
(IaaS)
Platform
(PaaS)
Software
(Saas)
Utility
manages
Vendor
manages
Key
Figure 3.2: Typical platform operating model choices
In the vast majority of these models, ownership for customer data definition and business¶43
J
T
H
C
Proprietary and Confidential Page 11 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 12
of 89
Expert Report
processconfigurationspecificationremainswiththeutility, thoughagainthereareexceptions
and the landscape is changing rapidly.
Notethattherearenotcommon, universallyaccepteddefinitionsoftheseservices; themarket¶44
is evolving rapidly and there are many variations in the specific components included, or
excluded, from a specific “XaaS” offering from a particular vendor. This obviously serves
to further complicate the landscape and the decision process for people and entities not
intimately versed in these nuances.
The choice of platform operating model is further complicated since not all CIS vendors offer¶45
their product in all operating models. In the City’s case, the chosen OSF product is available
in the On Premise, PaaS and SaaS configurations. The City initially selected an On Premise
implementation of OSF but, as discussed below, eventually sought to enhance the contract to
a blend of the PaaS and SaaS models4 .
Business process definition and scope
A utility needs to define the scope of what is to be procured, and how it will fit into the utility’s¶46
operation. This is often based on considering the business processes the utility undertakes,
and determining to what extent the new CIS system will need to support them. Other input to
this business process definition activity may include the utility’s organization chart (many
utilities are organized around business processes, so the organization chart may be a useful
proxy for a business process model). Similarly, in the case where a utility is replacing an
existing CIS, the functions supported by that system often provide a minimal set of processes
that the new solution must meet.
A related decision the utility needs to make is to identify how much business process trans-¶47
formation and modernization it wants to undertake as part of the CIS implementation. Some
utilities want the CIS implementation to have minimal impact and change to their existing
business processes. Others recognize that their existing business processes may be anti-
quated, and may have evolved over decades driven by the restrictions of the utility’s current
systems. The needs of the utility’s customers may also have evolved since the current pro-
cesses and systems were designed, thus the utility may want to take the opportunity offered
by a new CIS implementation to re-visit, and transform, its business processes.
With the evolution of the CIS product marketplace over time, the leading CIS implementation¶48
practices now focus on transforming and adopting the utility’s business and processes to the
features and functions provided by the chosen product, as opposed to adapting the product
to the existing utility’s business processes. There are several drivers for this trend, including:
1.Reducing total cost of ownership to the utility by reducing the amount of custom code
that is specific to that utility, and for which the utility will need to carry the maintenance
costs.
2. Leveraging the CIS product vendor’s investment, amortized over multiple clients.
3.Adopting proven business processes, as provided by the product, which are common
and accepted across the industry.
4 Rule 30(b)(6) Deposition of City of Fort Collins (Storin Rough) at 80:22-81:21.
J
T
H
C
Proprietary and Confidential Page 12 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 13
of 89
Expert Report
This does require a willingness on the part of the utility to change business processes which¶49
may have been in use for decades, and thus the leadership of the utility may need to be
prepared to lead its business team through a (potentially challenging) organizational change
process as part of a CIS implementation.
Requirement definition
In this activity, the initial requirements of the desired solution are created. This process takes¶50
place well before the process to identify a product and implementation partner, though it will
continue and evolve during the selection and initial implementation processes. Requirement
definition is usually driven by, or conducted in parallel with, the business process definition
stage described above.
The objective is to define a list of “must have” features (“requirements”) that the chosen¶51
solution must possess in some form. Often utilities engage the services of expert advisory
consultants, familiar with the product and implementer landscape to assist in this process.
Many of these advisors have their own approaches and methodologies for producing a re-
quirements list, and many have pre-existing requirement libraries based on their experience
from other engagements that can serve as a template for, and jump-start, the requirement
definition process.
In addition to “must have” requirements, utilities often create a list of “nice to have, but not¶52
critical” requirements that the utility would like to see in the solution.
In terms of quantity, a requirement definition process for an RFP may produce from many¶53
hundreds to several thousand of requirements.
There are several key objectives of the initial requirement definition, including:¶54
1.To provide potential product and implementation vendors with a general description of
the ideal solution the utility seeks, to enable them to provide a responsive solution (and
price) for meeting the utility’s needs.
2.To trigger, facilitate and document discussions within the utility about the impact of
the implementation on its business. This is also linked to the business transformation
discussion described earlier (paragraph ¶47).
3.To provide the utility (and potential vendors) with some level of certainty around the
completeness of the solution the utility seeks. This is typically validated by the utility
mapping the requirements to its business processes, ensuring that all business pro-
cesses are covered by an appropriate requirement or set of requirements. Business
processes with few or no associated requirements are “yellow flags” indicating potential
omissions in the requirement list, or incompleteness in the desired solution scope.
Requirements, as expressed in the vendor selection documents, are typically concise declara-¶55
tive statements regarding a specific feature the solution must possess. For example (taken
verbatim from the City of Fort Collins’s “Functional requirements matrix” as provided to
bidders):
System provides the ability to display amounts due and due dates on primary CSR
J
T
H
C
Proprietary and Confidential Page 13 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 14
of 89
Expert Report
screen5 .
This is a very typical requirement description; short, and undetailed, generally calling for a¶56
“yes-or-no” type answer from potential vendors.
One weakness of this approach to requirements is that, by virtue of their conciseness, the¶57
requirements are typically imprecise and open to multiple interpretations particularly con-
sidering that they are written by people with deep knowledge of the specific utility producing
them, yet read by vendors operating under tight time constraints, who possess deep industry
and product knowledge, but inevitably less or no in-depth knowledge of the specific utility’s
practices and procedures.
A (hypothetical) potentially ambiguous requirement could be:¶58
The system will support payments made from an agency as part of a Low Income
Heating Assistance Program (LIHEAP).
In many ways this is just another payment source to the CIS, so a “yes, we support” answer¶59
is accurate. However there may be details in practice, such as that payments for multiple
clients are bundled into one “mega payment” from the LIHEAP agency, which make the
requirement’s implementation so unwieldy as to be effectively not capable of being supported
by the system. The devil is in the details, and terse requirements often cannot be sufficiently
detailed to provide full clarity into the need, and thus the response.
Proposals generally are made based on the information in the RFP and subject to revision¶60
during negotiation and project execution, as more details specific to the utility’s business are
discovered.
Other RFP components
Although the requirement list forms the core of the RFP body, there are other components¶61
which are critical and which need careful consideration and writing. These include:
1.Background information regarding the utility, its size, its number of customers and the
number of services it bills for (e.g. electricity only, electricity and gas, other services
such as broadband).
2.Key business drivers for the utility including any desired time-frames, or other con-
straints.
3. Details on the procurement process the utility will follow.
4. Often, specific legal terms and conditions that must be part of any final contract.
5.Possibly sample template contracts that the successful bidder will be expected to use as
a basis for negotiation.
6.The required format of the response. This, in many cases, can be quite restrictive and
specific, with vendors who do not comply with the required response format being
penalized or having their response disqualified from further consideration.
5 CFC_000237. “CSR” stands for Customer Service Representative.
J
T
H
C
Proprietary and Confidential Page 14 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 15
of 89
Expert Report
3.2.2 Vendor evaluation and due diligence
Once the RFP has been prepared it is sent to a selected group of potential candidate firms. It¶62
may also be published via the utility’s procurement process to increase the list of potential
responders. The utility will most likely conduct at least one “bidder’s conference” to both
answer questions that potential bidders may have, and also provide commentary on, and clar-
ification to, the utility’s selection process. Sometimes these meetings result in amendments
to the RFP being issued, sometimes not.
Vendor evaluation
TheRFPwillcontainadeadlinebywhichvendorsarerequiredtosubmitresponses. Following¶63
receipt of responses the utility will perform an evaluation of the responses received. This
process is largely driven by the utility’s procurement team.
An initial review process may include an administrative review of the responses to make sure¶64
they comply with any mandatory requirements regarding format, content etc. Responses
passing this screening (not all do) are forwarded to an evaluation team within the utility to
competitively rank and score the responses. This evaluation team typically consists of utility
business and technical staff members, operating under strict rules to maintain confidentiality.
The evaluators often work in a combination of individual assessments and group discussions,
to maintain alignment and consistency while evaluating multiple responses over multiple
days, weeks or sometimes months. If a third party advisory team is helping the utility, then
that team may take an overall facilitation lead, or the facilitation role may be performed by a
representative of the utility procurement organization. The references supplied by the vendor
are checked, often at this stage, sometimes later in the process.
After an initial evaluation, the utility typically invites vendors to oral presentations and dis-¶65
cussions. Depending on the number of respondees and the utility’s procurement process, this
may involve only the top ranked respondees or it may involve all of them. These oral meet-
ings can range from relatively short (one half-day or whole-day meetings) to more extensive
multi-day events. These are one-on-one meetings between the utility and the vendor.
The conduct of the meetings is usually dictated by the utility, in terms of agenda, flow, topics¶66
to be covered, and topics not to be discussed (for example, many utilities forbid the discussion
of pricing details at oral presentations, handling that topic in a more confidential manner).
Often a large group of utility employees (larger than the core evaluation team) will attend the
oral presentations, thus giving exposure and soliciting input from a broad component of the
utility’s staff. For example, core evaluation teams of around 10 utility staff members may be
typical, while 20 or 30 or more are often invited to the oral presentations.
A typical oral presentation will include demonstrations of the vendor’s product, presentations¶67
on the proposed implementation methodology, introduction of key individuals and partner
firms, as well as general statements, often from senior leadership of the vendor, about the
vendor’sfutureintentionsregardingtheproductandthemarket. Implementationdiscussions
oftenincludesectionsontheroletheutilitystaffwillbeexpectedtoplayintheimplementation;
thiscantriggerintensediscussionwithintheutilityteamifthereisagapbetweenthevendor’s
position and that of the utility.
J
T
H
C
Proprietary and Confidential Page 15 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 16
of 89
Expert Report
There is typically plenty of opportunity for the utility staff to ask questions, raise concerns¶68
and interact with the vendor’s team. The discussions are usually lively and informative, and
often result in follow-on material or clarifications being submitted by the vendor.
Due diligence
Following oral presentations from all the invited vendors, the utility evaluation team regroups¶69
and refines its evaluations and recommendations based on the oral meetings. The team
then proceeds to conduct a series of activities to produce a recommended selected vendor
to engage in detailed contract negotiations. These activities are often referred to as “due
diligence”.
The specific activities performed by the utility in this phase vary across utilities. The outcome¶70
of the activities is a “selected” vendor (and sometimes an official “runner up” vendor). There
is typically a requirement that the process be thorough, fact-based and well documented in
terms of both the reasons to select a particular vendor, and the reasons to reject a vendor.
Ideally the selection team will (quickly) align and coalesce around a candidate vendor, with
all team members largely in agreement as to both the choice and rationale for the selection.
Often the due diligence process is iterative; some vendors may be eliminated quickly leaving¶71
either a clear leader, or a small (two, possibly three) number of viable candidates which will
be subject to a more lengthy and thorough vetting process by the utility.
In addition to the core evaluation team, the utility may engage the services of internal or¶72
external experts to assist in this due diligence (for example, engaging specialized accountants
to opine on the vendor’s financial stability).
Typical due diligence activities may include:¶73
1. In-depth conversations with the client references a vendor provided.
2.Conversations with other utilities who may have experiences with one or more of the
vendors.
3. Consultation (formal and informal) with industry experts and consultancies.
4. Review of the vendor’s financial stability.
5. Evaluation of the impact of any potential or active litigation involving the vendor.
6. Further conversations with one or more of the vendors, with the goals of (for example):
• Clarifying questions the evaluation team may have.
•Exploring options not included in depth or not addressed in the vendor’s proposal,
but which may be of interest to the utility (for example, the vendor’s ability to
support an alternative platform operating model, as discussed in section 3.2.1 on
page 11).
7.Starting the process of socializing the preferred vendor(s) within a broader group of
utility executives and management to identify any concerns or issues which may need
to be considered as the due diligence process advances.
J
T
H
C
Proprietary and Confidential Page 16 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 17
of 89
Expert Report
8.Adetailedanalysisofthevendor’sfinancialproposal,tobothvalidateitscompliancewith
the utility’s business case, and to prepare for in-depth negotiations in the contracting
process.
9.An in-depth analysis of the vendor’s technical proposal, implementation methodology
and approach, to validate detailed alignment with the utility’s desires, and also prepare
for the contracting process.
3.2.3 Vendor selection, contract negotiation and award
Once a vendor has been selected, a detailed contract negotiation process commences. This¶74
can range from several weeks to multiple months in duration. There may be multiple vendors
to negotiate with, for example it is possible that the CIS product and the implementation
vendor are separate firms6 .
Key components to be negotiated include:¶75
1. A license framework for the software being procured.
2.A master services agreement, which provides a general framework under which the
implementer will provide services.
3.One or more Statements of Work (SOWs) that describe in detail specific work items to be
carried out by the implementer and the utility. Typically these SOWs carefully define
not only what the implementer will do, but also what the utility must do to enable the
implementation to succeed.
Often as part of this negotiation process the parties gain additional information regarding¶76
the manner in which the product chosen may not fully meet particular business needs of the
utility. This is a result of the parties working together in a more cooperative, open manner
compared with the competitive nature of the bidding and evaluation process.
In the event that material gaps are identified, the utility and the vendor usually work to¶77
determine potential solutions, similar to the steps described in section 3.3.2 on page 20. Any
such resolution may, in turn, result in the parties agreeing to modified contractual scope
and / or pricing details to accommodate the revised solution. In some (relatively rare) cases,
the uncovered gap(s) may be so great (or the cost of bridging them so high) that the utility
chooses to break off contract negotiations with the vendor, and pursue an alternative vendor’s
solution.
At the successful completion of the negotiations the project will proceed to the execution¶78
phase, oftenwithabreaktopermitthesuccessfulteamtoregroupandorganizefortheproject.
Typical SOW components
The SOW is a critical document for the success of the project. The SOW sets the scope, tone,¶79
mutual agreements, assumptions, and expectations for the lifetime of the CIS implementation
project (which may range from multiple months to several years in duration). Typically the
6This was not the case for the City’s procurement
J
T
H
C
Proprietary and Confidential Page 17 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 18
of 89
Expert Report
parties recognize that during the course of the project the assumptions and agreements
contained in the SOW will need to be modified in the light of project (and other) events. Thus
the SOW typically will contain procedures for modifying the SOW, often known as “Change
Controls”, or another synonym such as “Change Requests” or “Change Orders”.
The form of a SOW varies widely across vendors and utilities, often driven by procurement¶80
or contracting practices, prior SOWs the firms may have negotiated and the individual pref-
erences of the teams negotiating them. However, most SOWs will contain in some form the
following components, many of which will have been initially described in the utility’s RFP
and / or the vendor’s proposal, and which may have been modified, clarified or extended
during the negotiation activities:
1.An executive summary, summarizing the project objectives and the scope of work to
be performed, outlining the general manner in which the parties will work together to
achieve those objectives and the broad division of responsibilities between the parties.
Additionally, crucial assumptions, partnerships and other items may be called out in
this section.
2.A more detailed description of the scope of the project, in terms of business processes
and requirements which are considered to be within the scope of the project. This is
often a copy of, or reference to, the scope descriptions in the utility’s RFP or the vendor’s
response.
3.Adescriptionofthemannerinwhichthevendorandutilityproposetocarryoutthework.
This is typically the result of negotiations between the utility and the vendor to align
on the methodology to be followed during the project, and is often a combination of a
schedule and sequence of activities (the initial Project Plan) and associated descriptions
of the activities.
The project plan in the SOW will typically be at a fairly high level, perhaps with one- to
two-hundredactivitiesanddeliverables. Itwilldescribethecontemplatedtimelineofthe
entire project. Parts of the plan will typically be expressed in the form of a schedule or
Gantt chart, other parts will often be expressed in tabular form (e.g. lists of deliverables
to be produced, or lists of integrations to be built).
4.A clear delineation of the roles and responsibilities to be assumed by each party (utility
and vendor) for each of the activities in the plan. Different activities may be carried out
by one or the other party alone, or by a combination of both parties. Similarly, activities
carried out by one party may need to be approved as “fit for purpose” by the other party.
Often a “roles & responsibilities” matrix is built into the SOW to clearly delineate the role
and responsibility of each party with respect to each activity. This is often expressed as
a “RACI” table, The definitions in table 3.1 are based on those in the SOW agreed-to by
the City and Open.
Table 3.1: RACI table from statement of work
R:Responsible for completing the activity or Deliverable. Respon-
sibility can be shared; the degree of responsibility is determined
by the individual with the ‘Approve’ role.
J
T
H
C
Proprietary and Confidential Page 18 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 19
of 89
Expert Report
A:Approve and accept the deliverable (includes being consulted
in its preparation).
C:Consulted when required (e.g., review and verification). Con-
tributes to input for completing the activity or deliverable.
I:Informed about the status or outcome of an activity.
5.A description of the project management and executive oversight processes, along
with the roles and responsibilities of the parties involved. This will typically include
agreements covering (among other items):
• Project status reporting within the project team (frequency, typical content).
• Project planning updates (frequency, staffing plans, typical process).
•Executive / Steering Committee process (participants, frequency, typical agendas).
• Tools and processes to track and manage the project.
6.A staffing model, showing the anticipated required levels of staffing which will be as-
signed to the project by each party.
7.A description of the facilities and technical infrastructure to be provided by each party.
8.A description of the way in which components of the project will be ’accepted’ as being
completed.
9.A description of the payment schedule; the manner in which the utility will pay the
vendor, and any associated milestones which must be met for the payment to be made.
10. A description of the Change Control process.
11.Any other key assumptions being made which, if not met, may affect the project timeline
or cost, and require changes to the SOW, or which may even jeopardize the project’s
success.
3.3 Project Execution
ThissectiondescribestheactivitiesofatypicalCISprojectfollowingagreementandexecution¶81
of the contracts. At this stage, with the competitive procurement activities completed, the
parties need to commence a process of building mutual trust and respect, to permit them to
work openly together to achieve implementation success.
3.3.1 Project initiation and planning
Most projects begin with an initiation or organization phase. Typically the implementer¶82
starts with a small team of key staff working directly with their counterparts in the utility,
who lay the groundwork for the remainder of the project. Some tasks are logistical (e.g.
establishing physical facilities, providing computer system access) while others are focused
on elaborating the activities described in the applicable SOW (e.g. creating project plans
which name individuals who will carry out specific tasks and scheduling needed meetings
andworkshopswithspecificrequiredattendees). Oftentheimplementerwillconducttraining
sessions for utility staff, training them not only in the product capabilities, but also in how to
carry out their role on the project.
J
T
H
C
Proprietary and Confidential Page 19 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 20
of 89
Expert Report
3.3.2 Scope confirmation and solution definition
Once the initial project organizational activities are concluded the teams move on to confirm¶83
the scope of the project, and elaborate on the way in which the solution will satisfy that scope.
Each implementation team has its own approach to this step, but the core activities and
objectives are common to all implementations. They include activities to:
1. Review the scope of the project as described in the RFP and SOW.
2.By comparison of product capabilities with the utility’s current (or desired future) busi-
ness processes, identify any substantial gaps where the product may have difficulty in
supporting the business process as the utility defines it. This builds on the work done
earlier in the procurement and negotiation processes to develop a greater level of detail,
since at this stage of the project the schedule can permit an in-depth exploration of
processes, contingent on the utility having well-defined processes and/or staff available
to discuss them.
3.Using knowledge of the product’s capabilities, and possibly a reference process model,
determine at a high level what the utility’s future business processes should look like to
maximize the alignment between the product and the business process.
The proposal and procurement work is often time-limited and unable to explore to the
required level of detail the full complexity and depth of the utility’s processes. In this
solution definition activity the utility and the vendor conduct a more detailed explo-
ration and discussion with input from subject-matter experts on both sides. Often this
discussion will continue during the project as more details regarding complex processes
are discovered and need to be resolved.
4. Suggest potential ways to eliminate or mitigate any identified gaps:
a) Change the utility business process.
b) Propose the vendor modify the product.
c) Develop an alternative approach:
• Manual process.
• Additional software brought into the solution.
• Other work-around.
The output of these activities is typically a set of “requirements”, based on the features¶84
requested in the RFP but elaborated to provide more detail and specificity (recall from para-
graph ¶57 that RFP requirements tend to be terse and very high-level). The requirements in
the RFP and/or SOW may grow significantly in number during this activity as more details
emerge as to the required capabilities. The number may also fall as the utility refines its
understanding of its own needs and desired functionalities, in effect reducing the scope of
the project.
Once the requirements have been confirmed and reviewed, people (primarily from the ven-¶85
dor, but often including participation by utility staff) with knowledge of both the product’s
capabilities and the requirements work to determine in detail how best to meet each require-
ment in light of the product’s capabilities. In many cases there is a direct, simple connection
between a requirement and the most appropriate solution, a simple “fit” of the solution to
J
T
H
C
Proprietary and Confidential Page 20 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 21
of 89
Expert Report
the requirement. For example, a requirement to support a particular tax may be a matter of
entering the tax rate in a tax table within the product, while a more complex rate structure
would require complex configurations to be designed, documented, implemented and tested
(see also paragraph ¶97).
In other cases, there may be a direct connection between a requirement and solution, but it¶86
requiresadditionalartifacts7 tobeconstructed. Asanexample,arequirementtopostfinancial
transactions to the utility’s General Ledger (GL) may require product configuration and code
to be written (known as an interface) to take the GL data from the CIS system and move them
into the utility’s GL system. This type of integration development is fully anticipated and
normal in CIS implementations.
Some requirements may have solutions that do not involve the CIS system at all, they may be¶87
entirely manual, or they may be carried out with the support of other software not in the CIS
solution’s scope, for example a Geographic Information System (GIS) could be used to support
the supply of service to newly constructed premises, with the CIS not being involved until the
service is fully installed and ready to be turned on.
Finally there will be some requirements for which there is no clear single best solution; rather¶88
a series of alternatives with various tradeoffs are identified. Typically the project team will
review these alternatives – most likely involving utility business team input and management
escalation, review and approval – to select an approach that most appropriately compromises
between the various alternative options. Very often in this situation one solution will involve
changing or modifying a utility business process to more closely align the business process to
the product’s capabilities. As an example, the utility’s process used to “start service” for a
new customer may not align directly with the new CIS’s standard behavior. A combination of
changes to the utility process and modifications to the CIS’s approach (and thus configuration)
is often adopted in this case.
One possible outcome of this process is the identification of requirements which are (a) re-¶89
quiredtobeautomatedintheCIS,but(b)arenotcapableofbeingautomatedusingcapabilities
of the CIS as it stands. These solutions become “product customization requests”, where
the CIS vendor makes the desired capabilities available in the product in order to meet the
utility’s requirement. The inclusion of product changes may be subject to a separate, more
formal, agreement process with the CIS vendor as changes to the product can impact other
customers using the product.
As the requirements and their solutions for particular groups of business processes are¶90
solidified, they are documented and “signed off” by both the utility and the implementer. In
Open’s methodology this is called the “Solution Scope” sign-off, though other terms are also
used. This sign-off (which may be a more or less formal process depending on the particular
utility) serves several purposes:
1.It provides the utility with additional confirmation regarding the manner in which the
solution will operate.
7“Artifact” is a term for any tangible thing that needs to be built, defined, or written as part of a CIS implemen-
tation. For example, code, documentation, training videos, test cases, test results, or policies and procedures all
can be considered to be artifacts.
J
T
H
C
Proprietary and Confidential Page 21 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 22
of 89
Expert Report
2.It provides the implementer with a basis for moving forward with the technical imple-
mentation.
3.It serves as a control point to minimize the potential amount of change for both the
utility and the implementer.
This latter point is important. Until the solution is signed off, there is some degree of iteration¶91
between the utility and implementer regarding the most appropriate combination of business
process, requirement and solution as the team works to optimize the implementation design.
However,atsomepoint,thisiterationneedstocease;certaintyisneededtopermittheutilityto
knowhowthesolutionwilloperate(andthusbuildthenon-systemcomponentsofitsbusiness
processes), and to permit the implementer to configure and build a cohesive solution.
The solution definition activity results in a series of decisions that refine the scope of the¶92
solution,typicallyproducingmoredetailedbusinessprocesses,anexpandedandrefinedsetof
requirements, and a list of artifacts that the project team needs to provide, with responsibility
for each artifact’s delivery being assigned to the utility or vendor in accordance with the
agreements. This latter list will include, for each part of the solution, some or all of the
following types of artifacts:
1.Configuration elements: details of what configurations need to be applied to the package
to deliver the required functionality. These are typically data concerning rate structures,
codes, tax-rates, GL account strings, and the thousands of other items which have to be
specified to enable correct operation of a CIS.
2.Report descriptions: descriptions (both content and layout) of any reports to be devel-
oped, or customizations that must be made to the standard reports that are delivered
with the product. Many CIS products come with a few (perhaps 50) standard reports;
many utilities’ businesses require many more reports than this (perhaps several hun-
dred); the additional reports will need to be specified and later built.
3.Interface descriptions: these technical specifications describe in depth the mechanisms
by which the CIS system exchanges data with other systems (both within the utility
and external to it). In a given CIS implementation there may be anywhere from 10–20
interfaces at the low end, to more than 150 interfaced systems, to be considered. In-
terfaces may be inbound (data flowing in to the CIS), outbound (data flowing out of the
CIS) or bi-directional. Examples of each type of interface would be payment information
from a bank (inbound), financial information to the GL (outbound) and field-orders
(bi-directional).
Additionally, interfaces may be real-time or batch. For a real-time interface, the data is
sent or received at the moment it is ready, typically one transaction at a time. A batch
interface queues up data to be sent, and then sends many transactions in one job, a
’batch’ of transactions.
4.Conversion descriptions: these specifications describe the Extract, Transform and Load
(ETL)specificationsformigratingdatafromexistingsystemstothenewCIS.Forexample,
details of customers’ accounts may need to be moved from the old to the new-CIS.
5.Enhancement descriptions: these technical specifications describe the code that the
project team will need to write to support specific requirements.
J
T
H
C
Proprietary and Confidential Page 22 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 23
of 89
Expert Report
6.Product customization descriptions: these specifications are similar to enhancement
descriptions, but they describe features that the product vendor will need to provide, by
enhancing the capabilities of its product.
Taken together, there will typically be several thousand artifacts that are described and¶93
delivered from the solution definition phase to the solution construction activities which
follow.
3.3.3 Solution construction
Fromatechnicalperspective,thevariousartifactsidentifiedaboveneedtobebuilt(configured,¶94
coded, unit-tested). This will involve taking the high-level descriptions produced as a result
of the activities described in paragraph ¶92, creating (very) detailed technical specifications
and then building the appropriate code and configurations.
This section, and the ones which follow, describe at a high level the various software construc-¶95
tion and testing activities which take place in a typical CIS implementation.
The solution definition activities will have identified components which need to be built¶96
or configured in the CIS product to support the agreed-to functionality, as expressed in
the detailed requirements and specifications. Some requirements can be met by system
configuration, others will require algorithms to be developed and tested, perhaps including
the development of background batch processes to perform bulk actions, unique to the
utility’s business, on a large population of its customer base. Unique, often legally mandated,
credit and collection processes are a typical example where periodic scans of the customers’
accounts may be necessary to implement utility-specific policies.
One often overlooked nuance is that “configuration” artifacts (paragraph ¶30) will need to¶97
be designed and documented, in addition to any custom code artifacts. A common mistake
is to think that, since configuration is simply entering data in screens in the application,
and the screens have labeled fields to put the data in, then no design or documentation is
needed. However, many configured objects in a CIS system have many, many inter-dependent
components, and they need just as much planning and documentation (design) as do artifacts
which are to be coded in a programming language. Examples of complex configuration
include:
• Rates: how consumption is translated to charges.
•Deposits: how to determine the amount of money, if any, a customer should be required
to deposit, before being able to start service with a utility.
•Credit & collection processes: what should the process be for progressing a delinquent
customer, from warning letters through penalty charges, potential disconnection of
service, referral to a collection agency and legal action.
Integration
The integration construction activities create the interfaces and integrations needed between¶98
the CIS and other systems, as described in item 3 in paragraph ¶92.
J
T
H
C
Proprietary and Confidential Page 23 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 24
of 89
Expert Report
Responsibility for integration construction is often shared to some degree between the utility¶99
and the vendor, since typically the utility will be more knowledgeable about the systems to be
interfaced with, while the vendor is more familiar with the manner in which the CIS operates
and how best to integrate to it. The SOW will define the exact roles and responsibilities, and
the point at which the responsibilities meet and hand-off.
In many CIS implementations, there is a layer of software known as “middleware” that sits¶100
between the CIS and the other systems, and acts as a broker mechanism to facilitate and assist
in the management of the data transfer. Often this middleware layer, if it exists, serves as
the handover point for integration implementation. The utility is responsible for integration
activitiesbetweenthemiddlewareandthetargetsystem,whilethevendortakesresponsibility
for activities between the CIS and the middleware. Responsibility for activities within the
middleware layer are typically agreed to as part of the SOW negotiation process.
The middleware layer may be a commercially provided product, or a component developed¶101
by either the utility or the product vendor8 .
Conversion
A critical part of any CIS implementation is the conversion activities. These activities are¶102
sometimes referred to as migration, or data-conversion, activities. These activities move
needed information from existing utility sources to the new CIS to enable its operation. Some
examples of information (often called data entities, or just entities) that will most likely be
needed in the new system, and which thus need to be converted, include:
1.Customer details: Information about the customers of the utility, such as names, mailing
addresses, phone numbers, email addresses etc. There may be “special needs” informa-
tion (the customer has critical medical equipment, or is a medical or emergency facility),
and other key data.
2. Payment history: for the customer.
3. Account balances: for each service provided to the customer.
4.Contact history: information about past interactions the utility may have with the cus-
tomer.
5.Premise details: Information about the physical locations where the utility provides
service, such as address, map coordinates etc.
6. Service details: Information about what service the utility provides to each premise.
7.Meter details: Information about the type of meter(s) installed at a Premise to deliver a
Service.
8.Usagehistory: Informationabout the historical usage foreach service (prior meter reads
for, say, the past 12- or 24 months).
The exact list of data entities to be converted, and how much historical information for each¶103
8 In the City’s case, Open provided their Open Smartflex Connector Kit (OSCK) component to provide this
middleware functionality.E.g. Open_Intl_00084109 at -122.
J
T
H
C
Proprietary and Confidential Page 24 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 25
of 89
Expert Report
of them is to be moved to the new system, is driven by the utility’s business and regulatory
requirements, and will need to be agreed on relatively early in the project. Note that even for
newservices,someinformationwillneedtobeconvertedandloadedintotheCIStojump-start
service provision. For example, for the Broadband initiative undertaken by the City of Fort
Collins, the list of premises that could be supplied with service, and their location, equipment
details etc. would most likely need to be pre-populated into the CIS prior to go-live.
There are many activities which need to take place during the conversion process, the key¶104
ones of these are summarized below:
1.Detailed conversion scope definition. This activity determines the specific entities to be
converted, the definitive source of the entity in the utility’s existing application portfolio,
the volume of historical data to be converted for each entity.
2.Dataqualityanalysis. Thisactivityassessesthe‘quality’ofeachdataentity,andidentifies
needed remediation and clean-up that may be required prior to conversion. Examples
of data quality assessments include:
• Name and address standardization.
•Duplicatedataremediation(e.g. duplicatemeterserialnumbers,whicherroneously
showthesamephysicalmeterbeing, impossibly, attwoseparatephysicallocations).
•Incorrectly entered data (i.e. data entry errors which may cause confusion or errors
if propagated to the new CIS without correction).
3.Data mapping. This activity maps the data elements needed by the new CIS to their
source in the legacy list of entities identified in the scope definition phase. Gaps in the
mapping (i.e. source elements with no place in the target system, or target system needs
with no associated source) indicate possible scope definition errors, or solution gaps,
which need to be investigated and resolved.
4.ETL construction & test. Following data mapping, the actual conversion code needs to
be written and tested. This code is often known as “ETL” code, standing for “Extract,
Transform and Load”, a phrase which describes the process of extracting data from
legacy systems,transforming it to the format required by the new CIS, and loading it
into the new CIS.
5.Balancing and audit. This process defines the audit checks and balances to be put in
place to assure that the data is converted correctly and completely, in accordance with
the agreed-upon conversion scope. Examples of balancing reconciliation include:
• Financial balance by service type.
• Count of customer accounts converted.
• List(s) of data elements not converted.
The balancing process can be complex and time consuming, but it is essential. Making
sure the new CIS contains all the necessary data needed, and that the data is correct, is
vital to ensure the CIS’ ability to operate correctly. Each unexpected result discovered in
the balancing process must be investigated, and either accepted as a valid deviation, or
remediated prior to go-live. The results of the balancing process form one key artifact
thattheutility’sauditteamwillusetocertifythatthenewCISis, indeed, readytosupport
the utility’s business.
J
T
H
C
Proprietary and Confidential Page 25 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 26
of 89
Expert Report
6.Performance / timing. Even with state-of-the-art technology, the conversion process
can take considerable time to execute. Ranges from 6–10 hours, to 3–5 days, are not
uncommon. The conversion timing is often the critical path in the go-live process; it
is critical that the time take for conversion is well understood and factored into the
go-live plan. Multiple conversion runs should be executed, timings carefully monitored,
and strategies deployed to make sure conversion takes place within the required time
window.
Responsibility for conversion tasks
In each implementation the parties define their respective roles and responsibilities for¶105
data conversion activities, often in the form of a “RACI” matrix (see table 3.1 on page 18).
The allocation of responsibilities will typically be initially defined in the SOW. Factors which
influence the assignment of responsibilities during SOW negotiation include:
1.Knowledge of the existing system(s). Typically the in-depth knowledge of the current
utility systems is concentrated in utility IT and business staff, thus they are the logical
choice to take primary responsibility for providing information and data from those
environments.
2.Knowledge of the new system(s). By similar reasoning, the vendor is best positioned to
carry out activities where knowledge of the target product is most required.
3.Staff availability. Carrying out the conversion activities can require significant effort
and allocation of staff, and potentially special skills, particularly if complex data quality
remediation or transformation activities are required.
In terms of the conversion activities listed earlier in this section, a typical assignment of¶106
responsibilities may be:
1. Scope definition. Joint agreement.
2.Data quality. Utility primarily responsible, vendor often provides both technical support
and advice as to leading practices and implications of the utility’s choices.
3.Data mapping. Joint. Mapping sessions typically led by vendor, with heavy involvement
of utility business and technical staff. Ultimately the mapping definitions should be
mutually agreed-to by both groups.
4.ETL construction. Often joint, with the utility providing extracts from existing systems,
and the transformation and loading activities being carried out by the vendor.
5.Balancing and audit. Utility responsible; vendor may provide technical assistance such
as the development of balancing reports running on the new CIS.
It can be seen from this discussion that the utility and vendor must closely cooperate on¶107
conversion tasks, and clearly agree to and deliver on their respective responsibilities.
J
T
H
C
Proprietary and Confidential Page 26 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 27
of 89
Expert Report
3.3.4 Report construction
Most utilities use a vast number of reports to manage, report on and monitor their business.¶108
A large utility may have over one thousand reports; for smaller utilities several hundred is not
uncommon.
These reports are often closely linked to the workflow and business processes of the utility,¶109
and are typically quite specific to an individual utility.
CIS products often come with a relatively small (perhaps 50 or so) standard reports. An¶110
implementation project will use these reports as the basis for building the other reports a
utility may need.
Often the vendor will agree to build a certain number of reports, usually less than the utility¶111
needs, as part of its responsibilities on the project. The remainder of the reports are typically
developed by the utility’s internal reporting team.
Specifications of reports are developed as part of the requirement and solution definition¶112
activities, with the specifications then given to the responsible party to develop and test.
3.3.5 Testing
Testing the solution (i.e. making sure it works as intended) is a complex task, and there are¶113
many ways it can be described and executed. For simplicity, this description divides testing
into two broad categories, System Testing, where components of the system are tested to
make sure they confirm to the specifications, and User Acceptance testing (UAT), where the
“Users” of the system (i.e. the utility) formally tests and accepts the system as being ready to
place into production.
Testing is often further divided into¶114
1.Unit testing: where small fragments of code and configuration are tested in isolation,
typically by the team which built them, to make sure they perform as expected.
2.System testing: where the small fragments are chained together into larger units, per-
haps supporting some or all of a business process, and tested to make sure they meet
the process specifications.
3.Integration testing: Unit- and System testing often are testing individual systems (e.g.
the CIS) in isolation. Integration testing connects systems together, and tests that they
are capable of inter-operating and exchanging information in the required manner.
4.End-to-end testing: this phase of testing strings all the processes, configurations, inter-
faces and other artifacts together, and runs through typical business processes from
“end-to-end”, making sure that everything works as expected – for example that correct
bills are produced, correct entries are made in the utility’s GL, field orders are correctly
issued and disposed-of and all the other processes work as intended.
5.Performance testing: testing the system to make sure it performs adequately (e.g. the
billing cycle completes in a timely manner, the system is appropriately responsive
to user requests). This is a complex activity, and can take significant time, planning
J
T
H
C
Proprietary and Confidential Page 27 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 28
of 89
Expert Report
and resources especially in the case of highly complex formal performance testing
requirements.
Typically, while the utility’s and vendor’s technical teams are engaged in construction activi-¶115
ties, the business team at the utility, supported by the vendor, has to prepare the business
process driven test cases used to validate that the solution is “fit for purpose”, i.e. works as
intended. Test cases typically are the responsibility of the utility because utility staff are in a
much better position to understand the utility’s business processes and thus to create test
cases which reflect the reality of the utility’s business. Having the utility prepare the test cases
also is advantageous because it acts as a check-and-balance on the work the vendor has done,
avoiding the vendor “grading its own homework”. Equally critically, test case preparation
typically increases the number of people in the utility familiar with the new CIS and the
associated new or changed business processes.
These test cases are fairly fine-grained scripts that follow the path of a part of a business¶116
process, and provide both detailed instructions on what to do, and also the expected result.
For example table 3.2 shows a possible (shortened and simplified) test case for testing part of
an “apply partial payment” business process.
Table 3.2: Simplified example of a test case
Action Expected result
Search for account of John Doe Account is displayed
Note the account balance (say, $123.45)
Apply a payment less than the ac-
count balance (say, $100.00)
The displayed balance is reduced by
the amount of the payment (in this
case, to $23.45)
In reality, test cases are often much more complex and specific, and can comprise 10’s of¶117
steps instead of the few shown above. Also, many utilities require rigorous evidence of test
case results, such as screen-captures showing that the CIS performed as expected.
An important group of test cases are those which verify how the solution reacts to error¶118
conditions, to make sure it fails gracefully, and does not crash, or cause inappropriate actions
tobetaken. So,tobuildonthepaymentexampleabove, additionaltestcasesshouldbewritten
to validate behavior when:
• “John Doe” does not exist in the CIS.
• The payment is more than the amount owed.
• The payment is dated for an “impossible” date such as February 29, 2022.
• The payment is for a negative amount (“-$100.00”).
and so on.
In the event that the result observed is different from the expected result, a “defect” for the¶119
J
T
H
C
Proprietary and Confidential Page 28 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 29
of 89
Expert Report
test is typically logged in a test management system, for further investigation and potential
remediation by the implementation team. Depending on the contract for the project, the
utility may have personnel who conduct an initial screening of registered defects to ensure
the issue is in fact a defect before it is passed to the vendor.
A typical utility CIS implementation may have 2,000 or more test cases; 5,000 would not be¶120
unusual for a large, complex utility.
Todevelopthesetestcases,itisnecessarytorelyondocumentationforthebusinessprocesses¶121
that the new solution provides. For a CIS replacement, this may be simply updating existing
business processes; for a greenfield implementation the processes may have to be built from
scratch.
A side effect of this focus on business process-based test development may be the identifica-¶122
tion of additional gaps in the implementation scope. Some level of this is inevitable and to be
expected, no matter how complete and rigorous the initial solution definition. Early definition
of test cases leads to early identification of these gaps, which minimizes any potential project
disruption caused by their implementation.
Testing is typically carried out by combinations of utility and implementer staff; often more¶123
technical testing (such as Unit Testing above) and defect remediation is reserved for the
implementer’s specialists, while business process related testing (such as end-to-end testing)
is carried out by utility staff members. Typically utility staff are needed to quickly identify
and resolve issues related to the technical environment, external systems and potentially
testers skill-sets which may arise. The specific split of responsibilities between utility and
vendor staff is normally one of the elements in the eventual contracts between the parties.
Having utility staff execute the business related testing also has two main advantages:¶124
• They become familiar with the new system.
•They may identify gaps in the implementation that the technical implementation staff
may overlook.
The reason testing is layered into multiple sub-phases is due to the complexity of a CIS. A key¶125
purpose of testing is not just to show the system works, but also find (and correct) areas where
it does not work. By verifying small components, then linking them together in increasing
layers of scope and complexity, the process of pinpointing the root cause of incorrect behavior
is made significantly easier.
The final testing activity in a CIS implementation is UAT. UAT is when the utility runs tests to¶126
formallyacceptthatthesystemisreadytobeplacedintoproduction. Thisdoesnotnecessarily
mean there are no defects, or that the project is complete, rather it means that the utility
accepts that the benefits of placing the system into production as-tested outweigh the risks
and problems associated with any defects or missing features. Typically there will be a clearly
documented and agreed-to list of such defects and missing features, with an associated
detailed schedule and plan for remediation and delivery following go-live.
In some contracts, UAT is extensive, and exhaustive, occupying many months and requiring¶127
J
T
H
C
Proprietary and Confidential Page 29 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 30
of 89
Expert Report
substantial commitments of utility staff. This testing would encompass executing test cases
that were developed independently from the test cases executed in other project phases to
ensure objectivity, independence and support the “ready to go-live” decision. Other contracts
takealighter(intermsofdurationandeffort)approach, oftenrelyingoninspectingtheresults
of earlier testing to decide if the system is ready for go-live.
3.3.6 Organizational readiness and training
Depending on the amount of business transformation needed to adapt the utility’s processes¶128
to the new CIS, there may be significant organizational change needed, and thus a need for
organizational change management (OCM) strategies to be developed and implemented. The
project needs to execute a series of activities to prepare the utility’s business (and technical)
teams to use, support and manage the CIS once it is in production. Depending on the utility’s
current practices, the readiness process may be more or less extensive. For example, for a
new line of business, there may need to be organizational structures established and staffed
(How many people will we need? Where will they come from? Who will manage them? Do we
have facilities for them?). A utility making a smaller change from current to new business
processes will have fewer of these structural activities to perform.
For all users of the new CIS, there will need to be training in how the system will support them,¶129
and to the extent that business processes may change, training in the new processes and
assessment of their impact on the team. For example, billing exceptions (problems with a
bill) may have been managed from a report in the old system, whereas they will be managed
via on-line workflow in the new CIS. Changes such as these will need to be trained out into the
organization.
Development and delivery of training material, and the manner in which it is delivered, will¶130
be specified in the agreements between the implementer and the utility, and often involves
combinations of staff from both parties. For example, training material preparation may be
led by the implementer, while the utility’s existing training team may be responsible for the
delivery and maintenance of the training.
3.3.7 Deployment and go-live
As part of placing the new system into production, detailed deployment and go-live plans will¶131
need to be developed describing all the needed steps, and their sequence and dependencies.
A typical deployment plan may have several hundred activities, some of the technical sub-
plans may cover critical activities at the 10- or 15-minute interval level of detail. Tasks to be
considered include but are by no means limited to:
1.Are there some business processes that need to be shut down, or switched to manual
solutions, during the cutover period?
2.Are there some clean-up activities that should be completed in the old system prior to
cutover?
3.When and how will external third-parties (such as banks) switch to sending information
to the new system?
4.Will the contact center be shut down for the go-live event? If so, how will emergencies,
J
T
H
C
Proprietary and Confidential Page 30 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 31
of 89
Expert Report
safety matters etc. be handled?
5. Is the workforce sufficiently trained? How will we know?
6.Have the Finance / Audit team signed off on the balancing process between old and
new systems, to ensure the utility’s finances continue to be accurately reported and
recorded?
7.What communications need to be sent to the utility’s customers and to the utility’s
regulatory or oversight organization? Is a media awareness / advertising campaign
needed (for example, if the layout and content of a bill from the new CIS is materially
different from that of the current bill).
8.Perhaps most importantly, what are the go-/no-go decision points during the go-live
event, who will make those decisions, and what do we do to recover part-way through
(for example, if, half-way through go-live, the new CIS is found to be out of financial
balance with the old system).
The preparation of the go-live plan typically takes place during the final
1⁄3 or 1⁄4 of the project.¶132
Actual execution generally will start with preparatory activities 2-4 weeks before the go-live
date,withthego-liveitselfoftenbeingexecutedoveraweekend, orholidayperiodtominimize
the impact on the utility’s operations. Some disruption is inevitable, but a well-designed and
well-executed go-live plan will minimize and manage this disruption.
3.4 Post-live support
When the new CIS is placed into production there is typically an intensive support program¶133
put in place to manage both the organizational and technical changes associated with the
event. This is often referred to as the “hypercare” or stabilization phase, and it typically
lasts a month or more. During this period staff from the utility, and the implementer (and if
applicable and appropriate, the product vendor) are standing by to quickly react to problems,
system defects, gaps in training or business processes or any other incidents that contribute
totheinevitableshort-termdipinproductivityandeffectivenessofthe utility’sbusinesswhen
the new system is launched.
Normally the project team remains in place to provide this hypercare support, though at a¶134
reduced capacity compared to the full project size. Often the hypercare team establishes
specific support infrastructure to enable users of the new CIS to quickly access the hypercare
team; specific email addresses and phone lines are common, and often hypercare team
members are colocated with specific end-user groups to provide in-person support (for
example in contact centers or billing back-office locations). These special capabilities are
typically deactivated at the end of the hypercare period.
Going live with a new CIS causes a short-term dip in productivity and performance of the¶135
business. This is inevitable given the degree of change associated with a CIS implementation.
The goal of the hypercare activities is to minimize both the depth and the duration of the dip.
This is shown diagrammatically in figure 3.3 on the following page.
Following the completion of the hypercare period the new CIS is managed and supported¶136
J
T
H
C
Proprietary and Confidential Page 31 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 32
of 89
Expert Report
through the utility’s typical support infrastructure (typically help desk and escalation support
teams). In some cases the escalated support function may be provided by utility staff, in other
cases it may be provided by a third party such as the product vendor or consultant entities.PerformanceTime
Naïve
Expectations
With
Hypercare
Without
Hypercare
Figure 3.3: Impact of hypercare on post go-live performance
The overall goal of hypercare is threefold:¶137
• Minimize the drop in performance at go-live.
• Minimize the duration of the performance drop.
• Maximize the improvement in performance once hypercare ends.
3.5 Governance and project management
The sections above discussed the key project activities of a CIS implementation. This section¶138
describes some of the key aspects of management and control that have been shown to be
effective in directing, managing and monitoring those activities.
3.5.1 Executive leadership and governance
In order to direct the wide-ranging activities of a CIS implementation, the project team needs¶139
effective leadership on a daily basis, and the support of an effective governance team both
within each of the parties, and jointly across the program. Project managers need to be em-
powered to effectively direct their teams, and make decisions within appropriate and defined
scopes of delegated authority. When decisions are to be made outside of their delegated
authority, project managers need swift access to informed executives who can make those
decisions or otherwise eliminate obstacles to project progress.
Thesegovernancestructuresand(ideally, named)individualsshouldbedescribedandagreed¶140
J
T
H
C
Proprietary and Confidential Page 32 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 33
of 89
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 34
of 89
Expert Report
4.If a test case (or group of test cases) fails, is there a common artifact that may be at the
root of the issue?
5.If a solution option proves difficult to deliver to the requisite quality level, what is the
business process impact?
The traceability model is also helpful in managing the tradeoffs that happen when a proposed¶145
solution component is not available in the product. As shown by the feedback loop at label A
in figure 3.4 on the preceding page, options available to the team at this stage include modi-
fying the business process, constructing a complex (expensive, perhaps error prone) piece
of custom code or having the product vendor enhance the product. Having data to support
the analysis of the potential impact of each choice greatly enhances the quality of decision
making.
An additional situation in which the traceability model proves valuable is when gaps in the¶146
solution are identified late in the project lifecycle. It is not uncommon for User Acceptance
Testing (UAT) to uncover gaps in the solution – UAT may be the time when a broad population
of users of the new CIS first get in-depth exposure to the solution, and with increased scrutiny
gaps are inevitably uncovered.
When a gap is discovered in UAT, a solution definition process needs to be carried out to¶147
determine how best to meet the gap. This is the red path labeled B in figure 3.4 on the
previous page. The impact of this can be significant if many business processes and the
associated requirements and build artifacts need to be revisited, or possibly constructed
from scratch. One (hypothetical) example: if the utility required an alert to be displayed on
multiple screens for a customer on “life support” equipment, and this was missed before UAT,
then as well as adding and testing the capability to display the alert, many of the screen-shots
in the training material may need to be recreated, which could take significant time and effort.
3.5.3 Execution lifecycle management
These activities are related to managing and tracking the progress of the project, making sure¶148
that the work done is contributing effectively to accomplishing the implementation activities
described above.
As with other aspects of CIS implementations, there are many variations and styles of effec-¶149
tive execution management, and many different tools available to support it. However the
following are some key components are common to most approaches:
1.An overall project activity plan, showing the activities of the project. This needs to
include work by both the implementer and the utility, as both teams must work together
if success is to be achieved. Typically there will be several plans, with different levels of
detail. For example there could be:
•A high-level plan showing the overall project, which may cover multiple months /
years and describe the high level project phases. This is typically only updated at
major project events, and may be the view used to communicate with leadership
outside of the project.
•A more detailed plan showing the upcoming 3 months of activities described in
J
T
H
C
Proprietary and Confidential Page 34 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 35
of 89
Expert Report
more detail. This plan could be updated 3-4 times over the 3 month period, and is
a rolling future look at plans, adjustments and work. It is typically the working plan
that the program managers use to help understand upcoming risks.
•A very detailed plan showing activities planned for the next month, including spe-
cificevents,workshopsetc. Thislevelofplanoftenaddressesthespecificavailability
of individual team members, showing (e.g.) vacation, and other planned time when
they are unavailable to the project. This is very much a living plan, managed and
adjusted on a weekly or potentially daily basis as progress is made, obstacles are
identified, team members work patterns change and so on. Rather than one sin-
gle project-level plan, depending on the project’s size, individual sub-teams often
maintain their own plans at this level of detail.
2.A staffing plan showing how the project will be staffed in alignment with the activities
describedabove. Thismaybepartoftheactivityplan,oritmaybemaintainedseparately.
Ideally staffing plans are shared between the utility and the implementer, though in
many cases this does not happen since there can be concerns about the confidentiality
of HR-related information that can be deduced from a staffing plan. Ultimately, each
party is responsible for allocating the staff necessary to meet its own commitments.
3.An updated list of key risks, shared between the utility and the implementer, identifying
major project risks and agreed-to mitigation and contingency actions. It is important
for success that risks are not only identified, but that plans are put in place to mitigate
them. If the risk “happens” (risks are things which may happen, not things which will
happen), then the mitigation plan will be executed to address the issue.
As a hypothetical example, a risk could be:
Ifwedonotcompletethesolutiondefinitiononschedule,duetostaffunavailability,
then the start of building the solution will be delayed potentially impacting our
go-live date.
The associated mitigating action could be:
Tomitigatethis,wewillstartbuildingdefinedcomponentsbeforethecompletionof
solution definition, recognizing that this may necessitate rework when the solution
is fully defined. We will also look to reassign staff temporarily to this activity,
recognizing it will impact their other work, and investigate bringing additional
contractor staff onboard to back-fill these individuals’ other duties.
Morecriticalthanthecreationofthesetoolsistheirfrequent,reviewanddiscussionamongthe¶150
leadership of all impacted parties (utility, implementer and any other groups participating).
These discussions would typically be weekly or bi-weekly, and would include review of the
plans, the projections, the risks and their mitigations. Typically the output of these meetings
also form the basis of communication to senior leadership outside of the day-to-day project
team, though often in a summarized, distilled manner.
Continuous management
The activities involved in a CIS implementation are numerous, complex and involve myriad¶151
combinationsofutilityprojectstaff,otherutilitystaffandvendorstaff. Theseactivitiesrequire
rapid, knowledgeabledecisionmakingandissueresolutiononadailybasistokeeptheproject
J
T
H
C
Proprietary and Confidential Page 35 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 36
of 89
Expert Report
on track. Typically this requires the utility and the vendor to dedicate skilled managers full-
time to the project (at least one from both the utility and the vendor, often several for complex
CIS implementations). This team of project leadership is often referred to as the Project
Management Office, or PMO. In addition to project managers, there are usually one or more
analysts assigned to support the PMO’s activities. A relatively small project may have 3-5
members of the PMO, a large project may have up to 10 or more and numbers in excess of
15-20 may be required on very large, complex implementations.
For a CIS implementation to succeed many key components need to be managed and coor-¶152
dinated. There are always conflicts between competing demands for people, and pressure
to make timely, well-considered decisions. The role of the project managers is crucial to
navigating the technical and organizational challenges that will occur. A competent, experi-
enced full-time project manager is essential to completing an implementation. Providing that
manager with support, in terms of implementation experience and organizational knowledge
of the parties involved, is also highly important.
For decisions which significantly impact other parts of the utility, or which may critically¶153
impact the project in some way, the PMO needs access to senior, empowered decision makers
in both the utility and the vendor organizations. At least some of these executives should be
able to devote 10% or more of their time on a long-term basis to the project’s oversight. This
helps ensure the project can swiftly address risks and issues as they arise, and help to avoid
the negative impact that long-running issues inevitably cause.
J
T
H
C
Proprietary and Confidential Page 36 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 37
of 89
Expert Report
4 Project overview and summary of opinions
Below I have provided a brief overview of the project to provide a framework in which to¶154
place the specific opinions and analysis which are summarized in section 4.2 on page 39 and
described in detail in section 5 on page 42.
4.1 Overview of the project
In late 2017 / early 2018, the City decided to proceed with a project to replace its utilities¶155
CIS with a new system capable of supporting both the existing utilities business, and also a
nascent broadband business the City was in the process of establishing. The project to replace
the City’s aging utilities CIS had been planned for several years. With the citizens’ approval
(in November 2017) of a ballot measure to add broadband as a City-provided service, the City
decided to expand the scope of the CIS replacement to include needed capabilities to support
broadband functionalities in addition to traditional utilities functionalities.
Accordingly in February 2018, the City issued an RFP for a combined broadband and utility¶156
CIS product, and associated implementation services. Open submitted what proved to be the
selected proposal, in mid-March, and after a period of negotiation signed agreements with the
City (in August 2018) to provide, and implement, Open’s “OpenSmartflex” (OSF) CIS product.
OSFhasbeenrecognizedsince2014asamajorCISproduct,havingbeenincludedinGartner’s¶157
Magic Quadrant (now Market Guide) since that date, with a strong presence and reputation
in the South American market in particular. As of the City’s RFP, the City would represent
Open’s first customer in the continental United States, though Open now has others, including
a successful implementation for Tualatin Valley Water District and Clean Water Services in
the Portland, Oregon metropolitan area 9 .
In order to deliver the agreed-to functionality, Open partnered with Milestone Consulting who¶158
would play a major role in the implementation, and work closely with Open’s own product and
services staff. Open also licensed Milestone’s Customer Self Service portal to replace Open’s
own component, as the Milestone product was already implemented in several utilities in the
US and was judged to be configurable to meet the City’s requirements.
The MPSA contained an initial anticipated timeline (shown in figure 4.1 on the following page)¶159
that proposed a go-live for the broadband implementation in late June of 2019, and a follow-
on go-live for the utilities-related CIS at the end of September, 2019. The agreements also
providedforaperiodofpostgo-livesupportandstabilizationservicesbeingprovidedbyOpen
to the City, ending in January 2020. Additionally the agreements defined anticipated staffing
levels, roles, responsibilities and activities that the parties would individually and jointly be
responsible for executing, a process for mutually agreeing to changes to the agreements, and
a resolution and escalation procedure for addressing situations where mutual agreement
could not be reached.
9 Open_Intl_00357584, at ¶¶2–3, 46–47.
J
T
H
C
Proprietary and Confidential Page 37 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 38
of 89
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 39
of 89
Expert Report
mentation project entirely on supporting the broadband initiative, pausing all utilities related
implementation work until a future date.
The broadband business went live in August 2019, albeit with a very limited set of products¶163
compared to the initial intent (approximately 10% of internet products, 20% of telephony
products and 0% of television products had been configured by the City in the CIS at the
broadband go-live date
12 ). City staff continued to add more products to the OSF broadband
product catalog, in parallel with supporting the new broadband business and implementing
new broadband business functions and processes. In September of 2019, the City decided, in
addition to continuing to work on the broadband related activities just mentioned, to re-start
the utilities component of the CIS implementation.
The City was again unable to sustain the level of effort and resources required for all these¶164
activities, and in June 2020 the Parties signed the “First Amendment” to the agreements
which formalized a revised target utilities go-live date of October 2020, along with various
financial components associated with the revision.
Again, the demands of creating the new broadband business in parallel with implementing¶165
the CIS for the utilities business proved beyond the capacity of the City to deliver. Thus in
November 2020, again with the goal of focusing on the broadband business, the City executed
a “Pivot to Broadband” which once more paused the utilities implementation in favor of
broadband.
As part of the “Pivot to Broadband” the City was required to deliver a prioritized list of out-¶166
standing broadband issues for Open to resolve in order to pursue the pivot
13 . The City failed
to complete the list for months after the January deadline
14 , and in May 2021 Open sent the
City a Notice of Default based on this and other specific breaches of the MPSA. The MPSA
provided that the City would have 30 days to cure these breaches. The City responded by
issuing its own Notice of Dispute and Notice of Termination, and then filed the current lawsuit,
which effectively halted the project.
4.2 Summary of opinions
4.2.1 The City’s Broadband business definition was initially rushed then significantly de-
layed
The City decided to add broadband to the already-planned CIS replacement RFP less than¶167
3 months before that RFP was issued. This did not give the City team adequate time to
prepare the broadband sections of the RFP, and thus resulted in an incomplete and ill-defined
definition of the broadband business in the RFP when it was issued.
The CIS procurement process and the agreements for the CIS replacement assumed the City’s¶168
Broadband business was substantially established by the project’s commencement, and thus
12 Defs.’ Resp. to Pl.’s Interrog. No. 7;see also CFC_130952.
13 CFC_061163.
14 Deposition of Aaron McClune (Rough) at 159:17-22, 164:7-20 (testifying that, as of May 20, 2021, City had
just completed “initial grooming” of broadband issues and explaining that City had finished defining and refining
broadband issues from the City’s perspective, although not necessarily prioritizing them “into our sprints,” and
were prepared to begin discussing the remaining broadband issues with Open).
J
T
H
C
Proprietary and Confidential Page 39 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 40
of 89
Expert Report
would be able to provide information to the project with a relatively high degree of certainty.
The reality, that the Broadband business was ill-defined and evolving, especially during the
first critical 6–9 months of the project, made it impossible for the project approach defined
in the agreements to succeed. The City’s inability to complete project definitions, even deep
into the project timeline, also lead to confusion within the City’s project team and contributed
to an abnormally high number of out-of-scope observations being logged after the planned
period (SSP) for logging such observations was over.
The City’s failure to meet its obligations to timely deliver broadband business definitions and¶169
platforminformationhadacatastrophiceffectontheproject’sabilitytomeetthetargetgo-live
date and led to other project delays, by delaying the parties’ ability to construct required
artifacts and by necessitating regular shifting and reallocation of resources away from other
project activities. These issues were further compounded by the fact that the broadband
operations team did not appreciate the information and effort needed to configure their
products into the CIS system.
4.2.2 The City did not provide adequate staffing
The agreements specified staffing levels and roles that the City would provide. Based on ¶170
my experience the level of staffing, and roles to be performed, by the City and Open staff
as described in the Agreements is typical for a project of this nature. On a continual basis
throughout the project, the City failed to provide this level of staffing. The City staffing levels
were required, among other reasons, to provide an adequate level of informed and authorita-
tive knowledge concerning the City’s business, thus merely augmenting the shortfall of staff
with non-City individuals (which was done) was not sufficient to mitigate this failure. Open
repeatedly flagged this risk to the project, City officials repeatedly acknowledged it, but the
City never rectified its critical resource shortcomings.
The City’s chronic failure to adequately staff the project led to staff burnout and frustration,¶171
and it negatively affected the City’s ability to complete tasks and meet its deliverable obli-
gations. This latter impact manifested perhaps most notably in the area of system testing,
where the City failed to meet its test writing and test execution obligations for months, during
the critical time period leading up to the target utilities go-live date.
4.2.3 The City’s governance, oversight and management of the project was flawed
There was little or no experience of CIS implementation projects on the City’s procurement¶172
and execution teams. While AAC Utility Partners was engaged by the City for approximately
two years prior to the issuing of the RFP, they were no longer engaged during the vendor
selection, negotiation and execution phases.
This lack of an experienced advisor to the procurement and project teams meant that Fort¶173
Collins had no CIS-specific experience to guide them through what is a complex and chal-
lenging procurement and project process. Of course, it is not possible to say with certainty
that an advisor would have enabled the project’s issues to be avoided, but it is my experience
that such an advisor immeasurably reduces risks, and helps the utility and vendor(s) achieve
mutually satisfactory outcomes from CIS projects. The City’s own executives, looking back,
J
T
H
C
Proprietary and Confidential Page 40 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 41
of 89
Expert Report
admitted this was a “key miss.”15
This is particularly relevant given the opinions stated above regarding each of the failures of¶174
the City to meet its obligations under the agreements. An experienced advisor would have
been able to steer the City along a different course and mitigate the outcome of this project. A
more effective governance team likely would have recognized and addressed this shortfall,
and required the project team to retain such an advisor.
Effective, dedicated project management and governance would have mitigated the effects of¶175
some of the project issues; unfortunately this was missing from the Fort Collins’ team for the
vast majority of the procurement and project execution.
Projectsofthiscomplexityandcriticalityrequireexperienced,continualanddedicatedproject¶176
management to succeed. The City (in the Agreements) agreed to provide a full-time manager.
ThisfailurewasaprimarycontributingfactortotheCity’sinabilitytosuccessfullymitigatethe
project risks, both in general and specifically those associated with their overall inadequate
staffingandrapidlychangingBroadbandbusiness. Additionally,theCity’srotatinganddiffuse
executive team was largely inactive through large portions of the project and failed to address
shortcomings in project management, staffing, and broadband planning and coordination.
Without proper management and governance, the City simply could not right the ship after
starting the project with inadequate planning and resources.
4.2.4 OpenandtheOSFproductperformedsubstantiallyinaccordancewiththeagreements
Theagreementsdefinedthescopeofthesolutiontobeimplemented, theapproachtobetaken¶177
to deliver that implementation, and the roles and responsibilities of the parties in executing
that approach. The role of both Open and the City were critical to that success; it would not be
possible for the project to succeed without both parties delivering the obligations they agreed
to.
Open’s product substantially met the agreed-to scope. The City knew and accepted from¶178
the RFP process that it would be Open’s first customer in the United States, and the City had
multiple, relatively lengthy, opportunities during the procurement process to see the system
in action, and form its own opinions as to the system’s suitability. There were other products
available to compare Open’s product to. More than two years into the project, multiple City
executives expressed internally and in public that the product was the right fit for the City
16 .
Open worked diligently and flexibly to deliver the solution in line with the agreed–to approach.¶179
Open also made significant financial concessions (e.g. in the fees associated with the First
Amendment to the MPSA) to help the City remain on-track. It was the City’s failure to deliver
on its part of the agreements that was the major contributor to the project’s failure. This
failure to deliver commenced on or about the project’s inception, and continued until its
eventual halting.
15 CFC_085150.
16 See TMG_000001 (October 2020 industry presentation by City Broadband Director Colman Keane stat-
ing, at 21:45, that “OSF is the right tool for us”); June 2, 2020 Mike Beckstead presentation to City Council,
available at https://reflect-vod-fcgov.cablecast.tv/CablecastPublicSite/show/1204?channel=
1&seekto=15829, (stating at approximately 4:35:00 that the partnership with Open was giving the City a system
that met its needs).
J
T
H
C
Proprietary and Confidential Page 41 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 42
of 89
Expert Report
5 Opinions
This section compares key elements of the project’s execution with the implementation¶180
overview contained in section 3, with particular attention to the aspects of the execution
that led me to the opinions summarized in section 4.2 on page 39. The sources for informa-
tion regarding the project events consist primarily of the parties’ shared project drive with
approximately 16,000 project documents, other contemporaneous project documents like
updates and reports, email communications from the project period, and sworn testimony
and statements from project participants. These are listed in appendix B on page 80. I have
assumed the information provided to me regarding events and dates is accurate, and the
conclusions and inferences made from that information are drawn based on my experiences
with CIS implementation projects.
TakentogethertheCity’sdeviationsandomissionsfromleadingCISimplementationpractices¶181
created multiple project issues which, as a whole, caused the failure of the project.
5.1 The City’s rushed addition of Broadband to the RFP, failure to timely
and clearly define Broadband services, and failure to timely supply
Open with Broadband business definitions materially impacted the
project’s success
5.1.1 The late addition of Broadband to the CIS replacement RFP resulted in insufficiently
detailed and incomplete broadband requirements
In November 2017 the City decided to establish a new Broadband service. This was a “green-¶182
field” operation, meaning there was no existing Broadband service provided by the City to
its citizens. The broadband initiative would require “CIS-like” services to allow it to bill its
customers and to otherwise support its business, although with a broadband focus rather
than a utility orientation. Although the City had been contemplating a broadband initiative
since 2015, it was not until the citizens of Fort Collins approved the provision of broadband
services, as part of the November 2017 ballot, that the City formally initiated the project17 .
Separately from the broadband initiative, since at least 2015 the City had been working on a¶183
strategy to address its existing aging traditional utilities CIS system (Banner). As part of these
activities the City had retained an expert consultant, AAC Utility Partners, to help evaluate
the existing CIS and develop a plan to upgrade or replace that system
18 . By July 2017 the City
had arrived at a decision to seek bids on a new (utilities only) CIS system and had completed a
draft RFP package for seeking a product vendor and related implementation services, which
the City was not planning to issue until the first quarter of 2019
19 . As of late September 2017
the City was intending to retain AAC throughout the vendor selection and contract negotiation
17 https://fcconnexion.com/about/.
18AAC 1-000385.
19 CFC_139964 at -964.
J
T
H
C
Proprietary and Confidential Page 42 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 43
of 89
Expert Report
processes for a utilities only CIS replacement20 .
This is a usual practice and timeline within the utility industry in my experience.¶184
However, the City’s approach changed in late 2017 as a result of the broadband initiative. In¶185
early November 2017, AAC issued a Change Control (#4) for its agreement with the City, to
include services to support exploring options for broadband billing
21 . The City Council had
not yet approved moving forward with the new broadband program at this time22 .
By the time of the approval of the broadband initiative in November 2017, the utility re-¶186
placement project was well advanced in the preparation of the RFP for a new utilities-only
CIS23 .
By late-November 2017 the City decided to combine the two CIS needs, Utility and Broadband,¶187
and procure a single system to support the needs of both its Broadband and Utility businesses.
Thisdecisionresultedinthelast-minuteadditionofbroadbandtothepreexistingRFPpackage
for a utilities-only CIS implementation
24 . The City tasked AAC with putting together draft
updated RFP documents and a baseline functional matrix for broadband, which AAC sent to
the City in late November
25 . These draft documents indicated that fundamental decisions
aboutthenatureoftheCity’snascentbroadbandbusinesswereyettobemade26 . Theaddition
of broadband to the CIS RFP also resulted in the City needing to issue the RFP much sooner
than its prior planned time frame of early 2019, because broadband needed to go live on an
aggressive timeline in order to meet bond requirements27 .
As a result, the City had only several weeks to incorporate broadband requirements into the¶188
RFP functional matrix and to review and cull the 3,500 requirements that AAC had developed
for the utilities RFP
28 . The City’s utilities staff had to largely cover Broadband’s needs with
existing utilities requirements, without documentation or understanding of Broadband’s
needs29 .
In addition, the City did the work of incorporating broadband into the CIS RFP largely without¶189
AAC’s (or another expert consultancy’s) involvement and advice. As noted, AAC provided
initial template documentation to the City in late November 2017, but it appears the City
effectively cut AAC out of the picture shortly thereafter, sometime in December 2017 or early
January201830 . IhavenotseendocumentsshowinganysubstantiveinvolvementbyAACafter
November 2017. This decision to cease working with AAC appears to have been a relatively
sudden decision; as late as November 23, 2017, a draft of the proposed RFP stated that “AAC
20AAC 1-000355 at -357 to -364.
21AAC 1-000169–174.
22AAC 1-000169 at 172.
23 CFC_139960at-964(notinginJuly2017thatthedraftoftheRFPforautilitiesCISreplacementwascomplete).
24 CFC_217210 at -211 (City project owner recounting to outside consultant that three-week window to develop
tacked-on broadband requirements for RFP led to “last-minute effort” “by the seat of their pants” and that “not
enough time” was “the achillees heal [sic].”).
25 CFC_141239 and attachments; CFC_087995-996.
26 E.g., CFC_141296 at -334 (noting that the City had not yet decided on its broadband service model).
27 Rule 30(b)(6) Deposition of City (Ward Rough) at 79:6-80:1.
28 CFC_046746–747; CFC_055875 at -876.
29 CFC_217210 at -211; CFC_055875 at -876.
30 CFC_090628-629; CFC_093005.
J
T
H
C
Proprietary and Confidential Page 43 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 44
of 89
Expert Report
Utility Partners is providing solution selection management consulting”31 .
As well as acting without the benefit of an expert advisor, the City completed the inclusion of¶190
the broadband requirements into the CIS RFP in a very short period (several weeks), with the
inevitable result that the broadband requirements were not as well thought out and complete
as the utility requirements, which had been developed over a much longer period and with
expert advice. This haste may also have contributed to the decision to undertake significant
de-scoping activities, which took place on at lease three occasions, at the RFP stage, and later
(separately) under Dr. Frey and Mr. McClune’s tenures as project manager32 .
I examined the Excel spreadsheet version of the Functional Matrix response submitted by¶191
Open33 . This showed approximately 1,920 requirements (eliminating some rows since they
contained organizational grouping headers rather than specific requirements). Of those,
a very small number (40 or 2%) were allocated to the “Broadband” application; the vast
majority (1,875 or 98%) were either non-functional requirements or associated with the CIS
application. There were a few (< 20) requirements where broadband-related items had been
added to, e.g. lists of services to be billed, but nothing that could be considered an in-depth,
thorough set of broadband-specific requirements34 .
Even allowing for some degree of miscategorization, and for some utilities’ requirements¶192
being relevant to the broadband operations (e.g. some requirements related to customer
management would be common to broadband and utilities businesses), this shows a remark-
able discrepancy in level of specificity between the requirements relating to the broadband
business and those relating to the utilities businesses’. This analysis supports myopinion that
the broadband-related component of the CIS RFP was developed hastily and with nowhere
near the depth of rigor and completeness of the utilities-related component.
This opinion is also supported by information from a key member of the City’s Broadband¶193
business team, Brad Ward:
WedidhaveBB[broadband]requirementsatcontract. BBgavehighlevelrequirements,
and the project would develop details to do configuration. I have not been happy with
the RFP requirements from the RFP they were broad, but they were there35 .
These activities indicate that the broadband requirements and capabilities were being added¶194
to the CIS RFP in a rushed and somewhat haphazard manner, driven by the broadband busi-
ness’s time frames
36 . In contrast, the utility RFP had been developed using a structured
approach of determining business needs and using those business needs to drive detailed
requirements, drawing on AAC’s comprehensive catalog of utility industry requirements. It
appears that to get the RFP reconfigured and issued in less than three months the broadband-
31 CFC_141296 at -303.
32 CFC_120440 at -805–809 (listing roughly 50 descoped requirements); Open_Intl_00034443–495 (recom-
mending descoping of additional 309 requirements); CFC_191533–534 (discussing descoping of additional 30%
of requirements).
33 Open_Intl_00083685;see also CFC_000237 (functional matrix as issued with the City’s RFP).
34 See also CFC_217099 (City’s functional lead–i.e., its keeper of the functional matrix–stating that only about
100 broadband functional requirements were added to the 2,000 existing utilities requirements).
35 CFC_202864.
36 See also CFC_087614 (one of the City’s project leaders stating, in December 2020, that the project was “over-
simplified and rushed from the very beginning” “due to broadband needing a billing system”).
J
T
H
C
Proprietary and Confidential Page 44 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 45
of 89
Expert Report
related components of the RFP were largely high-level compared to the very detailed require-
ments developed to support the utility business. Note also that this work was taking place in
November and December, which is both a holiday season, and often a period of extra work
related to year-end.
In summary, it appears that, in November / December 2017,¶195
1.The City decided to combine the Broadband and Utilities CIS RFPs only a few months
before the RFP was issued.
2.The City chose to dispense with the services of its external CIS advisory experts who had
been advising the City for 2+ years, and
3.These decisions resulted in a high-level, incomplete and poorly understood set of re-
quirements being added to the RFP.
5.1.2 The lack of detailed and timely-delivered broadband definitions and platforms sub-
stantially impacted project execution
The CIS implementation was one of many critical components of establishing the Broadband¶196
business, providing the ability to interact with, enroll, and bill, customers for the services they
received. Thus, aligning the availability of the CIS to the roll-out of the Broadband service
was important to enable the City to deliver on its Broadband-related commitments.
Quoting from the City’s RFP:¶197
Our preference is that a Comprehensive Solution has a go-live date of July 201937 .
The project plan described in the Agreements and refined and developed during the planning¶198
phase is depicted graphically in figure 4.1 on page 38. It shows a projected path which would
meet the City’s requested July 2019 Broadband go-live. The plan proposed to conduct many
activities in an overlapping, parallel manner. This is normal, and the degree of overlap is not
unusual. Overlapping activities in this manner does, however, eliminate much of the schedule
contingency that could be used to accommodate delays in the work. Thus the plan required
work to be conducted with minimal deviation from the Agreements, and with the agreed-to
prerequisites such as the Broadband definitions being available, in order that the schedule
be maintained.
The City was required to have procured the broadband, video and voice platforms required to¶199
provide actual service to customers. For purposes of the CIS implementation, these platforms
needed to be documented and defined, and the associated business processes and technical
integrations provided to the CIS implementation team, in order that the CIS implementation
could configure the specific details of those processes and platforms into the CIS workflows.
The documentation and specifications of the City’s broadband network electronics, video,
and phone platforms had to be delivered prior to the project starting.
The City failed to procure these required external platform contracts and specifications in a ¶200
timely manner
38 . This had an inevitable effect on the project’s ability to complete process
37 CFC_000150 at -165.
38 Defs.’ Resp. to Pl.’s Interrog. No. 5.; CFC_015310 at -315-317; Open_Intl_00170873 Tab ’Facts’.
J
T
H
C
Proprietary and Confidential Page 45 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 46
of 89
Expert Report
definitions, and impacted the ability of the project to have “all the definitions” available to it.
Specifically, the project required the Broadband definitions to be available so that the “Scope¶201
Solution Presentation” (SSP) activities could be conducted in time to support the remaining
activities needed to achieve the July 2019 target go-live date. This requirement was clearly
laid out in Section 13 of SOW #1:
One month after the project starts, all the definitions corresponding to the Broadband
platforms must be closed and delivered to the Open project team39 .
Based on a mid-August 2018 project start, the parties’ agreements required that “all the¶202
definitions” for Broadband be finished and delivered to Open by mid- to late-September. This
did not happen, as noted above. Indeed, the Broadband business requirements were still not
fully defined by January 2020, some 16 months after the project commenced:
There are still undefined business requirements for Broadband services within the
OSF environment, as well as finalizing Utilities business operations based on the OSF
system. In the case of Utilities, most of the unresolved business requirements are due
to a lack of time to articulate them fully with the OASIS project team and represent
a risk to the revised schedule as we endeavor to bring team members together to
fill these gaps. However, for Broadband operations, this blocks the CFC’s ability to
effectively deliver current and new services to the community for both residential and
commercial customers. This has been a challenge throughout the project but is even
more critical as we are now supporting Broadband operations in production, while
managing two other system environments; OASIS project and legacy system Banner.
These challenges consume project (OASIS) resources to address and delay efforts on
the revised Utilities Go-Live implementation40 .
Additionally, interfaces to the City’s Geographic Information System (GIS) and Fiber Manager¶203
were required to be defined and available, since these systems also provided key components
of the broadband service.
Given the last-minute addition of broadband to the City’s CIS project and the City’s related¶204
rush to issue the RFP, it is not at all surprising that this pattern of incomplete and delayed
business requirements occurred.
Analysis of Observations recorded in SAO
Figure 4.1 onpage 38 showsSSPasrunningfromSeptember4,2018throughFebruary1,2019.¶205
This 22 week activity is of an appropriate duration for the scope of work to be carried out if
the requisite preparatory work had been completed. From a purely duration perspective this
phase largely ran according to plan. The City signed off that the solution scope review was
completed in late December 2018, which satisfied the milestone
41 . Open and the City then
worked through the bulk of the observations identified during the review by February 2019
42 .
39 CFC_120440 at -523.
40 CFC_220079.
41 Open_Intl_00118399.
42 Open_Intl_00124806 at -807.
J
T
H
C
Proprietary and Confidential Page 46 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 47
of 89
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 48
of 89
Expert Report
needed to be included in the solution, and which were not included in the scope identified
during the procurement process. This is a typical event in a CIS implementation project.
Following that initial peak of 9 observations, the count of new in-scope observations falls off,
with no subsequent month in 2018 or 2019 having more than two such observations, and
several months (7) having no in-scope observations created.
The out-of-scope observations tell a very different story. The monthly volume of out-of-scope¶211
observations are depicted as green circles and lines () in figure 5.1 on the preceding page.
DuringtheSSPphase68out-of-scopeobservationswererecordedinSAO.Thisisveryunusual
and suggests strongly that there was poor alignment between the scope contracted for, and
the actual scope the City needed. This is not unexpected, given the rushed addition of the
undeveloped broadband business requirements to the scope of the RFP.
In the period March through December 2019, a further 193 out-of-scope observations were¶212
recorded. This is exceedingly unusual in two respects. Firstly, the sheer volume of observa-
tions compared with the in-scope observations during the same period indicates that the
gap between contracted-for and desired scope was substantial. Secondly the timing of the
identification with respect to the completion of the SSP phase (approximately 75% of the
out-of-scope observations were recorded after the conclusion of SSP) shows that the process
of fully and correctly identifying the solutions scope simply did not work as intended by
the parties and described in the SOW. The City failed to raise functional scope issues dur-
ing the dedicated solution-scoping phase, but it then raised scores of mostly out-of-scope
observations thereafter, distracting from other project tasks.
If we switch points of view from the behavior described in the Statement of Work, to the actual¶213
behavior aligned to project events, we gain additional insights. The Broadband business was
new, being created and staffed, and was subject to change. The broadband-specific require-
ments in the RFP were brief, relatively high level, and added before the broadband business
was fully established. Thus in the actual situation of the City, with a evolving Broadband
business, it is not surprising that new requirements would be continually discovered, or
existing requirements modified, in direct conflict with the stable scope definition agreed to
by the parties in the Agreements.
Impact of failure to define solution fully in SSP
Theimpactofthedelayanduncertaintyindeterminingthesolutionscopedescribedabovewas¶214
substantial. Referring to figure 3.4 on page 33, the construction and test preparation activities
of the project rely totally on the correct definition of scope to identify what artifacts are to
be built, which detailed interface transactions and report components are to be developed,
and what data is to be converted to the new system. Changes to these definitions require
re-evaluation of the designs, re-assessment of the impact of the changes, and potential re-
work to already completed, or in-flight, artifacts. All of this requires not only additional effort
to perform, but also impedes the performance of work in progress, and the testing of work
completed, as the impact of the changes are factored into the project’s work.
Additionally, effort from the functional (business-oriented) staff is required from both the¶215
City and Open to fully validate and develop solutions for new and changed requirements. The
piecemeal addition of new or changed requirements also potentially impacts the stability
and overall quality of the delivered solution. Completing the SSP results in a portfolio of
J
T
H
C
Proprietary and Confidential Page 48 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 49
of 89
Expert Report
requirements that can be considered as a whole, to look for overlaps, synergies and conflicts
which would enable Open to develop a holistically architected solution. Adding significant
amounts of new or changed scope, as happened in this project, inevitably means that the
holistic solution developed as a result of SSP is placed at risk as new or changed requirements
have to be accommodated.
Finally, the high degree of parallelism in the Setup phase, and its overlap with Scoping,¶216
noted in paragraph ¶198 on page 45, provided little or no buffer to accommodate this scope
identification delay.
In summary, the substantial new scope identified in the period following the completion of¶217
SSP had the following impacts on the project, none of which were part of the agreed-to project
scope:
1.Development of the new scope required functional effort from both City and Open staff.
2.Evaluating the impact on assumed-to-be-fixed scope required technical and functional
effort mainly from Open staff.
3. Developing new artifacts required mainly technical work from Open staff.
4.Reworking any in-flight or completed artifacts required mainly technical work from
Open staff.
5.Developing new or modifying existing test cases required effort from both City and Open
staff.
6.Planning, managing and coordinating this work required effort and skill from both City
and Open staff.
7.Funding for this work required alignment and potential funding from both Open and
the City.
The City responded to the delay in identifying Broadband scope and processes by prioritiz-¶218
ing effort away from the Utility implementation and towards Broadband. While helpful for
Broadband, this focus did not sufficiently address the root issue, that the Broadband business
definition was in a state of rapid evolution, and thus fitting a CIS solution to it using the ap-
proach agreed-to in the Agreements was difficult, if not impossible. The failure to adequately
assess and react to this lack of definition is a significant example showing the project was
inadequately managed and led by the City.
Even after Broadband go-live, the City’s broadband operations team appears not to have¶219
understood what was required for OSF to configure broadband products
44 . This led to the
broadband team delivering expected launch deadlines for new products at the last minute
and without product definition details, with no appreciation of the work needed to launch
those products or of how unrealistic the deadlines were
45 . This type of uncoordinated and
misinformed project behavior is highly problematic and can be expected to cause inefficiency
and delay as project resources are diverted to new and unanticipated tasks, which the City’s
44 Deposition of Dr. Michelle Frey (Rough) at 110:11-16.
45 Deposition of Dr. Michelle Frey (Rough) at 184:12-185:6; CFC_015224–225; CFC_211947.
J
T
H
C
Proprietary and Confidential Page 49 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 50
of 89
Expert Report
project manager later acknowledged had been a problem for the City throughout the project
46 .
5.1.3 Summary
In my opinion:¶220
1.The decision to merge the utilities and broadband CIS procurement and implementa-
tions into one project, only 3 months before issuing the associated RFP, resulted in a
lack of specificity and detail for the broadband requirements.
2.The City’s failure to provide basic platform information and “all the definitions corre-
sponding to the Broadband platforms” in the time-frame it agreed to had a catastrophic
effect on the project team’s ability to meet the target go-live date for Broadband. This
effect included both delaying the ability to construct the required artifacts to provide the
solution, and also diverting project team effort from tasks such as test case preparation
to the continued solution definition activities.
3.The level of definition of the broadband business was vague, superficial, incomplete and
subject to frequent change. This caused confusion within the project team as to what,
specifically, the detailed requirements were, and contributed to the abnormally high
volume of out-of-scope SAO Observations noted earlier.
4.TheCity’sinability(asstatedbytheCity’sprojectmanagerandquotedinparagraph¶202
on page 46) to complete the Broadband process definitions even 16 months after the
project’s start reflects an inability and unpreparedness to undertake the project in the
manner that the City contemplated in its RFP and agreed to in the Agreements with
Open.
5.The Broadband team lacked even a basic understanding of what was required of them
in order to implement an integrated CIS system serving all of the City’s utility services–
even while Broadband insisted on overly aggressive configuration schedules for new
products rolled out after go-live–which compounded the problems arising from the lack
ofplatforminformationandbusinessdefinitionsandcreatedpersistentfrictionbetween
broadband and utilities needs, after Broadband went live and the project attempted to
focus on utilities.
6.Open’s project approach, as described in the Agreements, relied on the City delivering
on its obligations, which the City failed to do.
5.2 The city failed to adequately staff the project
During contract negotiation, the parties agreed on a staffing matrix that set forth the required¶221
levels of staffing for both Open and the City
47 . In summary, the parties agreed that the City
provide 64,689 hours of full-time equivalent (FTE) City staff to be assigned to the project,
with an FTE staffing headcount of 22.45 for the majority of the project’s duration (with some
ramp-down in the last 3 months).
In addition, Section 13 of SOW #1 provided that both parties had specific obligations with¶222
46 CFC_213730 at -733.
47 CFC_083424–083425.
J
T
H
C
Proprietary and Confidential Page 50 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 51
of 89
Expert Report
respect to staffing. These obligations included48 :
1.The City and Open must have assigned the necessary human and technical re-
sources, according to the staffing matrix, which was created by each party, for the
execution of project activities in each of its phases.
2.Any change in the number of resources or its dedication must be documented
and managed with the change control management.
Since the City did not maintain regular data or records of its staffing resources dedicated to¶223
the project, I have not been able to measure precise City staffing levels or hours over time
for the full duration of the project. Most of the utilities in my experience maintain detailed
records of actual time spent by each staff member on a project, in the form of “time cards”
or other similar logs. It is unclear whether the City ordinarily tracks its employees’ time on
projects. One of the City’s executive leaders for the project testified that City personnel usually
record their time but just did not do so for this project
49 . On the other hand, the City has
stated that it does not track the time its employees spend on projects as a matter of course
50 .
The City also apparently did not maintain regular documentation of staffing assignments¶224
or levels for the project, and instead only compiled staffing allocation estimates at irregular
intervalsduringthecourseoftheproject51 . Itisunclearwhethertheestimatedfiguresineven
these documents are accurate, as in at least some of the documents the City did not clearly,
or perhaps accurately, distinguish between employees’ efforts on the CIS implementation
project versus their efforts on utility or broadband business operations52 .
The City’s lack of detailed staffing plans and records is unusual, and in my opinion is another¶225
exampleoftheCity’slackofcompetentprojectmanagement. Establishmentandmaintenance
of a detailed staffing plan with named resources and allocation is a critical practice in large
complex projects such as a CIS implementation. Sharing this allocation model across both
vendors and utility management teams is normal. Maintaining this plan, typically on a rolling
3-month basis, forms a crucial component of staffing related risk identification (and thus
serves as the first step in avoiding and mitigating risks). This is particularly true for projects,
such as this one, where the parties acknowledge that staffing is a clear issue.
Nonetheless, even in the absence of comprehensive actual and projected staffing plans and¶226
time records, there are numerous data points which make it abundantly clear that the City
continuallyfailedtomeetitsstaffingobligations. TherecordsIhavereviewedcontainmultiple
instancesoftheCity’sownrecognitionofitsstaffingfailuresthroughouttheproject. Ihighlight
a sampling of these data points below.
At the outset of the project, Open and the City created a Risk Management Plan and accompa-¶227
nying matrix that identified and assessed various project risks
53 . This risk matrix identified:
Client’s project team is over-allocated and understaffed. Currently must “absorb”
48 CFC_120440 at -522.
49 Deposition of Theresa Connor at 98:15-99:5.
50 Pl.’s Resp. to Defs.’ Interrog. No. 5.
51 Pl.’s Resp. to Defs.’ Interrog. No. 5.
52 E.g., CFC_011487–011488.
53 Open_Intl_00032040; Open_Intl_00107722–00107723.
J
T
H
C
Proprietary and Confidential Page 51 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 52
of 89
Expert Report
current responsibilities and project responsibilities.
as a risk with a high (70%) probability of occurring. The parties further defined this risk,
stating
Client’s project team lacks sufficient personnel for the execution of activities.
The City’s project staff also was raising the same concerns about the City’s team being under-¶228
resourced and over-allocated from the beginning of the project54 .
Parenthetically, as noted in paragraph ¶142, there are many tools available other than Excel¶229
to support risk management, staffing management, and many other aspects of project ad-
ministration. Open, in its RFP response, offered the City the use of its proven toolkit to help
with this, but the City declined to purchase the required software licenses to the Microsoft
components on which the tool was based.
In late 2019 and early 2020 the City’s lack of adequate resources remained a problem. In¶230
December2019,Dr. MichelleFrey,theCity’sprojectmanager,concludedthattheCity’slimited
resources–and those resources’ parallel operational job responsibilities
has created a conflict in the City’s ability to manage their contractual obligations in
support of the OASIS project55 .
Further, in January 2020, Dr. Frey informed the City’s leadership that
There have been and still are no fully dedicated IT resources to this project and only
a handful of fully dedicated business/functional team members. The City has under-
taken a project of this scope without providing the resources needed to deliver timely
intelligence, project activity and deliverables, and an overall Broadband deployment
project execution plan. The [City] had clear responsibilities assigned to it in the Master
Professional Services Agreement (MPSA) and is vulnerable to being held in violation of
the contract …56
In May 2020, the City continued to recognize its staffing issues. The City sought to “reboot”¶231
the project and, in connection with that effort, City executives Mike Beckstead and Kevin
Wilkins created a slide deck
57 . Page 3 of that deck contains a project organization chart, with
supporting detail on page 12. This shows some 16-17 full-time-equivalent staff assigned to
the project, with only 8 roles being staffed on a 100% basis–three of which were filled by Open.
The remaining roles are allocated across approximately 20 City staff on a part-time basis.
While some part-time staffing is inevitable, and appropriate, this level of full time, dedicated
staff to a program of this complexity is not appropriate.
The City called out the staffing topic in this presentation, on page 5, citing the “lack of fo-¶232
cused and dedicated resources,” and separately recognized around the same time that City
resources’ competing job priorities were impairing project progress and that more fully
54 Deposition of Theresa Connor at 250:16–252:14.
55 CFC_215431.
56 CFC_220079.
57 CFC_080989–081002.
J
T
H
C
Proprietary and Confidential Page 52 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 53
of 89
Expert Report
dedicated positions should be created58 .
Note that this level of FTEs is short of the 22.45 agreed to in the parties’ initial staffing matrix.¶233
The City also did not meet the agreed-upon staffing requirements in other ways, too, such as
by failing to assign a fully dedicated project manager or adequate IT resources in 202059 .
Also in May 2020 another City team member, Mona Walder, wrote that:¶234
…our pool of resources continues to cover new broadband product implementation,
operations/production support and the utilities project. I’m not sure how many more
resources we need now to cover everything …60
This memo also has attached a staffing plan, which was last updated on November 15, 2019,
some 6 months prior to the memo’s date
61 . Based on other documents
62 , it is unclear whether
the resources’ allocation to “Ops” in this staffing plan was intended to be project work. But
even assuming that it was, the City proposes to assign a maximum of 19.26 FTEs in October
and November 2019, and a minimum of 15.51 FTEs in September 2019. This maximum
plannedstaffinglevelislowerthanthe22.45FTEstafftheCityagreedtoprovideintheoriginal
staffing matrix, and, as just discussed, the City appears not to have met this planned target,
either63 .
In late 2020 the same issues persisted, with the City’s latest project leaders recognizing that¶235
the City’s project leadership had not been paying sufficient attention to resourcing, that the
City’s resource needs were not being dealt with, and that many of the City’s team members
were working part-time on the project while also fulfilling their regular daily operational
jobs64 .
In addition, the fact that staffing had been an issue for the City since the project started¶236
continued to be discussed among the City’s project leadership. On November 15, 2020,
Theresa Connor, then the Acting Utilities Director stated:
Colman and Mike Beckstead made the decisions to under-resource this project that
has absolutely been a factor in this project not achieving success. This was a very, very,
VERY short-sighted decision that Colman pushed on the behalf of Broadband. The
consequences have been tremendous and will continue to be felt for a while. There is
a huge cost to that decision65 .
The opinion in this statement implies the City deliberately did not deliver on its obligations¶237
under the Agreement, and that, in Ms. Connor’s opinion, this contributed to the project not
achieving success66 .
58 CFC_184603–184604.
59 Deposition of Mike Beckstead at 121:4-15, 204:7–205:21, 286:19–287:4, 288:15–289:1; CFC_205465; Depo-
sition of Coy Althoff at 53:9-22.
60 CFC_083424.
61 CFC_083426; CFC_083424.
62 CFC_011487–011488.
63 CFC_080989–081002.
64 CFC_205651; Deposition of Theresa Connor at 178:25-180:18, 181:20-182:17.
65 CFC_080119.
66 See also CFC_217359 (notes from 2021 interview of City PM stating that “AAC gave resource plan” and “Open
J
T
H
C
Proprietary and Confidential Page 53 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 54
of 89
Expert Report
Mr. Althoff responded, stating in part that:
…agreewithyourassessmentsonthepoorresourcingfromthebeginning…Iremember
that originally they tried to implement Connexion and Utilities in parallel; then they all
came to the conclusion that they had bit more off than they could chew …67
These comments provide direct support that not only was the project inadequately resourced,¶238
butalsothattherewasineffectivegovernanceandleadershipfromthebeginning. Aneffective,
experienced leadership likely would have been able to identify the over-ambitious scope, and
inadequate staffing, and enforce a more appropriate course of action.
Project change requests
The change requests from the project further demonstrate the City’s staffing shortfall and¶239
changes in scope over the course of the project. The City did issue several change requests
to procure more staffing assistance from Open. In the project’s change request log 22 of the
30 executed change requests are categorized as “Services” related, and represent a total of
9,846 hours of effort, a considerable amount of additional effort68 .
Note that the shortages mentioned in prior sub-sections of this section are shortages even¶240
allowing for the 9,846 hours provided here. Thus the true shortfall in City staffing is sig-
nificantly greater than 9,846 hours, though due to the absence of detailed records it is not
possible to estimate the actual shortfall. Indeed, the City’s May 2020 FTE count, discussed
above, already included the supplemental Open resources it had hired as of that time69 .
It is also important to recognize that, while augmenting City staff with Open people helps the¶241
shortfall, it is not a universal panacea. While Open staff are knowledgeable in matters related
to technology, the Open product, and the industry in general, they do not possess, and cannot
be a substitute for, City employees who have in-depth knowledge of City practices acquired
over (often) many years of work for the City, and knowledge of the City’s business objectives
for its services in the future. These supplemental resources typically also do not possess
decision making authority, and so will be dependent on City staff for approval of decisions
and direction of their work.
It is evident that Open raised concerns about City staffing levels on multiple occasions, in¶242
multiple forums. For example, Open’s Response to the City’s Interrogatory No. 5 identifies
documentation of at least seventeen separate instances in which Open raised concerns over
the City’s inadequate staffing, from late August 2018 to May 2021
70 . I have not located any
formal response from the City to any of these concerns, or evidence of a concrete, communi-
cated, actionable plan to remediate them. Informally, the City’s response seems to have been
(paraphrasing) “don’t worry, we’ll do what we need to do to make this work”. Note that this is
very much in contradiction with the SOW assumptions quoted in paragraph ¶222 on page 50
and agreed to by the City and Open.
had a resource plan” but that “executive management at the time said you do not need all of these resources”).
67 CFC_099930.
68 Open_Intl_00006656, PCR Nos. 1–30.
69 CFC_081001.
70 Defs.’ Resp. to Pl.’s Interrog. No. 5.
J
T
H
C
Proprietary and Confidential Page 54 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 55
of 89
Expert Report
5.2.1 The City failed to meet its testing obligations due in large part to its lack of adequate
resources
TheCity’sfailuretoadequatelystaffitsprojectteamhadvariousnegativeeffectsontheproject¶243
and directly contributed to the City’s failure to meet its deliverable deadlines and provide
quality deliverables. One of the clearest examples of this is in the area of system testing.
In January 2020, the parties had agreed that system testing for the Utilities functionality¶244
would commence on May 4, 2020. The project produced a document which contained a
detailed list of 60 prerequisites that needed to be completed for testing to commence, and
assigned responsibility for their completion to, in some cases, specific named individuals
and in other cases more generally, to the responsible party (the City or Open)71 .
In order to prepare for testing, according to the Agreements, the City was responsible for,¶245
and should have completed, among other tasks, the following items (taken from the list of 60
mentioned above):
1. Definition of all the Business Cases (BCs) to be tested72 .
2. Definition of all the specific Test Cases (TCs) to be tested.
3. A traceability matrix mapping TCs and BCs to the functional matrix of requirements.
4. The required integrations to systems outside of the CIS needed to support this testing.
AccordingtotheSystemTestPlancitedabove,thiswouldgivetheCity3–4monthstocomplete¶246
the required preparatory activities. In my opinion this is a reasonable elapsed time period
were adequate staffing and direction to be provided by the City.
The City was unable to meet its testing obligations as outlined above.¶247
As of May 6, 2020 (i.e. 2 days after the May 4 planned start of testing) only 30% of the BCs,¶248
and 20% of the TCs, had been completed, and in addition 10 of the 13 needed integrations
were also incomplete73 .
The parties negotiated PCR#24, which postponed the start of testing from May 4, 2020 to June¶249
15, 2020 (a 6 week delay) to provide more time for the City to complete its obligations outlined
above74 . The parties agreed to keep the planned October 5, 2020 go-live date unaltered. Thus
the net effect of this was that the testing period was shortened by 6 weeks, driven by lack of
City staff to adequately prepare for testing as described in the Agreements. PCR#24 explicitly
called this out, recommending that the City “should augment their team”.
In my opinion, the risks associated with a 6-week shortening of the time-period from testing¶250
commencing to go-live were not given enough weight by the City’s leadership. By the time
PCR 24 was executed, in June 2020, the project had been active for 21 months. During
71 Open_Intl_00327832.
72A Business Case is the term Open uses to describe what is often known as a Business Process in other
methodologies.
73 Open_Intl_00082561.
74 Open_Intl_00082929–931.
J
T
H
C
Proprietary and Confidential Page 55 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 56
of 89
Expert Report
that period there had been numerous instances of schedule slippage associated with City
resource shortfalls. It would have been appropriate for project leadership and governance
to recognize this and to move the projected go-live date rather than hoping that, contrary
to prior experience, the project could recover from what was expected (optimistically, as it
transpired) to be a 6-week delay in starting testing.
By June 1, 2020 (2 weeks before the testing was due to start in accordance with PCR 24) test¶251
case development was only 73% complete
75 . The test case completion rate was only 1% per
day, thus meaning some 25+ working days of effort would be required to complete the test
case preparation, with only 10 working days available before the scheduled start of testing on
June 1576 .
The same email chain also indicates that the City decided to start testing without having fully¶252
completed test case preparation, and complete the definition of test cases in parallel with
initial testing activities.
In general, while having 100% of the test cases completed is preferred, starting testing while¶253
some test cases remain to be written is, of itself, not unusual. However, in this situation, the
same City staff members would be expected to perform both tasks (test execution and test
case preparation) without any allowance being made for their increased workload. This is,
in my opinion, an unreasonable expectation of the staff. Indeed, it appears the City staff
would be expected to work on not only test case preparation and execution, but also support-
ing Broadband operations, supporting Utilities operations in the legacy system, continuing
Broadband configuration, and providing first-level support for the system testing process77 .
This, in my opinion, goes beyond realistic expectations for the City’s team and should have¶254
been recognized as such by City project leadership and governance.
During this period the City accepted that the October 2020 go-live was unattainable, and¶255
proposed a January 2021 go-live date
78 . Shortly after that decision, the City (under new
project direction) decided to pause the Utilities project entirely, and “Pivot to broadband”.
5.2.2 Summary
In my opinion:¶256
1.The City’s chronic failure to adequately staff the project in line with the Agreements,
despite Open’s continued requests and in contradiction to leading CIS implementation
practices and contract requirements set forth in the staffing matrix, significantly con-
tributed to the project’s failure. This failure to appropriately staff was pervasive through
the project from its planning stages to its cessation, across multiple activities, as noted
75 Open_Intl_00186087.
76 Open_Intl_00186087.
77 Open_Intl_00175598; CFC_031557–568; Deposition of Dr. Michelle Frey (Rough) at 99:24-100:20.
78 Open_Intl_00218607; see also CFC_150342 (Mr. Beckstead acknowledging in early May 2020, before First
Amendmentwassigned,thatOctober5go-livewasalreadyatriskandcitingCityresourceshortfalls); CFC_078629
(Mr. Amato noting in early August 2020 that late October go-live was “optimistic” in light of delayed system testing
issues and advising that mid-late November was more realistic); Open_Intl_00204094 (Open warning the City in
late August 2020 that even late November 2020 go-live was becoming unlikely due to delays in system testing,
and Mr. Beckstead acknowledging having “the same concerns”).
J
T
H
C
Proprietary and Confidential Page 56 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 57
of 89
Expert Report
in section 5.2 on page 50. There are some indications that this was a conscious decision,
with the concurrence and direction of City program leadership.
ThefailureonthepartoftheutilitytostaffCISprojectsadequatelyisafrequentrisk,inmy
experience, and often contributes to failures. The City’s own independent assessment,
by TMG, concurs, citing lack of adequate resourcing as #6 in its “Top Ten” reasons why
projects fail, and observing that “This project has also been inadequately resourced”
79 .
2.The failure to staff appropriately contributed not only to the failure of the project to
deliver its intended outcome, but also to a sense of staff burnout and overall frustration,
as noted in the assessment conducted by TMG Consulting in April 2021
80 , for example
Assessment Consideration #2 on page 5, and a general theme of City tiredness and
frustration running through the interview results. This frustration inevitably made any
attempted replanning or refocusing activities more challenging to execute than if the
staff still had the level of motivation expressed at the start of the project.
3.The failure of the City to adequately prepare for and execute its testing responsibilities
under the Agreements represents a significant departure from leading CIS implementa-
tion practices. It is my opinion that lack of City staff assigned to test case preparation
and execution played a very significant part in the City’s inability to complete this work.
5.3 The City failed to provide effective governance and oversight
5.3.1 The City lacked CIS implementation experience and expertise, and failed to address
that problem
There was little or no CIS implementation experience among the members of the City’s¶257
procurement and execution teams. While this is relatively common for a utility entering a
CIS implementation project, such utilities almost always make up for their lack of internal
experience and expertise by hiring an outside professional partner–typically, a utility-CIS
focused consulting company–to work with them.
The City initially recognized the importance of having an expert partner. The City hired AAC¶258
UtilityPartnerstoassesstheCity’slegacyCIS(Banner),makearecommendationastowhether
the City should replace that legacy CIS, and guide the City through the creation and issuance
of a RFP for implementation of a new CIS, the vendor selection process, and the contract
negotiation process
81 . However, as discussed in paragraph ¶189, the City ceased working
with AAC before the RFP for the CIS replacement project was issued.
TheCitystillplannedtohireanoutsideconsultancyviaseparateRFP,inaprojectmanagement¶259
role, during the vendor selection phase of the implementation procurement process
82 . The
City did not in fact do so, and instead proceeded through the procurement and contract
negotiation phases, and into project execution, without CIS implementation experience or
expertise83 . IllustratingtheinadequacyofprojectmanagementfromtheCity’sside, oneofthe
senior executives for the project, Broadband director Colman Keane, did not believe the City
79 Open_Intl_00082794.
80 Open_Intl_00082770.
81AAC 1-000355–364.
82 E.g., CFC_063162– 063163; CFC_055800–803.
83 DepositionofLoriClementsat79:3-13(City’sprojectmanagerfromAugust2018toDecember2019admitting
J
T
H
C
Proprietary and Confidential Page 57 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 58
of 89
Expert Report
had any need for a project manager
84 . The City later acknowledged that this was a mistake
85 ,
and I agree with this opinion.
In my opinion, the decision not to retain (or replace) the services of AAC Utility Partners is¶260
an additional indication of the City’s poor management and lack of effective oversight. A key
attribute of an effective management and leadership team is recognizing, and addressing, the
provision of additional support and guidance when appropriate. That did not happen on this
project. Retaininganexpertadvisor, andprovidingadditionalCISimplementationexperience
as part of the utility project team, contributes substantially to reducing risks and helping the
utility and vendor(s) achieve mutually satisfactory outcomes from CIS projects. A third-party
advisor would have also been able to recognize the amount of organizational change that the
introduction of a modern CIS would have on the business, and make recommendations in
that area to mitigate the impacts of that change.
It is also informative to note the testimony of Mr. Hickmann, the CEO of Tualatin Valley Water¶261
District TVWD), a North American utility where OSF was successfully implemented. TVWD
chose to retain AAC in an advisory role, and also contracted a full-time project manager from
AAC. He states, regarding the value of external advisors and skilled project management:
Throughout the implementation process, AAC also provided guidance for shaping the
RFP; for assembling, refining, and tracking the roughly 1,700 functional requirements
that Open agreed to satisfy; and ultimately for managing the project on a day-to-day
basis on our behalf. TVWD and CWS also retained AAC’s CIS-implementation specialist
AndyKrugmantoserveastheir full-timeprojectmanagerthroughouttheproject. Inmy
opinion it would have been very difficult to complete the project without his expertise.
…TVWD could not have led this unique project without the additional bandwidth and
expertise of a CIS-implementation specialist like AAC86 .
5.3.2 The City’s broadband business was not ready for a fixed price and scope implementa-
tion
The City’s overall approach to contracting for the project demonstrates its problematic in-¶262
no experience with implementing a project like this, and relying on general utilities operations, billing, and
customer service experience);accord id.at 14:8-22.
84 See Deposition of Colman Keane at 49:14-50:12 (testifying that City didn’t need a project manager and
actually assigned an internal resource only to serve as a “point person or a wrangler, not a true project manager”);
CFC_055875 at -876 (Rosintoski memo stating that City “had planned on AAC being our PM vendor but Colman
opposed,andthefundingdidnotsupport”);seealso DepositionofTheresaConnorat68:4-25,69:5-70:15(testifying
that City decided to use Open’s project manager but conceding that was “not the normal arrangement” where “the
City would hire its own third-party project manager”).
85 Deposition of Mike Beckstead at 79:4-8 (Q: “Do you think that the City’s decision not to hire a professional
projectmanagerwasakeymistake,sittingheretoday?” A:“Yes,Ido.”);seealso CFC_085150at-150-151(Citysenior
database analyst stating “I would hope that Utilities and the City has finally learned the lesson that professional
project management from day one, is critical!” and City project senior executive responding, “Also fully agree …in
a project of this size and complexity, a professional PM is required …. I recall we had a pm in the plan back when
we picked Open and I’m not sure where that fell out of the plan, but it was a key miss.”);see CFC_217345 at -345,
-347 (notes from 2021 interview of City CFO stating that City project manager lacked requisite skill set and that
“[d]ollars to donuts would never greenlight a project without 3rd party involvements; [d]on’t think there was any
appreciation for the needs of independent party”).
86 Open_Intl_00357584, at ¶¶19, 24.
J
T
H
C
Proprietary and Confidential Page 58 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 59
of 89
Expert Report
experience. The City’s RFP demanded a fixed price bid for both Broadband and Utilities
components of the implementation. In a fixed price bid the vendor provides a fixed price
for a particular scope. Any changes in scope would typically require additional negotiation
(potentially contentious and time-consuming) between the utility and the vendor to revise
the price to reflect the changed scope.
Fixed price approaches are most suitable for implementations where the scope is well defined¶263
and the vendor can reasonably confidently assess the risks associated with performing the
work. IntheCity’ssituation,thescopefortheUtilitiesCISimplementationcouldbeconsidered
to be reasonably well defined. However, the scope of the Broadband component of the work
was not well defined at the time Open responded to the RFP, nor during the subsequent
negotiation process (see section 5.1 on page 42).
The broadband business, in Q4-2018, was poorly defined and poorly established. Platforms¶264
were not in place, business processes and system requirements were not in place, and the
business overall was very much in a “start-up” model. In my opinion the business was not
ready to undertake a fixed-price, fixed scope CIS implementation for broadband in the Q4-
2018 time frame.
To enable agreement on scope and risk, Open and the City agreed, in the SOW, to a series of¶265
significant assumptions and conditions about the scope which were required to be met. Some
of these assumptions are called out in paragraph ¶222 on page 50 of this report. Assumptions
and conditions like these are typical, and crucial, for fixed price, fixed scope agreements. Par-
ticularly with respect to the broadband business, the City was not prepared for the approach
in the Agreements, which relied on the clear definition of scope being established in the first
few weeks of the project and which did not occur.
As reflected by the assumptions in the SOW, Open was not aware that the City lacked the¶266
Broadband platforms and business definitions until the project began. Also, the City’s second
project manager concluded that the Broadband team at the City did not appreciate the im-
portance of this information
87 . Advice from an experienced third-party advisor could have
guided the City down a different path, highlighting my opinion that the City lacked sufficient
competent governance and oversight.
5.3.3 The City provided inadequate project management
During the procurement process, and in line with the staffing requirements in the SOW, the¶267
City recognized the need for a full-time experienced project manager, and issued a RFP in
June 2018 to identify and procure the services of such a person. Overruling the Utilities
personnel who had planned to hire an external, professional project manager, the Broadband
Director concluded the role was unnecessary
88 . Instead of a qualified, experienced external
project manager, the City appointed an existing employee (Ms Lori Clements) to serve as its
project manager. Ms. Clements served as the project manager from the project’s inception
(August 2018) through November 2019.
87 Deposition of Michelle Frey (Rough) at 110:11-16.
88 Deposition of Colman Keane at 42:2-8, 47:20-50:7; Deposition of Lori Clements at 65:19-66:7 (testifying that
Mr. Keane thought a project manager for the City was unnecessary, despite mirrored approach of dual project
managers required by project contract).
J
T
H
C
Proprietary and Confidential Page 59 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 60
of 89
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 61
of 89
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 62
of 89
Expert Report
manager, even while he continued in that role and the project foundered, yet City project
leadership did nothing to rectify this critical lack of leadership and governance from the City’s
side of the project95 .
It is my opinion that the lack of experienced CIS-specific and fully dedicated project manage-¶277
ment experience in the two individuals who were leading the project for the vast majority
of its existence (for approximately 27 of its 35 months, or 77% of the time) contributed
significantly to its failure. Relevant experience is critical in terms of being able to proactively
identify risks, prioritize them and suggest and implement effective mitigation actions; this
was lacking on the City’s team.
5.3.4 The City’s project governance was not adequate for the needs of the project
Overall the City did not deliver on its obligations in the Agreements regarding clear and¶278
effective leadership, at multiple levels of the organization and over a prolonged period of time.
There are multiple examples of this provided in both testimony and written material, among
them Mike Beckstead’s assessment of the “Current Situation” on May 11, 2020 where he calls
out:
• Continue to miss deadlines…
• Leadership not focused on critical items
• Many on project team have day jobs–competing priorities impair progress
•Weekly project team meeting comprised of mid level managers– decisions im-
paired96
Note this is written 21 months after the project’s commencement, so can in no way be at-¶279
tributed to challenges with any start-up related adaption to the project’s needs
97 . Indeed,
the City’s governance and decision-making processes appear to have been disorganized
throughout much of the project period98 .
95 Deposition of Coy Althoff at 43:18-44:12, 46:8-17 (City’s project owner rating Mr. Amato’s performance as
“weak” yet acknowledging he raised no concern to Mr. Amato’s company); Deposition of Travis Storin at 28:15-24,
29:23-30:23 (rating Mr. Amato as “mediocre” and testifying that he concluded that Mr. Amato’s firm could not
“deliver on the technical requirements the City had expected” by Q4 2020 or Q1 2020—months before the City
replaced Mr. Amato); Deposition of Theresa Connor at 230:1-19 (concurring with Mr. Althoff’s email that Mr.
Amato’s “role on the project most certainly was not working” and noting his construction, rather than software,
expertise).
96 CFC_184603–184604.
97 See also CFC_055875 at -876 (Rosintoski internal memo stating there was “not strong representation from a
designated [executive leadership team] member to support the project” and that leadership “could not execute”
and that “leadership in Connexion . . . was poor”); City 30(b)(6) Deposition (Storin Rough) at 113:7-114:20
(testifying about his statement in TMG interview that two of City’s executive leaders for the project were “asleep at
the wheel” that he assessed “they weren’t carrying out the full duties that we expected of an executive sponsor”);
Beckstead at 124:18-126:1 (testifying that, up to mid-2020, the primary project executive had an ill-defined role
and had not been “acting as a project leader” but had been “handing off things and coordinating things,” not
being “timely in decision and active in solving for problems”); id. at 182:5-19 (“[T]hat was one of the root issues.
There was a project sponsor and there was an internal PM and then and external PM, but there really wasn’t a
boots-on-the-ground project leader.”).
98 E.g., Deposition of Theresa Connor at 122:9-21 (stating that, even as of late September 2020, “there wasn’t a
whole lot of structure” to the City’s decision-making and escalation processes).
J
T
H
C
Proprietary and Confidential Page 62 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 63
of 89
Expert Report
Throughout the bulk of the project, Lisa Rosintoski served as the project owner in title, but¶280
operated in a somewhat ineffectual and indecisive manner, declining to make decisions and
waiting for guidance from a group of City senior executives who were not involved with the
project on a day-to-day basis. According to the City’s CFO two of these three individuals were
“asleep at the wheel” during much of this period99 .
Eventually, a retiring senior executive from the City, Mike Beckstead, assumed a temporary¶281
role as lead executive for the project. He confirmed that decision-making systems for the
City’s team were disorganized. Mr. Beckstead noted that a flaw from the beginning of the
project was that no senior-level City personnel were responsible for working in the project
details100 , and that the City’s lack of management accountability was best illustrated by the
fact that “I’m searching for the project leader and I can’t find one.”101
Although Mr. Beckstead brought clarity of vision to the City’s team for his six-month stint, he¶282
was then replaced by a three-to-four person executive panel (the City is not clear who com-
prised that panel), Ms. Rosintoski as project owner and Coy Althoff as owner’s representative.
During this second period of diffuse executive leadership, the City’s strategic objectives kept
switching dramatically, from a focus on utilities implementation to broadband implementa-
tion, from using Open’s implementation methodology to attempting to use Agile methodology,
and from contracting for an On-Premise implementation to negotiating with Open for a modi-
fied PaaS implementation. The failures at the project management level are unsurprising
given the lack of consistent—or even identifiable—executive leadership overseeing the project.
The City’s failure to implement and deliver effective governance and leadership is noticeable¶283
in various aspects of the project, throughout the project timeline.
•Broadband approval. The City decided to go-live with broadband in August 2019, and it
formally approved the broadband-delivery milestone in October 2019
102 . The notifica-
tion of payment release associated with the Broadband go-live also indicates that
Broadband functionality has been substantially delivered by Open, and Open has
estimated 96.4% completion of software delivery103 .
Nonetheless, a full year later, the City presented Open with a list of over 100 alleged
problems with broadband functionality most of which were not within the scope of the
project or had already been addressed or were in process104 .
•Approach to 1st Amendment to MPSA. The City negotiated the 1st Amendment to the
MPSA to include deadlines that the City proved unable to meet and which, even at the
time the 1st Amendment was signed, the City should have known it likely would be
unable to meet.
99 CFC_217345 at -346; City 30(b)(6) Deposition (Storin Rough) at 112:7-113:20.
100 CFC_059756.
101 Deposition of Mike Beckstead at 280:7-18.
102 CFC_214397. And the City gave that approval only after receiving a detailed analysis of the functional matrix
requirements Open delivered, Open_Intl_00344983-984, 00344985_, and only after circulating that conclusion
internally and receiving feedback from the City’s project leadership, CFC_195765, 195767.
103 CFC_113577.
104 See Milestone_026564, 025655;see also City 30(b)(6) Deposition (Althoff Rough) at 25:5-28:13 (City testifying
it had no basis to dispute Open’s conclusions about the alleged problems identified by the City).
J
T
H
C
Proprietary and Confidential Page 63 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 64
of 89
Expert Report
•Project redirects. During the project, the City repeatedly shifted the project’s focus, and
all City resources, back and forth between broadband and utilities105 .
•Change of methodology. The City switched its internal methodology approximately 2
years into the project, from the traditional waterfall approach described in the Agree-
ments to a more “Agile” approach
106 . True agility in software projects requires a con-
siderable shift in mind-set, from leadership on down. Attempting to adopt an agile
approach at this stage of a project is challenging; aligning that internal methodology
with an in-flight project which was using a different approach is risky, and in my opinion
thischoiceplacedanadditionalburdenontheCity’salreadystretchedstaff. Thisalsoap-
pears to have been an ill-considered idea that ultimately was abandoned as unworkable
and thus proved to be little more than a needless distraction107 .
•Portal launch. The City failed to launch the customer self-service portal. Although Open
delivered the portal to the City in March 2019,and despite identifying only minor bugs
thatOpenfixed, theCityneverfullylaunchedtheportalatanytimeduringtheremaining
2 years of the project 108 .
5.3.5 The City was not prepared to undertake its obligations during post go-live and hyper-
care
This section considers an example of the consequences of poor management and governance¶284
by the City, focusing on the period when the Broadband business went live in July and August
2019, inparticularthroughthelensofincidents(potentialdefects)reportedbytheCityduring
and after that period. In summary, the events described below demonstrate further the City’s
lack of effective planning, execution, oversight and governance of its agreed-to role in the
project.
Process-wise, production-related problems identified by the City are reported to Open using¶285
Open’sSAOtool, inasimilarmannertoreportingpotentialfeaturemismatches(Observations)
as described earlier in paragraph ¶206 on page 47. One important difference is that potential
defects are categorized as “Incidents” in SAO, as opposed to the category of “Observations”
used for features. Incidents have specific life-cycles and agreed-to performance targets,
known as Service Level Agreements (SLAs) which are described in the Agreements.
Open, in addition to the SAO data on Observations analyzed earlier, also provided comparable¶286
data on Incidents109 . This data was used as the basis of the analysis that follows.
Contractually, the hyper-care post go-live support period for broadband ended 3 months¶287
after go-live, at which point the Annual Support Services (SAM) contract would control the
manner in which Open provided support. In practice the lines between project support and
105 CFC_213730 at -733–734.
106 CFC_213730 at -737–738; CFC_212113 at -150, 162.
107 Deposition of Coy Althoff at 137:23-138:14.
108 See CFC_089019 (after broadband go-live, in September 2019, identifying two issues related to making portal
“more user friendly”); CFC_084390 (noting delivery of second fix for the portal in September 2019 and that City’s
functional lead and broadband manager reviewed and “it looks good,” but waiting to “mov[e] to production” till
City marketing department reviewed); CFC_171319 (in March 2021 City noting necessary “fixes for the sales
screen” that were preventing it from “turning on the portal”).
109 Open_Intl_00367564.
J
T
H
C
Proprietary and Confidential Page 64 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 65
of 89
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 66
of 89
Expert Report
vices110 . Planning for post go-live support is a critical part of well-executed implementation
projects, and evaluating the readiness of the City to carry out those duties should have been
part of the pre go-live check-list to confirm readiness to go live.
The Agreements assigned the initial problem triage as being the City’s responsibility. This¶292
data suggests they were not prepared to effectively perform that role.
The rate of reporting of annulled incidents does not start to moderate until the second quarter¶293
of 2020, some 9 months after go-live (shown by the line marked “Incidents moderating” in
figure 5.3 on the previous page). Relatively small utilities (such as Fort Collins) with relatively
small organizations would typically reach stabilization in under 6 months, possibly within
2–3 months, post go-live if the project was well managed and led. Likely causes for this ex-
tended period of high percentages of annulled defects are inadequate staffing and inadequate
preparation.
In any implementation a certain volume of defects is normal and expected, as the inevitable¶294
result of the implementation of a complex solution such as a CIS. The Agreements recognized
this, and contain provisions for support from Open and a process, and service levels, for Open
and the City to manage these defects.
Considering the 54% of Incidents which were assessed to be valid defects, the total volume of¶295
these (180 requiring a fix, and 71 not requiring a fix for a total of 251 in a 24 month period) is
high. I conclude that the likely root causes for this were:
•Inadequate User Acceptance Testing of the solution by the City prior to go-live, caused
by the City’s decision to rush the Broadband launch.
•Changing, or undefined, business processes uncovering new or changed needs that the
implementation project was not aware of and thus could not address before launch.
Both of these causes support my opinion that a combination of poor preparedness, manage-¶296
ment and staffing contributed significantly to the project’s failure.
5.3.6 Summary
It is my opinion that:¶297
1.The City’s lack of CIS implementation experience, coupled with its decision to proceed
without the services of AAC Utility Partners, or an equivalent advisory consultancy,
exacerbated the impact of the City’s other mistakes by depriving the City access to
impartial third-party advice grounded in proven CIS implementation experience.
2.The City’s failure to provide a competent full-time project manager from the beginning
of the project
a) Contravened the Agreements between the City and Open.
110Although, as noted above, this false-reporting proportion is not unusual for the City.See also CFC_212113 at
slide 19 notes (Mr. McClune, the 4th project manager for the City stating that, in production incident reporting,
“[w]e currently use [Open’s SAO] system for items they shouldn’t need to look at”).
J
T
H
C
Proprietary and Confidential Page 66 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 67
of 89
Expert Report
b)Reduced the likelihood that the myriad complex issues which arose from the be-
ginning stages of the project could have been mitigated through prompt, decisive
action.
3.The City’s failure to provide executive governance that was empowered and responsive:
a) Contravened the Agreements between the City and Open.
b)Reduced the likelihood that the myriad complex issues which arose from the be-
ginning stages of the project could have been mitigated through prompt, decisive
action.
c)Lead to the project shifting back and forth between priorities, without consistent
direction or discipline.
5.4 Open fulfilled its obligations and did not cause the project’s failure
This section looks at Open’s performance of both its OSF product and its implementation¶298
services, in light of the obligations it had in the Agreements.
5.4.1 Selection, procurement and negotiation
In terms of the time-line events and Open’s participation, the overall procurement process¶299
starting with Open’s formal engagement on receipt of the RFP appears to have followed a
fairly typical series of events and timeline, perhaps shorter than some other procurement
processes, but not unusually so. Note this is in contrast to the events leading up to the issuing
of the RFP, which, as noted in section 5.1 on page 42, were rushed following the decision to
include broadband in the RFP scope. In summary the events and timeline of the selection and
procurement process, in my opinion, provided sufficient opportunity to enable the City to
fully understand the Open product, Open’s approach to the project, and the obligations placed
on the City by the Agreements. The City also knew and accepted during the RFP process that
it was going to be Open’s first customer in the continental United States and the first to use
OSF version 8.
The selection timeline is shown graphically in figure 5.4 on the next page. The City issued its¶300
RFP on February 10, 2018 and the procurement process ended with the signing of the Master
Professional Services Agreement (MPSA) with Open on August 9, 2018, an elapsed time of
approximately 6 months.
The RFP itself contains the elements and structure typically found in a CIS procurement RFP,¶301
including requests for information on the vendor’s company background, its implementation
methodology, the City’s requirements as expressed in the “Functional Matrix” section of the
RFP and information about the staffing roles and headcount that the vendor would require
the City to provide111 .
Open, along with other vendors, responded to the RFP on March 12, 2018, four weeks after¶302
the RFP was issued. The City’s initial evaluation of the responses was scheduled to complete
by April 4, a period of approximately 4 weeks from the RFP submission date112 .
111 CFC_000149–245.
112 CFC_076181.
J
T
H
C
Proprietary and Confidential Page 67 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 68
of 89
Expert Report
2018
Feb Mar Apr May Jun Jul Aug
5 12 19 26 5 12 19 26 2 9 16 23 30 7 14 21 28 4 11 18 25 2 9 16 23 30 6 13 20 27
Respond
to RFP Oral
presentation
Further on-site
meetings
Negotiation
Notice to
negotiate
Council
approval
MPSA
signed
Figure 5.4: Timeline of key events in Open selection process
Based on the structure and content in the City’s RFP-response Evaluation Workbook it ap-¶303
pears that eight vendors submitted responses
113 . Six of these were eliminated during initial
evaluation, leaving two vendors (Open and Cogsdale) to make oral presentations of their
solution as part of the City’s second phase of evaluation. This elimination, from 8 to 2, is not
unusual114 .
Open was invited to spend three days on-site in oral presentations from April 26–28, 2018.¶304
The three days of presentations for the Oral phase of the procurement is longer in duration¶305
compared with many other such sessions; some are as short as
1⁄2 day. The 3–day period
would provide significant time for both structured interactions and informal discussions
about the product, the implementation process and the vendor.
Following the oral presentations, the City commenced further evaluations to reach a rec-¶306
ommended vendor to present to executives for approval to enter negotiations to reach final
agreements.
In furtherance of this, the City invited Open back on-site for a further 4 day meeting from¶307
May 29 to June 1, 2018, four weeks after Open’s oral presentation session. Again, this is
not unusual particularly recognizing that the City was cautious about selecting Open, who
was introducing its product to the USA market for the first time. At the conclusion of these
meetingstheCityinformallytoldOpentheywouldmostlikelyproceedtocontractnegotiations
with Open as the preferred vendor. All of this is typical for a CIS procurement process.
Following the formal selection notification from the City to Open (on June 14, 2018), the¶308
contract negotiation process took two months, including developing the software licensing,
services agreement and initial statement of work, as well as negotiating final pricing, source
codeescrowprovisionsandothercommercialterms. TheCityCouncilapprovaloftheselection
took place on July 10, 2018 and the MPSA was signed on August 9, 2018.
All of these activities are typical for a CIS procurement process, and provided ample opportu-¶309
113 See CFC_076181.
114 CFC_114230, 114232.
J
T
H
C
Proprietary and Confidential Page 68 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 69
of 89
Expert Report
nities for exploration (by both parties) and resolution of any specific areas of concern. For
example, Fort Collins was concerned that Open’s product development team was based in
Colombia. The City researched the situation thoroughly, including contacting the US State De-
partment, exploring the company’s history, and negotiating contractual performance bonds
with Open’s USA business entity. This was considered to be especially important to the City,
as the City (as noted earlier in paragraph ¶157) would be Open’s first implementation in
the continental United States. The City produced an 18-page due diligence report on Open
International, which it appended to its executive recommendation presentation115 .
5.4.2 Open’s alignment with RFP scope
Open’s product was evaluated against the Functional Matrix of desired requirements con-¶310
tainedintheRFP.ThatshowedaveryhighdegreeoffitbetweentheproductandtheFunctional
Matrix, with 89% of the requirements being met by the base product via configuration,8%
as being available in the base product in time for use by the City,1% requiring work to
provide and 1% not being able to be met by Open’s solution 116 .
As noted above, the procurement and selection process followed a fairly typical timeline and¶311
series of activities, giving the City ample opportunity to explore in depth the product fit and
Open’s approach. The City evaluated eight vendor responses and shortlisted two for final
selection (Open International and Cogsdale). Thus , there were multiple products, providing a
range of functionality, presented to the City for consideration, four of which were selected for
a deeper review in Oral presentations prior to the final selection of Open.
During the evaluation process, the final decision was between Open International’s OSF prod-¶312
uct and the Cogsdale / ETI product. Based on the Executive recommendation presentation,
both Open OSF and Cogsdale ETI met the 15 Project Objectives established by the City. How-
ever, Open“Exceeded”11, and“Met”4, whileCogsdale“Exceeded”1, “Met”12and“Minimally
Met” 2. This is shown graphically in figure 5.5 on the following page117 .
Beyond the procurement process, during the implementation itself, Open and the City repeat-¶313
edly assessed OSF’s compliance with the functional matrix, including three thorough reviews
at critical points.
First, in fall 2018, the City and Open undertook the Solution Scope Presentation phase of the¶314
implementation, during which they “[r]eview[ed] all the answers and or disposition/direction
of the functional questions and scope observations” for OSF. At the end of that process, on
December 20, 2018, the City signed off that Open’s OSF solution met the functional scope
required by the City118 .
In September 2019, after Broadband went live, Open conducted a detailed, line-by-line review¶315
of the delivered product against the functional matrix and concluded that it delivered more
than 96% of the functionality promised by the functional matrix for broadband, and Open
delivered that analysis to the City on October 2, 2019
119 . The City spent two weeks assessing
115 CFC_081663.
116 Numbers rounded to whole percentages, so do not exactly total 100%
117 CFC_081663.
118 Open_Intl_00119874.
119 CFC_059235, 059236 (October 3, 2019 cover email and spreadsheet with functional matrix analysis).
J
T
H
C
Proprietary and Confidential Page 69 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 70
of 89
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 71
of 89
Expert Report
OSF product. A detailed description of the training provided is contained in Open’s Response
to the City’s Interrogatory No. 3.
From my review of training logs and recorded sessions, Open provided a substantial amount¶318
of instructor led, video conferenced classroom-style training on the product, which was
recorded for subsequent review and use. Sampling of the recordings and logs provided by
Open indicated that this appeared comparable in quality to other (CIS) vendors’ training
in terms of scope, approach, level of detail, and duration. Making recordings available for
subsequent review by participants (as a refresher) and new team members (as part of their
onboarding process) is a good practice which was followed on this project. The library of
recordings I had access to comprised approximately 48 hours of material delivered over 40
separate sessions. This amount of training, and the delivery mechanism, is typical in CIS
implementations.
Additionally, satisfaction surveys conducted by Open showed the City’s evaluation in week 5¶319
of the training as 4.88 out of 5, and later the overall initiation and planning process, which
included training, was evaluated as 8.74 out of 10127 .
This training, and the interaction it offered with Open staff knowledgeable in the OSF product,¶320
provided an additional mechanism whereby City staff could become familiar with OSF’s
capabilities and start to consider how it would integrate into their business.
5.4.4 Open concessions and investments
Open made several concessions and investments in the OASIS project, from its early stages¶321
until shortly before the project was halted. These include:
1. Cost allocation in Amendment 1 to the MPSA
The First Amendment to the MPSA recognized that both parties shared responsibility
for the events addressed by the Amendment
128 . The parties agreed in the Amendment
to allocate the costs associated with the events 55% to the City, and 45% to Open. This
represents a significant concession by Open. Open’s internal assessment was that the
City bore 88% of the responsibility for the delays, and thus should bear 88% of the
costs129 . The City’s project manager, Dr. Frey, assessed that the City was responsible for
73% of the delays (and thus Open would be responsible for 27%). Even using Dr. Frey’s
numbers, Open’s concession to pay 45% is 66% more favorable to the City in relative
magnitude than the 27% assessed by Dr. Frey (and is 375% more favorable to the City
than Open’s internal calculations)130 .
2. Development of MPaaS Proposal
127 Milestone_133242, 133243 (cover email and satisfaction survey).
128 CFC_048315.
129 23_CFC_083203.
130 See also June 2, 2020 Mike Beckstead presentation to City Council, available at https://reflect-vod-
fcgov.cablecast.tv/CablecastPublicSite/show/1204?channel=1&seekto=15829, (stating at approxi-
mately 4:36:00 that the “willingness of Open to cover 45% is truly a reflection of their commitment to this
partnership” and that “more of the responsibility lies on our side than on their side in terms of where we are today,
but they got comfortable with the 45% really in the realm of ‘this is a partnership, we both have to be successful,’
and this is part of their contribution to making sure that works”).
J
T
H
C
Proprietary and Confidential Page 71 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 72
of 89
Expert Report
In December 2020 and January 2021, at the City’s request, Open invested in developing
a detailed, 39-page proposal to move the City from on-premise hosting of OSF to a
Managed Platform as a Service (MPaaS) model131 .
3. Development of the “Reset Proposal”
In late June of 2021, at the City’s request, Open developed and shared a comprehensive
proposal to realign and refocus the project; rather than respond to the proposal it
solicited, the City filed the current lawsuit132 .
4.Continuation of broadband project-level support after completion of the stabilization
period
The broadband stabilization period ended in January 2020
133 . Open continued to sup-
port the City’s broadband production system with a higher level of support than it was
contractually obligated to provide under the terms of the Agreement134 .
In addition to the pure financial concessions and investments associated with these events,¶322
in my opinion they represent the actions of a partner willing to invest in its partnership with
the City, and work towards a long-term and mutually beneficial relationship.
5.4.5 Compliance with production incident SLAs
The Agreements provided Service Level Agreements (SLAs) describing the response times¶323
Open agreed to for responding to production incidents with the OSF product. Open’s statistics
indicate that it complied 100% with the agreed-to service levels as required by the Agree-
ments135 .
Note that “Observations” do not have contractual SLAs associated with them. However, anal-¶324
ysis of the Observations data shows that 35% of them were closed within 30 days of being
opened, and 64% within 90 days (using the “Date last updated” field in the SAO data as a proxy
for close date). In my experience this is not an unreasonable overall closure rate for project-
driven requests, questions and potential gaps, which is what “Observations” represent.
5.4.6 Delivery of the Self-service portal and OSF generally
Considering the self service portal, this is a component of the product that Open decided to¶325
license from Milestone and include as an integrated part of OSF in releases of the OSF product
starting with Version 8.0, the version proposed in Open’s RFP response and agreed-to by the
City in the Agreements136 .
131 Open_Intl_00230561, 00230574; Open_Intl_233319 (minutes from City-Open meeting regarding MPaaS
proposal);see also Deposition of Theresa Connor at 257:5-20 (stating that Kevin Wilkins asked Open to put the
MPaaS proposal together).
132 CFC_26315; Rule 30(b)(6) Deposition of City (Storin, Rough) at 91:7-11, 92:18-94:18.
133 See Open_Intl_00003732, at -732-734 (City memorandum acknowledging delivery of Broadband and Open’s
three-month post-go-live stabilization requirements).
134 See e.g., Milestone_25654, 25655 (December 2020 cover email and attached spreadsheet reflecting Open’s
in-depth attention to City concerns about broadband and other functionality concerns).
135 Open_Intl_00077510 at -520–522.
136 See generally Defs.’ Resp. to Pl. Interrog. No. 15. At the time of the demonstration to Fort Collins this license
relationship was in its early stages. The cooperation was informally agreed to on April 13, 2018 (before Open’s
oral presentation to Fort Collins) and the license signed on June 14, 2018, at the time of the City providing “intent
J
T
H
C
Proprietary and Confidential Page 72 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 73
of 89
Expert Report
An individual CIS product or solution is often comprised of multiple separate components,¶326
potentially developed by several separate entities and integrated by the primary product
vendor. The sophistication of many customers, the short time-to-market required, the highly
competitive business climate and the multiple regulatory and customary expectations from
potential purchasers often makes “license and integrate” a more compelling proposition to a
product vendor than “build from scratch”.
Similarly, the pattern of behavior adopted by Open, developing license relationships in asso-¶327
ciation with a specific deal is not unusual, especially in a situation similar to this one where
a vendor is entering a new market. Continuing the integration and development of product
features in parallel with customer specific implementation activities is similarly a typical
activity.
Open invests approximately 300,000 hours per year in OSF-related development efforts
137 .¶328
Spending 10% of this effort on self-service portal related activities is, in my opinion, rea-
sonable and in no way excessive. This is especially true given that self-service capabilities
have moved from the “nice to have” to “must have” list of CIS features as the expectations of
utilities’ customers are evolving rapidly in that direction.
I did observe a demonstration of the portal, on August 2, 2022. This was an unscripted¶329
informaloverview whereI wasshownthe portal in anon-production environment, andso was
able to direct the conversation and demonstration in a variety of paths. While not an in-depth
evaluation,theimpressionofwhatIsawwasofaproductthatprovidedmanyofthecapabilities
I would expect to see, smoothly integrated and operational around account establishment,
product ordering, historic usage, payment processing etc. Although not demonstrated, I was
informed that the design permits access from desktop and mobile devices, and integrates
with vendors such as Kubra for handling credit card payments, which is a standard approach
to support credit card security concerns.
Further, I understand that the Open self-service portal went live in July 2022 for the Tualatin¶330
Valley Water District and for Clean Water Services—two water, waste water, and surface water
service providers that serve several hundred thousand users in Oregon. As of October 2022,
more than 15,000 of those providers’ customers had registered to use the self-service portal,
which has performed there without complaint138 .
Givenmypersonalassessmentoftheportal,thefactthattheportalissimplyanOSF-integrated¶331
version of a market-tested and -accepted portal that Milestone has supplied for several other
CIS users, and the fact that Open’s OSF-integrated portal has performed successfully at scale
for two other service providers, I conclude that Open delivered a portal that complied with
its obligations and that the City’s failures described above—in particular, the City’s failure to
define the product and processes for Broadband—caused the City’s problems with the portal.
Similarly, I conducted a high-level overview of OSF version 8, and I did not observe any¶332
aspects of the system that support the City’s assertion that it was not ready for the North
American market. To the contrary, the system appeared to check the boxes I expect from a
to negotiate” notification to Open.
137 Defs.’ Resp. to Pl.’s Interrog. No. 15.
138 Open_Intl_00357584, at ¶34.
J
T
H
C
Proprietary and Confidential Page 73 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 74
of 89
Expert Report
North American CIS, and I am aware that the same version of OSF is successfully operating
for two other U.S. customers of Open. I am also aware that version 8 was delivered to the City
in early 2019, several weeks behind schedule, but that delay does not appear to have been a
material problem for the project139 . .
5.4.7 Open exhibited great flexibility in accommodating project issues
As described in section 5.2 on page 50 on multiple occasions Open provided additional staff¶333
to help the City meet its obligations under the Agreements. Not all the shortfalls in City
staffing could be addressed by third party staff however it appears that Open were flexible
and understanding in its attempts to help the City.
Open also worked diligently to help the City resolve its fundamental project issues, notably by¶334
proposing a managed PaaS arrangement to complete the implementation
140 and eventually,
a “Reset Proposal” dated June 28, 2021 which contained a 41–page proposed approach to
continue the project, based on a joint discussion between the City and Open on June 17
2021141 . This appears to me to be a well-thought-out approach and potentially could have
formed the basis for a negotiation and agreement to move the project forward. This approach
didnotproceedbeyondthisinitialproposal, astheCityinitiatedthislawsuitshortlythereafter.
5.4.8 Summary
It is my opinion:¶335
1.That the activities carried out by the City which resulted in the execution of the Agree-
ments with Open provided ample time and opportunity for the City to evaluate the
capabilities of Open’s OSF product to meet the City’s needs as expressed in the RFP and
as subsequently negotiated into the Agreements. These activities were also sufficient to
enable the City to understand its obligations with regards to its role, especially regarding
staffing, and agree to provide the expertise described in the Agreements.
2.That Open delivered a product that substantially satisfied the functional requirements
established by the City in the functional matrix, including as to the self-service portal,
as refined throughout the project.
3. That Open performed ample, high-quality training to the City and met its service-level
obligations for reported production incidents after the broadband system went live, and
promptly addressed the City’s observation reports for the system in development.
4.That Open worked diligently to help address the project challenges, and in many cases
went beyond the strict requirements of the Agreements (for example in shouldering a
substantial additional portion of the costs associated with the First Amendment).
139 Open_Intl_00170563 (email describing parties’ agreement that delayed software delivery accounted for less
than 2% of overall delays through early 2020).
140 Open_Intl_230561, 230574.
141 CFC_026315.
J
T
H
C
Proprietary and Confidential Page 74 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 75
of 89
Expert Report
6 Conclusion
CIS implementation projects typically exist at the intersection of legacy technology, large com-¶336
plex software packages, modern computer system architectures, utility customer experience,
andsubstantialorganizationalentanglements. Theyareclassicexamplesofso-called“wicked
problems”142 characterized by incomplete, contradictory and often unknown requirements,
coupledwithstronginteractions andinterdependencewith othersimilarlycomplex problems
which are often owned by stakeholders with substantially different reference frames.
Even relatively small implementations, such as Fort Collins, are fraught with risks and highly¶337
complex interdependencies both internally and externally to the project.
In my opinion from the early months of the project, it had a low probability of success. The¶338
failure of the City to meet its obligations, particularly with regards to the stability of the
Broadband business, the inexperience and detachment of its management team and the
inadequate level of staffing made it difficult, if not impossible, to succeed.
The City’s choice to not engage an experienced third part advisor to guide it through this¶339
complex process was short-sighted. Advice from such a party, if taken, would most likely have
mitigated some or all of the issues the project experienced.
The misalignment of the project and contract structure to the reality of the City’s capabilities,¶340
expectations and execution, most significantly with respect to the broadband business, also
contributed significantly to the challenges faced by the project, especially when coupled with
the lack of experience in the City’s management team.
The sheer volume of missteps greatly reduced the likelihood of any corrective action being¶341
effective. Many projects successfully recover from a few significant roadblocks and issues,
however in my opinion there were simply too many critical structural defects in play on this
project–including ill-defined and poorly constructed broadband functional requirements,
undefined broadband business processes, lack of City knowledge of its own broadband and
utilities systems, grossly inadequate City resource allocation, diffuse and unaccountable City
leadership structure, and a lack of CIS-implementation experience on the City’s team–to
enable mid-course corrections to be effective, no matter how well intentioned they were.
For its part, Open faced an impossible challenge to succeed when the keys to success were¶342
largely outside of their control. I understand that Open raised with the City the problems
I have identified with the City’s approach to this implementation, including by requesting
change controls and amending the MPSA. In hindsight, Open would have been better served
by stopping the implementation when it learned that the City’s broadband team was not ready
to launch that product and the City’s project leadership and staff were falling short of what
142 https://en.wikipedia.org/wiki/Wicked_problem
J
T
H
C
Proprietary and Confidential Page 75 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 76
of 89
Expert Report
was required, rather than acquiescing to the City’s demand that broadband had to go live
in Q3 2019. Instead, over and over, Open trusted the City’s assurances that it would course
correct, but the City never did.
Signature:
Name: John R. Hutchinson
Principal, The John Hutchinson Consultancy
Date: October 24, 2022
J
T
H
C
Proprietary and Confidential Page 76 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 77
of 89
Expert Report
Appendices
A Professional Resume
Summary
John Hutchinson specializes in assisting clients in the E&U industry (gas, electric, & water)
to successfully accelerate the time-to-value associated with complex (typically $5M–$20M+)
billing and customer service related projects.
He most often achieves this as a solution architect or a project executive, establishing himself
as a trusted advisor to senior client management.
He has managed large, global, profitable portfolios of people and projects, as the leader of the
Delivery arm of a software product company (leading 400 people in 9 countries delivering
$63M revenue).
He has outstanding communication, leadership and implementation skills, built on top of a
highly technical background, which drive effective, result oriented outcomes. John works
locally, regionally and globally, in both the US and other countries, achieving superior results
in a cost-effective manner leveraging integrated on-site, near-shore and off-shore teams.
John’s experience as a Senior Vice President at SPL WorldGroup has led to a natural focus
on the product portfolio now sold by Oracle following their acquisition of that firm. He thus
most frequently works with Oracle based solutions involving both cloud (CCS, OIC, OSC) and
premise-based (CC&B, MDMS, CSS, Documaker, OBIEE, eBiz and Fusion) products. He has
also worked with several other vendors and packages, and has significant experience in aging,
legacy custom software environments.
Selected Accomplishments (not chronological)
As a solution architect, thought leader and delivery executive, John has:
1.Fora“Tier1”USutility,retaineddirectlyastheProgramArchitectforamulti-yearlegacy
system replacement, replacing a 30+ year old mainframe CIS with Oracle products
(primarily CCS). Acted as the Client’s primary advisor on the solution architecture,
coordinated and facilitated cross-workstream architectural topics and led the “Design
Council”. In addition he coordinated the Client’s engagement with Oracle Consulting
Services, and in general advised and facilitated work between the Client, Oracle and the
Systems Integrator. Due to COVID related restrictions, and the location of the work, this
was a 100% remote role.
2.FunctionedastheSI’sco-ProjectLead/SolutionArchitectforaCC&Bupgrade(toversion
2.7) for a North American deregulated electricity retailer. The upgrade was from a rela-
tively old (COBOL-based) version of CC&B, and included data truncation and migration,
interface reintegration, and redevelopment / re-evaluation / refitting of customizations
in its scope. The work was performed approximately 50% on/off site overall.
J
T
H
C
Proprietary and Confidential Page 77 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 78
of 89
Expert Report
Activities included leading the scope definition workshops, architecture definition, data
migration strategy definition and co-management of the development and testing of
the solution. Development and solution testing was almost entirely conducted offshore.
Other activities included providing key leadership / advice to the client regarding overall
testing strategy and management (using HP-ALM as the part of the management toolkit).
3.Functioned as the Solution Architect during sales and design phase of a $13M customer
experience transformation for an English water company based on Oracle products.
The solution supported a greenfield wholesale business, integrating with a still nascent
national water market operator and multiple retail companies, while simultaneously
serving the regulated residential customer base as the English water market entered
deregulation.
4.Led the pursuit, and then functioned as the Solution Architect, for the Customer Ca-
pability component ($15M) of a $53M contract to act as the System Integrator in the
establishment of a greenfield water utility for Ireland, based on Oracle CC&B, MDMS,
CSS, Documaker, eBiz, IBM Maximo and other industry leading software products.
5.Helped a North American Utility architect the integration of their service order workflow
between IBM’s Maximo work and asset management solution, and their in-flight Oracle
Utilities implementation.
6.Led the pursuit, then the initial delivery solutioning, for the customer workstream of a
major ($100m+) business transformation initiative for a wholesale electrical provider
serving over 3M customers in a North American deregulated market, including ar-
chitecting automated BPNM-driven workflow assistance for common market driven
transactions.
7.Led the pursuit, contract negotiation and initial delivery of a $55M, 8-year Oracle CC&B
implement / manage / host project for a 450,000 customer US Municipality supporting
7 separate billable services.
8.Jointly led the contract negotiations and led the project team through the Requirements
phase of a legacy CIS merge / migration project for the US arm of a major international
utility.
9. Served as Solution Architect / SME for “first-of-a-kind” integration of IBM Watson with
Oracle’s CC&B to provide contact center “chat” services.
10.Led the competitive pursuit, negotiation and implementation of a municipal utility
billingsystem. Thebillingplatformwasnewlyintroducedtothemarket. Mr. Hutchinson
worked closely with the client and product vendor staff over the first stage of this two-
stage $16M project. At the completion of the definition phase, the client decided not to
proceedwiththeimplementationduetocompetingdemandsontheirinternalresources.
11.Participated as a billing SME in a 3-month engagement leveraging IBM Design Thinking
to reimagine the billing experience for all stakeholders in a major US custodial bank.
12.Was retained by the COO of a FinTech startup to advise and assist in the establishment
of the Business Operations team, including billing and customer support / service
functions.
J
T
H
C
Proprietary and Confidential Page 78 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 79
of 89
Expert Report
13.Managed a $63M P&L as a consulting services executive, responsible (in 2001–3) for
over 400 consultants in 9 countries implementing and supporting Utility billing systems.
Under Mr. Hutchinson’s leadership the unit experienced significant head-count and
revenue growth, accompanied by profitability increases, while maintaining customer
and employee satisfaction and delivery excellence.
Employment history
2017 – Self-employed consultant.
2005 – 2017 IBM; Associate Partner, North American E&U Industry and Global Oracle
Center of Competence.
2004 – 2005 Independent project advisor.
1998 – 2003 Vice President, then Senior Vice President, Global Delivery, SPL WorldGroup
Inc.
Prior roles:Vice President, Origin Technology in Business (then part of of Philips Elec-
tronics NV).
Delivery Director, Vice President, AGS Information Services.
Consultant, AT&T Bell Laboratories, Holmdel NJ.
Various teaching, research and support positions in the Computer Science
department at the University of London.
Education and accomplishments
• Inductee, IBM “100% Club”.
•Co-author of IBM patent applying machine learning to AMI data to improve targeted
marketing activities.
• Developer and presenter of a “CIS Implementation” course at CS Week conference.
•B.Sc(Honours), ComputerScienceandPhysics, LondonUniversity(QueenMaryCollege),
UK.
Work eligibility
• U.S.A. Permanent Resident (“Green Card”), UK Citizen.
• Languages: English.
J
T
H
C
Proprietary and Confidential Page 79 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 80
of 89
Expert Report
B Materials considered
1.I had access to, thoroughly reviewed, and generally considered the contents of the
project shared drive, Open_Intl_00002983-00077507, including project plans, project
status reports, project memoranda, test cases, business cases, and the like. Where I
have purposefully considered specific documents from the shared drive, I cite them
specifically.
2. Travis Storin deposition dated 06.21.2022
3. Travis Storin 30(b)(6) deposition (rough) dated 10.05.2022
4. Theresa Connor deposition dated 07.29.2022
5. Michael Neil Beckstead deposition dated 07.12.2022
6. Lori Clements deposition dated 06.30.2022
7. Edith Mercado deposition dated 07.28.2022
8. Dr. Michelle Frey deposition (rough) dated 10.12.2022
9. Coy Althoff 30(b)(6) deposition (rough) dated 10.05.2022
10. Coy Althoff deposition dated 06.23.2022
11. Colman Keane deposition dated 07.13.2022
12. Aaron McClune deposition (rough) dated 10.19.2022
13. Brad Ward 30(b)(6) deposition (rough) dated 10.06.2022
14.Open’sResponsestotheCityofFortColllins’sSecondSetofInterrogatoriestoDefendants
dated 08.01.2022
15.Open’s Responses to the City of Fort Collins’s Third Set of Interrogatories to Defendants
dated 09.12.2022
16.Open’s Responses to the City of Fort Collins’s First Set of Request for Admissions to
Defendants dated 09.12.2022
17.Responses and Objections to Defendant’s First Set of Discovery Requests to Plaintiff
dated 11.29.2021
18.Open’s Responses to City of Fort Collins First Set of Interrogatories to Defendants dated
11.26.2021
19. Exh. 3 to Open’s Responses to City of Fort Collins First Set of Interrogatories
20. Exh. 2 to Open’s Responses to City of Fort Collins First Set of Interrogatories
21. Exh. 1 to Open’s Responses to City of Fort Collins First Set of Interrogatories
22. Defendant’s Answer and Counterclaim dated 08.27.2021
23. Complaint and Jury Demand dated 07.02.2021
24. PCR Nos 1-30
25. AAC 1-000169
26. AAC 1-000385
27. AAC 1-000355
28. CFC_00013408
29. CFC_000149
J
T
H
C
Proprietary and Confidential Page 80 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 81
of 89
Expert Report
30. CFC_000150
31. CFC_000199
32. CFC_000200
33. CFC_000201
34. CFC_000202
35. CFC_000203
36. CFC_000204
37. CFC_000236
38. CFC_000237
39. CFC_000238
40. CFC_000239
41. CFC_000240
42. CFC_00059907
43. CFC_00103804
44. CFC_00114229
45. CFC_00114230
46. CFC_00114232
47. CFC_00174996
48. CFC_00174997
49. CFC_010322
50. CFC_011487
51. CFC_011488
52. CFC_011488
53. CFC_012217
54. CFC_012277
55. CFC_012762
56. CFC_015224
57. CFC_015310
58. CFC_021724
59. CFC_021726
60. CFC_021727
61. CFC_021728
62. CFC_021729
63. CFC_025421
64. CFC_026315
65. CFC_026529
66. CFC_026696
67. CFC_026738
68. CFC_031557
69. CFC_034022
70. CFC_034023
71. CFC_035510
72. CFC_043371
J
T
H
C
Proprietary and Confidential Page 81 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 82
of 89
Expert Report
73. CFC_046746
74. CFC_046747
75. CFC_048315
76. CFC_048594
77. CFC_048653
78. CFC_049891
79. CFC_049967
80. CFC_051017
81. CFC_051055
82. CFC_055800
83. CFC_055874
84. CFC_055875
85. CFC_059235
86. CFC_059756
87. CFC_061163
88. CFC_062800
89. CFC_063162
90. CFC_063163
91. CFC_065044
92. CFC_067853
93. CFC_069036
94. CFC_070559
95. CFC_073864
96. CFC_074717
97. CFC_074719
98. CFC_076181
99. CFC_078269
100. CFC_080119
101. CFC_080119
102. CFC_080568
103. CFC_080989
104. CFC_081001
105. CFC_081491
106. CFC_081663
107. CFC_082415
108. CFC_082458
109. CFC_083203
110. CFC_083424
111. CFC_083425
112. CFC_083426
113. CFC_084390
114. CFC_084447
115. CFC_084735
J
T
H
C
Proprietary and Confidential Page 82 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 83
of 89
Expert Report
116. CFC_085150
117. CFC_086666
118. CFC_087614
119. CFC_087995
120. CFC_089019
121. CFC_090628
122. CFC_092717
123. CFC_093005
124. CFC_099930
125. CFC_099930
126. CFC_099942
127. CFC_100747
128. CFC_102679
129. CFC_103158
130. CFC_107251
131. CFC_110101
132. CFC_110102
133. CFC_111208
134. CFC_111210
135. CFC_111390
136. CFC_111391
137. CFC_111581
138. CFC_113577
139. CFC_119367
140. CFC_120440
141. CFC_120832
142. CFC_122108
143. CFC_122732
144. CFC_122733
145. CFC_130951
146. CFC_130952
147. CFC_130954
148. CFC_130955
149. CFC_139533
150. CFC_139534
151. CFC_139960
152. CFC_141239
153. CFC_141240
154. CFC_141241
155. CFC_141296
156. CFC_145665
157. CFC_145666
158. CFC_149761
J
T
H
C
Proprietary and Confidential Page 83 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 84
of 89
Expert Report
159. CFC_150342
160. CFC_150928
161. CFC_150929
162. CFC_150930
163. CFC_159116
164. CFC_159118
165. CFC_161155
166. CFC_161156
167. CFC_171319
168. CFC_174211
169. CFC_174212
170. CFC_184603
171. CFC_191533
172. CFC_191533
173. CFC_195765
174. CFC_195767
175. CFC_200815
176. CFC_202864
177. CFC_205465
178. CFC_205651
179. CFC_209828
180. CFC_209832
181. CFC_211947
182. CFC_212113
183. CFC_212297
184. CFC_213622
185. CFC_213730
186. CFC_213940
187. CFC_213941
188. CFC_214397
189. CFC_215431
190. CFC_216348
191. CFC_217099
192. CFC_217099
193. CFC_217151
194. CFC_217167
195. CFC_217210
196. CFC_217345
197. CFC_217349
198. CFC_217359
199. CFC_217363
200. CFC_217448
201. CFC_219578
J
T
H
C
Proprietary and Confidential Page 84 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 85
of 89
Expert Report
202. CFC_220079
203. CFC_220428
204. Cloud 1-000004
205. Milestone_025654
206. Milestone_025655
207. Milestone_025655
208. Milestone_026564
209. Milestone_029306
210. Milestone_119782
211. Milestone_133242
212. Open_Intl_00000001
213. Open_Intl_00003732
214. Open_Intl_00006656
215. Open_Intl_00006758
216. Open_Intl_00031983
217. Open_Intl_00031983
218. Open_Intl_00032007
219. Open_Intl_00032007
220. Open_Intl_00032025
221. Open_Intl_00032025
222. Open_Intl_00032040
223. Open_Intl_00032040
224. Open_Intl_00032040
225. Open_Intl_00032051
226. Open_Intl_00032061
227. Open_Intl_00032070
228. Open_Intl_00032077
229. Open_Intl_00032084
230. Open_Intl_00032084
231. Open_Intl_00032091
232. Open_Intl_00032091
233. Open_Intl_00034178
234. Open_Intl_00034184
235. Open_Intl_00034413
236. Open_Intl_00034421
237. Open_Intl_00034426
238. Open_Intl_00034437
239. Open_Intl_00034443
240. Open_Intl_00034496
241. Open_Intl_00034502
242. Open_Intl_00034504
243. Open_Intl_00034506
244. Open_Intl_00034563
J
T
H
C
Proprietary and Confidential Page 85 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 86
of 89
Expert Report
245. Open_Intl_00077508
246. Open_Intl_00077510
247. Open_Intl_00077523-00078036
248. Open_Intl_00082423
249. Open_Intl_00082561
250. Open_Intl_00082573
251. Open_Intl_00082651
252. Open_Intl_00082662
253. Open_Intl_00082665
254. Open_Intl_00082667
255. Open_Intl_00082678
256. Open_Intl_00082770
257. Open_Intl_00082794
258. Open_Intl_00082920
259. Open_Intl_00082929
260. Open_Intl_00082929
261. Open_Intl_00082929
262. Open_Intl_00083685
263. Open_Intl_00084109
264. Open_Intl_00085632
265. Open_Intl_00089459
266. Open_Intl_00090576
267. Open_Intl_00106816
268. Open_Intl_00106958
269. Open_Intl_00106959
270. Open_Intl_00107722
271. Open_Intl_00107722
272. Open_Intl_00107723
273. Open_Intl_00118399
274. Open_Intl_00119874
275. Open_Intl_00124806
276. Open_Intl_00159772
277. Open_Intl_00170563
278. Open_Intl_00170563
279. Open_Intl_00170565
280. Open_Intl_00170872
281. Open_Intl_00170873
282. Open_Intl_00175598
283. Open_Intl_00186085
284. Open_Intl_00186087
285. Open_Intl_00204094
286. Open_Intl_00212310
287. Open_Intl_00216662
J
T
H
C
Proprietary and Confidential Page 86 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 87
of 89
Expert Report
288. Open_Intl_00218607
289. Open_Intl_00219811
290. Open_Intl_00227007
291. Open_Intl_00230561
292. Open_Intl_00230574
293. Open_Intl_00233319
294. Open_Intl_00241239
295. Open_Intl_00252185
296. Open_Intl_00252260
297. Open_Intl_00252263
298. Open_Intl_00252284
299. Open_Intl_00252287
300. Open_Intl_00252289
301. Open_Intl_00252291
302. Open_Intl_00252293
303. Open_Intl_00252314
304. Open_Intl_00267457
305. Open_Intl_00271075
306. Open_Intl_00291472
307. Open_Intl_00294833
308. Open_Intl_00300426
309. Open_Intl_00304670
310. Open_Intl_00315501
311. Open_Intl_00315501-00315504
312. Open_Intl_00318868
313. Open_Intl_00323418
314. Open_Intl_00326020
315. Open_Intl_00327830
316. Open_Intl_00327832
317. Open_Intl_00328057
318. Open_Intl_00329015
319. Open_Intl_00329659
320. Open_Intl_00329743
321. Open_Intl_00329902
322. Open_Intl_00329929
323. Open_Intl_00330018
324. Open_Intl_00330975
325. Open_Intl_00332002
326. Open_Intl_003344983
327. Open_Intl_003344985
328. Open_Intl_00336943
329. Open_Intl_00337766
330. Open_Intl_00337766
J
T
H
C
Proprietary and Confidential Page 87 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 88
of 89
Expert Report
331. Open_Intl_00338160
332. Open_Intl_00343288-00343300
333. Open_Intl_00343452
334. Open_Intl_00343452
335. Open_Intl_00343477
336. Open_Intl_00343736
337. Open_Intl_00343770
338. Open_Intl_00343812
339. Open_Intl_00343833
340. Open_Intl_00343849
341. Open_Intl_00343849
342. Open_Intl_00343920
343. Open_Intl_00343928
344. Open_Intl_00343933
345. Open_Intl_00345050
346. Open_Intl_00352949
347. Open_Intl_00357576
348. Open_Intl_00357564
349. Open_Intl_00357565
350. Open_Intl_00357584
351. TMG_000001
352. TMG_000055
353.Trainingvideos(Open_Intl_00343301-343400,Open_Intl_00343452,Open_Intl_00343477,
Open_Intl_00343731, Open_Intl_00344024-344090)
354.https://en.wikipedia.or/wiki/wicked_problem
355.https://fcconnexion.com/about
356.https://reflect-vod-fcgov.cablecast.tv/CablecastPublicSite/show/1204?
channel=1&seekto=15829
357.https://www.fcgov.com/
358.https://www.gartner.com/
359.https://www.openintl.com/
J
T
H
C
Proprietary and Confidential Page 88 of 88
Case No. 1:21-cv-02063-CNS-SP Document 217-1 filed 05/09/23 USDC Colorado pg 89
of 89
21407038_v1
IN THE UNITED STATES DISTRICT COURT
FOR THE DISTRICT OF COLORADO
Civil Action No.: 21-cv-02063-CNS-SP
CITY OF FORT COLLINS,
Plaintiff/Counterclaim Defendant,
v.
OPEN INTERNATIONAL, LLC
Defendant/Counterclaim Plaintiff,
and
OPEN INVESTMENTS, LLC,
Defendant.
PROPOSED ORDER GRANTING JOINT MOTION TO UNRESTRICT
This matter comes before the Court on the Joint Motion to Unrestrict filed by the parties
in this case. Having been fully advised on the matter, the Court hereby GRANTS the Motion
and ORDERS that all briefing and exhibits currently maintained under restriction in this case
(ECF Nos. 43, 45, 54, 58, 63, 67, 74, 76–78, 101, 104–106, 116, 119, 123–25, 128–29, 150–58,
173–76, 184, and 185, and all accompanying exhibits) will be unrestricted and made available on
the public docket, with the exception of the affirmative expert report of John Hutchinson (ECF
Nos. 128-1, 153-1, 175-2). The Court will make available on the public docket only the redacted
version of that report attached as Exhibit 1 to the parties’ Joint Motion to Unrestrict.
DATED: _________________, 2023. BY THE COURT:
Case No. 1:21-cv-02063-CNS-SP Document 217-2 filed 05/09/23 USDC Colorado pg 1 of
2
21407038_v1
2
Case No. 1:21-cv-02063-CNS-SP Document 217-2 filed 05/09/23 USDC Colorado pg 2 of
2