HomeMy WebLinkAboutMemo - Mail Packet - 1/9/2018 - Memorandum From Terri Runyan Re: Performance Excellence Journey -� 2017 Feedback Report Summary And Communication Info Sheet (Baldrige Framework)OUR JOURNEY
TO WORLD CLASS
MEET MALCOLM BALDRIGE!
WHY BALDRIGE?
17-18967 fcgov.com/excellence
RESULTS OF BALDRIGE
The Malcolm Baldrige National Quality Award is the nation’s highest presidential
honor for performance excellence through innovation, improvement and
visionary leadership. Since 1988, 106 organizations have earned the award
named after Malcolm Baldrige, the 26th Secretary of Commerce. Congress
created the award in 1987 to enhance the competitiveness and performance of
U.S. businesses.
Thanks to Baldrige, the City is more e cient
and e ective, thus providing residents and
businesses with world class services and greater
trust and transparency in local government. The
City selected the Baldrige framework for its
journey to excellence based on:
Customer/Community
Focus:
Pursuing excellence starts with
the needs, expectations and
satisfaction of the customer.
International Best
Practice Criteria:
The Baldrige evaluation
process is one of the best,
most cost-e ective, most
comprehensive performance
assessments organizations
can find.
Recent TOP 10 Rankings:
of residents rate Fort Collins’
quality of services “good or
very good”
*Data from Community Survey
*
• The highest rating in a decade
• 13 percent increase in
satisfaction since 2008
90%
of residents love
the quality of life
in Fort Collins
*91%
of residents feel
safe in Fort Collins
*90%
SMARTER CITY
FOR SUSTAINABILITY
BEST BICYCLE
CITIES
MOST EDUCATED
CITIES
ENVIRONMENTALLY
FRIENDLY CITIES
HEALTHIEST
CITIES
CITIES WITH ENERGY
STAR BUILDINGS
FEEDBACK IN ACTION JOURNEY
TIMELINE
Strategic Planning Process:
In its third iteration, the City developed
a comprehensive strategic planning
process, which included resident and
business input, because of Baldrige’s
feedback.
Community Dashboard:
Thanks to Baldrige, community members
can look at the City’s transparent
Dashboard to track progress on
community goals and keep the City
accountable.
Volunteer Program:
In 2015 the City launched its Volunteer
Program, FC Volunteer, to plug more
than 9,000 engaged residents into
various volunteer opportunities to make
Fort Collins a better place to live.
Business Engagement Plan:
Based on Baldrige feedback in 2016,
the City is working to better understand
and anticipate business needs.
• First BFO Cycle
• Created Vision
• Submitted Application
for Rocky Mountain
Performance Excellence
(RMPEx) High Plains Level
• Submitted RMPEx
Foothills Application
• Worked on
Key Processes
• Achieved RMPEx
Timberline Award
• Achieved RMPEx
• Submitted Peak Award
1st Baldrige
Application
• Submitted 2nd
Baldrige Application
• 1st Site Visit
• Submitted 3rd
Baldrige Application
• 2nd Site Visit
• ACHIEVED
BALDRIGE
AWARD
Our Journey
Continues...
05
10
11
12
13
14
15
16
17
City of Fort Collins
Malcolm Baldrige National Quality Award—2017 Feedback Report 3
Preparing to read your feedback report . . .
Your feedback report contains Baldrige examiners’ observations based on their understanding
of your organization. The examiner team has provided comments on your organization’s
strengths and opportunities for improvement relative to the Baldrige Criteria. The feedback is
not intended to be comprehensive or prescriptive. It will tell you where examiners think you
have important strengths to celebrate and where they think key improvement opportunities
exist. The feedback will not necessarily cover every requirement of the Criteria, nor will it say
specifically how you should address these opportunities. You will decide what is most
important to your organization and how best to address the opportunities.
If your organization has not applied in the recent past, you may notice a change in the way
feedback comments are now structured in the report. In response to applicant feedback, the
Baldrige Program now asks examiners to express the main point of the comment in the first
sentence, followed by relevant examples, in many cases resulting in more concise, focused
comments. In addition, the program has included Criteria item references with each comment
to assist you in understanding the source of the feedback. Each 2017 feedback report also
includes a graph in Appendix A that shows your organization’s scoring profile compared to the
median scores for all 2017 applicants at Consensus Review, as well as the median scores at Site
Visit Review of those 2017 applicants that received site visits.
Applicant organizations understand and respond to feedback comments in different ways. To
make the feedback most useful to you, we’ve gathered the following tips and practices from
previous applicants for you to consider.
• Take a deep breath and approach your Baldrige feedback with an open mind. You applied to
get the feedback. Read it, take time to digest it, and read it again.
• Before reading each comment, review the Criteria requirements that correspond to each of
the Criteria item references (which now precede each comment); doing this may help you
understand the basis of the examiners’ evaluation. The 2017–2018 Baldrige Excellence
Framework containing the Business/Nonprofit Criteria for Performance Excellence can be
purchased at http://www.nist.gov/baldrige/publications/business_nonprofit_criteria.cfm.
[Don Chalmers] had a vision of providing a better customer experience. He wasn’t in it for the
award but for the feedback to get better. … If we can do this, anyone can. We are a testimony to
the Baldrige framework [capacity] to provide organizational sustainability and success.
Gary Housley, President and Dealer Principal
Don Chalmers Ford
2016 Baldrige Award Recipient
Malcolm Baldrige National Quality Award—2017 Feedback Report 4
• Especially note comments in boldface type. These comments indicate observations that the
examiner team found particularly important—strengths or opportunities for improvement
that the team felt had substantial impact on your organization’s performance practices,
capabilities, or results and, therefore, had more influence on the team’s scoring of that
particular item.
• You know your organization better than the examiners know it. If the examiners have
misread your application or misunderstood information contained in it, don’t discount the
whole feedback report. Consider the other comments, and focus on the most important
ones.
• Celebrate your strengths and build on them to achieve world-class performance and a
competitive advantage. You’ve worked hard and should congratulate yourselves.
• Use your strength comments as a foundation to improve the things you do well. Sharing
those things you do well with the rest of your organization can speed organizational
learning.
• Prioritize your opportunities for improvement. You can’t do everything at once. Think about
what’s most important for your organization at this time, and decide which things to work
on first.
• Use the feedback as input to your strategic planning process. Focus on the strengths and
opportunities for improvement that have an impact on your strategic goals and objectives.
Using Baldrige to improve was, I think, one of the smartest things we did in our
business. It really gave us a touchstone; it really gave us an opportunity to learn
about [how the Baldrige framework and criteria for excellence] could be adapted to
our organization … and to constantly measure ourselves and evaluate how
we’re doing.
Scott McIntyre, Managing Partner
PricewaterhouseCoopers (PwC) Public Sector Practice
2014 Baldrige Award Recipient
Malcolm Baldrige National Quality Award—2017 Feedback Report 5
KEY THEMES
Key Themes–Process Items
City of Fort Collins (the City) scored in band 6 for process items (1.1–6.2) in the Site Visit Review
of written applications for the Malcolm Baldrige National Quality Award. For an explanation of
the process scoring bands, please refer to Figure 6a, Process Scoring Band Descriptors.
An organization in band 6 for process items typically demonstrates refined approaches
generally responsive to the multiple Criteria requirements. These approaches are characterized
by the use of key measures, good deployment, and innovation in most areas. Organizational
learning, including innovation and sharing of best practices, is a key management tool, and
there is some integration of approaches with current and future organizational needs.
a. The most important strengths or outstanding practices (of potential value to other
organizations) identified in the City’s response to process items are as follows:
• The City ensures a focus on success now and in the future through a mature, well-
deployed Leadership System (Figure P.2-2) that aligns the organization to support Plan
FC, a 25-year vision for the City that was developed through public input and city
leadership. As part of the Leadership System, strategic objectives and action plans are
identified through the biennial strategic planning process (SPP) to achieve the long-term
vision of the organization. Systematic processes for resource allocation, workforce
engagement, and performance analysis and improvement support plan achievement. To
ensure long-term benefit to the community, the City has invested in resources such as
those for flood mitigation and community-wide emergency preparedness, as well as
investing in community assets such as bicycling paths and hiking trails that are focused
on supporting healthy lifestyles. The City tracks its progress to planned actions through
Strategy MAPs, Quarterly Service Area Reviews (QSARs), and financial Monthly
Operating Reports (MORs), with analysis occurring at weekly, monthly, quarterly, and
annual intervals. Use of systematic processes to guide, monitor, and respond to the
needs of key customers and stakeholders may support the City’s long-term vision of
providing world-class municipal services through operational excellence and a culture of
innovation.
• The Budgeting for Outcomes (BFO) process helps the City deliver value and results by
balancing priorities for community stakeholders. Through BFO, the City ensures that
financial and other resources are available to support key strategic initiatives while it
still meets its ongoing obligations. Once strategic objectives are determined through the
SPP, the BFO “bidding” process invites the various service areas throughout the
organization to submit “offers,” competing with other service areas for funding. These
offers are posted publicly to enable input from residents and businesses, and the city
council ultimately votes on budgeting priorities, balancing the needs of citizens and
other stakeholders. Successful “bidders” are then required to develop initiatives to
achieve the proposed outcomes, including workforce- and supply-chain-related plans.
Malcolm Baldrige National Quality Award—2017 Feedback Report 6
BFO was implemented in 2005 and has since become increasingly integrated with Plan
FC and the SPP, as well as becoming increasingly transparent through the BFO scorecard
to support accountability. The BFO process is aligned with the City’s Strategy MAPs and
QSARs to ensure its full deployment and integration with daily operations.
• Customer-focused excellence is evident in the seven key Outcome Areas (Figure P.1-1),
which provide the City with a framework to align its complex operating environment
with the competing demands of its community. Key requirements of residential and
business customers are aligned with the City’s key work and support processes in each
of its service-delivery areas. Strategic objectives are developed for each of the seven
Outcome Areas, which then cascade to service areas and departments through the BFO
and QSAR processes, and to individual employee goals through the Quarterly
Performance Alignment (QPA) process established in 2015. Various leadership
communication methods and customer listening approaches provide feedback on the
City’s performance within each of the seven Outcome Areas, while citizen involvement
on 27 advisory boards and commissions provides support to the city council. The seven
Outcome Areas provide the City with a common set of objectives that operationalize its
mission and vision, helping to engage the community and provide transparency in
support of its “Triple Bottom Line” approach for considering the environment, social,
and economic aspects of decision making.
• Senior leaders reflect visionary leadership in setting the direction for the future state of
the City based on their co-creation model of governance with the community, which
promotes collaboration and transparency in government actions, stakeholder
communications, strategic partnerships, and trusting relationships. Community
members interact with the City’s various boards and commissions, the city council, and
the Executive Lead Team (ELT) in identifying issues, conducting short- and long-term
planning, and making decisions on issues of concern to community residents. Through
interactions over time, community residents have come to expect timely, respectful,
authentic, and transparent responses from the City to their concerns in each venue.
Leaders also inspire and cascade the organizational values through outreach activities in
the city’s neighborhoods, including the use of community policing for building trust,
citizen participation on BFO teams, and public access to city government information via
the transparent Community Dashboard and Access FC app. The City’s leadership reflects
a clear commitment to all stakeholders of the community.
b. The most significant opportunities, concerns, or vulnerabilities identified in the City’s
response to process items are as follows:
• Organizational learning is not embedded in key systems and processes of the City. In
particular, it is not evident that processes in the areas of leadership communications,
voice-of-the-customer (VOC) listening methods, workforce, or operational cost-control
and safety are systematically evaluated for learning and improvement opportunities.
Additionally, systematic cycles of learning are not evident in the design of key work
Malcolm Baldrige National Quality Award—2017 Feedback Report 7
processes; use of projections, comparisons, and benchmarks for performance measures;
strategic objectives and initiatives; and approaches such as QPA, QSAR, and “Hot, Help
and Well.” In regard to several changes and improvements that have been made in the
City’s processes, it is not evident that they occurred as a result of systematic evaluation
and improvement, evaluation of intelligent risks and opportunities for innovation, or
benchmarking. Ensuring systematic organizational learning may allow the City to
achieve higher levels of return on its invested resources, in order to deliver greater
value to its engaged, supportive community.
Key Themes–Results Items
The City scored in band 5 for results items (7.1–7.5). For an explanation of the results scoring
bands, please refer to Figure 6b, Results Scoring Band Descriptors.
For an organization in band 5 for results items, results typically address most key
customer/stakeholder, market, and process requirements, and they demonstrate areas of
strength against relevant comparisons and/or benchmarks. Beneficial trends and/or good
performance are reported for most areas of importance to the overall Criteria requirements
and the accomplishment of the organization’s mission.
c. Considering the City’s key business/organization factors, the most significant strengths
found in response to results items are as follows:
• Several results for key indicators support the City’s vision to provide world-class
municipal services, including good-to-excellent levels of customer satisfaction. The City
ranks in the top 10% of the nation as a desirable place to live and to work, and it ranks in
the top 1% against national comparisons for welcoming and listening to its citizens. In
addition, measures of customer satisfaction with community amenities show favorable
performance levels and trends against relevant comparisons, including results for the
quality of culture and recreation, natural areas and open spaces, and ease of traveling
by public transportation and bicycle. Among national recognition received, the City has
been designated sixth among Best Places to Live, second among Best Cities for Small
Business, eighth as a Best Performing City, and fourth as a community promoting
residents’ overall well-being. Such results highlighting the City’s customer-focused
excellence may help it leverage its strategic advantage of having a supportive and
engaged community.
• The City’s focus on delivering value and results is reflected in indicators related to key
customer requirements and environmental health, which demonstrate good-to-
excellent results levels and trends, as well as showing leadership against regional and
national comparisons. For example, electricity costs for both business and residential
customers of the City are among the lowest in the state, while results for the reliability
of the City’s electric utility services demonstrate favorable improvement trends over the
past three years. In addition, the City ranks first in the nation for citizen satisfaction with
water quality, while its results for water conservation have favorably doubled over the
Malcolm Baldrige National Quality Award—2017 Feedback Report 8
past two years. Likewise, the City ranks first among regional (Front Range) comparisons
for air quality, with favorable improvement trends demonstrated for indicators of
outdoor air quality particulate, pollution prevention, and abatement of greenhouse
gases. These results reflect the City’s visionary leadership in regard to commitment to
the community for the future and align with the organization’s values of outstanding
service and stewardship.
• The City’s financial results reflect its concern for ensuring success now and in the future
through fiscal stewardship and efforts to maintain a sustainable organization in spite of
fluctuating revenue streams. The City has been able to effectively manage and grow
revenue while contributing to the long-term economic vitality of the area; for example,
City revenue has exceeded the budget over the past five years, with increases in sales
and use tax income supported by the success of six different tax referendums passed by
community voters. In addition, results related to the accuracy of budgetary expenses
have improved over the same period, helping the City create a financial surplus during
this time. The City’s financial performance, including liquidity results that outperform
the U.S. city median comparison, contributes to its high credit rating by Moody’s.
Continuing to ensure the availability of financial resources to fund its action plans and
strategic objectives may help the City strengthen its core competency of commitment to
the community.
d. Considering the City’s key business/organization factors, the most significant
opportunities, vulnerabilities, and/or gaps (related to data, comparisons, linkages) found
in response to results items are as follows:
• Several expected results that may be important to the City are not included for
evaluation. For example, results related to supply-chain management—such as on-time
delivery, fair and competitive pricing, and quality of services—are missing or limited. In
addition, some expected marketplace results are missing, such as those for new
programs or services offered to the homeless community and programs designed to
advance the small and mid-sized business communities. Some workforce-focused results
are also missing, including results related to needed skills, staffing levels, and
recognition for workforce members. Further, results for indicators of customer
dissatisfaction and for some indicators of effective leadership and ethical behavior are
missing. As the City develops data and information to track results in these areas,
opportunities for greater alignment, collaboration, and integration may become
apparent that could contribute to providing a higher return on invested resources from
the City’s supportive and engaged community.
• The City may benefit from comparisons and benchmarks that better reflect its strategic
context and its vision to achieve world-class levels of performance. For example, the City
has an opportunity to compare segmented results for customer satisfaction and
engagement to those of local surrounding municipalities with whom the City competes
for sales and lodging tax revenues and economic development and expansion
Malcolm Baldrige National Quality Award—2017 Feedback Report 9
opportunities. Additionally, benchmarks and comparisons are missing for workforce-
focused results such as turnover rates and days to fill positions, which may help the City
identify improvements and innovations to achieve its key workforce plans of hiring and
retaining high-quality staff members with appropriate capabilities. Further, it is not
evident that the City uses comparisons to industry high performance, such as “top
decile” or “best in class,” in its performance measures. Use of world-class comparisons
and benchmarks may support the culture of excellence and continuous improvement
driven by the City’s mission, vision, and values.
• Some results for key indicators of performance aligned with the seven key Outcome
Areas demonstrate adverse or inconsistent trends. In particular, for the Outcome Areas
of Safe Community, Transportation, High Performing Government, and Culture &
Recreation, these include results for Commercial Vacancy Rates, Development Review
Final Plans, My Role and the City’s Mission and Vision, Overall Turnover, and Volunteer
Hours. Other results measures showing mixed or adverse trends include Police
Response Time, Police Officer Training, and Fleet Maintenance and Recreation
Participation. These results highlight opportunities for the City to leverage its
performance improvement system to support its vision of operational excellence and its
culture of innovation.
Malcolm Baldrige National Quality Award—2017 Feedback Report 10
DETAILS OF STRENGTHS AND OPPORTUNITIES FOR IMPROVEMENT
The numbers and letters preceding each comment indicate the Criteria item requirements to
which the comment refers. Not every Criteria requirement will have a corresponding comment;
rather, these comments were deemed the most significant by a team of examiners.
Category 1 Leadership
1.1 Senior Leadership
Your score in this Criteria item for the Consensus Review is in the 70–85 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• c Senior leaders create an environment for success and a focus on action through
the Leadership System (Figure P.2-2) while encouraging innovation and intelligent risk
taking to increase value for the City’s stakeholders. The strategic planning and BFO
processes cascade a focus on action through Strategy MAPs to work teams and
employees, with this focus further supported by quarterly QPAs and QSARs and
Community Dashboards and financial MORs. These systematic processes facilitate
stewardship and may enable the City to more rapidly attain its vision of world-class
municipal services through operational excellence and a culture of innovation.
• a(2) The ELT uses multiple approaches to demonstrate leaders’ commitment to a legal
and ethical environment, including use of open records and data, the BFO process,
presentations at new-employee orientation, the “Raise the Bar” program, the ELT ethics
360 assessment, a hotline, an ethics policy and training, and the Citizen Review Board.
These approaches promoting ethical and legal behavior reinforce the City’s core value of
integrity in the eyes of community members served.
• a(1) Senior leaders deploy the City’s vision and values to the workforce, suppliers,
partners, and customers through various communication methods, as well as through
the personal commitment of senior leaders to world-class service. Outstanding ELT
service is reflected in such practices as direct follow-up by leaders on citizens’ concerns
and the ELT’s transparency regarding its open 360-degree feedback process. Senior
leaders also personally model the organization’s values as a foundation for the
workforce culture. This may help the City advance exceptional service across the
organization.
OPPORTUNITIES FOR IMPROVEMENT
• a(1) There is no evidence of a systematic approach for senior leaders to deploy the
City’s vision and values to volunteers, and the City does not systematically evaluate and
improve its methods for deploying the mission, vision and values (MVV; Figure 1.1-2).
Malcolm Baldrige National Quality Award—2017 Feedback Report 11
Ensuring that all stakeholders understand the organization’s MVV through a systematic
approach that includes volunteers may strengthen their relationships with the City and
reinforce the City’s core competency of community commitment.
• b The City’s Communication Methods (Figure 1.1-3) are not systematically evaluated
to ensure understanding of what is being conveyed or to improve their effectiveness.
Ensuring effective communication to all stakeholders through systematic evaluation
may advance excellence and support the City’s core competency of commitment to the
community.
Malcolm Baldrige National Quality Award—2017 Feedback Report 12
1.2 Governance and Societal Responsibilities
Your score in this Criteria item for the Consensus Review is in the 70–85 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• a(1) The City reflects responsible governance for its residents and businesses
through accountability, transparency in all transactions, and frequent stakeholder
interactions. The city council’s accountability is supported through quarterly work
plan reports, internal and external audits, and annual financial disclosures. Use of
these systematic processes may enhance governance and further earn the public trust.
• b(2) To promote and ensure ethical behavior, the City deploys systematic processes
such as training on ethics for leaders, Public Purchasing Ethics policies and procedures,
the “Raise the Bar” ethics program, and internal and external employee accountability
reviews. This systematic approach focused on ethical behavior in areas of importance to
customers may help the City advance public trust among community members.
• c(1) The City uses multiple approaches to integrate societal well-being and benefit at
strategic and operational levels. The Sustainability Assessment Tool (SAT) helps the City
balance competing priorities as they arise among the “triple bottom line” of
environmental, social, and economic systems. The Sustainability Assessment Summary
(SAS) provides a mechanism to involve the city council in key issues and opportunities.
These approaches support the City’s stewardship value and vision of operational
excellence.
• c(2) Senior leaders contribute to the City’s identified key communities of residents
and businesses through their participation on local boards and in professional, through
their volunteering in schools and social organizations, and through police involvement in
the Shop With a Cop initiative. Other methods that the City’s leadership has established
to encourage community support and assist key communities include allowing four
hours of paid time for employee volunteering on Volunteer Day and recently hiring a
community liaison to support access to City services in low-income neighborhoods.
Outreach service efforts targeted to the needs of key communities help demonstrate
the City’s commitment to the community through engaged and committed employees.
OPPORTUNITIES FOR IMPROVEMENT
• c(2) There is no evidence of a process to systematically measure and evaluate the
City’s support for its key communities. For example, the City supports multiple
community interests through an enterprise relationship with the United Way, while
individuals support these and other interests for community benefit; there is no
evidence that these activities align to strategic areas of interest within the organization
Malcolm Baldrige National Quality Award—2017 Feedback Report 13
or the City’s focus on the “triple bottom line.” Having a systematic process in this area
could help the City prioritize its support efforts in areas targeted most directly to its
strategic goals or direction.
Malcolm Baldrige National Quality Award—2017 Feedback Report 14
Category 2 Strategy
2.1 Strategy Development
Your score in this Criteria item for the Consensus Review is in the 70–85 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• a(1) The City’s systematic, seven-step Strategic Planning Process (SPP) (Figure 2.1-2)
encompasses a two-year cycle led by a Strategic Plan (SP) Core Team with input from
many stakeholders. The SPP shapes the City’s biennial budget through its BFO process,
and cycles of improvement have added a priority dashboard, updated risk
assessments, and community comment sessions. The SPP guides the City’s strategic
priorities and use of resources, supporting long-term goal achievement of the City’s
mission of providing exceptional service for an exceptional community.
• a(2) The City uses its Intelligent Risk Process (Figure 2.1-4) and competitive “budget
offer” process (the BFO) to stimulate innovation. The Intelligent Risk Process guides
leaders’ identification and support for innovation. The BFO process is guided by the
biennial SPP and supports SP priorities. Beyond the SPP, interdisciplinary teams can
submit innovative ideas in relation to a reserved $50,000 fund that is earmarked to
support valuable new ideas. Such engagement of the workforce and leadership at all
levels in stimulating innovation helps the City reinforce its values of excellence and
stewardship through innovation in action.
• a(3) The City has many inputs to its SPP (Figure 2.1-3), including an environmental
assessment, extensive financial analysis and projections, and a strategic risk
management (SRM) process. Leaders have expanded the SPP inputs through cycles of
learning to help identify changes in the regulatory environment, develop strategic
objectives, identify strategic challenges, and reduce blind spots. Analyzing multiple data
sources and findings as part of the SPP may help the City leverage its resources toward
infrastructure and amenities, strengthening a strategic advantage.
• b(2) The City’s process to balance and allocate resources among competing
organizational needs aligns with strategic objectives for the seven Outcome Areas and
supports citizen input. Budget “offers” compete for available dollars, with potential
offers posted for citizen and business input. Votes by the city council determine shorter-
term and longer-term planning horizons and balance citizen and stakeholder needs. This
balancing process helps the City address the strategic challenges of a fluctuating
revenue stream and changing customer expectations.
Malcolm Baldrige National Quality Award—2017 Feedback Report 15
OPPORTUNITIES FOR IMPROVEMENT
• a(1) The various service areas within the City often have their own planning
processes, and the service-area plans in some cases are not aligned with the biennial
Strategic Plan of the City. Integration of planning processes may assist the City in
reaching its vision to provide world-class municipal services through operational
excellence and a culture of innovation.
Malcolm Baldrige National Quality Award—2017 Feedback Report 16
2.2 Strategy Implementation
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• a(3) The City uses the BFO process to allocate assets and ensure that financial and
other resources are available to support achievement of the various funded initiatives
while the City also meets all current financial obligations. Financial risks are managed
through daily monitoring of division budgets and analysis of financial reports against
projections. This approach for ensuring resource availability to achieve key initiatives
helps the City balance its strategic challenges of competing citizen desires and
changing customer expectations amid fluctuating revenue streams.
• a(1, 2) Systematic development and deployment of action plans is accomplished
through the BFO process. Various budget offers are submitted as means of
accomplishing the specific strategic objectives within the Strategic Plan. Action plans,
which the City calls “initiatives,” are developed within the approved “budget offers,”
following a competitive review. Submitted budget offers are reviewed by committees
responsible for putting budget offers within a single service area into priority order. The
recommended budget offers are then reviewed by senior leaders, who determine which
offers to submit to the city council in the city administrator’s recommended budget. Use
of the SPP and the BFO process to guide deployment of key initiatives support’s the
City’s vision of world-class municipal services.
OPPORTUNITIES FOR IMPROVEMENT
• a(1, 2) There is no evidence of an approach to prioritize strategic objectives or to
systematically track measurable progress in achieving strategic objectives. Use of a
systematic approach to prioritize objectives with measurable outcomes may help the
City focus limited resources on areas of greatest impact to the community through
operational excellence and a culture of innovation.
• a(6) The City does not have performance projections for most of its shorter- and
longer-term horizon measures, as seen in the Community Scorecard, and few
performance measures and indicators reflect systematic use of comparisons with key
competitors or benchmarks. Use of a systematic process to evaluate and improve
performance measures in comparison to projected targets, benchmarks, and top-
performing competitors may help the City advance its performance levels in alignment
with its vision of world-class municipal services.
• a(5) Some listed performance measures in the Community Scorecard do not align with
the strategic objectives and the initiatives set out in the funded budget offers. As an
Malcolm Baldrige National Quality Award—2017 Feedback Report 17
example, “Safe Community” initiatives funded in the current budget include “(1) Police
Personal Protective Equipment and (2) SWAT replacement Negotiator’s Console,” and
the key related performance measures are “Part 1 Crimes per Capita Compared to Cities
of Similar Size” and “Average quarterly response time of priority one calls.” Alignment of
the funded initiatives with relevant performance measures may assist the City in
identifying performance problems and support cycles of improvement for a culture of
operational excellence.
Malcolm Baldrige National Quality Award—2017 Feedback Report 18
Category 3 Customers
3.1 Voice of the Customer
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in an increased percentage range of 70–85. (Please refer to
Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• a(1) The City has a systematic approach to select, collect, align, and integrate
customer listening methods (Figure 3.1-2). Through cycles of evaluation and
refinement in 2013, the City launched the Community Dashboard and the Peer City
Project; in 2014, the City established the Community Outreach and BFO processes and
hired a civic engagement officer; in 2015, improvements included customer
segmentation of new activities in utilities, policing services, social media, and a
mobile-friendly website. The completed Public Engagement Strategic Plan led to an
IAP2 process implementation, and in 2017, to the “It’s YourMyOur City” campaign
(Figure 3.1-1).
• b(1) The organization selects and uses multiple VOC listening methods such as
customer-focused surveys, face-to-face communications, community outreach, market
data, and social media to build a more customer-centric culture and support operational
decision making. Results are shared with the city council, service leaders, staff members,
and customers in the municipality. Collectively, these approaches to gather, evaluate,
and improve the residential experience support the organization’s customer-centered
focus, reinforce its values of outstanding service and integrity, and may expand
relationships with customers and give the City a competitive market advantage for
future growth.
OPPORTUNITIES FOR IMPROVEMENT
• b(1) The City does not systematically determine customer dissatisfaction or identify
best practices for learning and improving based on customers’ dissatisfaction
experiences. When satisfaction concerns are identified within service areas, processes
for managing dissatisfaction are not systematically deployed or shared across the
organization. A systematic approach to determining and managing customer
dissatisfaction across service areas, leveraging best-practice sharing, may enhance
customer engagement by demonstrating the City’s commitment to the community.
• a, b(2) The City does not systematically obtain and use data to evaluate customer
satisfaction and dissatisfaction across the customer lifecycle and relative to competitors
with similar products and services, in order to advance organizational performance. For
example, information about aspects of services that are not meeting customer
expectations is not routinely reviewed to support future customer satisfaction. Without
Malcolm Baldrige National Quality Award—2017 Feedback Report 19
a systematic process for data-driven decision making and improvement in these areas,
the City may limit its ability to overcome the strategic challenges of balancing competing
citizen desires and changing customer expectations.
Malcolm Baldrige National Quality Award—2017 Feedback Report 20
3.2 Customer Engagement
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in an increased percentage range of 70–85. (Please refer to
Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• b(1) The City’s approaches to engaging customers and building relationships reflect
its commitment to the community. Through cycles of learning, the City launched a
community outreach program. It also hiring a civic engagement officer and a
community liaison to focus on building relationships. In addition, the City’s police
service created the neighborhood-focused Engagement Team. The completion of the
Public Engagement Strategic Plan led to implementation of the IAP2 process and to
initiation in 2017 of the “It’s YourMyOur City” campaign (Figure 3.1-1) to work toward
co-creation and increase perceived value of City services.
• a(1) The organization uses a systems approach to capture customer-related
information to learn about customer requirements, expectations, and desires. Through
cycles of evaluation and learning, many programs have been adopted to engage
customers throughout the full life cycle of the relationship. The City uses technology,
social media, and mobile-friendly websites to give the public access to its offerings.
These approaches to meeting and exceeding customer expectations help the City build
customer loyalty.
• b(2) The City has a systematic process in place for collecting information from
residents on issues of concern and transferring those to the appropriate service. For
example, community policing officers develop one-on-one relationships with citizens
and serve as a resource for residents’ issues and concerns. Information collected from
residents is reviewed, with results trends monitored. Use of a systematic process for
identifying, responding, and resolving customer concerns supports the City’s value of
outstanding service and promotes customer engagement.
OPPORTUNITIES FOR IMPROVEMENT
• a(3) Customer segments are not readily identified or prioritized for specific actions
within service areas, for resident or business customer process improvement, or for
business growth. Use of systematic approaches to identify and segment information by
customer groups may help the City enhance residents’ experience and develop new
offerings to serve new customers.
• b(2) The organization does not systematically manage complaint resolution.
Complaints are identified within service units, and a systematic process to identify
factors causing complaints or to prioritize them for action is not evident. Efforts to
resolve issues and learn from them to improve customer-focused processes are not
Malcolm Baldrige National Quality Award—2017 Feedback Report 21
systematically integrated across the City, with the exception of the city council.
Systematic approaches to identify, refine, and enhance complaint management
processes may advance customer engagement while supporting the organizational
value of service excellence.
Malcolm Baldrige National Quality Award—2017 Feedback Report 22
Category 4 Measurement, Analysis, and Knowledge Management
4.1 Measurement, Analysis, and Improvement of Organizational Performance
Your score in this Criteria item for the Consensus Review is in the 70–85 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• a(1) Key performance measures are tracked at varied frequencies based on measure
type and Outcome Area, guiding the City’s performance to goals. Measures and targets
are evident in the seven Outcome Areas (Figure 4.1-2), while multiple measurement
review methods (Figure 4.1-3) provide mechanisms to act on results. Tracking tools
include Strategy MAPs (2017), QSARs (2015), the financial MOR (2012), and Community
Dashboards (updated in 2016). On a two-year cycle, the budget director meets with
department heads and tracks BFO offers as well as the QSAR process. The Futures
Committee approves the measures, which are then reported on the Community
Dashboard. These well-deployed measurement tools and processes may strengthen the
City’s data-driven culture and support its vision of operational excellence.
• a(3) The City uses multiple approaches to select customer data on its key resident and
business customer groups, and these efforts are aligned to the seven Outcome Areas.
Examples include benchmarked surveys, inquiry and complaint analysis, involvement of
the city council, tracking of and responses to social media comments, and community
outreach. In 2013–2014, the City’s SPP-related community outreach for hard-to-reach
population groups included objectives such as providing safe, accessible housing and
transit access. Another example of a response is a tax rebate program established to
ease financial burdens on low-income residents. The City’s approaches to gathering and
responding to VOC data may help it assess and support the “triple bottom line”
objectives of economic, environmental, and social equity.
• b The City assesses its progress and responds to rapidly changing needs through
multiple methods of reviewing its performance for key measures (Figure 4.1-3) at
varying frequencies based on the nature and use of the information. Staffing, revenue,
and productivity measures are evaluated daily, along with customer complaints and
feedback as received. QSAR, Strategy MAPs, and financial MOR reports are reviewed
monthly, with actions taken at weekly ELT and city council meetings. In the process of
benchmarking, the budget office looks at each of the measures and comparison data to
assess the appropriateness of the comparisons. Timely review and action on key
performance measures helps the organization optimize its processes for effectiveness,
supporting operational excellence.
• b Senior leaders review key data and information to assess organizational success,
competitive performance, financial health, and progress on achieving strategic
Malcolm Baldrige National Quality Award—2017 Feedback Report 23
objectives. The city council also reviews the organization’s performance and its progress
on strategic objectives. ELT members identify and select measures by testing whether
the measures are meaningful (adequate) and valuable (in driving improvement). The city
council’s Futures Committee approves the measures, which are then reported on the
Community Dashboard. Daily measures come from input to the citizens survey. These
processes may support and enhance the City’s culture and align the MVV to individual
goals.
OPPORTUNITIES FOR IMPROVEMENT
• c(2) The City lacks a systematic process to deploy its priorities for improvement and
opportunities for innovation to suppliers, partners, and collaborators. In regard to direct
communications by employees with suppliers, partners, and collaborators, there is no
evidence of a process to align and use this communication to engage suppliers and
partners in process improvements or innovation. Use of a systematic approach to
deploy priorities and opportunities for innovation to suppliers, partners, and
collaborators may help the City optimize results across all outcome areas.
Malcolm Baldrige National Quality Award—2017 Feedback Report 24
4.2 Information and Knowledge Management
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• b(1) Multiple approaches to collect and transfer knowledge and data across the
workforce include a centralized IT department and system with a City Connect webpage,
a learning and development system, a data repository for policies and meetings
information, as well as more traditional approaches such as guidebooks and checklists.
For example, employees report that the “Fort Shorts” e-newsletter, the “Issues and
Answers” streamed briefings, email updates, and many face-to-face meetings with
departmental staff members, supervisors, and divisions keep them informed of key
information. Use of such enterprise-wide tools may help the City bridge communication
and processes across multiple departments and workforce areas, while supporting clear
expectations, meaningful work, growth opportunities, and role accountability.
• b(2) Multiple approaches to promote and share best practices include intentionally
bringing together leaders from all service units to share best practices across the
organization and sharing of such practices at the All Supervisors planning team. For
example, the Safety Team recognized that some City employees were being issued
chainsaws, without having received training, for removal of broken branches after a
heavy snowfall; given that the arborists had been certified in chainsaw use, a best
practice, arborists then trained employees on safe and efficient use of a chainsaw. Best
practices are also identified through City participation in trade organizations such as
ICMA, attendance at conferences such as Transforming Local Government and the
Baldrige Program’s Quest for Excellence, visits to Baldrige Award recipient organizations,
and employee blogs. These multiple approaches to share best practices may help the
City continually improve its processes and support a culture of innovation.
• a(2) The City uses multiple approaches to ensure data and information availability
(Figure 4.2-1), including user-friendly IT system formats and menus, standardized
reporting tools and formats, and Internet access to the Enterprise Resource Planning
(ERP) system and other website tools. Customers, stakeholders, partners, suppliers, and
employees can access extensive, role-specific information and services via a web-based
system at fcgov.com using any type of electronic device with Internet access. The Access
Fort Collins app provides a convenient method for employees and residents to report
issues within the City. The IT department provides rapid responses to requests for help,
and customer satisfaction with this service is high. Leveraging key technology assets to
produce timely, relevant information helps the organization meet its customers’ needs
and expectations regarding the availability of information.
Malcolm Baldrige National Quality Award—2017 Feedback Report 25
OPPORTUNITIES FOR IMPROVEMENT
• b(3) There is no evidence that the City’s multiple approaches to embed and advance
learning are systematically deployed and integrated across the organization. For
example, it is not evident that learning related to the QPA, QSAR, and “Hot, Help &
Well” are leveraged to systematically advance process performance and results. In
addition, “learning” is often being interpreted solely as leadership development.
Deployment and integration of systematic approaches to embed and advance processes
of evaluation and improvement may help the City attract and retain high-quality
employees while advancing a culture of operational excellence and innovation.
• b(1) A systematic process to guide the transfer of relevant knowledge to suppliers,
partners, and collaborators is not evident. The multiple approaches used do not
consistently transfer knowledge to and from involved parties, and there is no evidence
that they are evaluated and improved over time. Systematic, well-deployed processes
for sharing knowledge with suppliers, partners, and collaborators may strengthen
mutual support and relationships, building community value.
Malcolm Baldrige National Quality Award—2017 Feedback Report 26
Category 5 Workforce
5.1 Workforce Environment
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in an increased percentage range of 70–85. (Please refer to
Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• a(1) The City uses systematic, well-deployed approaches for assessing workforce
capability and capacity needs. Through the BFO process, service areas and departments
analyze needed staffing levels and skill sets as they develop their “offers” for service
delivery, with seasonal workers and volunteers included and monitoring occurring
through the QSAR process. In service areas such as Police, Parks and Utilities,
determination of workforce capacity is fact-based, using ratios based on population,
acreage, and demand. The BFO offers also include needed capabilities identified through
the SPP. Individual capabilities are assessed as part of the QPA process, and
development opportunities are established as QPA goals, as appropriate. These
approaches help the City ensure that needed skills sets and resources are available to
provide exceptional service for an exceptional community.
• a(2) The City’s nine-step recruiting and hiring process (Figure 5.1-2) for recruiting,
selecting, and hiring new workforce members. An integrated talent management
software system, branded as FC Career Connect, was implemented in 2014 to increase
the efficiency and effectiveness of the process, which includes an applicant tracking
module and automated screening. Group interviews are often used to ensure
prospective workforce members will be a good fit with the organization’s culture and
values. New-employee orientation training is designed to retain new hires by engaging
them in the culture, values, mission, and vision. A dedicated position for talent manager
was added in 2017 to further enhance and manage the process. This series of integrated
processes may help the City address the strategic challenge of attracting and retaining
high-quality employees.
• a(3) Systematic approaches to prepare the workforce for changing capability and
capacity needs help the City maintain its strategic advantage of engaged and committed
employees. Planned changes are identified through the SPP and BFO processes,
affording the City the opportunity to communicate with affected employees and to
create development plans to redirect them into other available positions across the
organization if a reduction is needed, or to acquire needed workforce skill sets should
growth opportunities present themselves. For unplanned changes, a systematic
Reduction in Force process was developed in 2010 to ensure open communication with
any affected employees and to minimize the impact of any needed reductions. Seasonal,
contract, and part-time hourly employees are used to manage fluctuating demands.
Malcolm Baldrige National Quality Award—2017 Feedback Report 27
• b(2) The City developed its Total Rewards Strategy in response to the comprehensive
Compensation and Career Path Assessment conducted in 2014. This framework
encompasses a diverse suite of benefits, services, and policies to support varied needs
and desires of the City’s workforce. One of the centerpieces of the Total Rewards
framework is the wellness program, for which the City received the Governor’s Award
for Worksite Wellness as a Top 5 finalist. The wellness program addresses a variety of
healthy lifestyle choices and includes health screenings and other offerings that have
been attributed to 28% lower health claims for program participants relative to non-
participants. The Total Rewards Strategy is designed to attract, retain, and engage high-
quality employees, which supports the City’s vision of high performance.
OPPORTUNITIES FOR IMPROVEMENT
• a(1, 4), b(2) The City may benefit from a more fact-based approach to evaluating the
effectiveness of some of its approaches related to workforce capability and capacity,
work accomplishment, and workforce benefits and policies. Despite evidence that some
degree of evaluation takes place, there is little indication that the evaluation process is
supported to any significant degree with data-driven decision making. Evaluating these
key workforce processes in a fact-based manner may demonstrate the City’s value of
stewardship and enhance its ability to deliver world-class services through operational
excellence.
• a(2) The City lacks a systematic approach to ensure that the workforce represents the
diverse ideas, cultures, and thinking of the hiring and customer community. In regard to
the multiple sourcing options that the City uses as part of the hiring process to attract
potential qualified candidates, there is no element within that approach to ensure that
the workforce is representative of the City’s community. As a result, The City may be
missing opportunities to more fully engage the residents and businesses it serves.
Malcolm Baldrige National Quality Award—2017 Feedback Report 28
5.2 Workforce Engagement
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in an increased percentage range of 70–85. (Please refer to
Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• a(3) The City’s multiple formal and informal approaches and mechanisms for assessing
workforce engagement may help it identify opportunities to strengthen its strategic
advantage of engaged and committed employees. Until 2017, the Q14 workforce
engagement survey was deployed to all employees twice a year; through a cycle of
improvement, the City moved to the Core 34 workforce survey in order to gain more
beneficial comparisons and increase insight into drivers of engagement. A volunteer
engagement survey was also implemented in 2017. Workforce survey results are
analyzed at the departmental and service-area levels and deployed through the QSAR
process and leadership communication mechanisms. Indicators of workforce
engagement are also identified through QPA discussions, through exit interviews when
turnover occurs, and through participation in various events and programs offered.
• b(3) The City supports career progression through a suite of leadership development
programs. In 2011, Lead 1.0 was piloted to accelerate the development of emerging
leaders, and it has resulted in participants receiving promotions within 24 months of
graduation. In the same year, Lead 4.0 was implemented to improve the effectiveness of
senior leaders and service area directors, and recently Lead 2.0 and 3.0 have been
developed to support frontline and department leaders. All Lead program participants
are assigned a designated coach as well as receiving DiSC profiling and training to
accelerate their learning and development. Behavioral change is measured by pre- and
post-360 degree evaluations and ratings on Upward Feedback from direct reports.
These approaches may help the City address its strategic challenge of retaining high-
quality employees and improve overall engagement.
• a(1) The City fosters an organizational culture that is characterized by open
communication, high performance, and an engaged workforce through the QPA process
and leadership communication mechanisms. Through the QPA process, employees are
engaged in the mission and vision of the organization by setting individual goals that
align with strategic objectives and BFO offers. Each quarter, employees and their direct
supervisors evaluate employees’ progress toward achieving the goals, then bring those
evaluations together in an open discussion. Senior leaders’ communication mechanisms
(Figure 1.1-3) provide for transparency in relation to the City’s goals, objectives, and
results. Through the QPA and leadership communication approaches, employees may
be empowered to assume a degree of ownership of their performance and
development, reflecting the organization’s value of collaboration.
Malcolm Baldrige National Quality Award—2017 Feedback Report 29
• b(1) Multiple approaches are used to evaluate the effectiveness of the learning and
development system. The Kirkpatrick model provides evaluation against objectives for
individual development offerings at the enterprise level, and the workforce engagement
survey reflects satisfaction with availability. Leadership training, which commands more
than two-thirds of the current development budget, is evaluated using a financial ROI
assessment. At an operational level, learning and development is evaluated through
QPA, as most training is focused on enhancing job skills for key operational processes.
The learning and development system has undergone multiple cycles of improvement
since its origin in 2002, including implementation of FC Career Connect in 2014 to
improve the efficiency of administration processes and the current four-quadrant
framework to align with key organizational needs.
OPPORTUNITIES FOR IMPROVEMENT
• a(2) The City has limited approaches for determining drivers of workforce
engagement. The current set of engagement factors (P.1-3) were identified in 2012
based on focus group input from a cross-section of the workforce. These drivers have
not been updated since that time, nor have they been refined to address unique
engagement drivers for different workforce segments. This improvement opportunity
was highlighted in 2017 when the Core 34 employee survey was implemented.
Additional potential workforce engagement drivers were identified from the survey
results, including drivers for specific workforce segments, such as those based on
generational preferences. An effective approach to determining and periodically
evaluating the drivers of workforce engagement may enable the City to make targeted
adjustments and improvements to ensure workforce talent retention and growth.
• a(4) The City is in the early stages of considering workforce compensation, rewards,
recognition, and incentive practices within its workforce performance management
system. In response to a Career Path and Compensation Assessment conducted in late
2014, the City identified priorities for improvement in workforce compensation,
rewards, and incentive practices; it is currently in the process of benchmarking its
compensation structure in relation to the competitive market to support the recruiting,
hiring, and retention of high-performing employees. Beyond these efforts, opportunities
may exist for the City to evaluate its approaches for employee recognition and non-
financial rewards to support the performance management system as an incentive for
high performance, which may accelerate achievement of the City’s strategic objectives.
• b(2) The City does not correlate its intended learning and development outcomes
with findings from evaluation of key organizational results. For example, no evaluation is
conducted to correlate learning and development outcomes with improvements in
customer engagement, work process effectiveness and efficiency, or other measures
associated with the seven Outcome Areas. Such a process may highlight opportunities
to further improve the learning and development system in order to increase the City’s
Malcolm Baldrige National Quality Award—2017 Feedback Report 30
return on invested financial resources in achieving the macro-level objectives of the
organization.
Malcolm Baldrige National Quality Award—2017 Feedback Report 31
Category 6 Operations
6.1 Work Processes
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• c The Purchasing Department systematically guides and integrates supply-chain
management across the organization. Well-defined policies and guidelines ensure a
standardized, competitive, and ethical approach to managing the supply chain. Within
the City’s RFP (request for proposals) process, cross-functional teams use a score
selection process to make decisions based on weighted criteria, and reference checks
are completed in order to round out the selection process. Poorly performing
contractors are kept informed and terminated if no improvements are made. Use of this
systematic and integrated approach may advance the City’s achievement of its key
customer requirements of high-quality, reliable services.
• a(1, 2) The City has deployed and integrated key work process design outputs from its
Work Processes and Systems (Figure 6.1-2), aligning the seven Outcome Areas to key
work systems, with integration of key support processes, process requirements, and
measures. The process design stems from the inputs of interdisciplinary teams during
the BFO phase of the SPP, including VOC feedback. Use of this systematic approach to
designing and managing work processes may effectively connect the City’s daily
operations to its mission of providing exceptional service for an exceptional community.
• a(3) The City designs its services and key work processes to meet key requirements
through the SPP and BFO processes, aligning product delivery mechanisms with
Outcome Areas and linking department leaders of key work process areas to those that
support the related Outcome Areas. Agility in relation to new products or services is
supported by the City’s process for intelligent risks. The City’s systematic and aligned
approaches to work process design help ensure that essential resources are available to
support business or residential customers’ requirements, supporting the City’s
commitment to the community.
OPPORTUNITIES FOR IMPROVEMENT
• b(3) The City has not deployed a systematic approach to process improvement to
reduce variability and improve performance results. Despite workforce members’
evident improvement orientation and references to a Plan, Do, Check, Act improvement
approach in interviews on-site, knowledge and application of this approach is not
evident across most of the organization. Ensuring organization-wide use of a systematic,
Malcolm Baldrige National Quality Award—2017 Feedback Report 32
effective approach to guide cycles of process improvement may contribute to sustained
improvement trends in measures of importance to community stakeholders.
• b(1) Early stages of use of in-process measures to control and improve work processes
are evident in relation to productivity, cycle time, reliability, and efficiency. For example,
while Fleet and Facilities, Utilities, Traffic Operations, and Police Services measure some
of these, there is no evidence of use of related data for improvement of work processes.
Additionally, a systematic approach to measure and manage work processes for
consistency and reliability is not evident. Using a systematic approach to in-process
measures may enable the City to identify early opportunities for areas of improvement,
supporting the City’s values of stewardship and excellence and helping the organization
meet key stakeholder requirements.
• d A systematic approach to pursuing strategic opportunities that the City determines
are intelligent risks is not evident. Developing such an approach could help the City
manage innovation and may enable it to move faster within competitive service markets
while advancing toward its vision of providing world-class municipal services through
operational excellence and a culture of innovation.
Malcolm Baldrige National Quality Award—2017 Feedback Report 33
6.2 Operational Effectiveness
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5a, Process Scoring Guidelines.)
STRENGTHS
• c The City uses systematic processes to address safety, emergency preparedness,
and business continuity. The City provides a safe operating environment through
accident prevention, inspection, root cause analysis, and recovery approaches, and it
uses after-action reviews to prevent recurrences. The City’s Emergency Preparedness
Plan (EOP), Office of Emergency Management (OEM), Continuity of Operations Plan
(COOP), and flood mitigation plan to conserve land on the Poudre River ensure the
City’s preparedness for emergencies and disasters and guide evaluations and
improvements. Systematic workplace safety and emergency management processes
support the City’s current and future commitment to the community.
• b(2) Multiple approaches centered on control of people, technology, and
policy/procedure vectors ensure security and cybersecurity of sensitive or privileged
information. Access controls such as strong passwords, IT application controls,
integrated cybersecurity training, and intrusion systems are deployed through a
proactive strategy to reduce cybersecurity risks. Use of multitiered approaches to
protect data and information may enable the City to mitigate operational and strategic
risks and impacts to the community and key stakeholders in the event of a data breach.
• b(1) The City’s approach to ensuring the reliability of IT systems includes organization-
wide use of a service-level agreement; daily “Scheduled Server Uptime” reports as well
as long-term planning for IT systems; and daily surveys of customer satisfaction, with
monthly reviews by department leaders. These methods are consistently deployed and
evaluated for learning and improvement. This may advance departmental performance
and accountability, increasing reliability and user satisfaction.
OPPORTUNITIES FOR IMPROVEMENT
• a Limited approaches are evident to control the costs of operational processes across
the organization through use of cycle time, productivity, and efficiency measures or
balancing of cost-control measures to meet varied customer needs. Use of a systematic
and integrated process to control the process costs of operations across the
organization may improve process efficiency and effectiveness, reflecting the City’s
values of stewardship and excellence to the community.
• c(1) Cycles of learning are not evident within the City’s multiple safety approaches
that are designed and deployed to create a safe operating environment for key
stakeholders. For example, there is no evidence that the City’s approaches to accident
Malcolm Baldrige National Quality Award—2017 Feedback Report 34
prevention, inspection, determination of root cause of failures, and recovery have been
systematically improved through regular cycles of evaluation and learning. A systematic
process for evaluation and improvement of key safety practices may improve workforce
safety and support outstanding service to customers.
Malcolm Baldrige National Quality Award—2017 Feedback Report 35
Category 7 Results
7.1 Product and Process Results
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5b, Results Scoring Guidelines.)
STRENGTHS
• a Multiple measures of significant importance to the City’s customers reflect good-to-
excellent levels of performance, with favorable trends for three or more years.
Examples are SAIDI/SAIFI/CAIDI (Figure 7.1-1); Water Quality, Dissatisfaction and
Satisfaction (Figure 7.1-2); and Fixed Route Ridership and Ease of Traveling by Public
Transportation (Figure 7.1-3). Consistently favorable performance in these key customer
requirement areas may help the City compete with rapidly growing regional
communities.
• b(2) Several City measures of safety and emergency preparedness show favorable
performance levels and trends in aspects of importance to the community. Examples
include Flood Mitigation (Figure 7.1-25), where results demonstrate future-focused
planning for the protection and preservation of green spaces against floods; and
Emergency Management Performance Grants (Figure 7.1-30), where results have
increased favorably. This beneficial performance on meaningful measures in relation to
the City’s future sustainability may help it favorably differentiate its services to support
its reputation as a desirable location to live, work, and play.
• b(1) Some results for key work process effectiveness show beneficial trends and/or
comparisons. For example, the City’s electricity costs are among the lowest in the state,
both for business and residential customers (Figures 7.1-10 and 7.1-11); water
conservation has more than doubled since 2014 (Figure 7.1-12); and citizen survey
responses related to the community’s visual attractiveness place the City in the top 5%
nationally and have created a comparative advantage since at least 2008 (Figure 7.1-15).
This performance on work process effectiveness and efficiency measures reflects the
City’s commitment to community and may contribute to meeting customer
requirements for high-quality, reliable services, accuracy, and responsiveness.
OPPORTUNITIES FOR IMPROVEMENT
• b(1) The City lacks some expected results on the effectiveness and efficiency of key
work and support processes, such as cycle time and other appropriate measures of
process effectiveness. Gathering and using in-process measures may help the City
determine whether a process is delivering the desired results and enable timely
adjustments to meet City stakeholders’ expectations of high-quality, reliable services.
Malcolm Baldrige National Quality Award—2017 Feedback Report 36
• c Some expected supply-chain management results are missing, such as results for
measures of on-time delivery, fair and competitive pricing, quality products, and
accuracy. Tracking such results in order to evaluate and continuously improve
performance in these areas may enhance the City’s ability to deliver services that meet
the requirements of customers and the City’s goals for operational excellence, and
results for indicators for supplier performance also may highlight opportunities for
improvement and innovation within key work processes.
• b Some results levels and trends reflect flat or unfavorable performance by the City in
areas of significance to customers. Examples are results for the Snow Removal Response
(Figure 7.1-18), Fleet Maintenance (Figure 7.1-19), Internet Availability (Figure 7.1-20),
and Police Response Time (Figure 7.1-27). Focusing on these areas of importance may
help the City deliver high-quality, reliable services to customers, particularly in light of
the organization’s strategic challenge of transitioning from a large town to a small city,
with associated transportation and traffic issues.
• a, b Comparisons are missing for multiple measures of results that are significant to
the City’s vision of operational excellence, such as average travel speed and community
greenhouse gas emissions. Without comparison results that could provide a benchmark
for high performance in such key measures, the City may find it difficult to engage
workforce members and teams in cycles of continuous improvement leading to world-
class performance levels.
Malcolm Baldrige National Quality Award—2017 Feedback Report 37
7.2 Customer Results
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5b, Results Scoring Guidelines.)
STRENGTHS
• a(1) Good-to-excellent performance levels are evident for measures of customer
satisfaction. For example, Fort Collins as a Place to Work (Figure 7.2-3), Quality of
Culture and Recreation (Figure 7.2-5), and Quality of Drinking Water (Figure 7.2-9) all
reflect performance levels that are consistently in the top 10% nationally from 2008 to
2015, favorable relative to comparisons with a national benchmark and cities in the
Front Range region. Such performance supports the City’s strategic advantages of a
supportive and engaged community and national recognition for the quality of life of
community members.
• a Several customer-focused results demonstrate strong performance relative to
leading national and/or regional comparisons. For example, the City ranked first in both
national and Front Range comparisons for Quality of Water (Figure 7.1-9) and Natural
Areas and Open Space (Figure 7.2-12), with results for both measures demonstrating
consistently high performance. Further, the City is in the top 10% of national and
regional comparisons for 15 out of 16 results measures in this area. These customer-
focused results reflect the City’s efforts to meet residents’ requirements for high-quality
reliable services, accuracy, and responsiveness.
• a(2) Some customer engagement results measures for residents show beneficial levels
and compare favorably to national and regional benchmarks. In particular, the measures
Ease of Traveling by Bicycle (Figure 7.2-24), Welcoming to Citizens (Figure 7.2-25), and
Listening to Citizens (Figure 7.2-26) all show sustained high performance from 2010 to
2015, with the City’s results in the top 1% or top 10% relative to both national and
regional comparisons. This high performance in relation to customer engagement helps
position the City to effectively address changing customer expectations, a strategic
challenge for the organization.
OPPORTUNITIES FOR IMPROVEMENT
• a Some expected results on customer dissatisfaction are missing. Despite the City’s
use of its website, the SARs process, and point-of-contact surveys to obtain
dissatisfaction data, no such results are evident. A focus on customer dissatisfaction
results may help the City address its strategic challenges of balancing competing
stakeholder desires and changing customer expectations.
• a Mixed performance trends are evident in some important measures of customer
satisfaction and engagement, such as Utilities Customer Satisfaction: Customer Needs
Malcolm Baldrige National Quality Award—2017 Feedback Report 38
(Figure 7.2-15), Utilities Customer Satisfaction (Figure 7.2-16), and Recreation
Participation (Figure 7.2-22). Focusing on results in areas of significance to customers
may help the City build community engagement and growth.
Malcolm Baldrige National Quality Award—2017 Feedback Report 39
7.3 Workforce Results
Your score in this Criteria item for the Consensus Review is in the 30–45 percentage range. The
findings from the site visit resulted in an increased percentage range of 50–65. (Please refer to
Figure 5b, Results Scoring Guidelines.)
STRENGTHS
• a(4) Some workforce development measures show favorable levels and trends over
five or more years. For example, Learning and Development (Figure 7.3-13) shows that
opportunities to learn increased from 60% in March 2011 to 75% in September 2016,
while five or more promotions have occurred annually as a result of participation in the
leadership development program (Lead 1.0 Program Participation, Figure 7.3-16). In
addition, over 50% of the current ELT members have been promoted as a result of
leadership development activities. These results show that the City is developing key
internal talent to provide a cost-effective workforce solution while supporting a best
cultural fit.
• a(1) Some workforce capability results show beneficial levels. For example, results
show 100% achievement from 2012 through 2015 of compliance certifications (Figure
7.3-3) that cover eight different roles (such as engineering, police, and
water/wastewater treatment), while certifications for water field operators are at 90%
for the same period. Achieving and maintaining essential workforce certifications and
licensure may assist the City in retaining and developing highly qualified workers,
providing a competitive service advantage.
• a(3) Most of the City’s workforce engagement indicators show good performance
levels and trends. For example, results for the City’s Engagement Survey (Figure 7.3-8)
show a range of 2% to 7% in improvement in the hourly worker category from
September 2011 to 2016, and favorable results levels are also shown for the Volunteer
Engagement Survey (Figure 7.3-9). In addition, measures of the employee engagement
factors of appreciation and meaningful work reflect favorable trends from 2011 to 2016,
and most results exceed national benchmarks. Achieving benchmark performance for
levels of workforce engagement may help the City leverage its strategic advantage of
engaged and committed employees in competitive labor markets.
OPPORTUNITIES FOR IMPROVEMENT
• a(1, 2, 4) Many expected workforce-focused results are missing, such as measures of
workforce health, needed skills and staffing levels, and recognition and celebrations for
employees. Tracking such results may help the City address its strategic challenge of
attracting, retaining, and developing high-quality employees.
• a Some workforce-focused performance measures show mixed, flat, or unfavorable
trends. For example, Overall Turnover (Figure 7.3-1) reflects an unfavorable trend from
Malcolm Baldrige National Quality Award—2017 Feedback Report 40
2013 through 2015, despite a favorable decline in 2016, while Custodial Inspection
Scores (Figure 7.3-6) show an unfavorable trend from 2012 through 2016 despite high
performance levels. Volunteer Hours (Figure 7.3-12) shows a decline in the most recent
year, as does Police Officer Training (Figure 7.3-14). Sustaining high performance levels
and continuously improving workforce-focused results may help the City retain its
advantage of having engaged and committed employees.
• a(1) Local or national comparisons are missing for some measures of workforce-
focused results, particularly Turnover Police Services (Figure 7.3-2) and Recruitment
(Figure 7.3-4). As a data-driven organization aspiring to world-class performance, the
City may find that using benchmarks aligned with its vision of world-class operational
excellence may help it guide improvements in areas of greatest value to its residential
and business customers.
• a Most workforce results are not segmented as expected relative to the Staff Profile
(Figure P1.4) provided by the City. With the exceptions of Turnover, Police Services
(Figure 7.3-2), Police Officer Training (Figure 7.3-14), Volunteer Engagement (Figure
7.3-9), and Volunteer Hours (Figure 7.3-12), results on most measures are recorded for
the workforce as a whole. Segmenting such results by workforce groups and segments
may allow the City to target improvements for specific employee groups in order to
maintain or increase their engagement.
Malcolm Baldrige National Quality Award—2017 Feedback Report 41
7.4 Leadership and Governance Results
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in no change in the consensus percentage range. (Please
refer to Figure 5b, Results Scoring Guidelines.)
STRENGTHS
• b Multiple strategy implementation results reflect good-to-excellent performance in
areas of importance to the City’s customers and stakeholders. Distinctions reflecting
the City’s high performance include being designated the sixth among Best Places to
Live, second among Best Cities for Small Business, eighth for Best-Performing City,
fourth among communities providing for Overall Well-Being, and in the Top 3 as a
Digital City (Awards and Recognitions, Figure 7.4-23). The City’s performance
recognition in multiple areas of significance to customers and stakeholders may reflect
and strengthen its commitment to the community and help it achieve its vision of
world-class operational excellence and innovation.
• a(2, 3) Resident and voter confidence in the City is reflected in the results for six
Successful Voter Tax Initiatives (Figure 7.4-6), as well as in the favorable results levels
and trends for External Financial Audits (Figure 7.4-7). Legal and regulatory results
showing excellent levels and trends include those for Regulatory Compliance (Figure
7.4-10) and External Financial Audits (Figure 7.4-7). In addition, results show that
intelligent risk assessments (Figure 7.4-9) are largely completed on time and within
budget. These results support the City’s value of integrity and customer requirements
for accuracy, responsiveness, and transparency.
• a(5) The City has beneficial results in relation to societal well-being and support of its
key communities of residents and businesses. For example, three City facilities have
received Gold Leader recognition from the Statewide Environmental Leadership
Program. In addition, results for Outdoor Air Quality—Particulate (Figure 7.4-14),
Pollution Prevention (Figure 7.4-17), and Abatement of Greenhouse Gases (Figure
7.2-19) all demonstrate favorable trends. These results reflecting a focus on the
environment demonstrate the City’s value of stewardship and support its strategic
advantage of being nationally recognized for a high quality of life.
OPPORTUNITIES FOR IMPROVEMENT
• a(1, 4) Some expected leadership results are missing. These include workforce
perceptions of organizational ethics and results of ethics reviews, based on
communications measures used by the ELT to engage workforce members and key
customers (Figure 1.1-3). Measuring such results may help the City support its values of
integrity and outstanding service, while meeting customer requirements for
transparency.
Malcolm Baldrige National Quality Award—2017 Feedback Report 42
• a(1) Mixed trends in results reflecting the organization’s alignment of its culture with
the workforce are evident over four years and nine review periods, as shown in My Role
and the City’s Mission and Vision (Figure 7.4-1). Four service areas, representing two-
thirds of the workforce, show variable trends over this period. Improving these results
may the City help retain high-quality talent through its values of collaboration, integrity,
and outstanding service to the community.
Malcolm Baldrige National Quality Award—2017 Feedback Report 43
7.5 Financial and Market Results
Your score in this Criteria item for the Consensus Review is in the 50–65 percentage range. The
findings from the site visit resulted in an increased percentage range of 70–85. (Please refer to
Figure 5b, Results Scoring Guidelines.)
STRENGTHS
• a(1) Evidence of strong financial performance includes results for Actual Revenue
Compared to Budget (Figure 7.5-1), which show a surplus each year over five years;
other examples are results for Accuracy of Budget (Figure 7.5-2), Sales and Use Tax
Income (Figure 7.5-3), Successful Voter Tax Initiative (Figure 7.4-6), Financial Ratios
(Figure 7.5-4), and Increase Emergency Management Performance Grant (Figure 7.1-30),
which shows consistent receipt of the grant. This strong financial performance may
assist the City in effectively managing its strategic challenge of fluctuating revenue.
• a(2) Good results levels and trends are evident for several measures of marketplace
performance, which may support local business development, local employment, and a
healthy growing community. For example, Total Affordable Housing Units Inventory
(Figure 7.5-6) and Unemployment Rate (Figure 7.5-9) demonstrate favorable trends in
commercial property availability, and a steady reduction in unemployment over the past
four years. These results support the City’s strategic advantage of leveraged
investments in infrastructure and amenities.
• a The City’s performance compares favorably relative to its neighboring competitors
for many indicators of financial and marketplace performance. For example, results
show that the City has achieved a consistently lower debt per capita than Denver (Figure
7.5-4), has a superior bond rating relative to most Front Range cities (Figure 7.5-5), has
issued more new commercial permits (except in 2015) than Greeley and Longmont
(Figure 7.5-7), and has maintained since 2012 a consistently lower unemployment rate
than state and nation comparisons (Figure 7.5-9). These results may strengthen the
City’s competitive position for business development/redevelopment, primary
employers/jobs, expansion opportunities, and highly qualified workers.
• a(1) Aggregate measures of financial viability and liquidity demonstrate improving
trends and favorable performance by the City against comparisons. For example, the
City has the best credit position according to Moody’s, with an “Aaa” rating that is
significantly above the U.S. city median rating of “Aa3.” Net cash balances as a
percentage of the City’s operating revenues have consistently improved from 33.7% in
2012 to 40.5% in 2015, exceeding the U.S. city median of 34.4%, while the City’s net
direct debt relative to operating revenues (ratio) of 0.41x reflects leadership over the
U.S. median comparison of 0.94x. In addition, the City’s Available Fund Balance as a
percentage of operating revenues has improved slightly from 28.8% in 2012 to 33.7% in
2015. These results reflect the City’s core value of stewardship and support its ability to
fund strategic opportunities and intelligent risks.
Malcolm Baldrige National Quality Award—2017 Feedback Report 44
OPPORTUNITIES FOR IMPROVEMENT
• a(2) Some expected marketplace results are missing, including measures related to
the City’s market position in areas of key services. Examples include results on the
number of new programs or services offered to the homeless community and programs
designed to advance the small and middle-sized business community. Complete results
on its marketplace performance in the seven Outcome Areas may enable the City to
identify new products or services to address the needs of emerging strategic focus
areas.
Malcolm Baldrige National Quality Award—2017 Feedback Report 45
APPENDIX A
The spider, or radar, chart that follows depicts your organization’s performance as represented
by scores for each item. This performance is presented in contrast to the median scores for all
2017 applicants at Consensus Review, as well as the median scores at Site Visit Review of those
2017 applicants that received site visits. Each ring in the chart corresponds to a scoring range.
Each point in red represents the scoring range your organization achieved for the
corresponding item following your site visit. The points in blue represent the median scoring
ranges for all 2017 applicants at Consensus Review. The points in green represent the median
scoring ranges at Site Visit Review for those 2017 applicants that received site visits. Seeing
where your performance is similar or dissimilar to the median of all applicants may help you
initially determine or prioritize areas for improvement efforts and strengths to leverage.
Malcolm Baldrige National Quality Award—2017 Feedback Report 46
APPENDIX B
By submitting a Baldrige Award application, you have differentiated yourself from most U.S.
organizations. The Board of Examiners has evaluated your application for the Malcolm Baldrige
National Quality Award. Strict confidentiality is observed at all times and in every aspect of the
application review and feedback.
This feedback report contains the examiners’ findings, including a summary of the key themes
of the evaluation, a detailed listing of strengths and opportunities for improvement, and scoring
information. Background information on the examination process is provided below.
APPLICATION REVIEW
Independent Review
Following receipt of the award applications, the award process review cycle (shown in Figure 1)
begins with Independent Review, in which members of the Board of Examiners are assigned to
each of the applications. Examiners are assigned based on their areas of expertise and with
attention to avoiding potential conflicts of interest. Each application is evaluated independently
by the examiners, who write observations relating to the scoring system described beginning on
page 31 of the 2017–2018 Baldrige Excellence Framework.
Malcolm Baldrige National Quality Award—2017 Feedback Report 47
Figure 1—Award Process Review Cycle
Malcolm Baldrige National Quality Award—2017 Feedback Report 48
Consensus Review
In Consensus Review (see Figure 2), a team of examiners, led by a senior examiner or alumnus,
conducts a series of reviews, first managed virtually through a secure database called BOSS and
eventually concluded through a focused conference call. The purpose of this series of reviews is
for the team to reach consensus on comments and scores that capture the team’s collective
view of the applicant’s strengths and opportunities for improvement. The team documents its
comments and scores in a Consensus Scorebook.
Step 1
Consensus Planning
Step 2
Consensus Review in
BOSS
Step 3
Consensus Call
Step 4
Post–Consensus–Call
Activities
• Clarify the
timeline for the
team to complete
its work.
• Assign
category/item
discussion leaders.
• Discuss key
business/
organization
factors.
• Review all
Independent
Review
evaluations—
draft consensus
comments and
propose scores.
• Develop
comments and
scores for the
team to review.
• Address
feedback,
incorporate
inputs, and
propose a
resolution of
differences on
each worksheet.
• Review updated
comments and
scores.
• Discuss
comments,
scores, and all key
themes.
• Achieve
consensus on
comments and
scores.
• Revise comments
Malcolm Baldrige National Quality Award—2017 Feedback Report 49
option to choose to receive a Baldrige Site Visit Experience.
The Baldrige Site Visit Experience is a new offering in which a team of examiners conducts a site
visit and, while still on-site and face-to-face, can share with the organization’s leaders evidence
collected on-site and its impact on scoring, high-level takeaways, and other insights gleaned.
Organizations that receive the Baldrige Site Visit Experience are no longer in contention for that
year’s Baldrige Award.
Site visits are conducted for the highest-scoring applicants to clarify any uncertainty or
confusion the examiners may have regarding the written application and to verify that the
information in the application is correct (see Figure 3 for the Site Visit Review process). After
the site visit, the team of examiners prepares a final Site Visit Scorebook.
Step 1
Team Preparation
Step 2
Site Visit
Step 3
Post–Site–Visit Activities
• Review consensus
findings.
• Develop site visit issues.
• Plan site visit.
• Make/receive
presentations.
• Conduct interviews.
• Record observations.
• Review documents.
• Resolve issues.
• Summarize findings.
• Finalize comments.
• Prepare final Site Visit
Scorebook.
• Prepare feedback report.
Figure 3—Site Visit Review
Applications, Consensus Scorebooks, and Site Visit Scorebooks for all applicants receiving site
visits are forwarded to the Judges Panel for review (see Figure 4). The judges recommend which
applicants should receive the Baldrige Award and identify any non-award recipient
organizations demonstrating one or more Category Best Practices. The judges discuss
applications in each of the six award sectors separately, and then they vote to keep or eliminate
each applicant. Next, the judges decide whether each of the top applicants should be
recommended as an award recipient based on an “absolute” standard: the overall excellence of
the applicant and the appropriateness of the applicant as a national role model. For each
organization not recommended to receive the Baldrige Award, the judges have further
discussion to determine if the organization demonstrates any Category Best Practices. The
process is repeated for each award sector.
Malcolm Baldrige National Quality Award—2017 Feedback Report 50
Step 1
Judges Panel Review
Step 2
Evaluation by Category
Step 3
Assessment of Top
Organizations
• Applications
• Consensus Scorebooks
• Site Visit Scorebooks
• Manufacturing
• Service
• Small business
• Education
• Health care
• Nonprofit
• Overall strengths/
opportunities for
improvement
• Appropriateness as
national model of
performance
excellence
Figure 4—Judges’ Review
Judges do not participate in discussions or vote on applications from organizations in which
they have a competing or conflicting interest or in which they have a private or special interest,
such as an employment or a client relationship, a financial interest, or a personal or family
relationship. All conflicts are reviewed and discussed so that judges are aware of their own and
others’ limitations on access to information and participation in discussions and voting.
Following the judges’ review and recommendation of award recipients, the Site Visit Review
team leader edits the final Site Visit Scorebook, which becomes the feedback report.
Malcolm Baldrige National Quality Award—2017 Feedback Report 51
SCORING
The scoring system used to score each item is designed to differentiate the applicants in the
various stages of review and to facilitate feedback. As seen in the Process Scoring Guidelines
and Results Scoring Guidelines (Figures 5a and 5b, respectively), the scoring of responses to
Criteria items is based on two evaluation dimensions: process and results. The four factors used
to evaluate process (categories 1–6) are approach (A), deployment (D), learning (L), and
integration (I), and the four factors used to evaluate results (items 7.1–7.5) are levels (Le),
trends (T), comparisons (C), and integration (I).
In the feedback report, the applicant receives a percentage range score for each item. The
range is based on the scoring guidelines, which describe the characteristics typically associated
with specific percentage ranges.
As shown in Figures 6a and 6b, the applicant’s overall scores for process items and results items
each fall into one of eight scoring bands. Each band score has a corresponding descriptor of
attributes associated with that band. Figures 6a and 6b show the percentage of applicants
scoring in each band at Consensus Review.
Malcolm Baldrige National Quality Award—2017 Feedback Report 52
Figure 5a—Process Scoring Guidelines (For Use with Categories 1–6)
SCORE DESCRIPTION
0% or 5%
• No SYSTEMATIC APPROACH to item requirements is evident; information is ANECDOTAL. (A)
• Little or no DEPLOYMENT of any SYSTEMATIC APPROACH is evident. (D)
• An improvement orientation is not evident; improvement is achieved by reacting
to problems. (L)
• No organizational ALIGNMENT is evident; individual areas or work units operate independently.
(I)
10%, 15%,
20%, or 25%
• The beginning of a SYSTEMATIC APPROACH to the BASIC REQUIREMENTS of the item is evident. (A)
• The APPROACH is in the early stages of DEPLOYMENT in most areas or work units, inhibiting
progress in achieving the BASIC REQUIREMENTS of the item. (D)
• Early stages of a transition from reacting to problems to a general improvement orientation
are evident. (L)
• The APPROACH is ALIGNED with other areas or work units largely through joint problem solving.
(I)
30%, 35%,
40%, or 45%
• An EFFECTIVE, SYSTEMATIC APPROACH, responsive to the BASIC REQUIREMENTS of the item, is
evident. (A)
• The APPROACH is DEPLOYED, although some areas or work units are in early stages of
DEPLOYMENT. (D)
• The beginning of a SYSTEMATIC APPROACH to evaluation and improvement of KEY PROCESSES is
evident. (L)
• The APPROACH is in the early stages of ALIGNMENT with the basic organizational needs identified
in response to the Organizational Profile and other process items. (I)
50%, 55%,
60%, or 65%
• An EFFECTIVE, SYSTEMATIC APPROACH, responsive to the OVERALL REQUIREMENTS of the item, is
evident. (A)
• The APPROACH is WELL DEPLOYED, although DEPLOYMENT may vary in some areas or work units. (D)
• A fact-based, SYSTEMATIC evaluation and improvement PROCESS and some organizational
LEARNING, including INNOVATION, are in place for improving the efficiency and EFFECTIVENESS of KEY
PROCESSES. (L)
• The APPROACH is ALIGNED with your overall organizational needs as identified in response to the
Organizational Profile and other process items. (I)
70%, 75%,
80%, or 85%
• An EFFECTIVE, SYSTEMATIC APPROACH, responsive to MULTIPLE REQUIREMENTS in the item, is evident.
(A)
• The APPROACH is well DEPLOYED, with no significant gaps. (D)
• Fact-based, SYSTEMATIC evaluation and improvement and organizational LEARNING, including
INNOVATION, are KEY management tools; there is clear evidence of refinement as a result of
organizational-level ANALYSIS and sharing. (L)
• The APPROACH is INTEGRATED with your current and future organizational needs as identified in
response to the Organizational Profile and other process items. (I)
90%, 95%, or
100%
• An EFFECTIVE, SYSTEMATIC APPROACH, fully responsive to the MULTIPLE REQUIREMENTS of the item, is
evident. (A)
• The APPROACH is fully DEPLOYED without significant weaknesses or gaps in any areas or work
units. (D)
• Fact-based, SYSTEMATIC evaluation and improvement and organizational LEARNING through
INNOVATION are KEY organization-wide tools; refinement and INNOVATION, backed by ANALYSIS and
sharing, are evident throughout the organization. (L)
• The APPROACH is well INTEGRATED with your current and future organizational needs as
identified in response to the Organizational Profile and other process items. (I)
Malcolm Baldrige National Quality Award—2017 Feedback Report 53
Figure 5b—Results Scoring Guidelines (For Use with Category 7)
SCORE DESCRIPTION
0% or 5%
• There are no organizational PERFORMANCE RESULTS, or the RESULTS reported are poor. (Le)
• TREND data either are not reported or show mainly adverse TRENDS. (T)
• Comparative information is not reported. (C)
• RESULTS are not reported for any areas of importance to the accomplishment of your
organization’s MISSION. (I)
10%, 15%,
20%, or 25%
• A few organizational PERFORMANCE RESULTS are reported, responsive to the BASIC
REQUIREMENTS of the item, and early good PERFORMANCE LEVELS are evident. (Le)
• Some TREND data are reported, with some adverse TRENDS evident. (T)
• Little or no comparative information is reported. (C)
• RESULTS are reported for a few areas of importance to the accomplishment of your
organization’s MISSION. (I)
30%, 35%,
40%, or 45%
• Good organizational PERFORMANCE LEVELS are reported, responsive to the BASIC
REQUIREMENTS of the item. (Le)
• Some TREND data are reported, and most of the TRENDS presented are beneficial. (T)
• Early stages of obtaining comparative information are evident. (C)
• RESULTS are reported for many areas of importance to the accomplishment of your
organization’s MISSION. (I)
50%, 55%,
60%, or 65%
• Good organizational PERFORMANCE LEVELS are reported, responsive to the OVERALL
REQUIREMENTS of the item. (Le)
• Beneficial TRENDS are evident in areas of importance to the accomplishment of your
organization’s MISSION. (T)
• Some current PERFORMANCE LEVELS have been evaluated against relevant comparisons
and/or BENCHMARKS and show areas of good relative PERFORMANCE. (C)
• Organizational PERFORMANCE RESULTS are reported for most KEY CUSTOMER, market, and
PROCESS requirements. (I)
70%, 75%,
80%, or 85%
• Good-to-excellent organizational PERFORMANCE LEVELS are reported, responsive to
MULTIPLE REQUIREMENTS in the item. (Le)
• Beneficial TRENDS have been sustained over time in most areas of importance to the
accomplishment of your organization’s MISSION. (T)
• Many to most TRENDS and current PERFORMANCE LEVELS have been evaluated against
relevant comparisons and/or BENCHMARKS and show areas of leadership and very good
relative PERFORMANCE. (C
• Organizational PERFORMANCE RESULTS are reported for most KEY CUSTOMER, market,
PROCESS, and ACTION PLAN requirements. (I)
90%, 95%,
or 100%
• Excellent organizational PERFORMANCE LEVELS are reported that are fully responsive to
the MULTIPLE REQUIREMENTS of the item. (Le)
• Beneficial TRENDS have been sustained over time in all areas of importance to the
accomplishment of your organization’s MISSION. (T)
• Industry and BENCHMARK leadership is demonstrated in many areas. (C)
• Organizational PERFORMANCE RESULTS and PROJECTIONS are reported for most KEY CUSTOMER,
market, PROCESS, and ACTION PLAN requirements. (I)
Malcolm Baldrige National Quality Award—2017 Feedback Report 54
1 Percentages are based on scores from the Consensus Review.
Figure 6a–Process Scoring Band Descriptors
Band
Score
Band
Number
%
Applicants
in Band1
PROCESS Scoring Band Descriptors
0–150 1 0 The organization demonstrates early stages of developing and
implementing approaches to the basic Criteria requirements, with
deployment lagging and inhibiting progress. Improvement efforts are a
combination of problem solving and an early general improvement
orientation.
151–200 2 0 The organization demonstrates effective, systematic approaches responsive
to the basic requirements of the Criteria, but some areas or work units are
in the early stages of deployment. The organization has developed a general
improvement orientation that is forward-looking.
201–260 3 4 The organization demonstrates effective, systematic approaches responsive
to the basic requirements of most Criteria items, although there are still
areas or work units in the early stages of deployment. Key processes are
beginning to be systematically evaluated and improved.
261–320 4 42 The organization demonstrates effective, systematic approaches responsive
to the overall requirements of the Criteria, but deployment may vary in
some areas or work units. Key processes benefit from fact-based evaluation
and improvement, and approaches are being aligned with overall
organizational needs.
321–370 5 46 The organization demonstrates effective, systematic, well-deployed
approaches responsive to the overall requirements of most Criteria items.
The organization demonstrates a fact-based, systematic evaluation and
improvement process and organizational learning, including some
innovation, that result in improving the effectiveness and efficiency of key
processes.
371–430 6 8 The organization demonstrates refined approaches responsive to the
multiple requirements of the Criteria. These approaches are characterized
by the use of key measures, good deployment, and evidence of innovation
in most areas. Organizational learning, including innovation and sharing of
best practices, is a key management tool, and integration of approaches
with current and future organizational needs is evident.
431–480 7 0 The organization demonstrates refined approaches responsive to the
multiple requirements of most Criteria items. It also demonstrates
innovation, excellent deployment, and good-to-excellent use of measures in
most areas. Good-to-excellent integration is evident, with organizational
analysis, learning through innovation, and sharing of best practices as key
management strategies.
481–550 8 0 The organization demonstrates outstanding approaches focused on
innovation. Approaches are fully deployed and demonstrate excellent,
sustained use of measures. There is excellent integration of approaches with
organizational needs. Organizational analysis, learning through innovation,
and sharing of best practices are pervasive.
Malcolm Baldrige National Quality Award—2017 Feedback Report 55
1 Percentages are based on scores from the Consensus Review.
2 “Industry” refers to other organizations performing substantially the same functions, thereby facilitating
direct comparisons.
Figure 6b—Results Scoring Band Descriptors
Figure
6a—Band
Score
Band
Number
%
Applicants
in Band1
RESULTS Scoring Band Descriptors
0–125 1 0 A few results are reported responsive to the basic Criteria requirements,
but they generally lack trend and comparative data.
126–170 2 4 Results are reported for several areas responsive to the basic Criteria
requirements and the accomplishment of the organization’s mission.
Some of these results demonstrate good performance levels. The use of
comparative and trend data is in the early stages.
171–210 3 33 Results address areas of importance to the basic Criteria requirements
and accomplishment of the organization’s mission, with good
performance being achieved. Comparative and trend data are available
for some of these important results areas, and some beneficial trends are
evident.
211–255 4 46 Results address some key customer/stakeholder, market, and process
requirements, and they demonstrate good relative performance against
relevant comparisons. There are no patterns of adverse trends or poor
performance in areas of importance to the overall Criteria requirements
and the accomplishment of the organization’s mission.
256–300 5 17 Results address most key customer/stakeholder, market, and process
requirements, and they demonstrate areas of strength against relevant
comparisons and/or benchmarks. Beneficial trends and/or good
performance are reported for most areas of importance to the overall
Criteria requirements and the accomplishment of the organization’s
mission.
301–345 6 0 Results address most key customer/stakeholder, market, and process
requirements, as well as many action plan requirements. Results
demonstrate beneficial trends in most areas of importance to the Criteria
requirements and the accomplishment of the organization’s mission, and
the organization is an industry2 leader in some results areas.
346–390 7 0 Results address most key customer/stakeholder, market, process, and
action plan requirements. Results demonstrate excellent organizational
performance levels and some industry2 leadership. Results demonstrate
sustained beneficial trends in most areas of importance to the multiple
Criteria requirements and the accomplishment of the organization’s
mission.
391–450 8 0 Results fully address key customer/stakeholder, market, process, and
action plan requirements and include projections of future performance.
Results demonstrate excellent organizational performance levels, as well
as national and world leadership. Results demonstrate sustained
beneficial trends in all areas of importance to the multiple Criteria
requirements and the accomplishment of the organization’s mission.
Malcolm Baldrige National Quality Award—2017 Feedback Report 56
2017 BALDRIGE AWARD APPLICANTS
BALDRIGE AWARD RECIPIENT CONTACT INFORMATION 1988–2016
Baldrige Award winners generously share information with numerous organizations from all sectors.
To contact an award winner, please see http://patapsco.nist.gov/Award_Recipients/index.cfm, which
includes links to contact information as well as profiles of the winners.
Sector Total Number of Award
Applications
Number of Award
Applicants
Recommended for Site
Visit
Health Care 12 7
Nonprofit 4 2
Education 5 3
Business–Small Business 3 2
Business–Service 0 0
Business–Manufacturing 0 0
Total 24 14
and scores to
reflect consensus
decisions.
• Prepare final
Consensus
Scorebook.
• Prepare feedback
report.
Figure 2—Consensus Review
Site Visit Review
After Consensus Review, the Judges Panel of the Malcolm Baldrige National Quality Award
selects applicants to receive site visits based on the scoring profiles. If an applicant is not
selected for Site Visit Review, the final Consensus Scorebook receives a technical review by a
highly experienced examiner and becomes the feedback report, or the applicant may have the
OUR JOURNEY
CONTINUES...
The City is always working to improve, innovate and
provide better service to the community. While we have
received this prestigious accolade, it by no means
signifies the end of the journey. The City will continue to
solicit feedback and explore ways it can improve its
processes and o er world class services to residents.
We will take the Baldrige feedback report and implement
opportunities highlighted for improvement. At the same
time, the City will share the story of our journey with
residents and other communities in a continued e ort to
be a transparent and trustworthy organization.
LEARN MORE AT fcgov.com/excellence
BALDRIGE = BEST OF...