Loading...
HomeMy WebLinkAboutCORRESPONDENCE - RFP - 8433 COMMERCIAL & RESIDENTIAL ENERGY PROGRAMS THIRD PARTY CONSULTANTUtilities Work Order Form Official Purchasing Form Last updated 10/2017 WORK ORDER PURSUANT TO A MASTER AGREEMENT BETWEEN THE CITY OF FORT COLLINS AND OPINION DYNAMICS CORPORATION WORK ORDER NUMBER: 3 PROJECT TITLE: Efficiency Works Residential Billing Analysis ORIGINAL BID/RFP NUMBER & NAME: 8433 Commercial & Residential Energy Programs Third Party Consultant MASTER AGREEMENT EFFECTIVE DATE: May 1, 2017 OWNER’S REPRESENTATIVE: Brian Tholl WORK ORDER COMMENCEMENT DATE: November 18, 2019 WORK ORDER COMPLETION DATE: March 31, 2020 MAXIMUM FEE: (time and reimbursable direct costs): $23,000.00 PROJECT DESCRIPTION/SCOPE OF SERVICES: Apex Analytics will conduct a statistical billing analysis for the City of Fort Collins Efficiency Works Homes Program. Service Provider agrees to perform the services identified above and on the attached forms in accordance with the terms and conditions contained herein and in the Master Agreement between the parties. In the event of a conflict between or ambiguity in the terms of the Master Agreement and this Work Order (including the attached forms) the Master Agreement shall control. The attached forms consisting of five (5) pages are hereby accepted and incorporated herein, by this reference, and Notice to Proceed is hereby given after all parties have signed this document. SERVICE PROVIDER: Opinion Dynamics Corporation By: Date: Name: Title: DocuSign Envelope ID: 15E69954-AA72-47FE-B585-060A7C9916CD 11/20/2019 Marjorie McRae Executive Consultant Utilities Work Order Form Official Purchasing Form Last updated 10/2017 OWNER’S ACCEPTANCE & EXECUTION: This Work Order and the attached Contract Documents are hereby accepted and incorporated herein by this reference. ACCEPTANCE: Date: Brian Tholl, Project Manager REVIEWED: Date: Marisa Donegon, Buyer ACCEPTANCE: Date: John Phelan, Project Sponsor ACCEPTANCE: Date: Scott Dimetrosky, Apex Analytics DocuSign Envelope ID: 15E69954-AA72-47FE-B585-060A7C9916CD 11/19/2019 11/20/2019 11/20/2019 11/20/2019 Utilities Work Order Form Official Purchasing Form Last updated 10/2017 ATTACHMENT A WORK ORDER SCOPE OF SERVICES This Scope of Work details the statistical billing analysis for the City of Fort Collins (City) Utilities Efficiency Works Homes Program (“EW Homes”). Service Provider (Apex Analytics) will estimate energy impacts from the EW Homes program by statistical billing analysis. The objectives of the billing analysis will be to: ▪ Determine the electricity and gas impacts attributable to EW Homes participation ▪ Identify, if possible, whether energy impacts vary across participant types, customer segments, or equipment installations ▪ Setup the data processing and analysis to allow future iterations of the analysis The following document includes the three primary research tasks to be performed for Utilities. Task 1: Data Review and Assessment Service Provider will review four primary data sources: gas billing data, electric billing data, tracking system data (for program participation details) and customer/household characteristic data (from assessor dataset, if available). The goal will be to assess the feasibility and validity of the proposed billing analysis, and Service Provider will assess the following for each of the provided datasets: ▪ Thoroughness of data ▪ Crosswalks between the data sources (account or premise IDs allowing merging of the data) ▪ Inconsistent formatting, outliers, duplicates Providing data dictionaries will help reduce the potential for back-and-forth clarification questions, and consistent crosswalks between datasets (e.g., account ID, premise ID, customer ID, Project ID) will help ensure the data assessment can be accomplished within scope.1 Task 2: Billing Analysis Service Provider’s approach to consumption analysis includes the following: ▪ Use a control group. Ideally, a control group is developed by randomly assigning customers to treatment and control conditions. However, high quality control groups can also be developed using matching methods. Control group customer should have similar characteristics, be located in similar areas, experience similar weather and day to day conditions, and have similar energy usage patterns prior to installation of EW Homes equipment. The only observable systematic difference between the two groups should be that one installed EW Homes equipment while the other did not. • For gas participants, the only option for a control group will be using current year participants and comparing their usage in the years covering the equivalent participants from earlier years (2016-2018). 1 Service Provider also used the data discovery task to determine whether the billing analysis cost would fall into a lower of higher cost range. DocuSign Envelope ID: 15E69954-AA72-47FE-B585-060A7C9916CD Utilities Work Order Form Official Purchasing Form Last updated 10/2017 • For electric participants, Service Provider can replicate the gas approach (of using current year as controls) OR select a group of participants with “like” household characteristics and equivalent pre-installation electric consumption data. ▪ Include at least one year of pre-treatment data and post-treatment data for both the participant and control groups. The pre-treatment data is useful for assessing if energy consumption changed with introduction of the intervention. It also allows the use of more powerful statistical techniques such as difference-in-differences. ▪ Estimate impacts using a difference-in-differences (DiD) panel model regression. This technique makes full use of pre and post data as well as the control group. It also can be augmented with additional explanatory power by including additional explanatory variables. The core concepts of difference-in-difference panel models can be illustrated using a simplified example. Prior to the program participation, energy use patterns for the participant and control group should be very similar, with small differences. If EW Homes participation lead to reductions in consumption, Service Provider should observe a change in consumption for customers who participated, but no similar change for the control group. In addition, the timing of the change should coincide with the installation of the equipment as part of the EW Homes program. The energy impact is estimated as the difference between the participant and control group, net any pre-existing differences. The general form of a difference-in-differences panel regression is presented below. Separate models will be estimated for the electricity (kWh) and natural gas (therms or CCF). The cooling degree day terms would likely be omitted from the natural gas model. Equation 1: Fixed Effects Regression Specification 𝑈𝑠𝑎𝑔𝑒𝑖,𝑡 = 𝛽 0 + 𝛽1 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖 + 𝛽 2𝑃𝑜𝑠𝑡𝑖,𝑡 + 𝛽3𝑃𝑜𝑠𝑡 𝑖,𝑡 ∙ 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖 + 𝛽4𝐻𝐷𝐷 𝑖,𝑡 + 𝛽5𝐶𝐷𝐷 𝑖,𝑡 + 𝛽6 𝐻𝐷𝐷𝑖,𝑡 ∗ 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑖,𝑡 ∗ 𝑃𝑜𝑠𝑡𝑖,𝑡 + 𝛽7𝐶𝐷𝐷 𝑖,𝑡 ∗ 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖,𝑡 ∗ 𝑃𝑜𝑠𝑡𝑖,𝑡 + 𝛾 𝑖 + 𝛿𝑡 + 𝜀𝑖,𝑡 Where: 𝑈𝑠𝑎𝑔𝑒𝑖,𝑡 Is the usage by for each individual customer and time period 𝛽0 Is the model intercept 𝛽1 Corrects for pre-existing differences between treatment and control group customers 𝛽2 Controls for differences between the pre and post interventions period that are unrelated to the intervention or weather (e.g. changes in the control group’s base load) 𝛽3 Represents the impact, or treatment effect, of the intervention on non-weather dependent load after controlling for pre-existing differences and other factors. Expected to be minimal for EW homes participants which target heating and cooling impacts Utilities Work Order Form Official Purchasing Form Last updated 10/2017 energy consumption per HDD and CDD respectively. 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖 A binary variable to indicate if a customer is part of the participant or control group. It remains static throughout the analysis period. In practice, this variable drops out whenever fixed effects are included because fixed effects account for all unique customer characteristics that remain static. 𝑃𝑜𝑠𝑡𝑖,𝑡 A binary variable indicating if the time period is before or after the interventions. Because participation occurs across seasons and years rather all at once, each participant’s statistical double in the control group is assigned the same intervention date. 𝑃𝑜𝑠𝑡𝑖,𝑡 ∙ 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖 A binary variable that represents if a customer is in the treatment group and has had their home upgraded. 𝛾𝑖 This represents customer specific fixed effects. They account for all unique customer characteristics that remain static. 𝛿𝑡 This represent time effects for each time periods. This account for observed and unobserved factors that vary by time but affect all customers equally. 𝜀𝑖,𝑡 The error term for each individual customer and time period. To estimate weather-normalized annual energy impacts attributable to EW Homes participation, Service Provider will calculate a linear combination of β3, β6, and β7 and the number of days in a year, average annual HDD, and average annual CDD for Fort Collins service territory as shown in Equation 2. Equation 2: Calculation of Annual kWh Energy Impacts from Regression Coefficients 𝐴𝑛𝑛𝑢𝑎𝑙 𝑘𝑊ℎ 𝐼𝑚𝑝𝑎𝑐𝑡 = 365.25 ∗ 𝛽3 + 𝐴𝑛𝑛𝑢𝑎𝑙 𝐻𝐷𝐷 ∗ 𝛽 6 + 𝐴𝑛𝑛𝑢𝑎𝑙 𝐶𝐷𝐷 ∗ 𝛽7 One of the key challenges for this study will be adequate sample sizes. While at first glance sample sizes are adequately large for detecting overall impacts, they may be insufficient for disaggregating impacts into subgroups or specific measure categories. Results will be automatically separated by heating fuel, because of the separate models for gas and electricity. However, segmentation of homes by mechanical system within a heating fuel, or by installed equipment type are not guaranteed to result in different point estimates, given potential overlapping confidence intervals. Task 2A: Future Replication As part of Task 2, Service Provider will prepare the data assessment and billing analysis routines to ensure future updates to the analysis can be made seamlessly and at a considerably lower cost. The replication will allow data preparation and model setup to be conducted by Service Provider relying on the same code and database processes. Service Provider will setup import routines, establish future calendarized table structures, and apart tables and code to ensure future iterations of this analysis can be replicated at a lower cost. The statistical modeling, which will be coded in Julia, will be setup similarly, with anticipation for future iterations. DocuSign Envelope ID: 15E69954-AA72-47FE-B585-060A7C9916CD Utilities Work Order Form Official Purchasing Form Last updated 10/2017 Task 3: Reporting Service Provider will develop a summary memo and presentation with key methodological assumptions and findings. The memo will summarize primary billing analysis activities and findings. Service Provider will follow Utilities guidelines and ensure the memo contains sufficient detail to understand how the final recommended savings estimates was determined. Service Provider will also include a webinar presentation of the key findings so any questions can be reviewed with the team. The summary memo and presentation will be completed no later than March 31, 2020. DocuSign Envelope ID: 15E69954-AA72-47FE-B585-060A7C9916CD Utilities Work Order Form Official Purchasing Form Last updated 10/2017 ATTACHMENT B WORK ORDER COST DETAIL Below is the anticipated budget for EW Homes billing analysis. Service Provider has determined that the billing analysis costs should fall in between the previously submitted scenarios.2 This reflects the approximate average data cleaning and manipulation costs associated with reconciling two tracking systems and two distinct gas datasets while also managing a separate gas and electric analysis. The City’s payment terms are Net 30. Staff Rate Task 1: Data Assessment Task 2: Billing Analysis Task 2A: Replication Task 3: Reporting Noah Lieb $170 12 76 8 10 Jon Koliner $170 4 10 4 6 Scott Dimetrosky $225 1 1 0 2 Total Cost by Task $2,945.00 $14,845.00 $2,040.00 $3,170.00 Total Cost $23,000.00 2 In the draft version of this scope of work, Apex had specified low-and-high-cost billing analysis scenarios. DocuSign Envelope ID: 15E69954-AA72-47FE-B585-060A7C9916CD Utilities Work Order Form Official Purchasing Form Last updated 10/2017 ATTACHMENT C CERTIFICATE OF INSURANCE CONTRACTOR shall submit Certificate of Insurance in compliance with the Contract Documents. DocuSign Envelope ID: 15E69954-AA72-47FE-B585-060A7C9916CD 𝛽4 and 𝛽 5 Are parameters that account for the relationship between weather and energy usage in the baseline period 𝛽6 and 𝛽 7 The primary parameters of interest. These coefficients capture the reduction in DocuSign Envelope ID: 15E69954-AA72-47FE-B585-060A7C9916CD