Link to client energy results blog

https://tedkidd.blogspot.com/

JUNE CONTRACTOR UPDATE: Trust, Truth, Transparency

I got some surprisingly positive responses to my February update!  Many were surprised (and a little annoyed) that the program had not informed them about their numbers, particularly where they were weak.  Contractors seem to mistakenly assume if there are problems with results the program will let them know.  

But ultimately Contractors liked the idea of tracking their work, independent data showing where they excel they can share with clients, data where they are weak they can take to their crews, and having various measures of quality to compare where they stand against their competition.  

Who agrees a dramatic overhaul is needed?  Who think this program is not a great example of “When Your Incentive System Backfires.”   Does this EMPOWER silliness offers the answer, or is it simply more of this death by a thousand cuts?  How about a system that lets you sell whatever you want, where job approval incentive isn’t about "cost effectiveness pass/fail”? 

Seems dissatisfaction with NY HPwES is at an all time high.  Production is as low as it’s been in nearly a decade, and people are lining up to leave.  Is anybody having fun with SIR, TRC, ProForma Tools, or the ever more confusing  “deemed cost effective” tweaks by NYSERDA?  


How about a system where you don't continuously wonder what you can sell?

How about a program where the government gets energy savings at the price it wants to pay, 

...the consumer gets home improvements for the net cost they were told, 

...and we can focus on designs that meet needs, wants, and budget, rather than software games?

How we think about "Home Performance" creates a fundamental orientation to our view of how programs should structure incentive. It can create bias that prevents seeing alternatives....

...IN THIS UPDATE let us explore TRUST and TRUTH, and consider an outline for a different way of thinking. A different way programs might provide incentives.  A different way for them to view and value projects.  At the end I will share the outline of a completely different way the program could be designed.  A design that allows for simple job approval, unlimited measures (windows, water heaters, and heat pumps too), and most importantly recognition and reward for excellence.  

TRUST and TRUTH lead to really big sales.  I think the only way to really achieve TRUST and TRUTH is with TRANSPARENCY.  Past history of ability to deliver on promises could be the huge trust bridge Home Performance needs to really grow demand.  WE NEED JD POWERS FOR HOME PERFORMANCE.

"I saw a study ...that said most people basically only believe they will see 25% of what they are told they will save." "...perhaps a big part of why savings does not sell is that people (rightfully) discount the value."

Matt Golden, Efficiency.org

"It is too bad that there is not much quality assurance in this process, and most customers heavily discount our savings predictions. So it would be pretty helpful if there was quality assurance of some type for savings predictions so customers could tell the difference between lies and honest and reasonably accurate predictions. We have QA for contractors work why not for their savings?"

Greg Thomas, PSD

“I have all kinds of work I’d like to have done to my house.  It’s just every contractor experience I’ve had has been so terrible, so exhausting, so disappointing, I keep putting it off.  It always costs more then they say, and they never live up to their promises”  
(amalgam of comments from various interviewee’s)

This isn’t rocket science.  This work can be done by rookies.  After taking the BPI Classes I was recruited into this industry by Hal Smith.  In my year and a half at Halco my average sale was over $16,000, roughly twice the department average and well over double the NY HPwES average.  I was not a seasoned Home Performance Professional, I was a rookie.  These sales were not “free riders,” they were comprehensive jobs.  They were much more comprehensive than this program allows today.  

And I tracked some of the results.  They lived up to TREAT modeled savings projections!

Better, more comprehensive work is more likely to have immediately noticeable results.  Immediately noticeable changes to homes leads to happier clients.  I’ve met very few contractors who wouldn’t really love to do better work for the personal satisfaction of it.  I’ve met none who wouldn’t like to do better work if they could be recognized and rewarded for it.

BROKEN PERSPECTIVE - A Home Performance program oriented on “cost effectiveness” is simplistic and broken.  It constrains consumers and contractors from doing the work they want to do.  Who believes “Comprehensive Home Performance” and “Financial Cost Effectiveness” are compatible?  

LET’S FIX WHAT’S BROKEN - Incentives that encourage exaggeration over accuracy must be changed.  SIR is the mortal enemy of Comprehensive.  If the Public Service Commission is concerned about cost effectively buying Energy Efficiency, why not incentivize “energy savings” instead of “investment return”.  Simply pay for the Negawatts at whatever price is currently deemed cost effective.  

ENOUGH WITH THE CONFLICT OF INTEREST!  Clients want the cheapest work.  Contractors chasing low cost want to cut every possible corner in hopes of maintaining margins this economy has made razor thin.  Various program interests earn more as administrative overhead increases, so incentive for paperwork burden is directly opposed to contractor and consumer interests.  

Sure, the Public Service Commission wants Energy Efficiency to be “Cost Effective”, but cost effective for WHO?  

It’s certainly not cost effective for consumers to waste time finding out they don’t qualify for incentive programs they’ve funded.  It’s certainly not cost effective for sales people to run around telling homeowners; “can’t sell you this, or this, or this”.   If the PSC is concerned about cost effectiveness of incentive, why don’t they simply buy Negawatts at whatever price they feel is cost effective?  

Sure, public monies allocated for “energy efficiency” need to be spent effectively.   Instead of having rigid incentives that force rigid cost effectiveness pass/fail, let’s have a program that pays a floating incentive.   

THINK ABOUT THIS INCENTIVE - What if the program paid, say, 40¢ for every annual KWH saved, then incentive could float with ENERGY SAVINGS instead of job cost, and every job qualifies for something.  No approved measures, deemed cost effectiveness, none of that.  Program pays for savings, contractors are free to be innovative, to fix what homeowners want fixed.  Honey-do list and all...

If you like that, here are some more ideas to ponder and an outline of one possible way forward:

How about a program where the Contractor is recognized and rewarded for excellent results?  Where the consumer see’s their interest and the contractors are aligned, so the process is collaborative rather than combative?  A program where contractor ability to deliver accuracy is rewarded with a streamlined approval process, those who struggle get support, and those who cut corners face scrutiny of both program and peer?

DO AWAY WITH CLIFFS!  How About a Program Where Every Job Qualifies?  How much easier is it for everyone if the salesperson can sell whatever the customer wants?  How wonderful would life be if EVERY job could pass without turning energy models into pretzels?  Corruption and inaccuracy live under cliffs.  Wherever cliffs occur, we need to do away with them.

LET’S SIMPLIFY THIS THING!  After my December newsletter I was contacted by a number of people.   I was invited to present my thoughts on how to “Align Incentive” at the Dry Climate Building Performance Contractors Summit” in California.   Some thought I’d have to hire a bodyguard, but positive response from contractors was surprising and encouraging.   

Here’s one recent example:

  

I also wanted to let you know that we are stepping up our post-completion tracking efforts, largely at your urging.  Do you have a standard process for doing this?  Right now we are collecting two years of pre-improvement data from all customers (when available) as part of the audit and asking customers if we can contact them 1-2 years after completion to get the post-improvement data.  So far everyone we've asked has been agreeable.

Business owners are a competitive lot.  The idea of seeing results, measuring quality, and competing based upon ability to deliver on promises is appealing to the good ones.

My December Contractor Update also prompted Efficiency First to ask me to help develop a White Paper around the idea of a True Performance Based System.  The basic premise is Client, Contractor, and Program interests ALIGNED instead of conflicting.   

BELOW IS THE ROUGH OUTLINE.  PLEASE FEEL FREE TO LET ME KNOW WHAT YOU THINK EITHER PRIVATELY OR ON THE BLOG:

Performance Based Incentives Outline

Introduction

• What are performance based incentives?
• A method of scaling rebates to the amount of energy savings achieved from performing an energy efficiency project.
• Example Incentive: $.40 per annual kWh saved. This is roughly the extrapolated value currently, without discounting for current program realization failure rate.

Core Components

• How is this system different from current practices?
• Currently, we operate in a “Threshold” or “Cliff” system where EE projects (or individual “approved” measures) must meet a minimum Savings to Investment Return (SIR).  Rebates are based on a fixed percentage of the total project cost (10%, or 50% for lower income).  
• Jobs that save energy, but don’t meet “cost justification” thresholds are not eligible for any incentive.  
• Performance Based Incentives would remove “cost justification” thresholds while still allowing the incentive paid to meet energy savings value requirements of programs.  Incentive pays for the value of energy saved rather than backing into incentive cost effectiveness on project by project basis.  Thus, the incentive is always cost effective and requires much less momentum killing scrutiny and modification.   

• Cost of “At The Kitchen Table” opportunities is very high and should not be squandered due to programmatic designs attempting to “cherry pick” opportunity.  Ultimately it is the homeowner and the consultant that are best suited to collaboratively determine what solutions meet the needs of homeowner and home.  Every opportunity with an interested homeowner should be capitalized upon.  Instead of telling people “Not Cost Effective” or “Not an Approved Measure” this approach says “The program thinks this is a nice opportunity to save energy and is willing to contribute $X toward the project.”
• Idea of “Net Cost Design” leveraging improvement budget AND energy savings into larger, more effective projects.  
• Not cherry picking cost-effective measures and missing the opportunity to provide complete solutions.

• Contractor Predicted energy reduction will be compared to actual post-retrofit usage for each project that participates in the program.  A “Realization Rate” will be calculated for each project at regular intervals and aggregated to create a “Contractor Realization Rate.”  Contractor Realization Rates will be published and sorted highest to lowest in a Contractor or “Gerardi Registry” that is accessible by all members of the public.  The Contractor Registry will also allow other comparative, competitive, and quality management metrics using public project data (auditor name, project cost, estimated savings, blower door #, project zip code, etc.).  By allowing graphic representation of this data contractors can compete in ways meaningful to the consumer.   (reference to HPXML minimum dataset.)
• One of the purposes of the “Contractor Registry” is to inform consumers about how accurate contractors are at predicting their energy use reduction.

• This will provide homeowners with a way to select the contractor that best suits their needs based on all available information. (Need deep air sealing talent?  Sort contractors by leakage reduction results)
• Another purpose of the Registry is a Quality Control Feedback Tool for contractors.  It allows them to see how accurate their predictions are, and improve their modeling, construction, bonus best crews, train worst, etc.  
• Creates positive opportunities to correct issues when projections fall short, and further strengthen relationships with clients through proactive behavior.
• Creates a “Race to the Top” for contractors as they compete to have the best realization rate in the program.

• Eventually, the program may elect to use the realization rates as determining factor for automatic job approval.  Possibly top third of contractors would have the most streamlined approval process, providing yet another reason to excel.  The tool would be very effective for identifying contractors who need additional training and support.

How it Would/Could Work (And Why It’s Better)

• Project Design (Contractor)
• Still perform BPI Energy Audit and be required to model the project

• Contractors could design projects with all of the efficiency measures needed and include other improvements.
• Adding in measures that have NO energy savings don’t cost the program anything, but allow people to improve their homes

• Lets clients include Honey Do List items on their work scopes
• Increases the likelihood of sales
• Reduces the cost/lead of sales and marketing

• Comprehensive projects are more likely to result in customer, contractor, and Energy Efficiency success.
• Dramatically reduce sales and marketing cost by increasing close rate.  

• Remove restraints to design of innovative solutions.
• The Contractor Registry encourages/rewards contractors for being conservative with their modeling.
• Although contractors have the ability to overpredict their energy savings (yielding a marginally higher customer incentive), their realization rate will decrease making them appear less competent to future customers, and risking greater scrutiny and difficulty getting jobs approved.
• Contractors who do more diligent work can justify the higher costs through their standing on the registry in metrics of realization, air leakage reduction, and/or modeling accuracy.

• Harnesses the power of feedback loops, transparency, and naturally competitive spirit.
• Make Contractors accountable for Energy Savings.   Currently they are simply doing what is required by the program to get jobs approved.  This removes accountability for results from the contractor and shifts it to the program.
• Shifting accountability from the Program to the contractors arguably removes a huge and growing liability exposure programs have for realization failure.  Studies show programs SIGNIFICANTLY under delivering for a very long time.  The longer this goes on the more appealing a class action lawsuit becomes.  Lawsuits for unmet energy savings projections are becoming increasingly common and FTC is siding with consumers even when claims made are vague.   
• TRUTH and QUALITY are both Great Marketing Opportunities, and encourages salespeople to deliver both.

• Project Approval (NYSERDA)
• Review and approval involves only reviewing projects for health and safety, missed opportunities, and contract language (similar to pre SIR days).  
• Burden of project review for TRC, SIR, etc. and kick back for model or contract “redo” would no longer be necessary,
• Funding burn rate very easily adjusted.  Increasing or decreasing incentive per KW Saved the program can easily manage funding AND program employment.
• This is a simple adjustment mechanism that can be communicated through all layers of the system.  Eliminates complex rules.

• Ability to adjust employment levels state-wide (dial up incentive during lean times, similar to how the FED adjusts money supply) helps avoid layoff, and loss of talent.

• Automatic project approval will be made possible with this system
• A minimum data set will be necessary to allow this.
• Project Construction (Contractor)
• Because tracked realization is ranked and published, contractors are invested in the energy savings of the project and will perform better work
• BD # at or below target
• Contractors are motivated to save even more energy than they model - the idea is that contractors might go above and beyond for their clients (eg. do more air sealing, install CFLs without bother to model, etc)

• Crews have the ability to be individually tracked on the Registry and will be included as part of the competition for excellence.
• The more that the contractors elect to enter more data for the voluntary fields, the more actionable information they will be able to get back.

• Test-In and Test-Out Blower Door Operators will have the ability to be tracked (optional)
• Project Quality Assurance (NYSERDA + Contractor)
• Realization rates are a main driver of quality assurance, and help spotlight projects for review.  This offers both QC and learning opportunity.

• Work would still be inspected for quality, and the contractors needing assistance would be apparent, they’d be the ones with low on the realization registry or with significant results scatter.  

Software Requirements

• Energy modeling software could be flexible.  Contractors can use whatever software they want so long as minimum HPXML Dataset is provided.  Modeled savings by top of the registry contractors would not be held to scrutiny.  

Transitional Accommodations

• In order to effectively move to the Performance Based System, and ensure enthusiastic contractor buy-in, a number of transitional incentives should be considered.

• Contractor incentives for high realization projects
• ex. $500 for projects above .85 or the top 2000 projects a year.  $200 for the next level, etc...

• Awards for top performing contractors in a region/state-wide, top 1st year contractors, top 2nd year contractors.
• This is designed to encourage contractors to start using the Contractor Registry, and performance of good overall work

• Phasing in Automatic Project Approvals
• Communication of this to contractors

Conclusion

• This is a win-win-win for the customer, contractor, and the program. Contractor and homeowner interests align as solutions design, accuracy, and quality supplant "cost effectiveness". Program interests are served as administration and cost effectiveness goals, and adjusting program volume are vastly simplified.

Again, this is only the rough draft.  More work is needed, but only if there is interest.  Comments can be made below.

I’ve made the program outline available on Google Doc’s for public editing (attempting to follow the Wikipedia theme) anybody with constructive ideas is welcome to add or edit.

http://bit.ly/HPwESPerformanceBasedDesign

IN CONCLUSION:  Let’s stop this train wreck.  Let’s advocate for a program where the players are partners, not adversaries.  Where the best work gets rewarded.  No more “this is cost effective.”  “This isn’t cost effective.” “This isn’t cost effective but our magic wand deems it cost effective.” “This is cost effective but it’s not an approved measure.”  Let’s get a program where we can sell what the client WANTS, what the home NEEDS.

Windows?  Sure!  Doors?  Why not!  The whole “Honey Do” list?  NO PROBLEM, just sign here!  Government gets energy savings at the price it wants to pay, consumer gets home improvements for the net cost they were told, and we can focus on DESIGN rather than software games.

WANT BIGGER JOBS?  We need to leverage energy savings into the project budget.  In order to leverage projected energy savings into people’s improvement budgets, people really need those savings to be believable.  A system that rewards contractors for accuracy also creates huge customer confidence that savings will significantly match projections.   It provides the comfort needed for customers to allow those savings to be leveraged into project budgets.   

Sure this simplifies administration not just for contractors, but for administrators.  Sure some at NYSERDA and CSG may see this as threatening to bloated fiefdoms.  But programs that have $3 of total cost for every $1 spent on improvements are not sustainable, much less scalable.  And if this governor is serious about accountability and transparency, lack of accountability and bloat is not long for this world anyway. At some point cost per negawatt will be applied to their work, with states competing against states.  

The flip side of this for NY, if they can act, is they can look like hero’s for paving the way for our nation.  

TRUST, TRUTH, TRANSPARENCY.  Who’s in?