Sunday, February 10, 2013

February Contractor Update, some new data - Happy New Year!

Dear Contractors,

You may find these numbers from 12,559 HPwES jobs shocking.  They may make you uncomfortable.  Certainly this is a very different look than the day to day look at individual jobs.  When you look at larger numbers patterns emerge.  But don't you want to KNOW how you are doing?!?

TRACKING MATTERS - Bill Gates says just the act of measuring causes polio cases to drop in African Nations.  The premise is operators use tools more effectively when they know their success, and their failure, will be open to public scrutiny.  Carrot and stick.  I believe the same is true for energy which is why I've been digging for these numbers.  Taking a common phrase one step further; Measuring Causes Management.

AMBIVALENT. INCOMPETENT. FRAUDULENT. Is that how you want your company, or this industry to look?  Sure, there's no upside to diligence that nobody measures but that is at an end.  As you can see, measurement is happening.  There will be great reward for accuracy in the form of public confidence and capital access.  This is the path to monetizing Energy Efficiency as a resource.  I've got a carrot and a stick here, let's not have to use the stick.  

Some of you may be aware that last spring I requested and received a large amount of data under the Freedom of Information Act from NYSERDA regarding your HPwES projects.  I now have blower door numbers for all of your projects for the past two years through November.  Test in, projected, and test out.  Below is a representation of how well you have done against yourselves, and against your peers.  

You also may recall that I feel this program needs to move toward a new competitive metric - one of published ability to deliver savings projected.  This will do for our industry what MPG ratings and JD Power did to the auto industry, it will create incentive to excel around a metric that benefits the consumer and adds credibility to our promises.  It will allow us to market based upon truth and accountability, not smoke, mirrors and hope.  Sell to reason, not fickle emotion.  
A program that can deliver TRUTH is unique in today's marketplace.  This could elevate our profession immeasurably! We may have to go through slight discomfort to get to a better place.  

Tracking before and after energy would be ideal, but right now there is an access problem to energy consumption data "after" improvements.  Until we can solve this, do we have anything that might indicate what results to expect by contractor?  Proxy for energy projection accuracy?  Well, besides "reported energy use before" and "modeled energy use before," we have blower door leakage, projected leakage reduction, and actual leakage reduction.  


In my last e-mail I discussed the importance of accuracy of modeling energy input, and provided an image of roughly where you stand against your peers with respect to energy truing accuracy.  The images below/attached show were you stand against your peers with respect to blower door projection accuracy.    Below is PROJECTED reduction, ACTUAL reduction, and Percentage of promise delivered.  In other words, how good are you at projecting?  How good are you at tightening?  And do you sand bag, nail it, or over promise?  (1300 sf house with 10k test in, 9k promise, 3k test out, really?)

There are a lot of ways these early numbers might misrepresent actual work performed.  For example; exaggeration of test in leakage (probably to obtain SIR) could make some look like they are doing more than they are.  (No regression analysis was performed to determine if/which Test In numbers were exaggerated, but I know for a fact people are doing this and suggest that practice stop immediately.)  

The intent of this is to inform not embarrass, so inaccuracy of these numbers helps as they can only help inform, not be used to judge.   But good data matters so going forward there is no amnesty for ANYONE around the data.  


We all know there is a direct and significant correlation between energy consumption and air leakage.  So it is reasonable to assume leakage reduction has direct and significant correlation to energy SAVINGS.  Accuracy in your blower door projections will show up in your ability to accurately project savings.    

I know these numbers - like the ones I sent last time - may be new to many of you.   What they mean may take a little time to sink in.  But please get your heads around this if you wish to avoid looking incompetent when compared to your peers once this registry goes live!  Again, forewarned is forearmed.  After today, claims of ignorance of harm will not be a compelling defense.  (I keep being asked for names.  The media loves nothing more than to go after contractors taking advantage of "defenseless consumers".)   

Question - What happens when customers you've projected $800 annual savings call you saying they only saw $150?   What happens if your model for them is full of inaccuracies? Over 10 years that's $6500 short.  Let's head that problem off before we get there? 


We need good inputs to prove Residential Investment Grade Energy Modeling is achievable!  Then we can move to PERFORMANCE BASED INCENTIVES, and ultimately away from the need for any incentives to drive Comprehensive Home Performance.  How nice would it be to show inaccuracy and scatter is caused by poorly designed process and corrupting program incentives rather than by contractors or software?  

It is a forgone conclusion that you will be measured against your peers upon your ability to accurately project savings:  

Inline image 3

(to see these larger go here:, and

Do YOU Deliver On Promise?  If you want to be at the top of the "delivers on promise" list for energy savings (closest to 100%), make sure your inputs are accurate!  A rising tide raises all boats, except the ones with huge holes.  If you have big holes, plug them now because the tide is coming in.  It appears some people are definitely on the right track, and bravo to you! 


Apply pressure on CSG and NYSERDA to help you improve air sealing, and prediction.  Make them accountable for helping you succeed.  Position yourself to take advantage of transparency.  

Remember, progress will be a strong defense to poor performance, so let's start tightening things up BEFORE we have tools that expose this work to public scrutiny.  I believe NYSERDA intends to improve reporting.  Demand this so you will not be caught unaware of your numbers in the future.  

Do you really want ME to have your numbers before YOU have them? 

COMPETE FOR RESULTS:  By tracking and competing for results we make Quality measurable.  Great contractors are starting to do this:

I propose a competition for accuracy, which in turn will drive AND REWARD excellence.  It will allow HPwES contractors to TRULY differentiate from Joe Six Pack.

I hope this expresses the importance of blower door input accuracy, and how allowing inaccuracy to go unchecked will be a big problem for YOU when we achieve tracking of savings realization.  

This goal of tracking realization, BTW, may not be far away.  NYSERDA now has direct access to energy consumption whereas before they apparently had to hire 3rd parties to access bills and perform analysis.  

Do I need to spell out what that means?  The registry is right around the corner!   My next e-mail may unearth your ability to deliver savings on promise....  


Ted Kidd
(585) 205-8674 Direct 

Published Realization Rates - Let's take the "blind buy" out of home performance.
PS.  If anyone would like the spreadsheet, and the various worksheets I built to help me think about these numbers, it is available HERE.  Type (Cntrl+S) or click the down arrow to download.  For those with older versions of Excel, here's the 1997-2003 version.

This email will be posted to my blog as was the last one.  You can see those conversations here:

The closer to the grey line, the more accurate the prediction average.  This does not look at variance.



 update subscription preferences