Delta Partners Management Consultants
Your trusted advisors.

Efficiency Indicators: Some Recent Lessons Learned

Francis Loughheed

As we have spoken about previously, Treasury Board Secretariat (TBS) has added program efficiency indicators to the MRRS refresh for 2014-2015.

Lessons Learned

In March we launched our Efficiency Indicator methodology, and over the past two months have worked with over 60 programs.  The results have been very positive, and we have learned some interesting lessons:

  1. Outcome vs. Output:  For the last 10 years, program managers have been trained to focus on outcomes – the results and impacts of program activities and outputs.  This orientation has become so ingrained that initial attempts to articulate an efficiency indicator for a program usually focus on the outcome, not program outputs.  The result is an ‘effectiveness indicator’, not an efficiency indicator.

    Green-light-bulbEfficiency is about how well a program transforms inputs (money, people, data) through activities to produce outputs that contribute to the results and outcomes.
  2. Refine the value proposition:  Our methodology builds and confirms an output indicator linked to the value proposition of a program.  Existing TBS documents have described generic activity value chains.  However, the methodology we are using has given us tremendous insight into activity value chains that are more mature, and provide greater understanding of the value a program is able to produce.
  3. What you can control:  One of the biggest lessons we have learned is that a program efficiency indicator needs to report on the activities a program has direct control over. In many of the activity value chain analysis we have completed, outputs are created that are then approved or reviewed by outside stakeholders.  The efficiency of that process is outside the control of the program and needs to be excluded from the program efficiency indicator.
  4. Repeatable indicators:  The biggest confirmation of the value of our methodology is that similar programs can, indeed, have similar indicators.  Indicators are repeatable based on the output classification process being used. 

Immediate Benefits

We have also identified some immediate benefits to the methodology for program managers and executives:

  1. Causes program staff to revisit their logic model and confirm the validity of the model: The efficiency indicator process we are using links efficiency to effectiveness, and thus the efficiency indicator to expected results/outputs.  The resulting performance story enhances the value proposition of the program.  In doing this analysis, program logic models (which may have been ignored since the date of publication) need to be revisited and validated.  The benefit is a reworking of the logic model to truly link program activities and outputs, and the contribution they make to outcomes.
  2. Enhanced performance stories:  In the RPP and DPR, program performance stories have had a primary focus on impacts, expected results, and outcomes.  Efficiency indicators based on an activity value chain analysis adds a new dimension of authority and credibility to a program performance story.  The benefit will be much better program reporting and management.
  3. Output to outcome linkages:  This process will also enhance evaluation approaches by being able to more concretely link activities and outputs to the contribution they make to outcomes.  This has traditionally been a weak area of evaluation, and we see that our methodology supports a stronger evaluation capacity by being able to tell a more complete program storyline.

Finally, we have learned that this methodology provides effective support to programs struggling to define their efficiency indicator.  Once the basic principles and methodology are well understood, we can get to a draft efficiency indicator very quickly.  The process becomes one of refinement and validation with program management and staff.

The ultimate impact can be a more effective program management approach that measures answers two questions:

  • Are we doing the right things, and
  • Are we doing things right?

If you are interested in learning more about the Delta Partners methodology for developing efficiency indicators, the following presentation will outline the basic approach that we follow:

 

 

 

 

Add a Comment


Notify me of follow-up comments?


About this Article

Posted by Francis Loughheed
Posted on July 9, 2013
0 Comments

Share |

Categories: evalution, lessons learned, management, performance measurement