Outcomes, Quality, and Evaluation, OH MY!

The human services non-profit sector is in the midst of a management revolution, a revolution built on measurement.

axstj-evaluation-1013-1993.jpg

Words like evaluation, performance management, outcomes measurement, and performance based contracts are now joining the ranks of quality assurance, compliance reviews, and performance audits in the minds of nonprofit leaders.  With all of these concepts flying around, many non-profit leaders don’t know the difference between them; they just want to be running effective programs!    So let’s say you want to get in on all of this ‘outcomes’ stuff – where do you begin?  What does this all even mean?

This post is my attempt to cut through the confusion, and define some of the main differences among the different ways of measuring nonprofits.

Compliance vs. Excellence

One the most important distinctions to make is between measurement for compliance and measurement for excellence.  My work at Project Renewal has four main components, which I will use as examples to differentiate those two approaches.  Those components are (1) regulatory compliance, (2) quality assurance, (3) performance measurement, and (4) evaluation. 

The first two items are about making sure that we are doing the minimum necessary for our nonprofit to be considered doing ‘good’ work.  These are more traditional models of measuring the work of a nonprofit, and ones that most leaders should be familiar with.

  •  Regulatory compliance is about following the rules – if we didn’t meet these criteria, we could be facing some serious penalties.  We measure this through activities like doing compliance audits, monitoring activities, and investigating issues as they arise.
  •  Quality Assurance, in the human services world at least, means making sure that the services that we are providing meet certain quality standards.  We measure this through activities like reviewing service documentation and getting feedback from clients and staff.

The last two items on my list aren’t about doing the least that we can do, but about being as excellent as we can be.  Instead of just trying to clear the low bar, we strive for the gold metal and setting new world records for how amazing programs can be.

  • Performance Measurement and Management is perhaps the most useful measurement solution to nonprofit leaders, and if your organization is not currently doing this, it’s time to fix that.  Performance Measurement work involves identifying important performance measures, regular data collection, and using tools like reports and dashboards to monitor performance.  Performance Management is then using that data to manage your staff and programs.
  • Evaluation is a much more rigorous and well defined set of activities – in fact, there is even a professional association dedicated to this field.  Evaluations are generally conducted by professional evaluators to answer specific questions about the effectiveness of various programs, although there are lots of types of evaluations done in lots of different ways.  Generally, nonprofit capacity to conduct evaluations comes after capacity to do performance measurement.

The difference between performance measurement, evaluation, and research is often hard to understand, but hopefully the chart below outlines it a little clearer.

Hopefully this table makes it all a little easier to understand.

Hopefully this table makes it all a little easier to understand.

Your priorities and resources are will determine which way you want to approach measurement, as will the type of program that you run.  You may use one or more of the above types of measurement in your organization, and you might be doing things that I don’t talk about here.  There are lots of other ways to think about measurement in nonprofit organizations, but I hope this provided a framework for you to think about measurement in your nonprofit workplace.


Patrick Germain

As Director of Strategy and Evaluation at Project Renewal, Patrick lGermain leads a team of internal evaluators, conducts a wide range of evaluative activities, and manages any external evaluator relationships.  Patrick also runs a professional networking group on issues of performance measurement and management in the non-profit and public sectors which currently has over 120 members and has had ongoing bimonthly events since its inception. Patrick has an MA  in Public Administration from NYU Wagner.