April 29, 2014

How to Predict the Success of a Nonprofit Program Before It Starts

One of the greatest challenges facing nonprofits is proving how effective they are. To determine which social programs are making the most progress toward tackling critical social issues, most groups rely on inconsistent reports, cursory impressions, “brands,” or even politics. But there is a better way.

Measuring how much it costs to produce a single “unit” of impact, called “cost per outcome,” can tell with a high probability whether a nonprofit program will work. This method can save charities time and money and can make it easier for foundations to choose which programs to support.

Cost per outcome is common in other industries, like finance and medicine. It allows them to predict success by picking apart what factors correlate to achieve certain goals.

For example, if you take out an individual loan, a lender can’t tell with absolute certainty whether you will repay it, so the lender makes a judgment based on your income, your payment history, and other information. The lender is making a bet based on data.

Now nonprofits are using cost per outcome, too. A program to get students proficient in math, for instance, can consider factors like location, teacher engagement, students’ language fluency, grade level, and parental involvement to predict how successful a program will be. It can then weigh the costs and effectiveness for different scenarios: Will it carry out the program five days a week with paid tutors in a location it will rent? Or will it ask parents to work with students at home two nights a week?

Then it considers the tradeoffs: One way of running the program may reach 10,000 students, but another way may reach only 5,000 students but with a much higher level of proficiency.

Cost per outcome doesn’t “prove” program effectiveness in the same way a historical multiyear evaluation study does, but it can pinpoint which programs are most likely to deliver the intended impact and forecast how much each will spend to achieve it. A foundation looking to allocate its funds can use those data to make decisions today that will prove 80-percent accurate rather than waiting five years for data that will be 100-percent accurate but perhaps outdated. That’s a pretty good trade-off.

Not all programs lend themselves to this kind of evaluation. Programs that deal with a brand new issue or that are highly innovative by definition won’t have a lot of existing data for comparison. Ten years ago, the data didn’t exist for programs to fight cyber bullying, for example. But cost per outcome does work for the vast majority of programs in the arts, health care, education, economic empowerment, and gender equality, among others.

Cost per outcome has many advantages over more traditional ways of showing a program’s impact.

It saves time. Other measurement approaches often take more than six months after the program is complete—and that’s six months added to the end of what is often three to five years of program activity.

By contrast, a cost-per-outcome analysis can be completed in a few weeks or months.

It can be done before a program even begins. Because cost-per-outcome measurements are based on existing analyses of thousands of programs, you can calculate the likely impact of a program based on its attributes or factors before it even begins.

It creates data you can compare. Cost per outcome makes it easy to compare the performance of programs aimed at the same outcome. So, for example, a foundation can see that it will cost Provider A $1,000 to develop science proficiency for one student, but it will cost Provider B $1,200 for the same outcome.

The more nonprofits use cost per outcome, the more it can help level the playing field among providers because it will matter less if they are big or small, old or new. What will matter most is how much it will cost to achieve results.

It saves money. Charities don’t have to break the bank to evaluate their programs. Cost per outcome can cost as little as $5,000 a program. Many nonprofits spend 5 to 10 percent of their program budget on measurement, so for programs costing well over $100,000, that can be a big savings.

In a nutshell, cost per outcome can help charities spend less time tracking and reporting on idiosyncratic performance measures and less money on programs that don’t deliver. That in turn frees them up to focus on making an even greater impact. Who wouldn’t want that?