Government and philanthropy have made big strides in their efforts to identify high-impact programs that, through rigorous testing, have been shown to deliver strong positive results for society.
But it’s not enough to identify what works. We have to understand how programs work so nonprofits can duplicate them nationwide.
To help find answers, we examined the federally funded Teen Pregnancy Prevention program, which has awarded $75-million in five-year grants to 75 grantees in 37 states. It shows real promise as a model for how donors—government or private—can support efforts to put proven programs in place. In fact, the prevention program is one of a small but growing number of federal programs that require grantees to choose from a list of practices that in scientific studies have been shown to work.
The challenge with any proven program is how to adopt it successfully in dozens of locations. That’s far easier said than done. Just ask anyone who has struggled to assemble an Ikea bookcase or reproduce a dish from Mastering the Art of French Cooking.
It’s important to follow the instructions as closely as possible to guarantee good results. Improvising with a recipe may seem inconsequential but can ruin the dish. Similarly, improvising with a proven social program’s “recipe” for success can undermine results, a lesson social scientists have learned from a growing body of research.
In the case of the pregnancy-prevention program, the federal Office of Adolescent Health announced that it would be an active partner with grantees in helping them run and evaluate programs. It trained and coached its own staff members to support grant recipients. It also gave grantees a full year to plan, test, and get problem-solving help, including consultations with the program’s designers or with outside experts.
The Adolescent Health Office also set out to ensure that grantees rigorously followed each program’s playbook. For example, it requires grantees to collect and report on a uniform set of performance measures.
Lessons from the pregnancy-prevention program point the way for any donor to follow when it comes to supporting local adoption of programs with proven impact. Among the lessons:
Spend money and time to identify approaches that are worth copying. The Office of Adolescent Health came up with 31 programs proven to reduce pregnancy among teenage girls. But it’s not possible to find that many options for many other causes, such as preventing youth violence or reducing childhood obesity. Foundations can help fill the void by financing evaluations that identify programs that work for a range of social problems.
Help nonprofits and government agencies figure out what approaches will work best in their communities. Some nonprofits and agencies realized too late they had not picked the prevention programs best suited their communities or that best matched their organizations’ skills. It’s not enough just to offer a menu of programs that works. Nonprofits need to better understand how to select an approach that is a good fit based on the people they serve, the budget they have, and the capabilities of their employees.
Find nonprofits and agencies that are strong enough to run a program well. Most of the pregnancy-program grantees are local nonprofits and government agencies. The grant-application process was fiercely competitive, with more than 1,100 applicants for 75 grants. Rigorous vetting of nonprofits is essential in finding organizations that know how to follow the steps of a new program and to make sure nobody cuts corners on essential components.
Donors need to provide continuous support and accountability. The Office of Adolescent Health sought to build strong relationships with grantees, give them enough time and money to plan adequately, and help them stay laser-focused on following every prescribed step to be successful. That blend of close monitoring and strong support—something that philanthropy can emulate—may be one of the most important things that donors can provide.
Hire experts to serve as coaches and trainers. For each of the 31 pregnancy-prevention approaches that nonprofits could choose from, the Office of Adolescent Health hired the program’s developer or an outside expert to create documentation and deliver training and coaching to grantees.
Keep looking for ways to improve programs. Even though the pregnancy-prevention efforts had all been subjected to testing to prove they worked, that doesn’t mean they cannot be strengthened. As more and more groups try an idea, it’s essential to pool the findings to learn what works best in different communities.
The trickiest part for the nonprofits that got money from the pregnancy-prevention program is what comes next. Many of the projects are not self-sustaining, so it’s unclear what will happen after the federal money dries up. Most of the grantees say they are already worried that they will not be able to keep running their programs and achieve high-quality results unless they can replace the federal money.
But finding that money is likely to be difficult. Suppose a new donor wants to support pregnancy prevention but doesn’t like the approach an organization has taken in the past. Another donor might encourage greater experimentation or shortcuts and not believe that carefully following a tested program’s steps is critical to success. Or new donors might not fully understand that effective training and coaching are essential, and not just a frill, when it comes to producing results.
Not all nonprofits or donors will want to embrace the textbook approach required to adopt a proven program. For many organizations, curtailing staff members’ autonomy is not natural, nor is a focus on consistency in delivering step-by-step programs according to design. But if nonprofits and their supporters want to get the results that evidence-based programs are shown to achieve, it’s important to follow the “recipes” that earned proven programs their stellar ratings in the first place.
Alex Neuhoff is a Bridgespan partner in New York, Laura Burkhauser is a senior associate consultant in San Francisco, and Bradley Seeman is a Bridgespan writer and editor in Boston.