Good Shepherd Services is proud of its program that provides transitional housing to young adults who are leaving foster care or are at risk of becoming homeless.
But measuring the impact of the 10-year-old program has always been a challenge for the New York charity. Last year, a partnership with a unit in the New York City mayor’s office—the Center for Innovation Through Data Intelligence—allowed the charity to conduct a study using data from several city departments, including children’s services, homeless services, and corrections, to assess the outcomes of the program.
The study found that those who participate in the transitional-housing program, called the Chelsea Foyer at the Christopher, were a third less likely to end up in an adult shelter and 55 percent less likely to go to jail than a control group.
The partnership with the Center for Innovation through Data Intelligence marks Good Shepherd’s first experience with big-data analytics—a “tool” that many nonprofits are interested in but few are using, according to a new survey conducted by the Bridgespan Group in collaboration with The Chronicle.
The new study identifies the types of management practices, or tools, that nonprofits are using and highlights which tools are proving most satisfactory for charities. Bridgespan, a nonprofit consulting group in Boston, intends to update the survey every two years. The survey is modeled on a similar study of businesses conducted since 1993 by Bain & Company, which incubated Bridgespan.
“The Bain study has become the Consumer Reports for management tools in the for-profit sector,” says Katie Smith Milway, a Bridgespan partner who oversees its research and publishing. “We see the need for something similar in the nonprofit sector.”
Big Groups Benefit
The Bridgespan study found that large organizations like Good Shepherd, which serves 29,000 people a year at 80 sites, are more likely than smaller organizations to use management tools.
Charities with at least $50-million in annual revenue used an average of 14 tools, compared with an average use of just nine tools for charities with $1-million or less in revenue.
The report cautions charities not to take on too many new management tools at one time. “Rather than go too far too fast, it’s best to take a hard look at which tools could bring value and focus on the few that can bring the most,” it says.
Several tools are already widely used by nonprofits, according to the survey, which is based on responses from nearly 500 organizations. More than two-thirds of organizations collaborate with other groups, use social media, and conduct strategic planning, program evaluation, and performance measurement.
Some newer tools are not yet widely used. Only a third of responding nonprofits engage in contingency planning, a process for handling unexpected outcomes that could have catastrophic consequences, and fewer than a quarter use design thinking, a way of finding practical, creative solutions for problems, with the goal of an improved future result. Only 15 percent are using big-data analytics, the approach that helped Good Shepherd assess its transitional-housing program.
Even so, the survey found that nonprofits are eager to experiment with up-and-coming tools. While fewer than one in six nonprofits are using big-data analytics now, nearly half say they intend to use the tool in 2015.
Good Shepherd turned to the Center for Innovation to get a much more data-intensive evaluation of the Chelsea Foyer program than it ever had before.
Previous studies of the program were based on education data, which provided only a narrow snapshot, says Miranda Yates, the charity’s director of program evaluation and planning. By using a much broader set of data, including information about homelessness and prison sentences, Good Shepherd was “able to look at the whole person across a variety of contexts,” Ms. Yates says.
“We were able to get really interesting insight into what goes on when young people leave the program,” Ms. Yates says. “The results speak to the impact of our intervention.”
Strategy Tools Popular
Bridgespan consulted with an external panel of two dozen experts to help identify the 25 management tools that make up the main focus of the new survey.
Among broad categories of management tools, Bridgespan found that organizations were most satisfied with tools that helped them clarify strategy, like developing a mission statement and a theory of change (the process of outlining how short-term goals lead to long-term progress) and engaging in strategic planning and contingency planning.
Tools that help charities take advantage of new opportunities, like partnerships and collective-impact collaborations (in which local organizations work together to solve systemic social problems), were next in popularity, according to Bridgespan. Charities were least satisfied with tools to help them understand their own operations, like market mapping (identifying key players in a sector and classifying them by relevant characteristics), donor segmentation, and financial models. At Hillside Family of Agencies, in Rochester, N.Y., one of the most useful tools has been Lean Six Sigma, a management process aimed at creating efficiencies and reducing waste.
Dennis Richardson, the charity’s president, says the process has led to significant reductions in the time it takes for the charity to accomplish two important tasks: providing services after a referral and moving adolescents from foster care into an adoptive home.
To date, the charity has used Lean Six Sigma on 27 projects, leading to an annual savings of more than $200,000. The efforts have been popular with donors, some of whom help subsidize the cost of arranging adoptions.
“We want to drive down waste so that we can use our philanthropic dollars more effectively,” Mr. Richardson says.
Financial Support
For each of the 25 tools included in the Bridgespan survey, a greater percentage of groups intended to use the tool in 2015 than actually used it in 2014. This trend was greatest among the smallest organizations. Charities with budgets of $1-million or less intend to use 18 of the tools in 2015, twice as many as they used in 2014.
Who Pays for Evaluation: Nonprofits Aren’t Getting Much Aid
81% say their supporters increasingly are channeling money to groups that can show evidence of success
But only 18% say their financial supporters are increasing the amount they provide for program evaluation
And 49% of nonprofits are stepping up their own spending on program evaluation and measurement
“That finding is indicative of a real appetite for tools that is out there in the sector,” says Chris Lindquist, a Bridgespan senior director and a co-author of the report.
In some cases, lack of money may hamper the use of tools.
For example, more than 80 percent of charities said their financial support will increasingly be tied to evidence of success, but fewer than one in five of the respondents said their donors were providing more support for program evaluation.
Even so, nearly 50 percent of the groups said they had significantly increased investments in performance measurement and program evaluation.
However, charities may be overlooking opportunities to use low-cost tools to improve their operations. Sixty percent of charity leaders said that hiring, training, and retaining staff members were among their organizations’ “greatest challenges.” Yet only 40 percent of charities were using tools to assess talent development, and only 38 percent were engaging in leadership-succession planning.
“That’s a call to action for nonprofits,” says Ms. Milway. “They need to stop complaining about losing talent unless they’re going to invest in developing their talent.”
Seeing Impact
The best management tools can provide nonprofits with systematic ways to improve their operations.
Boys Town, an Omaha charity that works with at-risk children, has adopted the “balanced scorecard,” a strategic-planning system that uses an array of measures to give a balanced view of organizational performance.
Though originally used by for-profit companies to weigh nonfinancial measures equally with profits, Boys Town found that using the system led to a greater focus on the financial bottom line.
Ten years ago, says Barb Vollmer, the charity’s senior vice president for governance and strategy, Boys Town regularly measured the effectiveness of its programs but paid less attention to financial metrics.
“We were running some programs that were costing us money and were having little impact on kids,” Ms. Vollmer says. “Now we’re much more business-focused. We’re thinking about the return on our investment.”
The report, “Bridgespan Nonprofit Management Tools and Trends 2014,” by Paul Carttar, Mr. Lindquist, and Amy Markham, is available on bridgespan.org.
