A small group of researchers has long pushed for foundations to bring the same equity-focused lens they might use in designing programs or hiring staff to their process for evaluating outcomes and impact.
But doing so can be challenging — in part, because it brings evaluation front and center to the core work of the project. Efforts like the Equitable Evaluation Initiative call for considerable community input. If you’re going to get that input, it needs to happen during the design of the program — not at the last minute. Program design and the evaluation process need to be ironed out in tandem, with input from the community the project is designed to serve.
Now the Annie E. Casey Foundation is out with a report on guiding questions to make it easier to work collectively with communities while being aware of racial and cultural contexts. The tips come from several years of work by Casey to turn its long interest in this topic into “actionable steps to negotiate what can feel like a large, overwhelming process.” (The Anne E. Casey Foundation is a financial supporter of the Chronicle of Philanthropy.)
Many of the guiding questions focus on making sure that funders and researchers consult and partner with people in the communities that are being studied. Are communities engaged in the design of the project? Are community members recognized for the unique expertise they might bring? Are they being compensated for their contributions? Are community assets being valued?
Kimberly Spring, the Casey foundation’s director of research and evaluation, says she hopes the paper will help other evaluators and funders use equitable practices and respond to the cultural backgrounds of communities.
“There’s no perfect way to do it — don’t worry about getting everything right,” she says. “But try — and have the humility to recognize that you’ll make mistakes.”
‘Adding Important Voices’
When Casey started this work, it initially partnered primarily with kindred spirits — including the Equitable Evaluation Initiative, which developed a framework that is described in the report as a “key influence” in the creation of Casey’s guiding questions. Now Casey is happy to assist foundations that have little experience incorporating equity into evaluation — including larger commercial research firms that may hesitate to reduce the influence of highly credentialed evaluators.
“We’re not excluding the expertise of the researchers,” Spring says. “We’re building on that and adding important voices and perspectives. This is really about doing human-centered design work.”
There’s a growing consensus among proponents of equity-centered evaluation that traditional philanthropic research can be extractive and exploitative. Spring points out that Casey and other funders have long worked in low-income neighborhoods in Baltimore, where community members were treated for too long like lab subjects. Researchers would go in, collect data from residents, and then leave — often sharing the data with other researchers and funders, but not making much effort to get information back to community members themselves, Spring says.
Casey’s effort and the Equitable Evaluation Initiative both started around the time that prominent leaders in the philanthropic sector were calling out this phenomenon — and the racial inequities it perpetuated. In 2017, while leading Grantmakers for Effective Organizations (she has since become president of the Council on Foundations), Kathleen Enright wrote that the “work to define effectiveness has typically come from white organizations — prominent consulting firms, think tanks, universities, philanthropy, and management support organizations.” That work, she wrote, “has unwittingly perpetuated or even exacerbated inequity in the nonprofit sector.”
Since the Equitable Evaluation Initiative was founded in 2019, more than 60 foundations have committed to the two-year process of using the initiative’s framework. Marcia Coné, a lead consultant who helped design the framework, says the Casey foundation’s guiding questions will help the many foundations that have not yet incorporated equity into their evaluation work.
“It’s a beautiful expression of how to ask questions that get us to new and different places in the work,” Coné says.
The EEI, founded by Jara Dean-Coffey and supported by several foundations, was designed to be a five-year project that would wind down this year. But it has been extended to 2028 to allow more foundations to go through the process. Coné says that after EEI sunsets, the framework will be maintained by the participating foundations.
Community Advisers
One of the most important contributions of bringing an equity approach to evaluation, Spring says, is the ability to uncover important elements of community progress that might be lost in traditional assessment methods.
She offers two studies of the work done at community colleges by the Working Students Success Network as an example. In the first study, researchers focused on the impact that the network’s interventions had on success rates for low-income students and students of color. In the second study, in 2020, the researchers sought input from students about what was helping them succeed. The students highlighted the importance of existing cultural programs for creating a sense of belonging that had helped them succeed.
“If we hadn’t taken the time to step back and think about this from more of an equity and inclusion perspective, we would have missed the value that these groups play for students of color,” Spring says.
The cultural programs had been undervalued within the community-college ecosystem, but the evaluation led to advocacy for those programs to play a more central role, Spring says.
Foundations and researchers don’t need to use all the steps in the guiding questions or go through them in any particular order. But they should be prepared, Spring says, for a process that takes time.
Many charitable programs serve low-income people who work multiple jobs. Seeking input from these community members requires flexibility on timing, and potentially child care, meals, and compensation. Casey has moved to a process in which community advisers are paid at levels that are comparable to other consultants hired by the foundation, Spring says.
“If we’re saying that their experience is important and valuable,” Spring says, “then we need to show that we really value that in terms of how we compensate them.”
