These days, many people who evaluate nonprofit programs emphasize the importance of involving program participants in efforts to measure how well programs work. After all, the participants are the true experts of their own experience; they know what makes a difference in their lives and communities. But the nonprofits that take this approach reap an unexpected reward as well.
Recently our organizations, Pace Center for Girls and MilwayPLUS, conducted a survey, interviews, and a series of focus groups with 15 nonprofits, including Pace, that are involving participants in program evaluations.
When program participants are full partners in the evaluation — including designing the research and gathering and analyzing data — our study shows that the benefits go well beyond more effective evaluations. Equity and inclusion are advanced throughout an organization, leading to more inclusive practices in hiring, service delivery, and using technology to reach and communicate with participants in ways that are easiest for them.
For instance, 60 percent of our study respondents said the use of participatory approaches in evaluation, such as surveys, focus groups, and town halls, led them to hire differently — specifically, to prioritize candidates who experienced the issues their programs aimed to solve.
On average, study participants also reported a 27 percent increase in efforts to actively seek feedback from everyone connected to their programs. Such efforts included creating participant advisory councils to weigh in on strategy, processes, and policies, and appointing former participants to their boards of directors.
Almost all the nonprofits surveyed found that using more equitable evaluation practices strengthened their ability to respond to the health, economic, and racial crises of the past year.
Here are some real-world examples to demonstrate how seeking input from clients or members can advance equity in other aspects of your work, along with steps you can take at your nonprofit to make your operation more equitable.
Hiring: Nonprofits should recruit staff members whose experiences are similar to those of program participants. The members of the focus group revealed that strong relationships between staff and participants are vital to participants feeling respected and wanting to persist in a program. (They also found causal links between that persistence and outcomes.)
For example, Nurse-Family Partnership (NFP), which pairs nurses with young first-time moms in low-income communities, staffs its outreach team primarily with moms who completed the program, women who know where to find potential participants and how to motivate them to join.
At Lake County Build a Generation in Leadville, Colo., which hires constituents to conduct research to improve local health and well-being, at least 25 percent of community researchers are people who have experienced challenges such as access to food or affordable housing. However, many potential applicants don’t see themselves as eligible. “What we’re really trying to think about is how to phrase job descriptions so that the people we serve see themselves in the job,” says executive director Katie Baldassar. One way to do that: share videos of program alumni working as staff members and explaining their jobs, so potential applicants see and hear from someone like themselves.
Delivering Services: When gathering participant feedback on program elements, elicit suggestions for improving them — the easiest, least expensive path to innovation. Union Capital Boston (UCB), which helps residents of low-income communities by rewarding them with Visa gift cards for participation in school and civic events, started surveying community members several years ago. The organization learned that participants wanted to meet each other. So UCB created weekly Network Nights and invited members to convene to discuss topics they chose and facilitated, and to exchange favors and goods.
“At the very start of the pandemic, we were able to raise $400,000 for Covid relief and distribute it quickly via $150 gift cards because we had the rewards model,” says Jalina Suggs, director of networks. “And we converted our Network Nights to virtual forums.”
After George Floyd’s murder in Minneapolis, Network Nights became even more crucial. “Our next Network Night was an opportunity for members to share how they were experiencing it,” says Suggs, who invited organizations working on restorative justice and defunding the police to attend.
Communications: Use technology to seek regular input and meet people where they are. Ben Goodwin, executive director of Our House, which supports people as they move from a shelter to living independently, began surveying clients in 2015 with a tool called Listen4Good. The insights gathered led to the creation of a client council. Today, the council interprets data from the group’s biennial survey into action agendas. The habit of regularly surveying clients has led to a culture in which staff regularly ask clients for input on big decisions. “When the pandemic hit, so much was changing and unknown,” says Goodwin. “Our staff just naturally went to: ‘Hey, let’s send out a survey. Let’s see how many participants are thinking they might put their kids in in-person school versus virtual.’”
Technology initially intended to gather feedback can become a lifeline for service delivery. For example, Pace expanded its videoconferencing technology during the pandemic to connect with 2,000 girls. Through videoconferencing, the group met with 75 percent of its girls, all of whom qualify for free or reduced lunch, seven or more times per week — delivering food, computers, and telecounseling. In 2020, 91 percent of the girls completed the program (vs. 81 percent in 2019).
At NFP when social distancing began, nurses shifted to communicating with their clients by phone or videoconference. Meanwhile, the organization partnered with Verizon to provide equipment to first-time moms who lacked phones, and logistics company Action Technologies Group delivered them at no charge for nurses to distribute. Despite some challenges, the corporate-nonprofit collaboration allowed NFP to reach, and nurses to stay connected to, 3,800 clients throughout the pandemic.
More Resilient Cultures
By conducting participatory evaluations, nonprofits will find that their organizational cultures support quick, smart shifts that create new ways of listening to, and working with, participants to respond to trauma, injustice, and emergencies.
Thirteen of the 15 nonprofits we studied said participatory methods made them more resilient in navigating the pandemic. All 15 said they benefit from having rapid ways to learn participants’ thoughts and needs.
However, developing a culture of feedback will open the door to difficult conversations as participants grow comfortable expressing dissatisfaction and pointing out what hinders them. After Floyd’s murder, staff at My Life My Choice, a nonprofit for sex-trafficked youth, worked with leaders to review policies and practices and address any that could unintentionally fail to empower its diverse and survivor-led staff. “We had important and very hard conversations in that period to build greater racial equity into our culture,” says AJ Espensen, manager of outcomes and evaluation.
These conversations will elicit important information that experimental evaluations don’t reveal: participants’ perceptions. “We are at an inflection point in evaluation methods where what counts as truth and what counts as data is shifting,” said Subarna Mathes, Ford Foundation’s senior strategy and evaluation officer, in an interview.
Bias is inherent in all measurement, but when organizations shift to participatory methods, the bias shifts to favor the people who must live with the results. Nonprofits that use this approach will find themselves creating more equitable evidence of impact and advancing equity throughout their organizations.
