To the Editor:
The recent Chronicle article “Data-Driven Program Evaluation Hurts Equity Efforts, Nonprofits and Critics Say” is an unfortunate and potentially harmful mischaracterization of the movement toward more equitable evaluation. In fact, equitable evaluation does not suggest that data hurts equity efforts, nor does it suggest any particular type of data is better. Rather, equitable evaluation addresses the underlying questions about the who, what, and why of evaluation.
Evaluation means “assigning meaning or value” to something — a practice, a program, a service, etc. When evaluation is equitable, we begin with questions about who gets to assign meaning or value, what needs to be evaluated, and why a particular evaluation is selected. We question who owns the data and who gets to say what the data mean.
As evaluation is currently practiced, too often those who are given the opportunity to answer those questions are the funder and the professional evaluator. Those whose experiences are being put under the microscope are not included in decision-making and very often do not have access to the information gathered.
Even if the evaluation is intended to address issues of equity, it embodies inequity in its very practice.
Evaluation uses the tools of science — systematic inquiry, rigorous data collection, and analysis — to help stakeholders understand the impact of the work they do. This can lead to the assumption that evaluation is “value neutral” because it is “scientific.” However, choosing what data to collect, how it is collected, and who interprets it are all value-laden choices. The understanding of impact will be incomplete, if not outright wrong, if the process is driven only by the interests and values of the most powerful stakeholders.
An equitable process includes and amplifies the perspectives of those living the issues, embodying equity and resulting in a more complete understanding of impact.
Current evaluation practice tends to favor quantitative measures, usually to report progress on outcomes by a funder or nonprofit in service of being strategic. We are not against strategy or quantitative data. However, when institutions with the most power decide what those measures are and what they mean without engaging with the least powerful to understand what those numbers actually mean in people’s lives, that is shortsighted and ethically compromised.
In today’s fast-paced world, we often want only the most relevant-to-us data in order to quickly assess the success or failure of our efforts. However, when we do this, we ignore the presence of nuance, the way that systems interact with people’s lives, and the multiple layers of cause and effect that have an impact on our outcomes. Equitable evaluation requires that we pay attention to, in fact elevate, the nuance.
When a funder or nonprofit professes that they are working toward equity, evaluation should be in service of that goal. We represent foundations, consultants, and academic researchers who understand evaluation can be a tool to achieve intended outcomes; it is not just an impartial assessment of cause and effect. Good data from and for all stakeholders is critical to assigning value and meaning and an example of equity in practice.
When you take two words that have many different interpretations — “equity” and “evaluation” — and put them together, you are bound to get confusion about what the phrase means. None of us have all the right answers about how to do equitable evaluation. The Equitable Evaluation Framework is emerging, but much remains to be learned, and we are grateful that the philanthropic sector is beginning to have this discussion.
Teri Behrens
Dorothy A. Johnson Center for Philanthropy
Grand Valley State University
Tracy Costigan
Robert Wood Johnson Foundation
Kim Leonard
Oregon Community Foundation
Trilby Smith
Vancouver Foundation
Kim Ammann Howard and Fontane Lo
James Irvine Foundation
Kimberly Spring
The Annie E. Casey Foundation
Kristy Klein Davis
Missouri Foundation for Health
Meghan Duffy
Grantmakers for Effective Organizations
Melissa Sines
PEAK Grantmaking
Meredith Blair Pearlman
David and Lucile Packard Foundation
Neeraj Mehta
McKnight Foundation
Sally Gillis
Seattle Foundation
Chris Cardona, Bess Rothenberg, and Subarna Mathes
Ford Foundation
Hanh Cao Yu
The California Endowment
Julia Coffman
Center for Evaluation Innovation
Kelci Price
The Colorado Health Foundation