News and analysis
September 01, 2010

Charities and Watchdog Groups Clash Over Monitoring Systems

By Raymund Flandez

New York

Battles between charities and the watchdog groups that help donors decide where to give escalated last week when a major trade association released drafts of two reports by scholars who say the watchdogs may do more harm than good.

The studies, paid for and released by the Direct Marketing Association’s Nonprofit Federation at a meeting here, charge that the watchdog groups use evaluation systems that are confusing and simplistic.

What’s more, they said, because the watchdog groups are influential, charities often take steps that are not smart in the long-term simply to garner high ratings.

The watchdog organizations they studied are the American Institute of Philanthropy, Better Business Bureau’s Wise Giving Alliance, and Charity Navigator.

Leaders of those groups say the studies were biased.

Ken Berger, chief executive of Charity Navigator, took issue with the idea that “our rating system can cause harm.” He charged that the Direct Marketing Association’s studies were designed to “foster the myth that all charities are good, that all charities are fundamentally equal. That is a lie.”

'Severe Shortcomings’

The studies of the watchdog groups took different approaches.

One, conducted by George E. Mitchell, a research assistant at the Transnational NGO Initiative at the Moynihan Institute of Global Affairs at Syracuse University, was based on in-depth interviews with top leaders from 152 nonprofit organizations.

  • Watchdog ratings may be biased against smaller organizations since economies of scale make larger charities seem more financially efficient.
  • The watchdogs “fail to provide adequate guidance about what their ratings measure.” The American Institute of Philanthropy, for instance, gives letter grades but “provides little guidance about what the scores actually mean” and does not “disclose how the raw scores are actually converted into their final designations.”
  • Benchmarks are arbitrary. Charity Navigator scores organizations that work on different missions in different ways. The BBB Wise Giving Alliance adjusts several of its standards if an organization protests or argues for special accommodations, the report says.

The watchdogs aren’t entirely at fault, Mr. Mitchell’s report says.

Lack of good data hinders their ability to put forth information that can be easily consumed and understood by donors. Most of the watchdogs rely on the IRS Form 990, which has financial data, and information from charity Web sites.

Overhead Measures

Ms. Sowa’s study criticizes the watchdogs for placing too little emphasis on programs’ operations and outcomes. For instance, their requirement of “low overhead as a measure of effectiveness” means that many nonprofit organizations decide it’s wise to offer low salaries for managers when they need highly trained managers “more than ever.”

“Financial health is important. However, it’s only one measurement of a broad spectrum of how organizations work,” says Ms. Sowa.

“It’s a delicate balance between holding nonprofits accountable on how they spend but not overemphasizing thrift for the sake of performance.”

She suggested that the watchdogs could use an accreditation process to evaluate and validate nonprofit organizations, require charities to release more information, and add more detailed information on their Web sites to counter the shortcomings of the rating system.

Focus on Results

Charity Navigator and the Wise Giving Alliance agree with the scholars that more needs to be done to focus on a charity’s accomplishments in meeting its mission.

But that is no easy task, Mr. Berger says.

“The sad state of reality is that there are no standardized agreed-upon measures of effectiveness that most charities gather,” he says. “The data isn’t quite there yet.”

His group announced two years ago that it would revamp its ratings system to look beyond nonprofits’ financial health and is now working to grade charities’ accountability, ethical practices, good governance. and transparency. Donors will see the results of those changes early next year, he says.

H. Art Taylor, chief executive of BBB Wise Giving Alliance. says the Wise Giving Alliance is working with GuideStar, a site that collects 990 data and other information on charities, and Independent Sector, a coalition of charities and foundations, to develop a format for organizations to report their effectiveness. It expects to release a new reporting format next year.

Mr. Taylor says the characterization of his group as making arbitrary exceptions is flawed. But, he says, some charities do ask to be excused from one or more of the 19 standards the alliance uses to rate charities. “We’ll certainly hear what the charities have to say,” he says. “If they’re legitimate reasons, we’ll certainly make some adjustments, or not.”

Lack of Disclosure

Daniel Borochoff, president of the American Institute of Philanthropy, says the biggest issue for charities and the watchdogs that review them is accountability.

“There’s a lot of bad reporting and poor disclosures going on,” Mr. Borochoff says. “Certainly, we’re open to realistic ways in which to improve,” but, he says, “for them to somehow say that financial ratios are not something of interest or something that donors don’t need in order to consider making a donation, it’s just stupidity. It’s very misguided for nonprofits to belittle the importance of the finances and how they’re spending their money.”

The Direct Marketing Association was not surprised by the reaction from the watchdogs. Christopher Quinn, executive director of the Nonprofit Federation, said in an e-mail: “These studies speak for themselves, and it’s only fair that so-called 'watchdogs’ are held to the same scrutiny as they expect of others,” he wrote. “There has to be a better way to evaluate charities than the current flawed and varied systems.”