Prominent Researcher Calls Interventions Without Evaluations ‘Immoral’

Courtesy Richard Nisbett

Don’t trust your assumptions. That’s the message of University of Michigan psychologist Richard Nisbett’s new book, Mindware: Tools for Smart Thinking, which challenges nonprofits to test programs and use experiments and research to better appeal to donors and grant makers.

Mr. Nisbett draws on social psychology, economics, and Western and Eastern philosophy to establish a framework for making decisions based on experimental evidence and sound reasoning, not biased observations.

Psychologists have identified dozens of mistakes people routinely make in assessments, including fundamental attribution error (failure to recognize how context affects outcomes) and mistakes due to loss aversion (reluctance to part with things we should let go) and the endowment effect (proclivity to assign more value to something because we own it).

How can nonprofits avoid these errors? Mr. Nisbett says they should systemically collect data and then carefully calculate the association between variables. That’s critical in philanthropy, he argues, because the results are costly when assessments of public policy or social programs are flawed.

Mr. Nisbett discussed his theories with The Chronicle in this interview, which has been edited for clarity and brevity:

You argue that situations and context influence individuals’ behavior more than their personalities. What does that suggest for designing social programs? Should we change environments to change behavior?

Yes, I do think that. When we put people in prison, I don’t know what we’re thinking of doing. Getting them off the streets, I suppose, was certainly a big part of the ’70s movement to lower crime rates. But there’s very little attempt to alter their situation when they’re let out of prison. I’m thrilled at today’s movement, a bipartisan one, to make the situations of ex-prisoners more conducive to lawful behavior.

There are philanthropic opportunities: What might be done to create situations for prisoners that would make a resumed criminal career less likely?

If I were in the business of charity, I would certainly be attentive to possibilities of not just succorance in the moment but the creation of situations which might make continued charity less necessary.

You show how organizations and governments can use small “nudges” to encourage positive behavior and discourage negative behavior. How can nonprofits set people up to succeed?

I’ve been struck by several interventions lately. I see homeless people, and you say, what’s going on here? Do these people want to be on the street? It’s Utah that’s come up with the idea [Housing First, a program that provides apartments to chronically homeless people], let’s make sure everybody has a room and a bed and a chair. Suddenly, there don’t seem to be many homeless people around. Give them a home, and in the end it can be a lot cheaper. That was just such a simple intervention.

Do you know the concept of “frequent flyers” in medicine? These people are going to make huge demands, hundreds of thousands of dollars a year. Let’s just accept that we’re going to have to give them a minder. The minder costs something, a few thousand dollars a year, for checking up on the person, making sure they’re taking medications, making sure the diet is reasonable. Society is hugely better off spending a few thousand for a minder than paying multiple thousands for visits to hospitals. The individual is surely better off.

You encourage people to adhere to sunk-cost theory, the idea that we shouldn’t make decisions based on resources we’ve already spent. What does sunk-cost theory mean for programs that aren’t working?

The fundamental problem is that we frame our activities in terms of the price we pay to carry them out. We don’t realize that once we’ve paid the price, there’s no way to be economical anymore. An example: I bought a $100 ticket to a basketball game. But the star isn’t playing, nothing hangs on the outcome, it’s started to snow, and the game is 40 minutes away. I don’t really want to go. But what a waste! A hundred bucks for nothing?

The economist has no trouble saying, “That’s right, but you can’t waste it; you can’t waste money that’s been spent. What you can do is pay twice: once for the ticket, and once to sit through a boring game.” Waste not, want not is good advice, but you can’t waste expended resources.

Another example: Suppose a town has to replace a hospital that’s obsolete. You could either build a new one or remodel the old one. It costs about the same, and you get the same quality hospital. But the old one was expensive to build, and most people say it would be a shame to waste that. The economist says, “Those resources are long gone. They can’t be wasted.” That thinking simply has to be pulled out of the equation.

The sunk-cost error gets amplified by the fact that, if I’m the one who designed this program, and it doesn’t look like it’s successful, I’m going to want to tinker with it or let it continue for a bit longer. I don’t want to look incompetent, so I’ll keep sending good money after bad in hopes it will turn good.

Some people make charitable contributions based on their passions, others on how they’ve always given or on how others give. Some do a cost-benefit analysis. What’s best?

With some charities, of course, if they’re on the up-and-up, you don’t have to do the research. Groups that fix cleft palates — it’s a safe assumption that the kid is better off without that cleft palate. There are some things that charities do that are manifestly beneficial.

Certainly for me, I do a cost-benefit analysis. It’s a combination of cost-benefit analysis and doing things that I can feel, see around me, see what the gains are. Both considerations are at work.

How can nonprofits use cost-benefit analysis to make decisions that could save lives?

There was an idea that was new to me when I was writing the book: The value-adjusted life span. It’s certainly worth much more money to save a young life than an old life. It’s a much better use of your money to reduce what will be the lifetime suffering of a young person than that of an old person.

Some people who are trying to figure out how to put a value on life say, “How much are we willing to pay to keep somebody alive? How much does it cost to keep alive a person with severe kidney problems on dialysis?” They are tacitly saying that the life of someone with that condition is not worth as much as a healthy person’s life, because their life is just not as much fun. Taken to the extreme, I would have to agree. How much am I willing to pay to keep some totally miserable person alive?

The quality-adjusted life — how much money you’re going to put in — is in part a function of the quality of the life.

Should foundations and donors put money into testing and experimenting, or should they just fund programs?

Philanthropy should never spend a nickel for [programs] they don’t have good evidence work.

Interventions without evaluations to me are just immoral. You’re spending your donors’ money on a hunch.

I’m happy that so many people are understanding this way of thinking. It’s just a lot easier I think to convince people of the need for testing interventions than it used to be.

According to the book, “more information about the behavior of others can motivate people to change their own behavior.” To encourage giving, should charities use appeals that say something like, “X percent of Americans donate to charity”?

It’s a wonderful thing to test. You put it into the appeal: “As you may know, 65 percent of Americans donate to charities. You might be interested to know that the average person gives 12.3 percent of their income to charities.” What’s the effect of that?

Can you explain the phenomenon of “natural experiments”? How might a nonprofit with a tight budget make use of them?

When people try to assess whether the death penalty is effective, you can look at the natural experiment of adjacent states that do and don’t have the death penalty. Or you can look at what happens to murder rates before and after abolition or reinstatement of the death penalty. These are not quite as good as actual experiments, but they can be pretty darn good.

This example is in the book: What does it do for really poor people in the inner city to have greenery in their lives? [A study of Chicago housing projects found those surrounded by greenery had fewer reported crimes.] That’s just the kind of environmental intervention that some philanthropy ought to be doing. So let’s do that experiment, and let’s do it right! Let’s take every nth housing project in Cleveland, Philadelphia, and Chicago and rip out the concrete and put in greenery. The effect in that natural experiment is massive. It’s just huge. It would be expensive to rip out concrete and put in trees, and probably cost you scores of thousands of dollars per housing project, but the potential payout is just colossal.

Given donors’ concerns about overhead, will they understand and financially support testing?

If a charity is testing something that a knowledgeable scientist can’t be certain is effective, that should surely not be thought of as overhead. It’s an investment in finding out. It’s an investment in getting rid of something even worse than overhead: that is, stuff that’s useless.

Suppose a philanthropy was to advertise in Silicon Valley: “We are an experimental philanthropy. We get smart people to come up with ideas that seem worth testing. We expect most are going to fail, but if some work, it’s the iPhone of philanthropy.” I bet you that would go gangbusters.

You wrote that cultural beliefs affect people’s decision making, that people from Western backgrounds often come to different conclusions than people from East Asian backgrounds. What does that suggest in terms of the composition of your board and staff?

Intellectual diversity is a good thing. Demographic diversity is fine, too, but I think we put more energy than probably is worth it into demographic diversity relative to intellectual diversity. Most of the upper-middle-class black people I know think exactly the same way I do. The East Asians I know don’t think the way I do; that’s both demographic and intellectual diversity.

Describing how people try to resolve cognitive dissonance — the incongruities between what people believe and how they act — you wrote: “Change people’s behavior and their hearts and minds will follow.” What are the implications for working with donors?

If people agree to sign a petition to save the redwoods, and later you approach them with a much bigger request for the same cause — like put this six-foot sign on your lawn — people will do it. If I’ve done something toward that cause, I begin to think of myself as someone who cares about the environment or as somebody who’s a “doer.” It’s an identity thing; you change the person’s identity. You’ve worn a behavioral groove there.

Andrew Carnegie gave a tremendous amount of money away. People would sometimes say to him, “If I had that kind of money, I’d do that too.” He said, “You don’t understand, I’ve always given money away.” Rich people who didn’t always give money away before they were rich don’t give it away now.