Jim Manzi has a new book out arguing that we need to leverage randomized control trials to test the efficacy of public policies:

In “Uncontrolled,” Manzi looks at two celebrated model-building exercises. Larry Bartels of Princeton produced a model finding that presidential policies exercise the single biggest influence on income distribution. The authors of “Freakonomics” produced a model showing legalized abortions subsequently reduced crime rates.

Manzi argues that by slightly tweaking the technical assumptions in these models, you eliminate the headline-grabbing results. He also points out that regression models that try to explain crime rates have not become more accurate over the past 30 years. All this model-building hasn’t even helped us get better at understanding the problem.

What you really need to achieve sustained learning, Manzi argues, is controlled experiments. Try something out. Compare the results against a control group. Build up an information feedback loop. This is how businesses learn. By 2000, the credit card company Capital One was running 60,000 randomized tests a year — trying out different innovations and strategies. Google ran about 12,000 randomized experiments in 2009 alone...

Businesses conduct hundreds of thousands of randomized trials each year. Pharmaceutical companies conduct thousands more. But government? Hardly any. Government agencies conduct only a smattering of controlled experiments to test policies in the justice system, education, welfare and so on...

This sounds like an obvious idea, but small steps to do exactly this meet with significant opposition, like this example from last December's New York Times:

The city’s Department of Homeless Services said the study was necessary to determine whether the $23 million program, called Homebase, helped the people for whom it was intended. Homebase, begun in 2004, offers job training, counseling services and emergency money to help people stay in their homes.

But some public officials and legal aid groups have denounced the study as unethical and cruel, and have called on the city to stop the study and to grant help to all the test subjects who had been denied assistance.

“They should immediately stop this experiment,” said the Manhattan borough president, Scott M. Stringer. “The city shouldn’t be making guinea pigs out of its most vulnerable.”...

One critic of the trial, Councilwoman Annabel Palma, is holding a General Welfare Committee hearing about the program on Thursday. “I don’t think homeless people in our time, or in any time, should be treated like lab rats,” Ms. Palma said.

But Seth Diamond, commissioner of the Homeless Services Department, said that just because 90 percent of the families helped by Homebase stayed out of shelters did not mean it was Homebase that kept families in their homes. People who sought out Homebase might be resourceful to begin with, he said, and adept at patching together various means of housing help.

Interestingly, the one example cited in the article of someone who didn't receive government assistance was able to get some elsewhere, showing that this policy, may not in fact be necessary. Back to the first article:

[T]he general lesson of randomized experiments is that the vast majority of new proposals do not work, and those that do work only do so to a limited extent and only under certain circumstances. This is true in business and government. Politicians are not inclined to set up rigorous testing methods showing that their favorite ideas don’t work.

It is not just a matter of wasting money on policies which don't work. It is possible that programs may actually harm their beneficiaries, making these types of studies a moral imperative.