Are RCTs Enough?
I’ve noticed that the gold standard in Microfinance research seems to be the “randomized control trial”, offering an unsurpassed level of experimental sophistication. From a statistical viewpoint, a randomized trial of a sufficiently large population allows researchers to ascribe causality. This is important in the impact evaluation of a policy.
While RCTs may be new to economics, they are nothing new to the natural sciences which have for a long time emphasized the importance of randomization. However, trials in the natural sciences (especially medicine and theoretical physics) control two one other crucial biases – the placebo effect and observer expectation. This is accomplished through the “double-blind trial” in which neither the sample nor the researcher know whether they belong to the trial or the experiment.
This is incredibly easier in the natural sciences (after all, the whole treatment is contained in a pill, and easy to replicate). However, recent speculation, suggests that the placebo effect might be quite as present and important for economic issues. The economist Tyler Cowen even suggests that the placebo effect could be employed to overcome issues of macroeconomic importance such as consumer confidence.
Is there a chance (researchers please comment) that this effect applies somehow to microfinance trials as well? Placebo effect, after all, is a result of expectation – do these trials somehow form an expectation that is self-fulfilling as it is in medicine?
Double-blind trials achieve one other thing – remove observer-expectation bias. Modern physics lends a beautiful example of how elegant research design can overcome this bias. In neutrino experiments, researchers have a bias of the total number of neutrinos, N, that they expect to see. Therefore, even if many different researchers are shown the whole sample, the observation would systematically conform to the expectation. Rather, each researcher is only shown a theoretically meaningless N’, which is a random fraction of the total sample. Because this is a meaningless value, there exists no expectation and hence the overall value (an aggregate of each answer) is not biased.
I wonder if something similar can be done in microfinance. I am not familiar enough with the actual research methodology to comment, but would be interested to hear from researchers regarding the above two points. And, if theoretically it is possible to conduct a double-blinded RCT – is it practically possible, or would it take too much effort. Inevitably a double-blinded trial has a large financial burden, but in medicine I would never trust to be reliable anything that has not been rigorously tested in a double-blinded setting. Would you say the same about economics?