An amusing thing about Friedmanite positivism is its remarkable naïveté about how empirical social science research is actually done. Quantitative empirical analysis in practice is nothing at all like the model described in Friedman (1953). Results are rarely conclusive. Disagreement (and fighting) is widespread. Some theories are so widely believed that no amount of empirical evidence will dislodge them. A renewed emphasis on identification and causal inference during the last few years has made things more interesting and some basic mistakes are being avoided. But Mises’s strictures still apply.
Lately there is a lot of discussion about academic dishonesty — fake data, mistaken (and even fraudulent) statistical inference, and more. A recent paper by Joseph Simmons, Leif Nelson, and Uri Simonsohn (via Paul Nightingale) discusses this problem in the context of applied psychology (as used in particular in management and marketing research). They emphasize the role of researcher judgment: choosing what data to collect, what tests to run, what results to report, and so on. They argue that common practices give researchers so much flexibility they can make almost any result look reasonable:
In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.
Follow Circle Bastiat
In a lot of ways, Friedman’s writing shows that he didn’t approach economics with the methodology he advocated. For example, we know that in A Monetary of the United States he and Anna Schwartz write that the evidence of the 1880s contradicts their prior conviction that deflation is related to contractions in productivity. Doesn’t this imply that they approached the data with preconceived theories and ideas, and not with a blank mind completely unaware of any theoretical causal relationships?
I don’t think Friedman had any theory to back him up. He just looked at the data and tried to draw conclusions.
As far as I know, positivism is not working properly even for the hard sciences. In the hard sciences the data without a theory doesn’t work at all, it is meaningless. And it is the theory which drives the research for data.
I remember an article by Mark Skousen. He said he reprimanded Friedman at a Mont Pelerin meting for his advice to Japan to keep inflating (no deregulation, etc.). Friedman replayed that “historically, increasing the money supply has stimulated economic recovery, and that fast monetary growth was necessary, given Japan’s fragile condition.”
Any way, you have to approach data with a preconceived theory, otherwise you wouldn’t know what to look for, or you wouldn’t look at it at all. The main problem is always the theory.