Let’s say that there is a space of statements we can make about the human world, some of which are true and some of which are not. (We’ll leave off things that are “not even wrong,” that aren’t even falsifiable, for the moment.)
The ideal social scientist stands somewhere in the space of possible thoughts and says to himself, “am I in the space of true things or in the space of false things?” And then gathers as much information as possible to figure out which is the case.
But at the same time as different possible thoughts have different truth values, they also have different degrees of social desirability:
So the sensible social scientist spends a lot of time in certain corners of the space of thoughts.
Sooner or later the space of things that are both true and socially desirable to believe will have been well-mapped (for various reasons, the space of social desirability may also move or constrict over time.) So social scientists find themselves walking on a very narrow ledge, between ideas that are not already well known and well described, but are still socially desirable or acceptable, and are at least potentially true, under some circumstances and at some times.
I think that’s more-or-less how you get the replication crisis in social science, quite apart from improvements in methodology and in addition to more open mechanisms for data-sharing and criticism.
In medicine, the socially desirable things to believe are often simply, “this treatment works for most people,” which can still have a distorting affect.
At his blog, the statistician Andrew Gelman asks the question, “Why Is The Scientific Replication Crisis Centered on Psychology?” and offers various, plausible methodological and sociology-of-science reasons. But psychology has two quite well-replicated areas, of behavioral genetics and IQ research. No one wants those to be true, so they’ve spent decades swimming in the opposite direction as fast as possible. It sets them up to find a lot of things that just ain’t so.