Science is the best tool we have for discovery and learning, because it encourages us to modify our theories when they don’t prove out.
As with any human endeavor, though, science is imperfect. Lee Smolin gives a great example of this in an Edge discussion of wrong scientific beliefs that were held for long periods:
Perhaps the most embarrassing example from 20th Century physics of a false but widely held belief was the claim that von Neumann had proved in his 1930 text book on the mathematical foundations of quantum mechanics that hidden variables theories are impossible.
These would be theories that give a complete description of individual systems rather than the statistical view of ensembles described by quantum mechanics. In fact de Broglie had written down a hidden variables theory in 1926 but abandoned work on it because of von Neumann’s theorem. For the next two decades no one worked on hidden variables theories.
In the early 1950′s David Bohm reinvented de Broglie’s theory. When his paper was rejected because von Neumann proved what he claimed impossible, he read and easily found a fallacy in the von Neumann’s reasoning. Indeed, there had been at least one paper pointing out the fallacy in the 1930s that was ignored.
The result was that progress on hidden variables theories in general, and de Broglie and Bohm’s theory in particular, was delayed by several decades.
I do think I have some insight into why a false belief about the possibility of alternatives to quantum mechanics could persist for more than two decades. During this period there was rapid progress in the application of quantum mechanics to a wide set of phenomena from astrophysics to nuclear and solid state physics.
Meanwhile the most popular interpretation of quantum mechanics was Bohr’s, which is now hardly taken seriously by anyone. Those who concentrated on the foundations of the subject were left behind, especially as it was convenient for the progress that was being made, to believe that the foundations were surer than in fact they were.
Perhaps there are periods in science where it makes sense for most scientists to sweep foundational worries under the carpet and make progress on applications, postponing the inevitable reckoning with the inconsistencies to a time when there are better hints from experiment.
Dr. Smolin also makes an analogy of how this sort of entrenched thinking led to our current economic mess.
The thought that I hope you carry away today is that we, individually, can become entrenched in our thinking and beliefs, too, failing to modify them even if there is an abundance of data to motivate us to.
During my regular (or irregular) periods of reflection, I ask myself “what assumptions/beliefs do I have that I need to re-examine?” Sometimes I learn from doing this, and sometimes I don’t. But, if I never asked the question, I would have virtually no chance of correcting wrong assumptions and beliefs. Need I say more?