The measurement problem is perhaps the best-known puzzle in quantum mechanics. A vast literature addresses this problem without ever resolving it. The measurement problem is also responsible for one of the most recognisable symbols of quantum mechanics, i.e. Schrödinger's cat.
The proposed solutions to this problem—the so-called "interpretations of quantum mechanics"—keep the mathematics as it is, and redefine reality to make sense of the maths. This procedure is problematic for several reasons: it inverts the scientific method of describing reality with mathematics (and it reifies mathematics), it's not testable, the different realities proposed are all mutually exclusive, and in the final analysis, changing reality to validate mathematics has no explanatory power.
Worse, the competing "interpretations" act more like ideologies in that they attract adherents who take sides and proselytise. I refer to these as "ideological" or "programmatic" interpretations because they are accompanied by a theoretical agenda. New ideological interpretations appear all the time, but none ever disappears. My intention here is not to spend time debating the merits of the ideological interpretations, all of which seem to me to be fundamentally unscientific, but to show that they are merely the froth on a much deeper set of philosophical issues.
The problem I wish to focus on is that the measurement problem does not, in fact, arise in the context of making measurements. Rather, it is a theoretical problem that arises from attempting to physically interpret a highly abstract mathematical procedure with no physical analogue.
In this essay, I will argue that the mathematical tail is wagging the metaphysical dog in quantum physics. Nature is not a mathematician. A problem arising from a mathematical procedure is being foisted onto reality rather than being dealt with in the mathematics.
Like many paradoxes, the associated difficulties seem to reside in how the problem is framed. So I want to begin this essay by attempting to accurately frame the situation in which the problem occurs.
What is the "Measurement Problem"?
The measurement problem hinges on precise distinctions between:
- What the formalism says,
- What the interpretations claim, and
- What the observations show.
So we need to be as clear as possible about each of these.
The Formalism.
By the 1920s, a combination of empirical results and theoretical breakthroughs—such as Max Planck's description of blackbody radiation and Albert Einstein's description of the photoelectric effect—had shown that existing theories were insufficient to explain atoms.
Light had started off as vague "rays" and, under the influence of James Clerk Maxwell, had become electromagnetic waves propagating in space. Then light was observed to be quantised and to have particle-like properties. In 1923, Louis de Broglie proposed that all matter particles are also wave-like and have a "wavelength". At the time, this crystallised as the wave-particle duality postulate: matter is both a particle and a wave. As we will see, the tension implied by this duality continued to be influential, even after the advent of quantum field theory (QFT), which proposed that all matter is fundamentally excitations in fields.
Atomic spectra were known to consist of a handful of frequencies rather than a continuous rainbow, and some progress had been made on clarifying the relationships between the frequencies. Both Werner Heisenberg and Erwin Schrödinger were trying to explain atomic spectra in the wake of Planck and Einstein. Schrödinger aimed at a realist explanation in terms of wave mechanics using a Hamiltonian, rather than a Newtonian, formulation. He was inspired by the Hamilton-Jacobi equation because it draws a formal analogy between classical mechanics and wave optics. This suggested that classical trajectories might arise from some underlying wave phenomenon.
The result was the famous Schrödinger equation, which describes a "wavefunction", 𝜓. The equation can be written in many ways, though the most familiar forms came later. Where Schrödinger's formalism relied on differential equations, Heisenberg used matrices to describe virtual oscillations. Schrödinger himself proved that the two approaches were equivalent. Max Born and Pascual Jordan developed the matrix formulation into a kind of algebra with "functions" and "operators". An operator is a rule that can be applied to a function and returns another function.
There's a disconnect between what Schrödinger set out to do and how quantum mechanics turned out, which he was never reconciled with. Schrödinger was seeking a realist theory and wanted to treat 𝜓 as a physical wave in space. Heisenberg was influenced by positivism and wanted to explain phenomena purely in terms of what could be observed. His approach treated atomic spectra lines as resulting from a virtual oscillator. These early ideas were soon replaced by the idea that 𝜓 is a vector in Hilbert space (named after the mathematician David Hilbert).
The modern formalism of quantum mechanics has two modes: evolution over time and extracting information. To make for a simpler narrative, I'm going to focus on what it tells us about the position of a particle.
Other things being equal, the wavefunction 𝜓 evolves smoothly and continuously in time according to the time-dependent Schrödinger equation. We can rewrite the Schrödinger equation to give us 𝜓 as a function of position: 𝜓(x). For Schrödinger, x was simply a coordinate and 𝜓(x) represented the amplitude of the wave at that coordinate. In his view, the modulus squared of the wavefunction at that point |𝜓(x)|² reflected the charge density.
However, Max Born had other ideas. As described by John Earman (2022):
Born rejected Schrödinger's proposal because the spreading of the wave function seemed incompatible with the corpuscular [i.e. particle] nature of the electron, of which Born was convinced by the results of scattering experiments.
Born introduced a further level of abstraction. In Born’s view, 𝜓 does not describe a physical wave. It is an abstract mathematical function. The quantity |𝜓(x)|² gives the probability density for the electron, which tells us how probability is distributed over space. To find the probability of detecting the particle in a given range of x, one must integrate |𝜓(x)|² over that range. This is called "applying the Born rule". (Note that the procedure is slightly different for discrete quantities like spin).
Paul Dirac and John von Neumann added yet more layers of abstraction. Dirac invented his own operator algebra. Von Neumann recast 𝜓 in terms of vectors in an infinite-dimensional space—a Hilbert space. The Dirac-von Neumann formalism is how quantum mechanics is taught.
We can also retrieve another kind of position information, which is the expectation value for x. This is the long-term average position of the particle. This involves applying the position operator x̂ to the 𝜓, producing another function.
In the standard textbook formulation of quantum mechanics, 𝜓 represents a vector in an abstract vector space, and 𝜓(x) is a function representing that vector in terms of positions in space. It evolves smoothly and continuously; then we apply the Born rule, ∫ |𝜓(x)|² dx, to extract probabilities of particles appearing in defined regions of space. The physical interpretation of this formalism presents seemingly intractable problems.
Interpretations
Having attempted to present the formalism in neutral terms, we could simply stop at this point. In practice, however, we are compelled by inclination and convention to say what the theory means. And this involves saying what the mathematics means in physical terms. As noted above, by "interpretations" I'm not referring to ideological views like "Copenhagen" or "Many Worlds". Rather, I'm trying to draw attention to the contradictory assumptions that underlie these ideological views.
Traditionally, scientists aim to explain some phenomenon P in terms of its causes. If A causes P, then A explains P. In the abstract mathematics of vectors in Hilbert space, causality is not defined. However, rather than this fact shutting down speculation, it has enabled a proliferation of competing interpretive frameworks, many of which are treated as axiomatic.
There are numerous ideas about what, if anything, 𝜓 represents, and these include some very fundamental tensions: particle versus wave, realist versus anti-realist; ontic versus epistemic, observer-independent versus observer-dependent.
As noted, the wave-particle duality postulate remains at the heart of quantum mechanics. Waves and particles are not simply different kinds of entities. The mathematics used to describe them is also fundamentally different. Particles have qualities like position and momentum, while waves have properties like displacement, wavelength, and period. Quantum mechanics purports to replace waves and particles with a single unified mathematics, but what this means in physical terms remains unclear. And in the end, we still use the wavefunction to extract information on position and momentum (though not at the same time).
Quantum field theory reformulates particles as excitations of fields, but the legacy of dualistic terminology and beliefs persists, especially in the use of the wavefunction to compute particle positions (which is partly why I chose the example of position above).
The realist view is that reality extends to the atomic world, even though we cannot directly experience it (by "directly" here, I mean without apparatus). In this view, atoms, electrons, and photons are assumed to be real entities with real properties that we can either measure or infer. We are so used to science being committed to realism that it may seem strange to insist on this. However, the considered opinion is that classical physics fails to describe the atomic and subatomic scales, and this opened the door to antirealism.
The antirealist view is that if we cannot see or measure something, then it makes little sense to assert that it is real. Non-observable entities, such as electrons, are merely mathematical objects that we use for making calculations to account for experimental observations. Quantum theory emerged in the early twentieth century when positivism was having its brief moment in the sun. Positivists were against metaphysics on principle (based on a rather naive reading of Kant). Where something could not be observed, they argued that it should not have a place in our descriptions of reality. And in the 1920s, this included atoms, electrons, and protons. Heisenberg initially set out to eliminate anything unobservable from his formulation of quantum mechanics.
As noted Schrödinger was a realist. As were Planck and Einstein, although Einstein had changed what "real" meant. Historian Mara Beller (1996) notes that Heisenberg, Pauli, Born, and Bohr all denied being positivists, but were given to making strongly positivist pronouncements in their work.
We can also note that, on the whole, scientific realism is strongly associated with a commitment to metaphysical reductionism. Reductionism holds that only substances are real. Substance is described as "fundamental". In this view, structures and systems are simply adventitious phenomena. Thus, physicists tend to view the "particles" of the standard model (or the fields of QFT) to be the ultimate building blocks, the foundations of reality. Although, ironically, the fields of QFT are not observable, even in principle.
Another level of debate occurs over the ontological status of the wavefunction 𝜓. How one thinks about this issue is naturally influenced by one's pre-existing view about realism versus antirealism. For example, anti-realists are not inclined to see 𝜓 as real; realists are so inclined.
A major issue with such views is that assumptions can concatenate: if A, then B; if B, then C; if C, then D; and so on. For example, if I assume a realist stance, then it may seem natural to conclude that the wave function is real. And if the wavefunction is real, then something real must happen to it when we measure the location of a particle. And so on.
Yet realism is not a given; it is a metaphysical stance—one that rests on its own chain of assumptions.
Most of us fall one way or the other in the realism versus antirealism divide, and this limits what views are available in other contexts. Views in which the wavefunction is a real entity are called 𝜓-ontic, and views in which it is only concerned with knowledge are called 𝜓-epistemic (I mentioned this in my essay about Probability).
A 𝜓-ontic view holds that the wavefunction is a real entity, which is obviously also a realist view. Some high-profile physicists, notably Sean Carroll, now routinely state "the wavefunction is real" without qualification. Such views are colloquially referred to as 𝜓-ontologies. For Carroll, this move is a prelude to introducing Hugh Everett's Many Worlds ideology, in which the reality of 𝜓 is axiomatic.
A 𝜓-epistemic view holds that the wave function only represents our knowledge of the system. The issue of what real process creates atomic phenomena is incidental. The problem here is that, e.g. 𝜓(x) on its own provides us no knowledge of position in terms of space or time. To get knowledge out, we have to apply the Born rule, and even then, we don't obtain knowledge about position, but only about probabilities.
None of these views resolves all the issues. For example, 𝜓(x) appears to represent all possible locations without distinguishing between them. And since this situation arises prior to applying Born’s rule, it is typically interpreted as describing the system "before a measurement is made". However, if all positions are represented and none is distinguished—let alone selected—then the interpretation itself seems to require further interpretation.
If each interpretation needs another to explain it, we’re not solving the problem; we’re just piling words on top of mathematics. And this process can lead to infinite regress. Adding interpretive layers does nothing to resolve more fundamental ambiguities and contradictions, even when it obscures them.
The next dichotomy brings us to the heart of the measurement problem.
On one view, by far the dominant view, the observer plays a central role in causation at the atomic level. Readers are likely familiar with the phrase that observation collapses the wavefunction, even if they are vague about what it means. In my essay Observations and Superpositions, I refuted the idea that observation can occur prior to the outcome of the events we are observing. Still, this issue continues to arise and cause confusion.
However, the term "observer" is a misnomer. What "observation" means in this context is that a particle is detected by a particle detector. The "observer" is never a human being—we cannot see particles. The "observer" is a Geiger counter, a photomultiplier, a photographic plate, etc.
Eugene Wigner went one step further and postulated that "consciousness" collapses the wavefunction, and this idea has proved irresistible to mystics. "Consciousness" is in scare quotes because it is undefined by Wigner, and decades later, there is still no standard definition of this often reified abstraction. Physicists, by and large, view Wigner's theory as an embarrassing moment in an otherwise distinguished career and sweep it under the rug. The idea that "consciousness" is involved in deciding outcomes is simply nonsensical.
It is also possible to take the view that quantum phenomena are not affected by observation. That the "collapse of the wave function" is, for example, random.
The Measurement Problem
The "measurement problem" can be viewed from a plurality of philosophical standpoints, and thus, there are many ideas about what it connotes. However, the basic problem is that to extract a probability, we have to switch from evolving the wavefunction in time to applying the Born rule.
This leads to a number of unresolved ontological questions, some of which are artificial. The ontological status of the wavefunction is generally decided prior to these questions, but there is an ongoing dissensus on this issue. A constant question for physics is how descriptions, especially mathematical descriptions, relate to reality (presuming our metaphysical commitments admit a reality). This leads to a number of deep questions:
- What does it mean for the wavefunction to "collapse"?
- If there is a change in mathematical procedure between time evolution and making a prediction, does this amount to a change in reality?
- If we cannot make any distinction in terms of position using time evolution, what is the ontological status of position prior to applying the Born rule?
- What is the ontological status of probability?
- Is making a prediction using the Born rule equivalent to making a measurement?
Each question has multiple serious answers, and is surrounded by a halo of versions for non-specialists that are more or less inaccurate. The idea of wavefunction collapse, in terms of position, is an interpretation of three facts:
- During time evolution of the wavefunction 𝜓, we obtain no definite information about position.
- When we apply the Born rule to 𝜓(x), we get the probability of a particle appearing in a range.
- When we measure position, we get definite position information.
An obvious question is, given how definitely we can measure position, why is the formalism that supposedly predicts this so vague and indefinite?
During time evolution, an ontic interpretation is that the particle has no definite position. An epistemic interpretation is that we don't know anything about the position prior to measuring it. A popular interpretation is that the particle is in all positions simultaneously.
Quantum physicists adopted the term "superposition" from wave mechanics to describe this situation. In wave mechanics, superposition refers to the simple mathematical fact that if two waves are described by functions f(x) and g(x), then their combined displacement at x is just f(x) + g(x). There is nothing remarkable about this; it is commonly observed and well understood in classical terms.
 |
| Superposition of waves |
Quantum superposition, in the case of a single particle, can be read as "no position", "unknown position", or "all positions". We often read about electrons being "smeared out" in space. Or forming a cloud around the nucleus. But these images raise more questions than they answer (not least questions about how such an atom can be stable). My impression is that most scientists now lean towards "no position".
The epistemic position is more or less intuitive, but this does not make it right. As we have seen, physical intuition can be a poor guide to the physics of things we can't physically experience (like galaxies or atoms). Ignorance is our natural state. Still, what can it mean for a particle to have no definite position in an ontological sense? Is it extended in space? Does it not exist? No one can say.
How we view superposition determines how we view applying the Born rule. An epistemic view says that applying the Born rule changes our state from ignorant to knowledgeable (again, this is intuitive, but so what?). But if we take the wavefunction to be something real, then we are forced to seek a realist interpretation of applying the Born rule. Somehow, and no one knows how, the Born rule causes the wavefunction to do or be something different. If we start with the idea that all positions exist simultaneously, then applying the Born rule appears to do something dramatic, which has been called "collapsing the wavefunction". Which of these views actually applies is unclear.
And finally, we have the fact that when we measure the positions of particles, they appear in one definite place. Again, depending on how we interpret the wavefunction to start with, and how we interpret the outcome of applying the Born rule, there are several possible positions one can take on this.
It's seldom explicit, but it seems to be common to assume that making a measurement is functionally equivalent to applying the Born rule. This is why people say "measurement collapses the wavefunction". However, the result of the former is a single location in space, while the result of the latter is a probability of occurring within a range of space. In fact, it's not at all clear how taking a measurement relates to the quantum formalism.
The idea that the wavefunction "collapses" is a rather dramatic way of framing the issue. The idea of "collapse" requires that we view the wavefunction as real, in a realist framework. The time evolution is viewed as pregnant with possibilities, and applying the Born rule gives birth to a single possibility. And how we frame this in physical terms depends on our prior philosophical commitments as sketched out above.
Conclusion
One way to view science is that it seeks to replace guesses and superstition with objective knowledge that is accurate and precise to certain limits. Belief is merely a feeling about an idea, and thus highly subjective. The antipathy between belief and science is never far from the news.
And yet, beliefs play a central role in the story of quantum mechanics, even if it is merely the belief in instrumentalism. Some physicists believe that the wavefunction is real; others believe it is not real. Some believe that the wavefunction represents reality; others believe it represents knowledge about reality. Some believe that measurement collapses the wavefunction; others believe that wavefunction collapse is an incoherent idea. After a century of trying, physics alone seems incapable of resolving such issues.
Part of the reason that quantum theory seems so complicated is that it involves multiple layers of abstraction, each of which is subject to multiple competing interpretations. Some interpretations are accepted as axiomatic, and others are treated as postulates. Just when you think you've understood one aspect, you discover another that contradicts it. Commentators often take philosophical stances without sufficient justification. And at the popular level, explanations of quantum mechanics often do more harm than good.
It's almost guaranteed that any given statement about quantum beyond the bare mathematics will be contradicted by another point of view. It is ridiculously difficult to give a concise account of quantum mechanics that won't be shot down by someone.
However, given the choices on offer, a thoughtful person might well choose none of the above. My sense is that the ideological/programmatic interpretations, which keep the maths and redefine reality, are all incoherent. Notably, as I have pointed out many times now, all of these interpretations are mutually exclusive. For example, the ontology of Many Worlds is completely unrelated to the ontology of, say, Pilot Wave theory, which is unrelated to spontaneous collapse interpretations, which are unrelated to information-theoretic interpretations. Given the plurality of choices on offer and the lack of any coherent ideas that can distinguish between them, I can see how it makes sense to focus on what works. In the absence of a viable ontology, adopting an instrumentalist stance—in which we have a functioning mechanism for obtaining probabilities—makes a lot of sense. However, it abandons the attempt at explanation, which I find unsatisfactory.
For realism, at least, reality itself is a point of reference. A good theory is not only consistent with reality, it is consistent with other theories about reality. When two theories are diametrically opposed, like realism and anti-realism, then only one of them can reflect reality. One of them must be a bad theory. But when it comes to quantum mechanics, we still cannot tell which is which.
~~Φ~~
Bibliography
Beller, Mara. (1996). "The Rhetoric of Antirealism and the Copenhagen Spirit". Philosophy of Science 63(2): 183-204.
Earman, John. (2022) “The Status of the Born Rule and the Role of Gleason's Theorem and Its Generalizations: How the Leopard Got Its Spots and Other Just-So Stories.” [2022 preprint]
