The history of string theory is not one of continuous progress along a clear direction. Indeed, much crucial work was done before physicists even realised the importance of strings. The story of progress can be divided into several eras.
As is so often the case, string theory arose from a collection of discredited ideas. Over a period of 50 years vital pieces of the puzzle came to light, only to be ignored in favour of more fashionable topics. When string theory became mainstream, physicists realised that these early insights were extraordinarily prescient.
The story begins in 1919 with a little known Polish mathematician, Theodor Kaluza. Inspired by Einstein’s revolutionary ideas, he attempted to overthrow a central tenet of physics. “What if there are extra dimensions we just can’t see?” he asked.
Working alone, he attempted to incorporate a hidden dimension into Einstein’s model for gravity. Unsurprisingly, his five-dimensional theory had more equations than the usual four-dimensional approach. Looking closely at the extra equations he had found, Kaluza spotted something remarkable. They were precisely Maxwell’s equations governing the electromagnetic field!
This unification of electromagnetism and gravity was completely unprecedented. Even Einstein had praise for the achievement, commenting, “I like your idea enormously”. But Kaluza didn’t have an explanation for why we don’t notice the fifth dimension. This discovery fell to another outsider, Oskar Klein.
Klein realised that the extra dimension could curl up to form a circle. If the circle were small enough we’d never realise it was there. Between them, Kaluza and Klein had taken the first baby steps towards a higher dimensional reality.
Sadly Kaluza-Klein theory was almost immediately shown to be wrong: its predictions differed vastly from experimental data. Alongside the birth of quantum mechanics, nobody had any time for a broken toy model of reality. And so Kaluza and Klein were forgotten. Forgotten until string theory, that is.
The second foundational fragment fared somewhat better. By 1943 quantum mechanics was established at the forefront of physics. Experiments had shown that electrons could be thought of as pointlike particles, but the atomic nucleus was causing some problems: protons and neutrons seemed to behave more like spheres than points.
Werner Heisenberg was among the first to tackle the issue. He proposed that our usual notions of smooth spacetime break down at subatomic scales, due to the uncertainly principle. This allows protons and neutrons to have spatial extent from the quantum perspective, but appear pointlike for general relativity.
Although this solved the problem of protons and neutrons, it raised yet bigger problems. Heisenberg was claiming that at the quantum scale, space and time are unreliable. It’s pointless trying to keep track of exactly how particles interact if you can’t rely on a fixed backdrop. “So”, said Heisenberg, “just calculate the probabilities of different interactions taking place”. In other words, don’t worry about exactly how things happen.
This bizarre new viewpoint became known as S-matrix theory. Unfortunately it was rather hard to do calculations in Heisenberg’s model. Although it had some success, it was eventually supplanted by the ideas of quantum field theory.
Nevertheless some researches persevered. One of these was Gabriele Veneziano. Using tools inspired by S-matrix theory, he kickstarted string theory. Heisenberg’s ideas of quantum spacetime breakdown eventually reached full fruition during the 1980s. These neglected ideas, and others, found joyous vindication in the humble string.
The first time strings were used to model particles, it was as a convenient way to look at data. In 1968 Gabriele Veneziano, a young researcher at CERN, was trying to describe the strong force. He realised that an equation, written by Leonard Euler several centuries earlier, seemed to do the job.
But while this approach worked well, no one really understood why. Several people began to work on an interpretation for Veneziano’s idea. By 1970, Yoichiro Nambu, Holger Nielsen and Leonard Susskind had all independently come to the same conclusion: the formula made sense if you thought of particles as tiny vibrating strings.
For several years, these ideas – then called dual resonance models – were very popular as a proposed model of the strong force. However this came to an end in 1973 with the discovery of quantum chromodynamics as the correct quantum field theory description of the strong force. Dual resonance models and string theory entered the scientific wilderness.
Nonetheless a few researchers persevered, motivated no longer by the strong force but by a deeper and more ambitious problem.
Their original string model of bosons (force particles) was consistent with special relativity (Einstein’s description of objects moving at very high speeds) and quantum mechanics, but required twenty-six dimensions! Their theory also predicted the existence of particles called tachyons, which had negative mass and could move faster than the speed of light. These properties seemed nonsensical to most people.
But in 1971 Pierre Ramond had modified the theory to include fermions (matter particles), and in doing so discovered supersymmetry. His superstring theory had no tachyons and reduced the required number of dimensions to ten.
Different string vibrations gave rise to different particles. One vibration was particularly interesting: it was a massless particle of spin two, precisely the properties of the hypothesised graviton.
So gravity seemed to emerge naturally from string theory. When John Schwarz and Joel Scherk discovered this in 1974, they suggested that string theory reinvent itself as a quantum theory of gravity.
With the discovery of the correct quantum description of the strong force hogging the limelight, their ideas were largely ignored by the physics community. String theory entered a fallow period, with few people continuing to work on it. To those in the know, there also appeared to be technical problems preventing any quantum description of both matter and gravity. This situation remained until the First Superstring Revolution in 1984.
For about a decade, string theory was a totally marginal subject in theoretical physics. Very few people worked on it and those who did found it very hard to get jobs. It had failed as a theory of the strong force. Although touted as a possible theory of quantum gravity, this seemed implausible to the few experts. Quantum theories of both gravity and matter particles were known to suffer from quantum anomalies that made them inconsistent, and there seemed no reason to think string theory was any different.
But in 1984 a landmark paper by Michael Green and John Schwarz changed the mood completely. They discovered an extra contribution to anomalies – now called the Green-Schwarz term – and showed that this term arose in string theory. Suddenly, the anomalies vanished due to a slew of cancellations. The theory became objectively more promising, and attracted the attention of some influential theorists, including Edward Witten.
Researchers flocked to the subject during the so-called first superstring revolution, and strings rapidly became fashionable. Very soon physicists noted that string theory could easily give models with the main features of the Standard Model plus gravity. For a brief intoxicating moment, it seemed that only one small push would be needed to obtain the final unified quantum theory of all the forces.
In retrospect, these hopes were naive. The more physicists studied string theory, the more they realised the depth of the underlying structure. There emerged not one, but five different consistent superstring theories. There were intrinsic connections –mirror symmetry was discovered during this period – but they were not fully understood.
The ten years from 1984 led to many discoveries about weakly interacting strings, where perturbation theory is a good approximation. The physics of string theory for strong interactions essentially remained a mystery. This all changed in the mid 1990s when physicists realised the importance of D-branes in describing non-peturbative dynamics.
The first superstring revolution left us with five consistent theories, where one-dimensional strings moved around in ten-dimensional spacetime. They all seemed to describe different worlds. Which, if any, was the correct one?
In 1995, Edward Witten gave a talk at the yearly “Strings” conference. He proposed that these five theories were actually all part of a single framework. But he could not fully describe his vision. Nevertheless he demonstrated how ordinary superstrings give us tantalising hints about the properties of this ultimate theory. He called it M-Theory.
Witten argued that there were dualities between the different superstring theories. Others had already suggested some of these, but Witten drew them all into a coherent picture. Each theory is an alternative way of looking at the same world. Which one you need depends on the values of certain physical parameters. He also showed that M-theory doesn’t have ten spacetime dimensions, but eleven!
There isn’t a string theory in eleven dimensions, but there is a supersymmetric theory of gravity, called supergravity. During the late 1970s, whilst their colleagues worked on incorporating supersymmetry into the Standard Model, some physicists tried to combine supersymmetry and gravity. The result was supergravity. It was largely ignored by string theorists, who worked in ten dimensions, not eleven! Ignored, that is, until Witten realised that supergravity is also part of the M-theory picture.
With supergravity came a supermembrane theory, describing two-dimensional membranes in an eleven-dimensional spacetime. For M-theory to hold together in eleven dimensions, it must also include surfaces called membranes.
If M-theory is correct, then why did it take physicists so long to spot it? The answer is simple. An eleven-dimensional theory with membranes looks just like a ten-dimensional theory with strings. Realising this requires a little imagination.
Suppose you live in an eleven-dimensional world, where one of your dimensions is a circle. Take a two-dimensional membrane sheet and wrap it around the circular dimension to make a cylinder.
Now imagine you make the circular dimension extremely small. From your perspective the world is now ten-dimensional and your cylinder has become extremely thin. In fact its thickness is now so small that it looks exactly like a one-dimensional string!
Shortly after Witten’s inspiring lecture, Joseph Polchinski realised that membranes with up to nine spatial dimensions had a very simple description in string theory. These so-called D-branes have been central to research ever since.
It quickly became clear that D-branes suggested new symmetries in M-theory. The most famous was introduced by Juan Maldecena in a 1997 paper. His result is known as the AdS-CFT Correspondence. It is essentially a duality between string theory and a type of quantum field theory.
D-branes, AdS-CFT and M-theory open the doors to the study of non-perturbative physics. They also provide a set of tools with applications from black holes to condensed matter. Much research today involves developing of the insights first articulated in the the mid-to-late 1990s.