Scientific discovery is unpredictable. History reveals that scientists rarely anticipate the nature or the source of new breakthroughs before they happen. One might think, then, that it is impossible to cultivate an environment that promotes discovery. But I argue otherwise: By encouraging open research without a programmatic agenda, we can establish a fertile ground for unexpected breakthroughs.
As for the donkeys you lost three days ago,
do not worry about them...
1 Samuel, Chapter 9, 20
In the biblical story, Saul goes searching for lost donkeys and winds up finding his kingdom by chance. This story has an important moral for scientists: You must open your mind to discovering something completely different from what you thought you were seeking, as something more exciting may be lurking at the periphery of your field of view.
For example, in 1965, Bell Labs engineers Arno Penzias and Bob Wilson were attempting to reduce the noise in their state-of-the-art radio antenna. But they couldn’t eliminate the noise entirely. No matter what they did, they were stuck with a “noise floor.” That “noise” turned out to be the radiation left over from the Big Bang. This watershed discovery forever changed our view of the universe. Like Saul, Penzias and Wilson went searching for donkeys and found a kingdom instead.
Breakthrough discoveries also happen when we open entirely new windows of exploration—even when we have no idea what we might find. In the early 1960s, for instance, NASA assembled a panel of “experts” to evaluate the merit of a proposal to launch an X-ray telescope into space. The panel concluded that the scientific justification for such a mission was weak, since all we could expect to observe were stars like the Sun emitting X-rays. The proposal was rejected.
But a decade later, when Uhuru, the first X-ray astronomy satellite, was finally launched, it blew those expectations away. Thanks to Uhuru and its successors, we now know that the X-ray sky is rich with radiation from accreting black holes, supernova remnants, galaxy clusters, and many other sources that NASA’s expert panel never imagined. The lesson is simple: Whenever there is a technological opportunity to open a new window for exploring the universe, we should do so without hesitation.
Today, we are attempting to open a window of exactly that character: gravitational wave astrophysics. Gravitational waves are ripples in spacetime predicted by Einstein’s theory of gravity. We have never detected them directly, though. While an existing detector called LIGO might just barely detect gravitational waves emitted by stellar-mass black hole or neutron star binaries in a few years, a more ambitious space mission called eLISA/NGO could unambiguously detect gravitational wave signals from the edge of the observable universe. We have some ideas about what eLISA/NGO might find—black hole binaries from distant galaxy mergers, for instance—but even more exciting is the possibility that the mission could discover new sources of gravitational waves that we have not even imagined yet. These discoveries could revolutionize physics in the century to come. Unfortunately, the funding agencies do not share this vision and eLISA/NGO is not being funded at this moment.
Funding in physics often targets guaranteed, short-term goals. After all, federal funding agencies like NASA and the National Science Foundation must justify their use of taxpayers' money within a period of several years, not several decades. They are naturally driven to fund low-risk research with predictable returns. But to maximize our long-term benefits, I believe that this approach has to change. Funding agencies should allocate a small fraction of their funds (10-20%) to open, data-driven research without programmatic reins tied to specific goals. They should award grants regularly to individuals with a proven track record of innovation rather than to projects with predictable results. Such a funding scheme is essential for promoting breakthroughs in the long run, since it encourages researchers to take on high-risk projects with potentially high gains but fundamentally unpredictable outcomes.
Why “data driven”? Without data, speculative theory bubbles which might have no real value in explaining nature are free to grow without restraint. Data guides physicists in the right direction and poses new puzzles that need to be resolved, keeping the scientific process honest and exciting. When theory and data don’t match up, we may be puzzled and disappointed, but these failures are a crucial aspect of our learning experience. They force creative individuals to come up with new ways of thinking about the physical reality. Practicing physicists should vow to work on testable predictions in analogy to The Hippocratic Oath in the medical sciences.
Over long periods of time, decades or more, the benefits from this approach are so great that even profit-oriented businesses may choose to support it. For example, between the 1930s and 1970s, AT&T nourished such a culture through its research and development arm, Bell Labs. AT&T stationed its creative scientists in the same corridor, gave them freedom, and harvested some of the most important discoveries in science and technology of the
20th century: the foundation of radio astronomy in 1932, the invention of the transistor in 1947, and the development of information theory in 1948. The invention of solar cells came shortly after in 1954, followed by the laser in 1958, the first communications satellite in 1962, the charged-coupled device (CCD) in 1969, and the fiber optic network in 1976. Such long-term benefits require patience and foresight.
Today, resources are abundant for projects with predictable results. Funding agencies are willing to invest billions of dollars on specific, programmatic questions, like the search for the Higgs boson—a major discovery that opens the door to exciting future advances. I am not advocating that funding agencies shift their primary focus to open research, but rather that they should not ignore it altogether; the range of possible outcomes must not be defined so narrowly. Right now, restricted by programmatic reins, large teams of astronomers are analyzing vast amounts of data with limited attention to the possibility of unexpected discoveries in aspects of the data unrelated to their main business agenda. It is as if Columbus were sailing away from America and ignoring any unexpected territory which is not the East Indies.
A few years ago, one of my PhD students worked with me on an elaborate project that took a year to complete. When the student showed me the first draft of our paper, I left many comments for him on the hardcopy. One of my comments was related to the Introduction section of the paper, in which we described the existing literature on the subject of our research. My comment said, “Please add a reference that discusses a particular possibility that we appear to ignore in our work.” The student came back to me a day later and replied, “Sorry, but there is no paper in the literature discussing this novel possibility.”
We immediately realized that this unexplored idea would be an excellent target for an exciting follow-up project. We ended up writing a short paper that was published a few months later in one of the most prestigious journals in fundamental physics. But when the student presented the research at his PhD research exam, he dedicated most of his talk to the first project and only a short amount of time at the end to the second project. In other words, he chose to organize his discussion based on the amount of time that it took to complete these two papers rather than on their scientific merit. After his exam, I told him, “Forget about the long project we worked on for a year. In your next presentation at a scientific conference, just focus on the exciting unexpected idea that we came across for our second project.”
Progress is not linear in time and sometimes it is even inversely proportional to the contemporaneous level of invested effort. This is because progress rests on lengthy preparatory work which lays the foundation for potential breakthroughs. When we discover something very different from what we set out to find, we should not grieve the time and money lost in pursuit of the original goal: What’s been found is far more valuable. No matter how long you have spent questing after lost donkeys, there is a moment at which you must forget about them and see the kingdom before you.
Author's picks for further reading
Gertner, J., "True Innovation," NY Times Sunday Review, February 25, (2012)
Isaacson, W., "Inventing the Future," NY Times Sunday Book Review, April 6 (2012)
Loeb, A., "Taking the Road Not Taken: on the Benefits of Diversifying Your Academic Portfolio," Nature 467, 358 (2010)
Loeb, A., "Rating Growth of Scientific Knowledge and Risk from Theory Bubbles," Nature 484, 279 (2012)