Where do good ideas come from?

Karan Chawla / January 10, 2020

7 min read

Throughout history, there have been hundreds of cases where significant discoveries have been made by multiple people independently at the same time — Calculus (1600s), Steam engine (1698), the jet engine (1840), the telephone (1876), etc. Why does this happen? How do breakthroughs happen? What kind of environments breed innovation? Are Eureka moments real?

Steven Johnson tackles these questions in Where Good Ideas Come From by identifying common patterns, and conditions among inventions like the lightbulb, incubator, world wide web, and many more. The entire narrative in pop culture and the media revolves around the idea that a small number of incredibly smart people have Eureka moments that leads to breakthrough innovations that change the course of human civilization. In this book, Johnson challenges this view.

We have a natural tendency to romanticize breakthrough innovations, imagining momentous ideas transcending their surroundings…But ideas are works of bricolage…We take the ideas we've inherited or that we've stumbled across, and we jigger them together into some new shape.

The adjacent possible

Billions of years ago, carbon atoms mulled around the primordial soup. However, these atoms did not spontaneously arrange themselves into complex life forms we see in nature today.

First, they had to combine to form simple structures like polymers, proteins, cells, single-celled organisms, etc. Each structure along the way opened up possibilities for new combinations, expanding the realm of what was possible. This is the idea of Adjacent Possible, coined by the biologist Stuart Kauffman to explain how complex biological systems emerge in the environment.

Steven Johnson expands on this idea and says that at any given time — in science and technology, only certain kinds of ideas are feasible, and this is determined by various combinations of events and discoveries that have happened before.

You begin in a room with four doors, each leading to a new room that you haven't visited yet. Those four rooms are the adjacent possible. But once you open one of those doors and stroll into that room, three new doors appear, each leading to a brand-new room that you couldn't have reached from your original starting point. Keep opening new doors and eventually you'll have built a palace. The history of cultural progress is almost without exception, a story of one door leading to another door, exploring the palace one room at a time.

Johnson gives the example of Stephane Tarnier, a French doctor, who upon seeing incubators for chicken hatchlings at the Paris Zoo, employed the zoo's poultry-raiser to build incubators for premature newborns at his hospital. She was the first to show how incubators significantly reduced the infant mortality rate, leading to their widespread use in France and beyond.

Good ideas are not conjured out of thin air; they are built out of a collection of existing parts, the composition of which expands over time.

According to Johnson, "the adjacent possible is as much about limits as it is about openings." Johnson suggests that one can't skip beyond the adjacent possible, and such leaps are rare. Had YouTube launched in the 1990s, it would have failed, since it relied on fast internet connection and the internet browser, neither of which had been developed then.

The bounds of the adjacent possible constrain evolution and innovation; in other words, the realm of possibilities at any given moment. And the ideas beyond the scope of the adjacent possible are simply "ahead of their time."

The predominance of multiples in innovation alluded to at the start of this newsletter, highlights how existing parts and knowledge constrain the adjacent possible.

The inventors of the transistor — Shockley, Bardeen, and Brattain admitted to the fact that had they not invented the transistor someone else might have. They understood that when you are exploring the adjacent possible, someone else will get there eventually. In this case, the transistor had come into the adjacent possible from the increased understanding of, e.g., the basic research in atomic structure and knowledge of electrons conducted by scientists such as Bohr and J. J. Thompson.

"There was little doubt, even by the transistor's inventors, that if Shockley's team at Bell Labs had not gotten to the transistor first, someone else in the United States or in Europe would have soon after." — Jon Gertner, The Idea Factory: Bell Labs and the Great Age of American Innovation

The slow hunch

In hindsight, great discoveries might look like sudden, definable Eureka moments; in reality, they tend to develop slowly over time. They are like slowly maturing slow hunches that need time and cultivation to bloom.

According to Darwin's autobiography, the theory of natural selection simply occurred to him when he was pondering about Malthus' writings on population growth. However, Darwin's notebooks reveal that far before this so-called epiphany, he had pretty much described a pretty comprehensive theory of natural selection. His slow hunch only matured into an epiphany over a decade of cultivation. Similarly, the World Wide Web was conceived over ten years and multiple side projects by Tim Berners-Lee.

Sustaining the slow hunch is less a matter of perspiration than of cultivation. You give the hunch enough nourishment to keep it growing, and plant it in fertile soil, where its roots can make new connections. And then you give it time to bloom.

However, one can only begin cultivating hunches when one remembers having an idea. Johnson gives the example of the "commonplace" book that intellectuals during the Enlightenment era maintained where they wrote down their thoughts, transcribed interesting or inspirational passages, and assembled a personalized encyclopedia of quotations. In the same vein, Johnson prescribes us to "write everything down."

The liquid networks

Carbon is the basis of all life on Earth because its atomic structure makes it great at connecting with other atoms enabling it to form long chains of compounds. These connections allow for the formation of polymers and proteins without which Earth would have likely remained lifeless. When we first began to organize ourselves into villages, towns, and cities, we became members of networks, which exposed us to new ideas and led to what is now known as idea-sex. Before this, a novel idea by a person could very well die with them. Great ideas arise in crowds.

To better understand the process of scientific breakthroughs, psychologists decided to record everything that went on in four molecular biology laboratories. One would imagine that in a field such as molecular biology, great discoveries would be made by peering through a microscope. However, it turned out the most critical ideas arose during regular lab meetings, where the scientists informally discussed their work.

Cities facilitate such large networks that allow for inter-mingling of ideas in novel ways. And this is one of the reasons why cities breed more creative ideas than smaller towns. However, the largest network in the 21st century is not a city at all, but the internet, making the creation and diffusion of ideas more effective than any network that has come before it.

In summary, the best way to have new ideas isn't to romanticize the genius inventor working in the isolation of a mountain cabin. Instead, it's to expand the range of your possible next moves – your surface area in the world – by exposing yourself to as much serendipity, as many rival and related ideas as possible; to borrow, to reimagine, to recombine.

Good ideas happen in networks; in one rather mind-bending sense, you could even say that "good ideas are networks." Or as Johnson also puts it — "Chance favors the connected mind."

Subscribe to the newsletter

Get emails from me about future of technology, philosophy, and human behavior.

- subscribers