When Networks Network

Half a dozen times each night, your slumbering body performs a remarkable feat of coordination.

During the deepest throes of sleep, the body’s support systems run on their own timetables. Nerve cells hum along in your brain, their chitchat generating slow waves that signal sleep’s nether stages. Yet, like buses and trains with overlapping routes but unsynchronized schedules, this neural conversation has little to say to your heart, which pumps blood to its own rhythm through the body’s arteries and veins. Air likewise skips into the nostrils and down the windpipe in seemingly random spits and spats. And muscle fluctuations that make the legs twitch come and go as if in a vacuum. Networks of muscles, of brain cells, of airways and lungs, of heart and vessels operate largely independently.

Every couple of hours, though, in as little as 30 seconds, the barriers break down. Suddenly, there’s synchrony. All the disjointed activity of deep sleep starts to connect with its surroundings. Each network — run via the group effort of its own muscular, cellular and molecular players — joins the larger team.

This change, marking the transition from deep to light sleep, has only recently been understood in detail — thanks to a new look at when and how the body’s myriad networks link up to form an übernetwork.

“As I go from one state to another, immediately the links between the physiological systems change,” says Plamen Ivanov, a biophysicist at Boston University. “It is quite surprising.”

And it’s not just in bodies. Similar syncing happens all the time in everyday life. Systems of all sorts constantly connect. Bus stops pop up near train stations, allowing commuters to hop from one transit network to another. New friends join your social circle, linking your network of friends to theirs. Telephones, banks, power plants all come online — and connect online.

A rich area of research has long been devoted to understanding how players — whether bodily organs, people, bus stops, companies or countries — connect and interact to create webs called networks. An advance in the late 1990s led to a boom in network science, enabling sophisticated analyses of how networks function and sometimes fail. But more recently investigators have awakened to the idea that it’s not enough to know how isolated networks work; studying how networks interact with one another is just as important. Today, the frontier field is not network science, but the science of networks of networks.

“When we think about a single network in isolation, we are missing so much of the context,” says Raissa D’Souza, a physicist and engineer at the University of California, Davis. “We are going to make predictions that don’t match real systems.”

Like their single-network counterparts, networks of networks show up everywhere. By waking up in the morning, going to work and using your brain, you are connecting networks. Same when you introduce a family member to a friend or send a message on Facebook that you also broadcast via Twitter. In fact, anytime you access the Internet, which is supported by the power grid, which gets its instructions via communications networks, you are relying on interdependent systems. And if your 401(k) lost value during the recent recession, you’re feeling the effects of such systems gone awry.

Findings so far suggest that networks of networks pose risks of catastrophic danger that can exceed the risks in isolated systems. A seemingly benign disruption can generate rippling negative effects. Those effects can cost millions of dollars, or even billions, when stock markets crash, half of India loses power or an Icelandic volcano spews ash into the sky, shutting down air travel and overwhelming hotels and rental car companies. In other cases, failure within a network of networks can mean the difference between a minor disease outbreak or a pandemic, a foiled terrorist attack or one that kills thousands of people.

Understanding these life-and-death scenarios means abandoning some well-established ideas developed from single-network studies. Scientists now know that networks of networks don’t always behave the way single networks do. In the wake of this insight, a revolution is under way. Researchers from various fields are rushing to figure out how networks link up and to identify the consequences of those connections.

Investigators including Ivanov are analyzing a deluge of data to understand how networks cooperate to make bodies function. Other researchers are probing the Earth around them to identify the links that keep the planet in balance. But it’s not all rainbows and butterflies. Much of the recent focus has been on the potential dangers that come with connection. In one landmark study, researchers at Boston University and elsewhere have developed math for explaining the way networks of networks can suddenly break down. Studying the bad along with the good may lead to a sort of “how to” for designing integrated systems that not only perform well in normal times, but also keep working when things go wrong.

Cascades of failure

A series of CNN news clips posted on YouTube highlight the vulnerability of interdependent systems. In what Wolf Blitzer repeatedly reminds the viewer is only an “exercise,” former U.S. government officials convene to respond to a simulated cyberattack. The War of the Worlds–esque report begins with a Russian computer infecting a smartphone with a virus. After jumping to other smartphones, the bug makes its way into U.S. computers. From there it crashes communication networks, which in turn take out power stations. The ensuing blackout shuts down transportation networks. Each failure leads to yet more failures as the effects of a single infection bounce back and forth between systems. Having no control over the Russian computer system and no authority to shut down smartphones, the U.S. government is powerless.

Shlomo Havlin of Bar-Ilan University in Israel sometimes shows portions of these clips during talks he gives on networks of networks. “If you have damage in one system, it can lead to damage in another system,” Havlin says. But he points out that concerns about such rippling damages are not entirely new. Several reports — such as the CNN coverage — have highlighted worries about how fragile interdependent systems might be. “What was not known was a systematic way to study this, a framework,” Havlin says.

He first became interested in the problem when a program reviewer from the U.S. Defense Threat Reduction Agency visited the Boston University physics department in 2009. The agency was funding Havlin and H. Eugene Stanley, along with Boston colleagues Gerald Paul and Sergey Buldyrev, to work on questions plaguing single networks. The reviewer mentioned a new topic that interested the agency: How resilient are interacting networks when something goes amiss? Proposals were due in a couple of weeks. Despite the short time frame, the team, later joined by Bar-Ilan colleague Roni Parshani, decided to tackle the issue.

Overnight Havlin came up with a way of thinking about it. Single networks are typically represented by dots joined by lines. The dots, called nodes, are the players in the network. The lines, called edges or links, represent connections between those players. Havlin’s insight was to connect some of the nodes in one network with nodes in another via a new type of line. His new lines, called dependency links, signal places where a node in one network relies on a node in the other to function — say, a computer that can’t get by without its sole power source. These key dependencies could allow a failure to propagate between systems.

Once Havlin outlined a way of thinking about the problem, Buldyrev worked through the math. It wasn’t simple. He had to use equations to explain each state of each network as the random removal of one node triggered the removal of other nodes. Buldyrev, whom Paul calls “a mathematical genius,” cracked it. Answering the program reviewer’s initial question took only about a week.

“One morning, I came in and Shlomo was — not quite dancing on the table — but he was very, very excited,” Paul says.

In their analysis of connected networks, the researchers found a type of mathematical behavior that couldn’t have been predicted from knowledge of single networks. When a node is removed from a single network, the failure tends to propagate gradually, the network coming apart bit by bit by bit. But removing nodes in a network of networks means the breakdown can occur abruptly. As nodes go offline, the system initially appears to be working properly. But all of a sudden, a threshold is reached. Lose one more node and — poof — the whole thing falls to pieces.

“Even if one more node fails, the network collapses completely,” Havlin says. “It makes the network a much more risky place.”

Stanley likens the single-network scenario to a drunken prisoner trying to escape with a pair of wire clippers. As the prisoner makes random cuts along a fence, a hole develops that gradually gets bigger and bigger. After a little while, maybe, the prisoner can stick an arm through, and with a few more snips, a head. Eventually enough snips may allow the prisoner’s whole body to fit through. But in the case of networks of networks, the prisoner cuts just one or two wires and then appears to hit on a magical one that makes the whole fence disintegrate. The prisoner can walk to freedom.

“It’s as if someone threw a switch,” Stanley says. “But there is no switch.”

After tweaking the math and running some simulations, the researchers submitted a paper to Nature. Since its publication, in 2010, more than 100 other papers have cited it.

Other teams have also found unexpected behavior in networks of networks. In 2009, D’Souza and a colleague showed that connecting a large portion of nodes in a network of networks takes fewer links than would be required for a similar single network. Other scientists have revealed that imposing travel restrictions may not reduce the spread of an epidemic as much as would be expected because of the interconnected nature of human mobility networks. And in 2008, Italian researchers reported that a power station shutdown led to a failure in the Internet communication network, causing the breakdown of more power stations and triggering an electrical blackout affecting much of Italy. In its Nature paper, the Boston group used this disaster as a real-world example to model how failures can cascade back and forth between networks.

What set the Nature paper apart from the others was that it offered a simple mathematical model to explain real-world phenomena. That finding meshed with others to give network-of-networks science a theoretical foundation.

“They have really figured out the framework of how to think about it,” says Albert-László Barabási of Northeastern University in Boston, who made seminal contributions to studies of single networks. “They came along and said, let me show how you calculate this and what are the consequences of coupling these networks.”

Since the discovery, the Boston cadre — along with a battalion of graduate students — has extended its framework to study the vulnerability of three or more interconnected systems. In another study, the researchers have found that terrorist-caused damage to an important power hub may differ from more arbitrary damage caused by, say, a rat chewing through an electrical wire.

Like a social scene in which all the popular kids hang out together, in some networks well-connected nodes are more likely to link up with other well-connected nodes. Stanley, grad student Di Zhou and colleagues have found that if one network in an interdependent system has this property, dubbed assortativity, then the whole system is more vulnerable to disturbance.

These early findings were unexpected based on studies of solo networks, leaving scientists wondering what other secrets networks of networks might hold. “There are many questions that appear immediately,” Havlin says.

It’s a small world

A similar burst of activity in network science occurred in 1998, after Cornell University’s Steven Strogatz and then-colleague Duncan Watts published a groundbreaking paper, also in Nature. Titled “Collective dynamics of ‘small-world’ networks,” it explained why the world seems so tiny.

At the time, “small-world phenomena” had already gained a degree of notoriety. In the 1960s, psychologist Stanley Milgram showed that a randomly selected person living in Nebraska could be connected via acquaintances to a target person in Massachusetts through just a few (typically six) other people. Students from Albright College in Reading, Pa., made the idea widely known in the mid-1990s when they invented a game known as Six Degrees of Kevin Bacon, based on the actor’s appearances in so many movies. With the links defined as coappearances in any single film, Bacon could supposedly be connected to any other Hollywood celebrity in no more than six steps. In the network of actors, moving from the node of Kevin Bacon to the node of, say, Hilary Swank would pass you over fewer than six films. (In fact, it’s hard to name an actor who is more than two or three degrees from Kevin Bacon. Try for yourself at www.oracleofbacon.org.)

Small-world, or Watts-Strogatz, networks exhibit two features: They are highly clustered, meaning the nodes clump together like cliques of middle school girls. And shortcuts connect those cliques, akin to a cheerleader who occasionally hangs out with a member of the nerdy group.

Much like the simple framework developed more recently by the Boston group, the Cornell duo’s findings had implications for how a network behaves. “Systems synchronize much faster, epidemics spread much more rapidly,” Strogatz says. “In the case of game theory — where you have people, companies, countries playing prisoner’s dilemma — we were able to show that the small-world structure would make a difference in how that game evolved.”

But what really launched the Watts-Strogatz revolution was the way features in their model matched multiple real-world networks. An electric power grid, actors connected to Kevin Bacon and the nerve cells in a worm were all in on a secret that scientists had only just uncovered.

“The legacy is the introduction of the idea of looking at the comparative anatomy of networks,” Strogatz says. “What we were able to show was there were universal principles that applied to different networks that scientifically were completely unrelated but mathematically were following the same architectural principles.”

Almost immediately, researchers from diverse disciplines abandoned existing projects and redirected their intellectual firepower to develop network math for proteins, planes, power stations and pathogens. Friends, film actors and financial players also got their fair share of attention. Over the last dozen years or so, this flood of effort has led to a better understanding of how nodes of all types come together to form networks and what happens when one gets plucked out.

But work so far has focused mostly on the comparative anatomy of single networks. Surprising behavior uncovered in networks of networks presents a new and still puzzling question: Do the übernetworks behind blackouts, stock market crashes, transportation gridlock and even sudden deteriorations in health — a particular worry of Stanley’s — conceal a deeper shared anatomy?

Stanley believes they might. When he walks down the stairs, he has a habit of holding the railing. Breaking a hip, he says, could trigger a series of disconnections in his body’s network of networks.

It’s widely known that an elderly person who fractures a hip faces a greatly increased chance of dying within the next year, even if repair surgery is successful. What’s not yet clear, though, is whether the cascading behavior outlined by the Boston team is behind this abrupt decline in health. An answer may emerge as scientists find out what networks of networks in the body, in finance and in nature have in common.

Plumbing networked networks

Of all the world’s network-of-networks problems, climate change is one of the most challenging to untangle. How much global temperatures will increase over the next century depends on patterns of behavior in the air, the ocean, the land and among all the organisms living on the planet. Natural cycles are influenced by human-driven networks — the economics governing greenhouse gas emissions, the political drive behind energy alternatives and the social recognition of global warming as a problem in need of a solution.

In a recent study, physicist Jonathan Donges of Germany’s Potsdam Institute for Climate Impact Research plotted hundreds of thousands of data points related to air pressure to study networks in just the atmosphere. By tracking how the data changed over time, he identified a series of horizontal networks that wrap around the Earth, layering on top of one another like Russian nesting dolls. The Arctic serves as the link, acting as a sort of atmospheric border patrol that controls mingling between the horizontal layers, he and colleagues reported last year in European Physical Journal B.

“The Arctic seems to be important in coupling atmospheric dynamics on the surface and in higher layers up in the atmosphere,” Donges says.

If networks of air molecules sound complicated, consider the network of goings-on in your cells, where the nodes and their links come in different forms. Within each cell of your body there is a constant dance among DNA, RNA and proteins. DNA encodes networks of 20,000-plus genes; at any one time many are being decoded into complementary strands of messenger RNA, which form their own networks as they guide the production of proteins. Those proteins can do-si-do with other proteins, interacting within their own network in a very physical way, or can connect with other networks by pulling genes onto or off the dance floor.

“You cannot look at these networks in isolation,” says Tom Michoel of the University of Edinburgh’s Roslin Institute. “Everything there is interconnected.”

Michoel tries to understand networked networks by studying small-scale patterns that show up more often than expected in a particular system, and thus say something about its overall functioning. Consider a common workplace pattern, in which an intermediary can serve as a point of contact between a boss and an employee. Michoel found many examples of a similar pattern in yeast cells. One of two linked-up networks included interactions that regulated gene activity, in which a protein (the boss) chemically tags a gene that codes for another protein (the intermediary). The other contained more direct protein-protein interactions (between the intermediary and an employee).

By looking at how the small-scale patterns clustered and overlapped, Michoel discerned that one boss interacts with one intermediary but that each intermediary represents many employees, sort of like a union spokesperson acting on behalf of union members. Without the übernetwork analysis, there was no way to understand the distinct roles of bosses and intermediaries, Michoel says. Important large-scale interactions would have remained hidden.

Exposing unknown interactions is not the only issue. Strengths of the connections linking networks are also important. The volume of buses traveling a route, for example, may ramp up during rush hour. Or in your social networks, you may see a coworker almost every day but a high school friend just once a year.

In his investigation of sleep cycles, Ivanov showed that changing how tightly two networks are coupled can affect physiology. Links don’t have to be newly created or severed to matter.

A former student of Stanley’s, Ivanov spent more than a decade collecting data on heart rate, breathing rate, muscle tone and eye movement to find out how the body’s networks interact during the various stages of sleep. Much like Donges’ approach with the atmosphere, Ivanov inferred links and the nature of those links by analyzing how measurable markers from each system parallel each other in time. His team found out how the networks hook and unhook, but also how those hookups vary.

Ivanov believes his problem, as well as other network-of-networks puzzles that show up in the body, is a bit more challenging than the ideal scenario tackled by Stanley and Havlin’s group.

“We could have failure even if a particular link between nodes doesn’t disappear,” Ivanov says. “We could still have all links present, but with different strengths, and the system can come to arrest.”

Such considerations inject further complications into the emerging field, suggesting just how much more there is to be learned.

Physicist and computational scientist Alessandro Vespignani of Northeastern University, who studies epidemics and other spreading processes in networks, compares the current state of knowledge to what the Romans knew about Africa 2,000 years ago. The Romans had a pretty good map of the world, but they didn’t journey deep into Africa. “There are lions, that was the only information,” Vespignani says.

Right now, scientists have a map of the future of network science, and networks of networks offer an exciting new area, but people are only beginning to travel there. “We need to define new mathematical tools,” Vespignani says. “We need to gather a lot of data. We need to do the exploratory work to really chart the territory.”

Linked resilience

D’Souza of UC Davis has made early strides in mapping a landscape different from the one where the Boston team planted its flag. When she and colleagues became interested in networks of networks, they focused on success rather than failure.

“We weren’t looking in the realm of something so catastrophic that the node goes away forever,” D’Souza says. “We are more interested in a dynamical thing that will keep the network still working.”

In a recent study, her team looked at how two linked power grids might interact, say a grid that covers much of the eastern United States and another that services the West. She constructed links between the grids that are similar to the links between individual nodes within each grid: The nodes interact, but the survival of one doesn’t depend entirely on the other. She calls them connectivity links.

Each node in each network was assigned a capacity, akin to the load a power plant can handle before it becomes overwhelmed by that demand. Links represent ways for a power plant to hand off its load. If a plant can’t meet a given demand, it can pass some on to another linked power plant, which can pass it on to another and then another. As the researchers gradually add demand, like sand being added to a pile, they look for “avalanches” of load. Load will take off running across nodes the way that sand added to a pile will eventually start tumbling down the sides. Fittingly, network scientists call these avalanches “sandpile cascades.”

In analyzing the mathematics of these cascades, D’Souza and her colleagues showed that having two networks can help take some of the burden off a single network, minimizing the threat of large avalanches. “A little bit of coupling was incredibly beneficial,” D’Souza says. “The second network acted as a reservoir where the first could shed some load.”

But add too many connections between the networks and larger avalanches become possible, the team reported in March in the Proceedings of the National Academy of Sciences.

Connected power grids are a good example of networks that cooperate, says D’Souza. Adding power lines to one network may boost the transmitting capabilities of the second. But such networks may also turn competitive, if, for example, an improvement in one puts the other at an energy-supplying disadvantage.

D’Souza’s efforts have highlighted other flavors that networks of networks can come in, too. In your social web, you probably have overlapping networks, in which you simultaneously belong to a friend group and a family group. Or there may be networks in which the nodes are the same, but the links differ; think of banks that borrow money from each other in one network and invest in each other in another.

Then there are systems in which one network is actually built on top of another, the way hyperlinked Web pages sit atop electric, fiber-optic and wireless communication channels. These “overlay networks” also show up in the brain. Its physical architecture, the very anatomy of the brain, provides the structural network from which function — thought, memory, reason — emerges.

“Functional activity for me is more of a fleeting, fast-changing, difficult to characterize and for that reason much more ethereal construct in some ways,” says Olaf Sporns of Indiana University. Sporns is a major player in the Human Connectome Project, which seeks to understand how all the nerve cells in the brain interact. “The structure of the brain, the anatomy is something that, if we have good enough instruments, we can measure,” he says. “It is actual wiring.”

Brain scientists agree that the functional network must somehow be rooted in the structural network. But exactly how one gives rise to the other isn’t clear. What’s more, the networks feed off each other, adding the element of evolution to an already hard-to-follow labyrinth of nodes and links. The architecture sculpts, constrains and molds the function, and the function leaves experiential traces on the structure over time.

Sporns proposes that these dynamics represent a constant balancing act between the wiring cost in the anatomical network and the desire for efficient outcomes in the functional network. “This process of negotiating, and renegotiating trade-offs,” Sporns and a colleague wrote in May in Nature Reviews Neuroscience, “continues over long (decades) and short (millisecond) timescales as brain networks evolve, grow and adapt to changing cognitive demands.”

As the brain changes in time, so does the behavior of the body — influencing all the larger networks in which a person plays a part.

That can expand the puzzles facing scientists. Questions extend to how a network of networks reacts to what’s happening within, and how people adapt to the system, says Vespignani. “If I know there is a blackout, I will do certain things. If I know there is an economic crisis, I will go to the bank and ask to get all my money back. If there is an epidemic, I will stay home.”

Some scientists speculate that currently available theoretical approaches for übernetworks may be too simplistic to be useful. One economist went so far as to warn of the dangers of applying the Boston team’s results too widely, assuming everything is a nail just because you have a hammer. Most researchers, though, offer a more measured take.

Toward better systems

While physicists and mathematicians strive for simplicity, engineers like Leonardo Due±as-Osorio of Rice University favor a more data-driven simulation approach, enriching tools from network science with realities from physical systems.

“When you have a complex problem, abstractions of the analytical kind can help you narrow down where to focus,” Due±as-Osorio says. “Then you need to add refinement, make things more realistic.”

Both approaches — theoretical and simulation-based — have some real-world payoff. With equations that are mathematically tractable, “you can do a lot of insightful derivations,” he says. “Those are very valuable, but sometimes you only achieve those at the expense of simplifying the systems.”

Due±as-Osorio and others instead build network models that pin every node into its proper geographic location and give each one a different likelihood of failing, depending on factors such as its age or activity level. Many of these researchers get their data on the ground.

During a trip to Chile after a 2010 earthquake there, Due±as-Osorio collected information about what transformers failed and what pipes broke. He talked to utility companies to track service interruptions. “This information allows us to get a sense of how strong the connections are between systems,” he says.

Such data also reveal ways in which systems are suboptimal and could be improved. Some areas hard-hit by natural disasters don’t have enough connections — with, for example, only one power plant supporting a pumping station.

Efforts by Havlin and colleagues have yielded other tips for designing better systems. Selectively choosing which nodes in one network to keep independent from the second network can prevent “poof” moments. Looking back to the blackout in Italy, the researchers found that they could defend the system by decoupling just four communications servers. “Here, we have some hope to make a system more robust,” Havlin says.

This promise is what piques the interest of governments and other agencies with money to fund deeper explorations of network-of-networks problems. It’s probably what attracted the attention of the Defense Threat Reduction Agency in the first place. Others outside the United States are also onboard. The European Union is spending millions of euros on Multiplex, putting together an all-star network science team to create a solid theoretical foundation for interacting networks. And an Italian-funded project, called Crisis Lab, will receive 9 million euros over three years to evaluate risk in real-world crises, with a focus on interdependencies among power grids, telecommunications systems and other critical infrastructures.

Eventually, Due±as-Osorio envisions that a set of guidelines will emerge not just for how to simulate and study networks of networks, but also for how to best link networks up to begin with. The United States, along with other countries, have rules for designing independent systems, he notes. There are minimum requirements for constructing buildings and bridges. But no one says how networks of networks should come together.

Ivanov hopes to develop a similar rulebook for the human body that shows actual designs. Many doctors’ offices display diagrams of the body that outline the different systems — the circulatory system, the respiratory system, the musculoskeletal system. But no diagrams show how those systems interact with one another, and that knowledge might be just as crucial for fighting disease.

As more data come in, the goals of those working on human-built systems and natural systems may merge. More important than whether biological, social and technological systems exhibit similar mathematical properties may be whether they should. Can people design better systems by learning from the systems that exist in nature?

Sporns predicts the answer could be yes. “These systems naturally, just by virtue of being here, actually having survived, have been optimized to a certain extent,” he says. “They are existing proof that you can have complex networks that are structurally buildable and realizable and sustainable, at the same time dynamically competent, resilient against perturbations and evolvable.”

How to maximize sustainability, resilience and evolvability in networks of networks are questions that are still largely open. Geneticists seek answers in the genes, physiologists in the broader body and ecologists in the interactions that govern all living things. Connections forming among these growing webs of knowledge, as well as with engineers’ models and theorists’ frameworks, will provide much-needed fuel for a burgeoning intellectual endeavor.

If the efforts prevail, one day preventing blackouts, interrupting epidemics and handling a complicated commute may be as easy as waking up in the morning.

---

Network catastrophes

While researchers have not yet analyzed them in detail, some recent real-world incidents highlight what can happen if disaster strikes within a network of networks.

© AP, Corbis

India blackout, 2012
Power grids collapsed in India earlier this year, leaving hundreds of millions of people without power. The outage triggered transportation failures as local and long-distance trains stopped running. Some sources speculate that the grid was overloaded because a weak monsoon had farmers using more electricity to pump water to fields.

Stocktrek Images/Richard Roscoe/getty images

Eyjafjallajökull eruption, 2010
Iceland’s Eyjafjallajökull volcano erupted in 2010, spewing ash that shut down air travel throughout Europe. But travelers weren’t the only ones affected: Manufacturers, medical suppliers and crop producers couldn’t move their goods. The effects of the grounding rippled into the fuel, hotel and car rental industries.

Eneas De Troya/flickr

Swine flu pandemic, 2009
When a swine flu outbreak hit Mexico in 2009, officials responded with travel bans and other control measures. But a drop in international air traffic to and from Mexico didn’t prevent a pandemic. Viruses travel through a complex global mobility über­network that is made up of long-distance flights as well as local commutes, and interacts with social and economic networks.