Princeton researchers working at forefront of 'exascale' supercomputing

PPPL Exascale index

Princeton researchers working with supercomputers operating at a speed called the "exascale" are aiming to harness the power of this technology to solve a broad range of research problems. For example, climate modeler Venkatramani Balaji is focused on designing software to organize climate data from a massive archive that includes numerous models from around the world. The above model of sea surface temperatures depicts the Agulhas rings, a well-known feature of the currents around Southern Africa.

Image courtesy of the Geophysical Fluid Dynamics Laboratory



Scientists at Princeton University are composing the complex codes designed to instruct a new class of powerful computers that will allow researchers to tackle problems that were previously too difficult to solve. These supercomputers, operating at a speed called the "exascale," will produce realistic simulations of dazzlingly complex phenomena in nature such as fusion reactions, earthquakes and climate change.

The capacity to deploy computations at such an extreme scale could put scientists closer to solving a wide range of research problems, including the quest to make fusion energy a safe, affordable power source, according to William Tang, head of the Fusion Simulation Program at the U.S. Department of Energy's Princeton Plasma Physics Laboratory (PPPL). The new computing power would also greatly enhance the realism of simulations used in that research.

PPPL Exascale group

Princeton scientists (from left) Venkatramani Balaji, Jeroen Tromp and William Tang — shown at the Visualization Laboratory created by the Princeton Institute for Computational Science and Engineering — have received funding from the Group of Eight's Research Councils Initiative on Multilateral Research Funding to support their work using exascale supercomputing technology. (Photo by Elle Starkman)

In recognition of the prospects for faster progress in the understanding of complex scientific issues — and as part of a new international collaboration — the Group of Eight's (G-8) Research Councils Initiative on Multilateral Research Funding has awarded grants to Tang and two other Princeton-based scientists: Jeroen Tromp, a geophysicist; and Venkatramani Balaji, a climate modeler. All are focused on helping to develop the advanced software that will put the coming new generation of computers to work on solving problems of global interest.

"What we hope to demonstrate is that this focused level of international scientific collaboration can help deliver breakthrough payoffs in high-performance computing," said Tang, who is also a member of the executive committee of the Princeton Institute for Computational Science and Engineering (PICSciE).

Computers approaching the exascale possess the capability of doing a quintillion — or a billion billion — calculations at once. Approximately a quintillion pennies laid out edge to edge would cover the surface of the Earth twice.

PPPL Exascale waves

Tromp and his collaborators are using computer images to map the interior of the Earth and create earthquake simulations. This sequence of three snapshots illustrates wave propagation in Southern California. (Image courtesy of Santiago Lombeyda)

The three-year awards for the Princeton researchers are part of a pilot program established by the G-8 in 2010 to foster multinational collaboration among scientists. The G-8 is an organization involving leaders of some of the world's largest industrialized nations, including Canada, France, Germany, Italy, Japan, Russia, the United Kingdom and the United States.

"There is recognition that science is international in scope and that there is an advantage to scientists from different countries partnering together," said Marc Rigas, staff associate for planning and coordination of the Office of Cyberinfrastructure at the National Science Foundation, which selected and funded the U.S.-based grantees for the G-8 council.

The Princeton grants are among six winning proposals selected from 100 international applicants' submissions. All the projects require partnering with scientists from other G-8 nations. According to Tang, the grant selections were made after an intense peer review process.

PPPL Exascale fibers

Tang's team is utilizing the emerging class of supercomputers to produce higher-fidelity simulations of the physics behind fusion reactions. This simulation depicts electron-scale plasma turbulence in the National Spherical Torus Experiment at PPPL. The colored "strings" represent regions of electron density. (Image courtesy of the University of California-Davis)

Understanding fusion reactions

Tang intends to use his $470,000 grant to develop advanced simulation codes that will be compatible with the emerging class of supercomputers. The goal of his project, known as NuFuSe, is to produce higher-fidelity simulations of the physics behind fusion reactions. That will mean solving problems it would not be possible to address in a timely way using present-day supercomputers.

Working with researchers in the United Kingdom, France, Germany, Japan and Russia, Tang hopes to learn how to better control the behavior of magnetically confined fusion plasmas. Though fusion reactions are responsible for energy release in the universe, harnessing fusion as a clean and sustainable supply of energy on Earth is a major scientific and technological challenge. It has commanded the attention of researchers at laboratories worldwide, including PPPL.

Fusion reactions are produced in experimental devices in which isotopes of hydrogen are heated and contained in a magnetic field. When these charged hydrogen particles are confined long enough at temperatures that exceed 10 million degrees, a large amount of energy is produced. However, the hot plasmas can behave erratically, a phenomenon known as microturbulence. Such an event can accelerate heat loss and compromise the efficiency of the reaction. Tang noted that "it is the balance between these heat losses and the self-heating rates of the fusion reactions which largely determines the size and cost of a fusion reactor."

As part of the quest to build efficient fusion power plants, Tang envisions using the new exascale computers to create sophisticated simulations of fusion reactions to gain new insights into how to reduce the heat losses brought on by microturbulence.

The international cooperation required by the G-8 council will have another benefit, Tang said. The research partners will be collaborating instead of working in separate silos. For example, new software developed within the international project could be tested on the collaborating partners' supercomputers. The grant will cover the salary of a postdoctoral student along with some travel expenses so research staff can attend conferences.

PPPL Exascale tube

Tang and his group are working to create simulations that will enhance understanding of how to control the behavior of magnetically confined fusion plasmas. This simulation illustrates flux-driven turbulence in the DIII-D tokamak experiment, a magnetic fusion research program developed by General Atomics. (Image courtesy of Choong-Seock Chang)

Organizing climate data

For his project, Balaji — head of the modeling system group in the University's Program in Atmospheric and Oceanic Sciences and the National Oceanic and Atmospheric Administration's Geophysical Fluid Dynamics Laboratory on the Forrestal Campus — is planning to use his $313,000 G-8 grant to design software that will organize the huge but unmanageable archives of climate data from around the world.

Producing accurate forecasts of a changing climate, especially on the regional scale, is of vital interest to those managing impacts on the environment and public health. For instance, "a scientist at work on malaria might want to know how many mosquitoes are likely to be in a region in the future, which means predicting temperature or humidity," Balaji said.

Complicating matters, different research groups have constructed different mathematical models to use in making such predictions, he added. There is a "giant data archive" that exists. But, with 20 different models in use, he said, scientists could come up with 20 different predictions. "We have to run them all and make an average, but then the problem is the data is now in 20 different places," he said.

There is currently enough data out there to "predict the temperature in Burkina Faso 25 years from now," Balaji said, "but you would need to analyze probable outcomes from many models and multiple scenarios of how society might change." It will take exascale computing to make that happen, he noted. His group is developing computer language that would instruct powerful computers to coordinate and streamline such an analysis.

"The G-8 gives us a unique opportunity to have science agencies around the world work together," Balaji said. He will be working with peers in Canada, France, Germany, the United Kingdom and the United States.

PPPL Exascale globe

This simulation from Tromp's research group shows surface ground velocities (red and blue circles) and seismographic recordings (yellow) of ground displacement following the 2004 Sumatra-Andaman earthquake. (Image courtesy of Santiago Lombeyda, Vala Hjorleifsdottir, Jeroen Tromp and Rick Aster)

Mapping the Earth's interior

In a similar approach, Tromp, the Blair Professor of Geology and director of PICSciE, will use his $500,000 grant to further his work mapping the interior of the Earth. Using data from seismograph readings from all over the globe, Tromp and colleagues in Canada and France construct computer images showing the structures underneath the Earth's crust to a depth of 700 kilometers. The images make pre-human history come alive.

Tromp, a professor of geosciences and applied and computational mathematics, noted that, for example, with a newly created image for Europe "we can see the subduction of Africa, see that slab being pushed into the mantle; we can see old and recent volcanism in the Czech Republic, and how Italy has rotated counterclockwise over the last 6 million years to where it is today."

Tromp and his colleagues are also known for taking seismographic findings and turning them into earthquake simulations, known as "ShakeMovies." Though earthquakes cannot be predicted, he said, the virtual earthquakes help assess potential hazards so that engineers can build structures to withstand the quakes.

Tromp sees the G-8 approach to funding multinational collaborations as "a wonderful model for computational science."

"This is how modern science should be conducted," he said.