Can a UNICORN outrun earthquakes?

Each year, anywhere from a few hundred to tens of thousands of deaths are attributed to the catastrophic effects of major earthquakes. Apart from ground shaking, earthquake hazards include landslides, dam ruptures, flooding, and worse–if the sea floor is suddenly displaced during an earthquake, it can trigger a deadly tsunami.

Although earthquakes can’t be prevented, processes involving the Earth’s tectonic plates that make up its crust and upper mantle can provide scientists with clues about the possible effects of these impending disasters before they arrive.

A team led by professor Tsuyoshi Ichimura at the Earthquake Research Institute (ERI) at the University of Tokyo (UTokyo) is studying the deformation of tectonic plates to aid physics-based forecasting of natural disasters such as earthquakes. Specifically, the team is simulating a tectonic plate boundary spanning from Vancouver, British Columbia, down to Northern California. At this boundary–called the Cascadia Subduction Zone–the coastal Explorer, Juan de Fuca, and Gorda plates move east and shift underneath the North American Plate, a process known as subduction that can trigger large-magnitude earthquakes and volcanic activity.

The team recently extended and optimized one of its scientific codes for the world’s most powerful and smartest supercomputer for open science, the IBM AC922 Summit at the Oak Ridge Leadership Computing Facility (OLCF), a US Department of Energy (DOE) Office of Science User Facility located at DOE’s Oak Ridge National Laboratory (ORNL).

By transforming the Unstructured fiNite element ImpliCit sOlver with stRuctured grid coarseNing (UNICORN) code into an artificial intelligence (AI)-like algorithm, the team ran UNICORN at 416 petaflops and gained a 75-fold speedup from a previous state-of-the-art solver by fully leveraging the power of the Tensor Cores on Summit’s Volta GPUs. Tensor Cores are specialized processing units that rapidly carry out matrix multiplications and additions using mixed precision calculations.

“The Tensor Cores aren’t available for just any type of calculation,” said Kohei Fujita, assistant professor at ERI. “For this reason, we had to align all of our data access patterns and multiplication patterns to suit them.” Data access patterns determine how data is accessed in memory by a software program and can be organized more efficiently to exploit a particular computer architecture.

Using UNICORN, the UTokyo team simulated a 1,944 km × 2,646 km × 480 km area at the Cascadia Subduction Zone to look at how the tectonic plate is deformed due to a phenomenon called a “fault slip,” a sudden shift that occurs at the plate boundary.

The team said the new solver can be used as a tool to aid scientists in the arduous task of long-term earthquake forecasting–a goal that, when realized, could lead to earthquake prediction and disaster mitigation.

Previously, the team demonstrated a general approach to introduce AI to scientific applications in the iMplicit sOlver wiTH artificial intelligence and tRAnsprecision computing, or MOTHRA, code–an achievement that earned them an Association for Computing Machinery Gordon Bell finalist nomination last year.

“For UNICORN, we optimized the code specifically for Summit,” said ERI doctoral student Takuma Yamaguchi. “New hardware with some specific features sometimes requires sophisticated implementations to achieve better performance.”

UNICORN performs denser computations, allowing it to take full advantage of the unique architecture of Summit, which features 9,216 IBM POWER9 CPUs and 27,648 NVIDIA Volta GPUs. The most computationally expensive piece of the code ran at 1.1 exaflops using mixed precision–a major undertaking for a code that is based on equations rather than deep learning computations. (Codes based on the latter are inherently optimal for systems such as Summit.)

For future earthquake problems, the team will need to apply UNICORN to analyze the Earth’s crust and mantle responses to a fault slip over time. This will require thousands of simulations then hundreds or thousands of additional iterations to compare the results with real-world earthquake events.

“To reach our earthquake forecasting goals, we will have to do many simulations of crust deformation and then compare our results with observed records from past earthquakes,” Ichimura said.

###

The team is presenting this work at the 2019 Supercomputing Conference, SC19, in a poster titled “416-PFLOPS Fast Scalable Implicit Solver on Low-Ordered Unstructured Finite Elements Accelerated by 1.10-ExaFLOPS Kernel with Reformulated AI-Like Algorithm: For Equation-Based Earthquake Modeling.” This work was conducted as joint research with NVIDIA, ORNL, the Japan Agency for Marine-Earth Science and Technology, the University of Texas at Austin, and RIKEN. Additionally, the team is presenting the work at the Workshop on Accelerator Programming Using Directives held in conjunction with SC19.

Related Publication: T. Ichimura, K. Fujita, T. Yamaguchi, A. Naruse, J. C. Wells, C. J. Zimmer, T. P. Straatsma, T. Hori, S. Puel, T. W. Becker, M. Hori, and N. Ueda. “416-PFLOPS Fast Scalable Implicit Solver on Low-Ordered Unstructured Finite Elements Accelerated by 1.10-ExaFLOPS Kernel with Reformulated AI-Like Algorithm: For Equation-Based Earthquake Modeling.” Poster to be presented at the 2019 International Conference for High Performance Computing, Networking, Storage, and Analysis (SC19), Denver, Colorado, November 17-22, 2019.