Keren Zhou and Binqian Yin, computer scientists at George Mason University, have received $362,095 from the U.S. National Science Foundation to develop a new generation of software for one of artificial intelligence’s most difficult computational problems: processing ragged tensors efficiently on graphics processing units.
The three-year project, titled “Collaborative Research: SHF: Hierarchical Optimization of Ragged Tensor Operators for Deep Learning Workloads,” focuses on a type of data structure that does not fit neatly into the fixed-size arrays commonly used by modern machine-learning systems. Unlike regular tensors, whose dimensions are uniform, ragged tensors contain sequences or groups with different lengths. This irregularity is common in real-world AI applications, including natural-language processing, recommendation systems, graph learning, speech recognition and scientific simulations.
Deep-learning frameworks are designed to exploit the massive parallelism of graphics processing units, or GPUs. These processors can execute thousands of operations simultaneously, making them central to the training and deployment of contemporary AI models. However, GPUs achieve their greatest efficiency when data is organized predictably. Ragged tensors disrupt that regularity: one input may contain a few elements, while another may contain thousands. As a result, GPU threads can be forced to wait, memory accesses can become inefficient, and computational resources may sit idle.
Zhou, an assistant professor of computer science in George Mason’s College of Engineering and Computing, and Yin, a professor in the same department, are seeking to address these limitations through a combination of algorithms, data structures, compiler technology and automated performance tuning. Their work will examine the fundamental computational behavior of ragged tensor operators—the routines that manipulate irregular data during AI workloads—and develop methods for making those routines faster, more scalable and easier to integrate into existing software systems.
A central innovation will be a multi-level, multi-phase auto-tuning module designed to adapt computing strategies to dynamic workloads. Auto-tuning allows software to evaluate multiple implementation choices and select the one most likely to deliver high performance for a particular input, GPU architecture or execution environment. For ragged tensors, that decision is especially complicated because the shape and distribution of the data can change from one operation to the next. The researchers’ hierarchical approach is intended to make these decisions at several levels, balancing global workload behavior with the detailed characteristics of individual operations.
The project will also investigate ways to reduce the overhead associated with irregular computation. In a conventional deep-learning pipeline, the time spent preparing data, coordinating GPU threads and moving information between processors can undermine the benefits of hardware acceleration. Zhou and Yin plan to develop mechanisms that overlap computation with communication, allowing useful processing to continue while data is transferred or other system tasks are completed. If successful, the approach could reduce idle time and improve throughput in applications that rely on large, constantly changing datasets.
Another major component is a compiler module that can incorporate ragged tensor operators into established deep-learning workflows. Compilers translate high-level programs into machine instructions, and a specialized compiler layer could automate many of the optimizations currently handled manually by programmers. The proposed module is intended to connect irregular operators with existing frameworks and hardware without requiring researchers or developers to redesign their entire software stack. That compatibility could be critical for bringing efficient ragged-tensor processing from experimental systems into practical AI applications.
The research arrives as the computational demands of artificial intelligence continue to expand beyond conventional image and tabular data. Large language models process sequences that vary in length, recommendation systems analyze users with different histories, and graph-based models work with networks whose nodes may have widely different numbers of connections. In each of these settings, forcing irregular information into uniform structures can require padding, which adds unnecessary memory use and computation. More direct support for ragged data could make AI systems more efficient, particularly when operating under constraints such as limited memory, high energy costs or the need for rapid responses.
The project is scheduled to begin in July 2026 and continue through late June 2029. Its results could influence the design of future AI libraries, GPU kernels and compiler systems by treating irregularity not as an inconvenient exception but as a fundamental feature of modern workloads. By combining adaptive optimization with improved communication and software integration, Zhou and Yin aim to help GPUs handle the messy, uneven data that increasingly defines real-world artificial intelligence. The research is supported by the National Science Foundation and represents George Mason’s broader contribution to high-performance computing and the foundations of next-generation AI.
Subject of Research: Efficient processing and optimization of ragged tensor operators for deep-learning workloads on graphics processing units.
Article Title: Collaborative Research: SHF: Hierarchical Optimization of Ragged Tensor Operators for Deep Learning Workloads
Web References: Mason Now: Power the Possible; George Mason University
Keywords
Ragged tensors, deep learning, artificial intelligence, graphics processing units, GPU optimization, auto-tuning, compiler technology, high-performance computing, irregular data, National Science Foundation, George Mason University
Tags: challenges of GPU parallelism with ragged tensorsdeveloping software for complex tensor computationsefficient AI computation with non-uniform dataGPU optimization for irregular data structuresgraph learning tensor operationshierarchical optimization of ragged tensor operatorsnatural language processing with ragged tensorsNSF-funded AI research projectsragged tensor processing in deep learningrecommendation system data structuresscientific simulations with irregular dataspeech recognition tensor processing
