Preprint
Machine Learning

Data Locality in High Performance Computing, Big Data, and Converged Systems: An Analysis of the Cutting Edge and a Future System Architecture

Sardar Usman, Rashid Mehmood(King Abdulaziz University), Iyad Katib(King Abdulaziz University), Aiiad Albeshri(King Abdulaziz University)
December 23, 2022Electronics28 citations
PDFDOI

28

Citations

0

Influential Citations

Electronics

Venue

2022

Year

Abstract

Big data has revolutionized science and technology leading to the transformation of our societies. High-performance computing (HPC) provides the necessary computational power for big data analysis using artificial intelligence and methods. Traditionally, HPC and big data had focused on different problem domains and had grown into two different ecosystems. Efforts have been underway for the last few years on bringing the best of both paradigms into HPC and big converged architectures. Designing HPC and big data converged systems is a hard task requiring careful placement of data, analytics, and other computational tasks such that the desired performance is achieved with the least amount of resources. Energy efficiency has become the biggest hurdle in the realization of HPC, big data, and converged systems capable of delivering exascale and beyond performance. Data locality is a key parameter of HPDA system design as moving even a byte costs heavily both in time and energy with an increase in the size of the system. Performance in terms of time and energy are the most important factors for users, particularly energy, due to it being the major hurdle in high-performance system design and the increasing focus on green energy systems due to environmental sustainability. Data locality is a broad term that encapsulates different aspects including bringing computations to data, minimizing data movement by efficient exploitation of cache hierarchies, reducing intra- and inter-node communications, locality-aware process and thread mapping, and in situ and transit data analysis. This paper provides an extensive review of cutting-edge research on data locality in HPC, big data, and converged systems. We review the literature on data locality in HPC, big data, and converged environments and discuss challenges, opportunities, and future directions. Subsequently, using the knowledge gained from this extensive review, we propose a system architecture for future HPC and big data converged systems. To the best of our knowledge, there is no such review on data locality in converged HPC and big data systems.

Analysis

Why This Paper Matters

This paper addresses a critical bottleneck in modern high-performance computing and big data systems: data movement. As systems scale toward exascale and beyond, the energy and time cost of moving data dwarfs computation costs. The convergence of HPC and big data ecosystems, driven by AI workloads, makes data locality even more essential. The paper's comprehensive review of data locality techniques across both domains is timely and valuable for system architects and researchers.

The proposed future system architecture, though not validated, offers a conceptual blueprint that could inspire practical designs. By synthesizing disparate research threads, the paper helps unify the field and highlights the importance of energy efficiency for environmental sustainability.

Technical Contributions

  • Extensive literature review: Covers data locality in HPC, big data, and converged systems, including cache hierarchies, intra/inter-node communication, process/thread mapping, and in situ/transit analysis.
  • Identification of key challenges: Energy efficiency as the primary hurdle; data movement costs dominate time and energy budgets.
  • Proposed architecture: A novel system architecture for future converged HPC and big data systems, designed to minimize data movement and improve energy efficiency.
  • Discussion of opportunities and future directions: Outlines research gaps and potential innovations in data locality for converged systems.

Results

The paper does not present experimental results or quantitative metrics. Its main output is a qualitative review and a conceptual architecture. The authors claim that data locality is a key parameter for HPDA system design, but no concrete performance numbers are provided.

Significance

This paper contributes a structured understanding of data locality in the context of converging HPC and big data systems. It emphasizes energy efficiency as a critical design constraint, aligning with global sustainability goals. The proposed architecture could influence future system designs, though empirical validation is needed. For AI practitioners, the insights on data movement minimization are directly applicable to large-scale training and inference workloads.

data-localityhigh-performance-computingbig-dataconverged-systemsenergy-efficiencysystem-architecturehpda

Related Papers