BROWSE
Detail
Conference Paper
OMB-CXL: A Micro-Benchmark Suite for Evaluating MPI Communication Utilizing Compute Express Link Memory Devices
Cited 1 time in 
Share 
Share
- Authors
- Tu Tran, Mustafa Abduljabbar, Hooyoung Ahn, Seonyoung Kim, Yoomi Park, Woojong Han, Shinyoung Ahn, Hari Subramoni, Dhabaleswar K. Panda
- Issue Date
- 2024-07
- Citation
- Practice and Experience in Advanced Research Computing (PEARC) 2024, pp.1-8
- Language
- English
- Type
- Conference Paper
- Abstract
- Compute Express Link (CXL) is a promising technology providing connectivity between host processors and peripheral devices like accelerators or memory modules. Compute nodes are usually connected through a high-speed network like Ethernet or Infiniband. CXL provides another way for connectivity by having compute nodes connected through CXL switches. CXL devices can connect to the switches, granting resource pooling and sharing across nodes. Access latency to CXL memory is ∼ 10x smaller than access to the memory of another node through network operations. This enables a more efficient way to communicate between nodes. Currently, no existing Message Passing Interface (MPI) libraries utilize CXL for inter-node communication. In this paper, we propose the usage of CXL memory devices to enhance message-passing communication across nodes. To demonstrate the benefit of CXL, we extend OSU Micro-Benchmark (OMB), a well-known MPI benchmark suite, to evaluate point-to-point communication going over CXL; the extended OMB is named OMB-CXL. As CXL technology is evolving and still under development, the availability of a CXL system with switches and memory devices is limited. We describe how to set up an emulated CXL system using QEMU, a virtualization software, for early experience. The experimental results show that communication over CXL is 15x better for small messages and 4x for larger ones in latency and bandwidth than over the network on average.
- KSP Keywords
- Access Latency, High-speed networks, MPI Benchmark Suite, Memory modules, Message Passing Interface, Network operation, Peripheral devices, Set up, inter-node communication, memory device, micro-benchmark