Presentation
Enabling Efficient Data Infrastructure and Analytics on HPC Systems
SessionDoctoral Showcase II
Author
Advisor
Event Type
Doctoral Showcase
W
TUT
TP
EX
EXH
Compilers
Data Analytics
Data Management
GPUs
MPI
OS and Runtime Systems
Doctoral Showcase
TimeTuesday, November 13th2pm - 2:15pm
LocationD221
DescriptionWe propose to leverage PGAS and one-sided communication for building data infrastructure and analytics frameworks on HPC systems. Specifically, we have developed SHMEMCache, a distributed in-memory key-value store and SHMEMGraph, a balanced graph processing framework. We have tackled unique challenges in data consistency, load balancing, etc. with novel design features. The experimental results show that SHMEMCache achieves significant performance improvements over other KV stores such as Memcached in terms of latency and throughput. In addition, SHMEMGraph achieves an average improvement of 35.5% in terms of job completion time compared to a state-of-the-art graph processing system called Gemini.
Archive