HPC/Cloud Hybrids for Efficient Resource Allocation and Throughput

HPC systems running massively parallel jobs need a fairly static software operating environment running on bare metal hardware, a high speed interconnect to reach their full potential, and offer linear performance scaling for cleverly designed applications. Cloud computing, on the other hand, offers flexible virtual environments and can be used for pleasingly parallel workloads.

Two approaches have been undertaken to make HPC resources available in a dynamically reconfigurable hybrid HPC/Cloud system. Both can can be achieved with few modifications to existing HPC/Cloud environments. The first approach, the "Nemo" system at the University of Freiburg, deploys a cloud-client on the HPC compute nodes, so the HPC hardware can run Cloud-Workloads for backfilling free compute slots. The second approach, the "Spartan" system at the University of Melbourne, generates a consistent compute node operating system image with variation in the virtual hardware specification which can be modified according to needs.

Presentation to Multicore World, Wellington, New Zealand, February 20, 2017