Skoltech HPC clusters comparison

HPC Cluster

Pardus

Arkuda

Zhores

General description

Vendor: IBM/LenovoPlatform: Intelligent Cluster

General purpose: students’ education, research requiring a small number of computing resources

Vendor: LenovoPlatform: LeSI/LeROM Best recipe 17D

Frontends
Server: 2x x3550 M5

VDI
Servers: 3x x3650 M5 equipped by 2x NVIDIA M60, or by 2x NVIDIA K6000, each

Vendor: DELLPlatform: Hybrid cluster platform

General purpose: for research requiring a large number of hybrid computing resources: GPU and CPU

Computing perfomance (Rpeak)

24,066 TFLOPS 135,95 TFLOPS 1 PFLOPS

The compute part

Compute blades: 18 (nx360 M4) + 5 (nx360 M5) + 4 (sd350)

  • Processor type A (nx360 M4): Intel Xeon Processor E5-2650 v2 (Ivy Bridge) 8C 2.6GHz
  • Processor type B (nx360 M5): Intel Xeon Processor E5-2697 v4 (Broadwell) 18C 2.3GHz
  • Processor type C (sd350): Intel Xeon Processor E5-2698 v4 (Broadwell) 20C 2.2GHz
  • GPU type: NVIDIA Tesla K80
  • Computing processors CPU quantity: 36 (A) + 8 (B) + 8 (C)
  • Computing processors GPU quantity: 2 (K80), i.e. 4 (K40)
  • Computing cores CPU quantity: 288 (A) + 180 (B)+160 (C)
  • Computing cores CUDA quantity: 9984

 

 

 

Peak performance CPU (Rpeak):
5,99 Tflops (A) + 6,624 TFlops (B) + 5,632 TFlops (C)

Peak performance GPU (Rpeak):
5,82 Tflops

 

Total CPU RAM: 1152 GB (4GB per Core type A) + 2560 GB (14GB per core type B) + 1024 GB (6.4GB per core type C)

Total GPU RAM: 48 GB

Compute blades: 54 (nx360 M5)

  • The monster SMP nodes: 2 (x3850 X6) +1 (x3950 X6)
  • Processor type A (nx360 M5): Intel Xeon Processor E5-2667 v4 (Broadwell) 8C 3.2GHz
  • Processor type B (nx360 M5): Intel Xeon Processor E5-2698 v4 (Broadwell) 20C 2.2GHz
  • Processor type C (x3850 M5): Intel Xeon Processor E7-4850 v4 (Broadwell) 16C 2.1GHz
  • Processor type D (x3950 M5): Intel Xeon Processor E7-8890 v4 (Broadwell) 24C 2.2GHz
  • GPGPU type A: NVIDIA K80s
  • GPGPU type B: NVIDIA M40
  • Computing CPU processors quantity:
    36 (A) + 72 (B) + 8 (C) +8 (D)
  • Computing GPU processors quantity: 6 (A) + 6 (B)
  • Computing cores CPU quantity: 288 (A) + 1440 (B) + 128 (C) + 192 (D)

 

CPU Theoretical peak performance (Rpeak):
14,74 TFlops (A) + 50,69 TFlops (B) + 4,30 TFlops (C) + 6,76 TFlops (D)

GPU Theoretical peak performance (Rpeak):
17,46 TFlops (A, FP64) + 42 TFlops (B, FP32)

Total CPU RAM: 4608 GB (16GB per Core type A) + 18432 GB (12.8GB per core type B) + 6144 GB (48GB per core type C) + 6144 GB (32GB per core type D)

Total GPU RAM: 144 GB (type A) + 72GB (type B)

Compute blades:

  • CPU Nodes – 44
  • GPU Nodes – 25

 

104 GPUs(Tesla V100-SXM2) with 16Gb of memory

The data storage system Parallel file system of data storage:
GPFSAvailable disk space (Operational): 15 TB (DRAID5) as fast scratch + 43 TB (DRAID6) for user data (aka /home)

Local scratch disk: 347 GB (nx 360 M4); 460 GB (nx 360 M5) + 16T (shared across 4x nx360 M5); 7.5TB (sd 350)

Backup: No

Parallel file system of data storage:
GSS24/GPFSAvailable disk spaces (Operational): 0.9 PB (GSS24) as fast scratch + 84 TB (DRAID6) for user data (aka /home)

Local scratch SSD disk: from 347 GB to 16000 GB

Backup: 84TB, GPFS/GSS asynchronous mirroring

Parallel file system of data storage:
GPFS 416T

Home directory SSD 6T NFS share /trinity/home

Data network:
InfiniBand FDR14 (56 Gb/s)Service and management networks:
1GbE
Data networks:
GPFS/GSS24 – InfiniBand EDR (100 Gb/s)Service and management networks:
1GbE
Data network:
InfiniBand EDR (100 Gb/s)Service and management networks:
1GbE
Operation System Linux Red Hat 6.8 (Santiago) Linux Red Hat 7.3 (Maipo) CentOS Linux release 7.6.1810 (Core)

Workload Manager

Moab HPC Suite v9.0.2 Moab HPC Suite v9.1.1 Slurm 19.05