Search

link to homepage

Institute for Advanced Simulation (IAS)

Navigation and service


JUGENE - Configuration

IBM BlueGene/P

Compute Racks

  • 72 racks (9x8) - 73728 nodes (294912 cores)

    • Rack: 2 midplanes a 16 nodeboards (4096 cores)
  • 2304 Nodeboards a 32 compute nodes (128 cores)
  • Overall peak performance: 1 Petaflops
  • Linpack: 825.5 Teraflops
  • Main memory: 144 TB
  • I/O Nodes: 600 (Connected to FORCE10 Switch)
  • Compute Card/Processor:

    • Power PC 450, 32-bit, 850 MHz, 4-way SMP
    • L3 Cache: shared, 8 MB
    • Main Memory: 2 GB, 13.6 GB/s bandwidth
    • Peak performance: 13.6 GFlops
  • Networks:

    • Three-dimensonal torus (compute nodes),
      bandwidth per link: 425 MB/s (total: 5.1 GB/s), hardware latency: 100ns - 800ns
    • Global tree and collective (compute nodes, I/O nodes),
      bandwidth per link: 850 MB/s (total 1.7 GB/s)
    • Barrier low latency and interrupt network
    • External: 10 GigE / Functional network (I/O Nodes)

Network Switches

  • 4 Force 10 E1200i Switches
  • Total ports: 896

Infrastructure Nodes

  • 2 Service nodes IBM p6 550:

    • Total number of processors: 8
    • Processortype: Power6 4.2 GHz
    • Total amount of memory: 128 GB
    • Operating System: SuSE Linux Enterprise (SLES 10)
  • 2 Login nodes IBM p6 550:

    • Total number of processors: 8
    • Processortype: Power6 4.2 GHz
    • Total amount of memory: 32 GB
    • Operating System: SuSE Linux Enterprise (SLES 10)
    • Internet address: jugene.fz-juelich.de

Position in Top 500 - History

  • June 2009: 3 (Europe: 1)
  • Nov. 2009: 4 (Europe: 1)
  • June 2010: 5 (Europe: 1)
  • Nov. 2010: 9 (Europe: 2)
  • Jun. 2011: 12 (Europe: 2)
  • Nov. 2011: 13 (Europe: 3)

Environment

  • Power consumption: max. 35 kW per rack (total 2.2 - 2.5 MW) + 20% for cooling
  • Average application: 23.9 KW, idle 9.9 KW per rack
  • Power cables: 72 x 35m, three-phase, 125A, total 2.5 km
  • Foot print: 130 m², incl. service area: 160 m² (240 tiles with 576 holes)
  • Weight per rack: 770-900 kg
  • Cables: 4352 datacables (23 km copper), 288 x 1GigE cables (7.5 km copper),
    600 x 10GigE cables (21 km fibre optics), total ca. 50 km
  • Air cooling: 60.000 m³/h, temperature: 16°C, fans per rack: 69 (total 4968)
  • Water cooling: 300m tubes, ca. 360 m³/h,
    temperature: max. 17°C inlet, 24-27°C outlet,
    use warm water to generate inlet temperature from 6°C cold water of cold water supply
  • Noise: 85 - 105 dB

Servicemeu

Homepage