Search

link to homepage

Institute for Advanced Simulation (IAS)

Navigation and service


JUQUEEN - Configuration

System Configuration

  • 28 racks (7 rows à 4 racks) - 28,672 nodes (458,752 cores)

    Rack: 2 midplanes à 16 nodeboards (16,384 cores)
    Nodeboard: 32 compute nodes
    Node: 16 cores

  • Main memory: 448 TB
  • Overall peak performance: 5.9 Petaflops
  • Linpack: 5.0 Petaflops
  • I/O Nodes: 248 (27x8 + 1x32) connected to 2 CISCO Switches

IBM BlueGene/Q Specifications

  • Compute Card/Processor

    IBM PowerPC® A2, 1.6 GHz, 16 cores per node

  • Memory

    16 GB SDRAM-DDR3 per node

  • Networks
    5D Torus — 40 GBps; 2.5 μsec latency (worst case)
    Collective network — part of the 5D Torus; collective logic operations supported
    Global Barrier/Interrupt — part of 5D Torus, PCIe x8 Gen2 based I/O
    1 GB Control Network — System Boot, Debug, Monitoring
  • I/O Nodes (10 GbE)

    16-way SMP processor; configurable in 8,16 or 32 I/O nodes per rack

  • Operating system

    Compute nodes: CNK, lightweight proprietary kernel

Miscellaneous

  • BG/Q Power
    Direct-current voltage (48V to 5V, 3.3V, 1.8V on node boards)
    60 - 70 kW per rack (average)
  • JSC power cables and usage
    app 4380 m power cables (63A/400V), app. 7900 kg
    app. 1.7 MW in 2012 (compares to app. 4000 households)
  • Cooling
    90% water cooling (18-25°C,demineralized,closed circle); 10% air cooling
    Temperature: in: 18°C, out: 27°C
  • JSC, Cable Trays
    48 m, 1900 kg
  • Square Footage
    83 m2 (+ 43 m2 JUST Fileserver)

IBM BlueGene/Q Characteristics

  • Integrated 5D torus provides tremendous bisection bandwidth.
  • Quad floating point unit (FPU) for 4-wide double precision FPU SIMD and 2-wide complex SIMD.
  • “Perfect” prefetching for repeated memory reference patterns in arbitrarily long code segments.
  • Multiversioning cache with transactional memory eliminates the need for locks.
  • Speculative execution allows OpenMPthreading with data dependencies.
  • Atomic operations, pipelined at L2 with low latency even under high contention, provide a faster handoff for OpenMP work.
  • A wake-up unit allows SMT threads to sleep while waiting for an event and avoids register-saving overhead.
  • A 17th core manages OS related tasks thus reducing OS related noise.

Network Switches

  • 2 CISCO Nexus 7018 Switches
  • Total ports: 480

Infrastructure Nodes

  • 2 service nodes and 2 login nodes (IBM Power 740):

    • Total number of processors: 16
    • Processor type: Power7, 8C, 3.55 GHz
    • Total amount of memory: 128 GB
    • Operating system: RedHat Linux V6.2
    • Internet address of login node: juqueen.fz-juelich.de

Installation Phase:

  • May 2012 - Jan 2013

Position in Top 500

  • June 2012: 8 (8 racks)
  • Nov. 2012: 5 (24 racks)
    28 racks:
  • June 2013: 7 (Europe: 1)
  • Nov. 2015: 11 (Europe: 3)

Servicemeu

Homepage

Logo

 

 

 

YOUR OPINION MATTERS!

 

Dear visitor,

To make our website suit your needs even more and to give it a more appealing design, we would like you to answer a few short questions.

Answering these questions will take approx. 10 min.

Start now Close window

Thank you for your support!

 

In case you have already taken part in our survey or in case you have no time to take part now, you can simply close the window by clicking "close".

If you have any questions on the survey, please do not hesitate to contact: webumfrage@fz-juelich.de.

 

Your Team at Forschungszentrum Jülich

 

Note: Forschungszentrum Jülich works with the market research institute SKOPOS to anonymously conduct and analyze the survey. SKOPOS complies with the statutory requirements on data protection as well as with the regulations of ADM (Arbeitskreis Deutscher Markt- und Sozialforschungsinstitute e.V.) and ESOMAR (Europäische Gesellschaft für Meinungs- und Marketingforschung). Your data will not be forwarded to third parties.