The CS500 S2600WF motherboard has 24 DDR4 DIMM slots with 12 DIMMs per processor. Each installed processor supports 6 memory channels via two Integrated Memory Controllers (IMC). Memory channels are assigned an identifier letter A thru F. On the S2600WF, each memory channel includes two DIMM slots.
Figure: S2600WF Motherboard Memory Architecture
The motherboard supports the following hardware:
Only DDR4 DIMMs are supported
Only Error Correction Code (ECC) enabled RDIMMs or LRDIMMs are supported
DIMMs organized as Single Rank (SR), Dual Rank (DR), or Quad Rank (QR):
RDIMMS – Registered DIMMS – SR/DR/QR, ECC only
LRDIMMs – Load Reduced DIMMs – QR only, ECC only
Maximum of 8 logical ranks per channel
Maximum of 10 physical ranks loaded on a channel
Supported Memory
Figure: S2600WF Motherboard DDR4 RDIMM and LRDIMM Support
Memory Slot Identification and Population Rules
A total of 24 DIMM slots are provided – 2 CPUs, 6 memory channels/CPU, 2 DIMMs per channel. The following figure identifies all DIMM slots on the motherboard.
Although mixed DIMM configurations may be functional, Cray only supports and performs platform validation on systems that are configured with identical DIMMs installed.Figure: S2600WF Motherboard Memory Slot Layout
Each installed processor provides six channels of memory. Memory channels from each processor are identified as Channels A – F.
Each memory channel supports two DIMM slots, identified as slots 1 and 2.
Each DIMM slot is labeled by CPU #, memory channel, and slot # as shown in the following examples: CPU1_DIMM_A2; CPU2_DIMM_A2
DIMM population rules require that DIMMs within a channel be populated starting with the BLUE DIMM slot or DIMM farthest from the processor in a “fill-farthest” approach.
When only one DIMM is used for a given memory channel, it must be populated in the BLUE DIMM slot (furthest from the CPU).
Mixing of DDR4 DIMM Types (RDIMM, LRDIMM, 3DS RDIMM, 3DS LRDIMM, NVDIMM) within a channel socket or across sockets produces a Fatal Error Halt during Memory Initialization.
Mixing DIMMs of different frequencies and latencies is not supported within or across processor sockets. If a mixed configuration is encountered, the BIOS will attempt to operate at the highest common frequency and the lowest latency possible.
When populating a Quad-rank DIMM with a Single- or Dual-rank DIMM in the same channel, the Quad-rank DIMM must be populated farthest from the processor. Intel MRC will check for correct DIMM placement. A maximum of 8 logical ranks can be used on any one channel, as well as a maximum of 10 physical ranks loaded on a channel.
In order to install 3 QR LRDIMMs on the same channel, they must be operated with Rank Multiplication as RM = 2, this will make each LRDIMM appear as a DR DIMM with ranks twice as large.
The memory slots associated with a given processor are unavailable if the corresponding processor socket is not populated.
A processor may be installed without populating the associated memory slots, provided a second processor is installed with associated memory. In this case, the memory is shared by the processors. However, the platform suffers performance degradation and latency due to the remote memory.
Processor sockets are self-contained and autonomous. However, all memory subsystem support (such as Memory RAS, Error Management,) in the BIOS setup are applied commonly across processor sockets.
For multiple DIMMs (RDIMM, LRDIMM, 3DS RDIMM, 3DS LRDIMM) per channel, always populate DIMMs with higher electrical loading in slot1, followed by slot 2.
