In October 2024 I switched from the Minisforum UM790 Pro I’ve been using as my main desktop “workstation” to a much beefier AMD EPYC-based one. My main goals were:

Although not super slow, I felt like I was still often hitting the limits of 8C 79040HS (and the 64GB of DDR5-5600)
I wanted to move in some GPUs back to the desktop system itself for local work - specifically, I was interested in being able to run https://pinokio.computer/ or https://github.com/av/harbor locally to play around with new things easily, and to have fast voice-to-voice and code completion on tap

Why I ended up going with EPYC:

I originally considered a new Zen5-based 9950X - it’s a huge performance boost over prior generations, especially for the sort of developer workloads I’m doing, but as I specced out a maxed out system, I realized that both the shortage of PCIe slots/lanes, and the really disappointing memory-bandwidth (still 2xDDR5) was really a shame, considering that the upgrade I’d be looking for would still cost $2-3K.
Threadrippers are AMD’s HEDT/workstation line, but the regular TR chips only have 4-channel memory. Threadripper Pros have 8-channel memory, but are actually way pricier than their equivalent EPYC chips (which have 12-channel memory and more PCIe lanes). The motherboards are also about the same price (if not more, but actually feel like worse values - they’re sleeker and have desktop QoL features, but the server boards actually tend to have a lot more real tech packed in)
While with the latest EPYC 9005s were launching around this time, their availability (besides Engineering Samples on eBay, etc) for individuals is limited (months at least to pick up individual chips). On the other hand, while better per-core perf is great and all, EPYC 9004 chips have come down from list price, are easily available, and one nice thing, the 9005s are a drop-in replacement if in the future I actually needed more CPU power. Looking a recent posting of STREAM TRIAD memory bandwidth testing , the “F” spec (high-frequency) chips looked like the best bet for both memory bandwidth and single-core performance

Build

My original plan was to go with a 16C 9174F, but the 24C 9274F can be found for basically the same price new (about $2200)
I had a tough time deciding between an ASRock Rack GENOAD8X-2T/BCM, which is compact and has a better layout for PCIe risers, but only 8 DIMM slots, and the Gigabyte MZ33-AR0 which has 24 DIMM slots (using 12 for optimal DDR5 speed), less 4 PCIe slots, but also has 5 MCIO 8i connectors. I ended up going for the Gigabyte board (about $1000 from Newegg) since instead of PCIe risers, I can uses C-Payne MCIO PCIe gen5 Device Adapters for additional expansion.
I bought 12x32GB (384GB) of DDR5-4800 RDIMMs from mem-store for $1600.
For cooling, I bought a CoolServer 4U-SP5-M99 air cooler from Ali Express for about $100. T h ere a re n^{'} t a l o t o f " d es k t o p c l a ss " SP 5 coo l in g o pt i o n s, t h eo t h er n o t ab l eo pt i o nb e in g t h e$ 350 SilverStone XE360-SP5 AIO liquid cooler
For now, I have put all this in a Phanteks Enthoo Pro 2 Server Edition case, which is reasonably priced, and has 11 PCIe slots. Note, due to the layout of the MZ33, I’ll still have trouble fitting even two GPUs in there, so it’s likely that I’ll likely eventually switch to one of those 6U multi-GPU mining-style chassis

The parts total was about $5 K, w hi c h w hi l e n o t c h e a p, w a s a c t u a ll y n o t so ba d (w i t h s hi pp in g an d t a x es t o J a p an, t hi se n d e d u p b e in g c l oser t o$ 6K).

2nd GPU

Currently I’m using an AMD W7900 I got from the AMD Pervasive AI Developer Contest - it in general performs in the ballpark (although slightly worse) than an A6000 (Ampere). While things were a bit rough earlier in the year, right now, I think the state of RDNA3’s AI/ML software support is pretty good.

Right now, Ampere A6000s cost about $4 Ke a c h, an d W 7900 scos t ab o u t$ 3K each. If you were buying two, then I think that unless you were extremely budget constrained (in which case you should probably be going for $800 MI100’s), then the A6000s are worth the extra money for the better performance and slightly easier setup and compatibility.

However, since I have a W7900 already it’ll be significantly cheaper for me to get a second W7900 vs 2 x A6000s to get to 96GB of VRAM.

Extra Bits

There is 1 M.2 NVMe slot for a boot drive. I am using a dirt cheap (but decently fast) Movespeed SSD 4 TB (YMTC EET1A X3-9070) (Maxio MAP1602A/YMTC 232L TLC)
Every single PCIe slot on the MZ33 can be configured for PCIe bifurcation, and I am also using one of the slots with a cheap 4 x NVMe adapter card for additional memory
As a server board, it doesn’t have a lot of built-in USB, although it does have a USB 3.2 Gen1 and USB 2.0 header onboard (plugged into my case front I/O). While this is “enough,” I had a Inateck KU5211 20Gbps USB 3.2 Gen 2 card lying around that I’ve plugged in as well. These is actually a dual 10Gbps board (VIA chips) and no plug will go faster than that, but I figured it wasn’t worth trying to fight with the new Asmedia 4242 expansion boards that look like compatibility nightmares. (There are decently priced (~$30) single-port 20Gbps USB 3.2 Gen2x2 cards that use ASM3242 chips).
I am using a short PCIe 4.0 X16 riser cable for my W7900 GPU in the bottom case slots (below the motherboard) as that’s the only way to avoid hitting the CPU cooler / RAM with this config
I picked up a couple Arctic P12 Max fans ( $7 e a c hin J P] (h ttp s : // s h o p . t s u k u m o . co . j p / g oo d s /4895213704076/)) a lt h o ug hi t^{'} s w or t hn o t in g t ha t a 3 - p a c k o f [S cy t h e G r an d T or na d o 120] (h ttp s : // h w b u s t ers . co m / coo l in g / scy t h e - g r an d - t or na d o - 120 mm - f an - re v i e w /) a re a v ai l ab l e f or [ab o u t$ 36 ($12 each). One fun thing, is I found a few tiny YouTube channels dedicated entirely to just recording fan noise profiles:
- https://www.youtube.com/@ydihtty/videos
- https://www.youtube.com/@ThermalLeft/videos

Install Notes

I’ve been assembling PCs for over 30 years, and have had my hands in a number of rackmount servers as well, but this was my first time building an EPYC system, and putting a server board in a desktop case, so I had a bit of trepidation, but installation was pretty simple.

The SP5 socket and carrier system was super simple
The Phanteks case had no problems w/ the EEB screw mounts.
I bought a vertical GPU bracket in hopes of being able to fit a second GPU in, but sadly it still didn’t quite fit with the CPU cooler in the way. However, I’ll probably use the bracket simply for standing up the card.

Performance

BTW, for those interested in the CPU specifics, the 9274F runs an all-core stress at 4.2GHz at 280W (RAPL) and about 80C on the dies, with a relatively cheap and quiet AliExpress CoolServer 4U-SP5-M99 air cooler. I got it cheaper than the TRPro equivalent (7965WX) and it has +50% more MBW and a lot more usable PCIe, so I think it’s actually a decent value. (although obviously if you just want I/O and don’t need as much raw CPU power, last-gen Rome chips are much better priced!)

https://browser.geekbench.com/v6/cpu/8342584 Single Core - Multicore

https://www.cpubenchmark.net/cpu.php?cpu=AMD+EPYC+9274F&id=5513

Nvidia and Multi-GPU Wonkiness

.config/fish/config.fish

# NVIDIA GPU preferences
set -x LIBVA_DRIVER_NAME nvidia
set -x __GLX_VENDOR_LIBRARY_NAME nvidia
set -x __NV_PRIME_RENDER_OFFLOAD 1
set -x __VK_LAYER_NV_optimus NVIDIA_only
set -x VDPAU_DRIVER nvidia
set -x XDG_SESSION_TYPE wayland
set -x GBM_BACKEND nvidia-drm
set -x __GLX_PREFER_BACKEND_DEVICE nvidia

/etc/environment

❯ more /etc/environment
#
# This file is parsed by pam_env module
#
# Syntax: simple "KEY=VAL" pairs on separate lines
#

# Ensures VA-API uses nvidia
LIBVA_DRIVER_NAME=nvidia

# VA-API driver setting https://github.com/elFarto/nvidia-vaapi-driver?tab=readme-ov-file#direct-backend
NVD_BACKEND=direct

# Use NVIDIA's DRM backend for GBM, which is essential for buffer allocation and management under Wayland with NVIDIA GPUs.
GBM_BACKEND=nvidia-drm

# Function: Ensures that when an application requests OpenGL via GLX, it uses the NVIDIA implementation.
__GLX_VENDOR_LIBRARY_NAME=nvidia

# Needed by Electron/Chromium to use Nvidia EGL
__EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/10_nvidia.json

# This setting is to allow render offloading to Nvidia GPU, but for some reason, required by some Wayland apps since it reorders Vulkan support? (gnome-control-center, pavucontrol)
__NV_PRIME_RENDER_OFFLOAD=1

# https://github.com/xeco23/WasIstLos/issues/330#issuecomment-1734885601
# Because the Nvidia proprietary driver does not fully implement DMABUF, it will cause failure to start and crash. Please downgrade or add environment variables to turn off the use of DMABUF /etc/environment (or other places where environment variables are set) WEBKIT_DISABLE_DMABUF_RENDERER=1
WEBKIT_DISABLE_DMABUF_RENDERER=1
# if we have more problems https://github.com/xeco23/WasIstLos/issues/404#issuecomment-2401989062
# WEBKIT_DISABLE_COMPOSITING_MODE=1

nvidia-wayland-diagnostic.sh gpu-manager.sh electron-audit.sh

https://claude.ai/chat/1fe83ef7-ead7-4e2e-b92c-e14e33ab415b

Vulkan

Some Vulkan rendering going to the AMD GPU still? Instead of messing with primus_vk or PRIME, we just remove this now:

# For Steam (see https://wiki.archlinux.org/title/Vulkan)
paru -S lib32-nvidia-utils 
# Remove AMD Vulkan
paru -R vulkan-radeon lib32-vulkan-radeon
# Check
paru -S vulkan-tools
vulkaninfo

Electron Apps

logseq and obsidian had problems w/ EGL. I’m able to run it with ANGLE_DEFAULT_PLATFORM=vulkan DISABLE_NATIVE_GPU_MEMORY_BUFFERS=true so to force this:

mkdir -p /etc/electron/env.d/

In this file: /etc/electron/env.d/gpu-acceleration.conf

  # Global Electron GPU acceleration settings for NVIDIA Wayland
  ANGLE_DEFAULT_PLATFORM=vulkan
  DISABLE_NATIVE_GPU_MEMORY_BUFFERS=true

  # Additional helpful settings for Wayland
  ELECTRON_OZONE_PLATFORM_HINT=auto

Hardware Info

numactl

🐟 ❯ numactl -H
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
node 0 size: 386507 MB
node 0 free: 280960 MB
node distances:
node     0 
   0:   10

hwloc-ls

🐟 ❯ hwloc-ls
Machine (377GB total)
  Package L#0
    NUMANode L#0 (P#0 377GB)
    Die L#0 + L3 L#0 (32MB)
      L2 L#0 (1024KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0
        PU L#0 (P#0)
        PU L#1 (P#24)
      L2 L#1 (1024KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1
        PU L#2 (P#1)
        PU L#3 (P#25)
      L2 L#2 (1024KB) + L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2
        PU L#4 (P#2)
        PU L#5 (P#26)
    Die L#1 + L3 L#1 (32MB)
      L2 L#3 (1024KB) + L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3
        PU L#6 (P#3)
        PU L#7 (P#27)
      L2 L#4 (1024KB) + L1d L#4 (32KB) + L1i L#4 (32KB) + Core L#4
        PU L#8 (P#4)
        PU L#9 (P#28)
      L2 L#5 (1024KB) + L1d L#5 (32KB) + L1i L#5 (32KB) + Core L#5
        PU L#10 (P#5)
        PU L#11 (P#29)
    Die L#2 + L3 L#2 (32MB)
      L2 L#6 (1024KB) + L1d L#6 (32KB) + L1i L#6 (32KB) + Core L#6
        PU L#12 (P#6)
        PU L#13 (P#30)
      L2 L#7 (1024KB) + L1d L#7 (32KB) + L1i L#7 (32KB) + Core L#7
        PU L#14 (P#7)
        PU L#15 (P#31)
      L2 L#8 (1024KB) + L1d L#8 (32KB) + L1i L#8 (32KB) + Core L#8
        PU L#16 (P#8)
        PU L#17 (P#32)
    Die L#3 + L3 L#3 (32MB)
      L2 L#9 (1024KB) + L1d L#9 (32KB) + L1i L#9 (32KB) + Core L#9
        PU L#18 (P#9)
        PU L#19 (P#33)
      L2 L#10 (1024KB) + L1d L#10 (32KB) + L1i L#10 (32KB) + Core L#10
        PU L#20 (P#10)
        PU L#21 (P#34)
      L2 L#11 (1024KB) + L1d L#11 (32KB) + L1i L#11 (32KB) + Core L#11
        PU L#22 (P#11)
        PU L#23 (P#35)
    Die L#4 + L3 L#4 (32MB)
      L2 L#12 (1024KB) + L1d L#12 (32KB) + L1i L#12 (32KB) + Core L#12
        PU L#24 (P#12)
        PU L#25 (P#36)
      L2 L#13 (1024KB) + L1d L#13 (32KB) + L1i L#13 (32KB) + Core L#13
        PU L#26 (P#13)
        PU L#27 (P#37)
      L2 L#14 (1024KB) + L1d L#14 (32KB) + L1i L#14 (32KB) + Core L#14
        PU L#28 (P#14)
        PU L#29 (P#38)
    Die L#5 + L3 L#5 (32MB)
      L2 L#15 (1024KB) + L1d L#15 (32KB) + L1i L#15 (32KB) + Core L#15
        PU L#30 (P#15)
        PU L#31 (P#39)
      L2 L#16 (1024KB) + L1d L#16 (32KB) + L1i L#16 (32KB) + Core L#16
        PU L#32 (P#16)
        PU L#33 (P#40)
      L2 L#17 (1024KB) + L1d L#17 (32KB) + L1i L#17 (32KB) + Core L#17
        PU L#34 (P#17)
        PU L#35 (P#41)
    Die L#6 + L3 L#6 (32MB)
      L2 L#18 (1024KB) + L1d L#18 (32KB) + L1i L#18 (32KB) + Core L#18
        PU L#36 (P#18)
        PU L#37 (P#42)
      L2 L#19 (1024KB) + L1d L#19 (32KB) + L1i L#19 (32KB) + Core L#19
        PU L#38 (P#19)
        PU L#39 (P#43)
      L2 L#20 (1024KB) + L1d L#20 (32KB) + L1i L#20 (32KB) + Core L#20
        PU L#40 (P#20)
        PU L#41 (P#44)
    Die L#7 + L3 L#7 (32MB)
      L2 L#21 (1024KB) + L1d L#21 (32KB) + L1i L#21 (32KB) + Core L#21
        PU L#42 (P#21)
        PU L#43 (P#45)
      L2 L#22 (1024KB) + L1d L#22 (32KB) + L1i L#22 (32KB) + Core L#22
        PU L#44 (P#22)
        PU L#45 (P#46)
      L2 L#23 (1024KB) + L1d L#23 (32KB) + L1i L#23 (32KB) + Core L#23
        PU L#46 (P#23)
        PU L#47 (P#47)
  HostBridge
    PCIBridge
      PCI 01:00.0 (NVMExp)
        Block(Disk) "nvme0n1"
    PCIBridge
      PCI 02:00.0 (NVMExp)
        Block(Disk) "nvme1n1"
    PCIBridge
      PCI 03:00.0 (NVMExp)
        Block(Disk) "nvme2n1"
    PCIBridge
      PCI 04:00.0 (NVMExp)
        Block(Disk) "nvme4n1"
    PCIBridge
      PCI 05:00.0 (NVMExp)
        Block(Disk) "nvme3n1"
    PCIBridge
      PCI 09:00.0 (Ethernet)
        Net "enp9s0f0np0"
      PCI 09:00.1 (Ethernet)
        Net "enp9s0f1np1"
    PCIBridge
      2 x { PCI 0b:00.0-1 (SATA) }
  HostBridge
    PCIBridge
      PCIBridge
        PCIBridge
          PCI 83:00.0 (VGA)
  HostBridge
    PCIBridge
      PCI c1:00.0 (VGA)
    PCIBridge
      PCIBridge
        PCI c7:00.0 (VGA)
    PCIBridge
      2 x { PCI c9:00.0-1 (SATA) }

lstopo

Defaults to GUI output w/o piping?

🐟 ❯ lstopo | cat
Machine (377GB total)
  Package L#0
    NUMANode L#0 (P#0 377GB)
    Die L#0 + L3 L#0 (32MB)
      L2 L#0 (1024KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0
        PU L#0 (P#0)
        PU L#1 (P#24)
      L2 L#1 (1024KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1
        PU L#2 (P#1)
        PU L#3 (P#25)
      L2 L#2 (1024KB) + L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2
        PU L#4 (P#2)
        PU L#5 (P#26)
    Die L#1 + L3 L#1 (32MB)
      L2 L#3 (1024KB) + L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3
        PU L#6 (P#3)
        PU L#7 (P#27)
      L2 L#4 (1024KB) + L1d L#4 (32KB) + L1i L#4 (32KB) + Core L#4
        PU L#8 (P#4)
        PU L#9 (P#28)
      L2 L#5 (1024KB) + L1d L#5 (32KB) + L1i L#5 (32KB) + Core L#5
        PU L#10 (P#5)
        PU L#11 (P#29)
    Die L#2 + L3 L#2 (32MB)
      L2 L#6 (1024KB) + L1d L#6 (32KB) + L1i L#6 (32KB) + Core L#6
        PU L#12 (P#6)
        PU L#13 (P#30)
      L2 L#7 (1024KB) + L1d L#7 (32KB) + L1i L#7 (32KB) + Core L#7
        PU L#14 (P#7)
        PU L#15 (P#31)
      L2 L#8 (1024KB) + L1d L#8 (32KB) + L1i L#8 (32KB) + Core L#8
        PU L#16 (P#8)
        PU L#17 (P#32)
    Die L#3 + L3 L#3 (32MB)
      L2 L#9 (1024KB) + L1d L#9 (32KB) + L1i L#9 (32KB) + Core L#9
        PU L#18 (P#9)
        PU L#19 (P#33)
      L2 L#10 (1024KB) + L1d L#10 (32KB) + L1i L#10 (32KB) + Core L#10
        PU L#20 (P#10)
        PU L#21 (P#34)
      L2 L#11 (1024KB) + L1d L#11 (32KB) + L1i L#11 (32KB) + Core L#11
        PU L#22 (P#11)
        PU L#23 (P#35)
    Die L#4 + L3 L#4 (32MB)
      L2 L#12 (1024KB) + L1d L#12 (32KB) + L1i L#12 (32KB) + Core L#12
        PU L#24 (P#12)
        PU L#25 (P#36)
      L2 L#13 (1024KB) + L1d L#13 (32KB) + L1i L#13 (32KB) + Core L#13
        PU L#26 (P#13)
        PU L#27 (P#37)
      L2 L#14 (1024KB) + L1d L#14 (32KB) + L1i L#14 (32KB) + Core L#14
        PU L#28 (P#14)
        PU L#29 (P#38)
    Die L#5 + L3 L#5 (32MB)
      L2 L#15 (1024KB) + L1d L#15 (32KB) + L1i L#15 (32KB) + Core L#15
        PU L#30 (P#15)
        PU L#31 (P#39)
      L2 L#16 (1024KB) + L1d L#16 (32KB) + L1i L#16 (32KB) + Core L#16
        PU L#32 (P#16)
        PU L#33 (P#40)
      L2 L#17 (1024KB) + L1d L#17 (32KB) + L1i L#17 (32KB) + Core L#17
        PU L#34 (P#17)
        PU L#35 (P#41)
    Die L#6 + L3 L#6 (32MB)
      L2 L#18 (1024KB) + L1d L#18 (32KB) + L1i L#18 (32KB) + Core L#18
        PU L#36 (P#18)
        PU L#37 (P#42)
      L2 L#19 (1024KB) + L1d L#19 (32KB) + L1i L#19 (32KB) + Core L#19
        PU L#38 (P#19)
        PU L#39 (P#43)
      L2 L#20 (1024KB) + L1d L#20 (32KB) + L1i L#20 (32KB) + Core L#20
        PU L#40 (P#20)
        PU L#41 (P#44)
    Die L#7 + L3 L#7 (32MB)
      L2 L#21 (1024KB) + L1d L#21 (32KB) + L1i L#21 (32KB) + Core L#21
        PU L#42 (P#21)
        PU L#43 (P#45)
      L2 L#22 (1024KB) + L1d L#22 (32KB) + L1i L#22 (32KB) + Core L#22
        PU L#44 (P#22)
        PU L#45 (P#46)
      L2 L#23 (1024KB) + L1d L#23 (32KB) + L1i L#23 (32KB) + Core L#23
        PU L#46 (P#23)
        PU L#47 (P#47)
  HostBridge
    PCIBridge
      PCI 01:00.0 (NVMExp)
        Block(Disk) "nvme0n1"
    PCIBridge
      PCI 02:00.0 (NVMExp)
        Block(Disk) "nvme1n1"
    PCIBridge
      PCI 03:00.0 (NVMExp)
        Block(Disk) "nvme2n1"
    PCIBridge
      PCI 04:00.0 (NVMExp)
        Block(Disk) "nvme4n1"
    PCIBridge
      PCI 05:00.0 (NVMExp)
        Block(Disk) "nvme3n1"
    PCIBridge
      PCI 09:00.0 (Ethernet)
        Net "enp9s0f0np0"
      PCI 09:00.1 (Ethernet)
        Net "enp9s0f1np1"
    PCIBridge
      2 x { PCI 0b:00.0-1 (SATA) }
  HostBridge
    PCIBridge
      PCIBridge
        PCIBridge
          PCI 83:00.0 (VGA)
  HostBridge
    PCIBridge
      PCI c1:00.0 (VGA)
    PCIBridge
      PCIBridge
        PCI c7:00.0 (VGA)
    PCIBridge
      2 x { PCI c9:00.0-1 (SATA) }

BIOS Recovery

2022-01-12 I decided to upgrade my BIOS from F13 to the latest R03_F33 to see if this might change my memory performance. Well, it did in the sense that the update bricked my machine. (Once up w/ F31, this did not make a perf difference)

Here’s the full recovery procedure: https://github.com/lhl/gigabyte-bios-recovery

mostlyobvious

Explorer

EPYC Workstation