Time for some Metal!!! Good Bye Hypervisor…

Let’s ditch the hypervisor and put our lab right on the metal!

First, go listen to this. It will put you in the right frame of mind for what’s to come.

OK, now let’s talk about this installation.

This project is not for the faint of heart. It will require some investment. You are going to need at least 3 Intel NUC machines with a minimum of 2 cores and 32GB RAM each.

Here is my prototype:

The control plane nodes are: NUC8i3BEK machines with 32GB of RAM and 500GB SATA M.2 SSDs.

The worker nodes are: NUC6i7KYK machines with 64GB of RAM, 256GB SATA SSD for FCOS, and 1TB SATA SSD for Ceph.

I’m going to show you how to build one with 3 nodes. With the following list of parts, you will be able to power your cluster off of two power supplies, and pack the whole thing up in a slim 15” laptop bag.

It will look like this:

Below is the complete bill of materials for your starter lab. I’ve included Amazon.com links to the gear that I have.

• Bare Metal Lab Gear:
  • Network adapter for your workstation
    • Noter: You will not be able to use the WiFi in your MacBook for the bootstrap network bridge. We’ll need a physical adapter.
  • GL.iNet GL-MV1000W (Edge Router with WiFi)
    • Note: You may also be able to use the less expensive GL.iNet GL-AR750S-Ext But, while it has dual-band WiFi, it is much more limited on internal storage, CPU, and RAM.
  • GL.iNet GL-MV1000 (Internal Router)
    • Note: The GL-AR750S-Ext will not work for this function.
  • Raspberry Pi 4b 8GB (Bastion Host)
  • SD Card for the Pi
  • Anker 63W 4 Port PIQ 3.0 & GaN Fast Charger Adapter
    • Note: I use this Anker power supply to run the routers and Pi off of one brick. It saves valuable bag space when travelling.
  • USB C cables
  • USB A to USB C cables
  • Network Cables
  • Gigabit Network Switch
  • Tomtoc 360 16-inch Laptop Shoulder Bag
  • 240W slim Power Supply
    • Note: Get a 320W Power Supply if you use the NUC10i7FNK machines. They need a bit more muscle for the 6 cores.
  • 7.4x5.0mm to 5.5x2.5mm Adapter
  • 5.5x2.5mm 1 to 4 splitter

  • 3 X Intel NUC10i3FNK configured with 32 or 64GB RAM & 1TB NVMe

    Other NUC models will work as well. The slim models are much more portable, so I tend to prefer those. I love the NUC10i7FNK because it has 6 cores. That’s 12 vCPUs for your nodes!!! Prices fluctuate so much on the NUCs, M2 NVMe, and RAM that I am not listing any links here. But I get most of my compute gear from B&H Photo Video, or Amazon.com.

Most of the bare-metal configuration is going to be just like the KVM setup that I posted previously. In fact, I have refactored the lab helper scripts to support both KVM and Bare metal installations.

The one significant difference for this setup, is that we are going to run the bootstrap node from our workstation. In this case, a MacBook Pro with 16GB of RAM.

Everything else is pretty much the same. The cluster nodes will boot via iPXE, retrieve their ignition configs, and run the installation.

Let’s get to it!

Go here first: Setting up Your Workstation for a Bare Metal OKD install