Build CodeReady Containers - With OKD

Prepare for the CRC Build

You will need a Linux workstation with a healthy amount of RAM for this build. I am using an Intel NUC8i3BEK that is one of the KVM nodes in my home lab. It is equipped with a 500GB SSD and 32GB of RAM. The host is running CentOS 8 Stream. Fedora should also work fine. If you choose to use a different Linux distribution, you will have to alter the initial setup that is described below. For more on my home lab setup, see this post: Let’s build an OpenShift Home Lab!

CodeReady containers is built from two code repositories, in a three step process.

  1. Deploy an OpenShift single node cluster on Libvirt
  2. Create a qcow2 disk image from the VM running the OpenShift cluster
  3. Compile crc and embed the qcow2 image.

Now, let’s build CodeReady Containers with OKD!

Initial Set Up For The CRC Dev Environment

  1. Starting with a minimal CentOS 8 Stream install, add the following packages, update and reboot you host:

    dnf -y module install virt
    dnf -y install jq golang-bin gcc-c++ golang make zip wget git bash-completion libguestfs-tools virt-install
    dnf -y update
  2. Configure that firewall to allow local access to libvirt:

    firewall-cmd --add-rich-rule "rule service name="libvirt" reject" --permanent
    firewall-cmd --zone=dmz --change-interface=tt0 --permanent
    firewall-cmd --zone=dmz --add-service=libvirt --permanent
    firewall-cmd --zone=dmz --add-service=dns --permanent
    firewall-cmd --zone=dmz --add-service=dhcp --permanent
    firewall-cmd --reload
  3. Enable libvirtd to listen on port 16509

    cat <<EOF >> /etc/libvirt/libvirtd.conf
    listen_tls = 0
    listen_tcp = 1
    auth_tcp = "none"
    tcp_port = "16509"
  4. Stop the libvirtd service, enable the tcp socket listener for libvirtd, and restart it.

    systemctl stop libvirtd
    systemctl enable libvirtd-tcp.socket --now
    systemctl start libvirtd
  5. Create a helper script to set environment variables:

    mkdir -p ${HOME}/bin
    cat << FOE > ${HOME}/bin/
    export OKD_VERSION=\$(curl | cut -d"/" -f8 | cut -d\" -f1)
    export CRC_DIR=${HOME}/crc-build
    export PATH=${HOME}/crc-build/snc/openshift-clients/linux/:\${PATH}
    export OPENSHIFT_PULL_SECRET_PATH="\${CRC_DIR}/pull_secret.json"
    export BUNDLE_DIR=\${CRC_DIR}/snc
    export OKD_BUILD=true
    export TF_VAR_libvirt_bootstrap_memory=16384
    export LIBGUESTFS_BACKEND=direct
    export KUBECONFIG=\${CRC_DIR}/snc/crc-tmp-install-data/auth/kubeconfig
    export OC=\${CRC_DIR}/snc/openshift-clients/linux/oc
    cat << EOF > \${CRC_DIR}/pull_secret.json
    {"auths":{"fake":{"auth": "Zm9vOmJhcgo="}}}
    chmod 700 ${HOME}/bin/
  6. Clone the code repos:

    mkdir ${HOME}/crc-build
    cd ${HOME}/crc-build
    git clone
    git clone

Step 1 - Create the Single Node Cluster

  1. Execute the helper script to set the environment:

  2. Ensure that you have the latest code, and switch to the okd-4.8 branch:

    cd ${CRC_DIR}/snc
    git fetch
    git checkout okd-4.8
    git pull
  3. Start the deployment of the single node cluster: (This will take a while)

  4. When the deployment is complete, you will see output like this:

    Cluster has stabilized
    + break
    + grep -v Completed
    + grep -v Running
    + oc get pod --no-headers --all-namespaces
    + retry ./openshift-clients/linux/oc delete pod --field-selector=status.phase==Succeeded --all-namespaces
    + local retries=10
    + local count=0
    + ./openshift-clients/linux/oc delete pod --field-selector=status.phase==Succeeded --all-namespaces
    pod "installer-2-crc-hqh5f-master-0" deleted
    pod "revision-pruner-2-crc-hqh5f-master-0" deleted
    pod "installer-2-crc-hqh5f-master-0" deleted
    pod "installer-3-crc-hqh5f-master-0" deleted
    pod "installer-4-crc-hqh5f-master-0" deleted
    pod "installer-5-crc-hqh5f-master-0" deleted
    pod "installer-6-crc-hqh5f-master-0" deleted
    pod "installer-7-crc-hqh5f-master-0" deleted
    pod "revision-pruner-3-crc-hqh5f-master-0" deleted
    pod "revision-pruner-5-crc-hqh5f-master-0" deleted
    pod "revision-pruner-7-crc-hqh5f-master-0" deleted
    pod "installer-3-crc-hqh5f-master-0" deleted
    pod "installer-4-crc-hqh5f-master-0" deleted
    pod "installer-5-crc-hqh5f-master-0" deleted
    pod "installer-6-crc-hqh5f-master-0" deleted
    pod "installer-7-crc-hqh5f-master-0" deleted
    pod "revision-pruner-4-crc-hqh5f-master-0" deleted
    pod "revision-pruner-5-crc-hqh5f-master-0" deleted
    pod "revision-pruner-6-crc-hqh5f-master-0" deleted
    pod "revision-pruner-7-crc-hqh5f-master-0" deleted
    pod "installer-3-crc-hqh5f-master-0" deleted
    pod "installer-4-crc-hqh5f-master-0" deleted
    pod "installer-5-crc-hqh5f-master-0" deleted
    pod "installer-6-crc-hqh5f-master-0" deleted
    pod "installer-7-crc-hqh5f-master-0" deleted
    pod "revision-pruner-5-crc-hqh5f-master-0" deleted
    pod "revision-pruner-6-crc-hqh5f-master-0" deleted
    pod "revision-pruner-7-crc-hqh5f-master-0" deleted
    + return 0
    + jobs=($(jobs -p))
    ++ jobs -p
    + '[' -n 24340 ']'
    + (( 5 ))
    + kill 24340
    ./ line 1: kill: (24340) - No such process
    + true
  5. Check out the OKD cluster:

    export KUBECONFIG=${CRC_DIR}/snc/crc-tmp-install-data/auth/kubeconfig 
    oc get pods --all-namespaces

    You should see a very clean listing of healthy pods.

    NAMESPACE                                    NAME                                                     READY   STATUS    RESTARTS   AGE
    openshift-apiserver-operator                 openshift-apiserver-operator-75f74d6456-zdjlv            1/1     Running   0          23m
    openshift-apiserver                          apiserver-746bff698b-hwgrc                               2/2     Running   0          27m
    openshift-authentication-operator            authentication-operator-66c5f8676c-9kvc5                 1/1     Running   1          27m
    openshift-authentication                     oauth-openshift-fcbb4554c-4v6w5                          1/1     Running   0          30m
    openshift-authentication                     oauth-openshift-fcbb4554c-5gpp8                          1/1     Running   0          30m
    openshift-cluster-machine-approver           machine-approver-77cc698544-cwlrk                        2/2     Running   0          23m
    openshift-cluster-node-tuning-operator       cluster-node-tuning-operator-598cfb69ff-25wg9            1/1     Running   0          23m
    openshift-cluster-node-tuning-operator       tuned-dqzwb                                              1/1     Running   0          25h
    openshift-cluster-samples-operator           cluster-samples-operator-78875bd7b4-5c8sr                2/2     Running   0          27m
    openshift-cluster-version                    cluster-version-operator-5794fc5fc-x547n                 1/1     Running   1          28m
    openshift-config-operator                    openshift-config-operator-57b84d4d9c-l2cg2               1/1     Running   0          23m
    openshift-console-operator                   console-operator-c74b976f5-f46hd                         1/1     Running   0          23m
    openshift-console                            console-86bb4d4dc8-92lrn                                 1/1     Running   0          24h
    openshift-console                            console-86bb4d4dc8-ztn96                                 1/1     Running   0          24h
    openshift-console                            downloads-6684b5d869-g7klk                               1/1     Running   0          27m
    openshift-controller-manager-operator        openshift-controller-manager-operator-6b65899945-7p5pd   1/1     Running   0          23m
    openshift-controller-manager                 controller-manager-xxtnj                                 1/1     Running   1          28m
    openshift-dns-operator                       dns-operator-5cfc8c7c89-c62vv                            2/2     Running   0          23m
    openshift-dns                                dns-default-w74lp                                        3/3     Running   0          25h
    openshift-etcd-operator                      etcd-operator-68f96d7bf8-cdv4b                           1/1     Running   0          23m
    openshift-etcd                               etcd-crc-hqh5f-master-0                                  3/3     Running   0          25h
    openshift-etcd                               etcd-quorum-guard-68c65dd785-fd6bf                       1/1     Running   0          25h
    openshift-image-registry                     cluster-image-registry-operator-69799f54dd-9vv5j         1/1     Running   0          23m
    openshift-image-registry                     image-registry-578bb6758d-p7k4b                          1/1     Running   0          27m
    openshift-image-registry                     node-ca-rb466                                            1/1     Running   0          24h
    openshift-ingress-canary                     ingress-canary-zq4m9                                     1/1     Running   0          24h
    openshift-ingress-operator                   ingress-operator-5bb8ffff48-lsf8k                        2/2     Running   0          23m
    openshift-ingress                            router-default-6fbf95b45-j5bbj                           1/1     Running   3          24h
    openshift-kube-apiserver-operator            kube-apiserver-operator-85ccb995cf-9bz9w                 1/1     Running   0          23m
    openshift-kube-apiserver                     kube-apiserver-crc-hqh5f-master-0                        5/5     Running   0          25m
    openshift-kube-controller-manager-operator   kube-controller-manager-operator-7794b64ddb-ksqx9        1/1     Running   0          23m
    openshift-kube-controller-manager            kube-controller-manager-crc-hqh5f-master-0               4/4     Running   6          24h
    openshift-kube-scheduler-operator            openshift-kube-scheduler-operator-c759dbbd4-5k9t9        1/1     Running   0          23m
    openshift-kube-scheduler                     openshift-kube-scheduler-crc-hqh5f-master-0              3/3     Running   4          24h
    openshift-marketplace                        community-operators-jnbx9                                1/1     Running   0          25h
    openshift-marketplace                        marketplace-operator-5594b977df-sz7p7                    1/1     Running   0          23m
    openshift-multus                             multus-admission-controller-kmnv5                        2/2     Running   0          25h
    openshift-multus                             multus-skj77                                             1/1     Running   0          25h
    openshift-multus                             network-metrics-daemon-27d8k                             2/2     Running   0          25h
    openshift-network-diagnostics                network-check-source-587567b644-85xv7                    1/1     Running   0          25h
    openshift-network-diagnostics                network-check-target-x54kh                               1/1     Running   0          25h
    openshift-network-operator                   network-operator-5b7884f84f-vwn7h                        1/1     Running   0          23m
    openshift-oauth-apiserver                    apiserver-74fc8d5cb8-9kxjr                               1/1     Running   6          25h
    openshift-operator-lifecycle-manager         catalog-operator-c4885b46c-rwqz8                         1/1     Running   0          23m
    openshift-operator-lifecycle-manager         olm-operator-554765c4cf-rpl7v                            1/1     Running   0          23m
    openshift-operator-lifecycle-manager         packageserver-64c9867f9c-2cgz6                           1/1     Running   6          25h
    openshift-sdn                                sdn-controller-89k44                                     1/1     Running   2          25h
    openshift-sdn                                sdn-q6tl2                                                2/2     Running   0          25h
    openshift-service-ca-operator                service-ca-operator-75c599f496-g7pf7                     1/1     Running   1          27m
    openshift-service-ca                         service-ca-54886b9fbc-cxb7c                              1/1     Running   2          25h

    If you see pods crash looping or otherwise not Running, then there may be issues with the cluster which will inhibit your crc build.

  6. Create the qcow2 Disk Image

    ./ crc-tmp-install-data
  7. Compile the CRC Executable with an Embedded Disk Image

    cd ../crc
    git fetch
    git pull
    git checkout crc-okd
    make clean
    make embed_bundle

    The crc executables will be in the ${CRC_DIR}/crc/out folder, arranged by architecture.


Post build clean up

  1. Delete the virtual machine resources that were created:

    CRC=$(virsh net-list --all | grep crc- | cut -d" " -f2)
    virsh destroy ${CRC}-bootstrap
    virsh undefine ${CRC}-bootstrap
    virsh destroy ${CRC}-master-0
    virsh undefine ${CRC}-master-0
    virsh net-destroy ${CRC}
    virsh net-undefine ${CRC}
    virsh pool-destroy ${CRC}
    virsh pool-undefine ${CRC}
    rm -rf /var/lib/libvirt/openshift-images/${CRC}
  2. Clean up the artifacts from the single node cluster build:

    rm -rf ${CRC_DIR}/snc/crc_*_${OKD_VERSION}*

That’s it!

You now have your own, personal copy of CodeReady Containers.

To get up and running quickly:

  1. Copy the appropriate binary to your workstation.
  2. Put the crc binary in your path.
  3. execute:

    crc setup
  4. execute:

    crc start
  5. Develop and Deploy code!
  6. When you are done:

    crc stop
  7. To delete a cluster instance:

    crc delete

The documentation for how to use it is here:

Note, that documentation is for the official Red Hat release of CodeReady Containers. Some of the documentation is not relevant to what you just built.