The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development.

overview components

This project can also be used as a template for writing your own metal-stack deployments.

<!– TOC depthFrom:2 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 –>

<!– /TOC –>


  • Linux machine with hardware virtualization support

  • Vagrant == 2.2.9 with vagrant-libvirt plugin >= 0.1.2 (for running the switch and machine VMs)

  • kvm as hypervisor for the VMs

    • Ubuntu 20.04:

      sudo apt install -y qemu qemu-kvm libvirt-daemon bridge-utils virtinst libvirt-dev
  • docker >= 18.09 (for using kind and our deployment base image)

  • docker-compose >= 1.25.4 (for ease of use and for parallelizing control plane and partition deployment)

  • kind == v0.8.1 (for hosting the metal control plane on a kubernetes cluster v1.18.2)

  • ovmf to have a uefi firmware for virtual machines

  • the lab creates a virtual network on your host machine, this hopefully does not overlap with other networks you have

  • (recommended) haveged to have enough random entropy (only needed if the PXE process does not work)

Here is some code that should help you setting up most of the requirements:

# Install Docker
curl -fsSL | sh
# if you want to be on the safe side, follow the original installation
# instructions at

# Install vagrant and other stuff
sudo apt-get install ./vagrant_2.2.9_x86_64.deb qemu-kvm virt-manager ovmf net-tools libvirt-dev haveged

# Ensure that your user is member of the group "libvirt"
# possibly you need to login again in order to make this change take effect
sudo usermod -G libvirt -a ${USER}

# Install libvirt plugin for vagrant
vagrant plugin install vagrant-libvirt

# Install kind from

The following ports are getting used statically:

PortBind AddressDescription
64430.0.0.0kube-apiserver of the kind cluster
44430.0.0.0HTTPS ingress
80800.0.0.0HTTP ingress

Known Limitations

  • to keep the demo small there is no EVPN
  • machine restart and destroy does not work because we cannot change the boot order via IPMI in the lab easily (virtual-bmc could, but it's buggy)
  • login to the machines is possible with virsh console, login to the firewall is possible with SSH from your local machine

Try it out

Start the mini-lab with a kind cluster, a metal-api instance as well as some vagrant VMs with two leaf switches and two machine skeletons.


Two machines in status PXE booting are visible with metalctl machine ls

docker-compose run metalctl machine ls

ID                                          LAST EVENT   WHEN     AGE  HOSTNAME  PROJECT  SIZE          IMAGE  PARTITION
e0ab02d2-27cd-5a5e-8efc-080ba80cf258        PXE Booting  3s
2294c949-88f6-5390-8154-fa53d93a3313        PXE Booting  5s

Wait until the machines reach the waiting state

docker-compose run metalctl machine ls

ID                                          LAST EVENT   WHEN     AGE  HOSTNAME  PROJECT  SIZE          IMAGE  PARTITION
e0ab02d2-27cd-5a5e-8efc-080ba80cf258        Waiting      8s                               v1-small-x86         vagrant
2294c949-88f6-5390-8154-fa53d93a3313        Waiting      8s                               v1-small-x86         vagrant

Create a machine/firewall with

make machine
make firewall

Alternatively you may want to issue the metalctl commands by your own:

docker-compose run metalctl network allocate \
        --partition vagrant \
        --project 00000000-0000-0000-0000-000000000000 \
        --name vagrant

# Lookup the network ID and create a machine
docker-compose run metalctl machine create \
        --description test \
        --name machine \
        --hostname machine \
        --project 00000000-0000-0000-0000-000000000000 \
        --partition vagrant \
        --image ubuntu-20.04 \
        --size v1-small-x86 \
        --networks <network-ID>

# Create a firewall that is also connected to the virtual internet-vagrant-lab network
docker-compose run metalctl machine create \
        --description fw \
        --name fw \
        --hostname fw \
        --project 00000000-0000-0000-0000-000000000000 \
        --partition vagrant \
        --image firewall-ubuntu-2.0 \
        --size v1-small-x86 \
        --networks internet-vagrant-lab,$(privatenet)

See the installation process in action

make console-machine01/02
Ubuntu 20.04 machine ttyS0

machine login:

Two machines are now installed and have status "Phoned Home"

docker-compose run metalctl machine ls
ID                                          LAST EVENT   WHEN   AGE     HOSTNAME  PROJECT                               SIZE          IMAGE                             PARTITION
e0ab02d2-27cd-5a5e-8efc-080ba80cf258        Phoned Home  2s     21s     machine   00000000-0000-0000-0000-000000000000  v1-small-x86  Ubuntu 20.04 20200331             vagrant
2294c949-88f6-5390-8154-fa53d93a3313        Phoned Home  8s     18s     fw        00000000-0000-0000-0000-000000000000  v1-small-x86  Firewall 2 Ubuntu 20200730        vagrant

Login with user name metal and the console password from

docker-compose run metalctl machine describe e0ab02d2-27cd-5a5e-8efc-080ba80cf258 | grep password

consolepassword: ...

If you want to access the firewall with SSH or have internet connectivity from the firewall and machine, you'll need to have a static route configured that points to the vagrant boxes of the leaf switches:

make route # shows you the route needed to access the network internet-vagrant-lab
add this route to communicate with the virtual internet network over leaf01 and leaf02
sudo ip r a nexthop via dev virbr0 nexthop via dev virbr0

# Connect to the firewall
ssh metal@

To remove the kind cluster and the vagrant boxes, run

make cleanup

Reinstall machine

Reinstall a machine with

docker-compose run metalctl machine reinstall \
        --image ubuntu-20.04 \

Remove machine

Remove a machine with

docker-compose run metalctl machine rm e0ab02d2-27cd-5a5e-8efc-080ba80cf258

Development of metal-api, metal-hammer and metal-core

To simplify developing changes for the metal-api, metal-hammer and metal-core, it is possible to use development artifacts from within the mini-lab. See the dev instructions for more details.

Page Tree