Apstra appliances

Juniper is not distributing appliances per se for KVM deployments, but OS disk images in QCOW2 format:

aos_server_4.2.0-236.qcow2

Main Apstra server appliance, also used for optional workers for the scaleout setup: you use the same image for the worker nodes (didn’t find this mentioned in the documentation but was mentioned by a Juniper guy)

apstra-ztp-4.2.0-34.qcow2

Zero Touch provisioning appliance. This is an optional ZTP appliance running PXE+TFTP. It can be used as pure TFTP with a pre-existing DHCP server.

Installation

Overview

For the VM sizing, the vendor has a recommendated sizing table:

Resource Value
vCPU 8
RAM 64GB + 500MB per installed offbox agent
Disk 160GB

In my experience, that many resources are not needed, especially the RAM. For my test environment, I’ll slash RAM to 32GB and most probably that’s not supported, I can live with that, YMMV.

We’ll create the empty virtual machines in Proxmox VE 7.4-15, importing afterwards the QCOW2 disks as needed. The procedure asumes you have downloaded all the files to a working directory (curl/wget are your friends) in a PVE node.

Apstra OS

VM creation

We’ll create a virtual machine with CLI, with 4 vCPUs and 32GB of RAM.

The network is being served via the default bridge and storage via local ZFS repository. Options are self explanatory, adjust as needed.

VMNAME=apstra-os-03.ipa.<my TLD>
VMID=$(pvesh get /cluster/nextid)
STORAGE=local-zfs
VCPU=4
VRAM=32768
VLANLAB=<my vlan>
BRIDGE=vmbr0

# OS doesn't support UEFI
qm create ${VMID} --name ${VMNAME} \
--bios seabios --machine q35 \
--ostype l26 --agent 1 \
--sockets 1 --cores ${VCPU} --cpu cputype=kvm64 \
--scsihw virtio-scsi-single \
--memory ${VRAM} \
--net0 model=virtio,bridge=${BRIDGE},firewall=0,tag=${VLANLAB}

Importing disk

Now, we’ll be importing the QCOW2 file provided by Juniper.

VMNAME=apstra-os-03.ipa.<my TLD>
VMID=$(qm list|grep ${VMNAME}|awk '{ print $1 }')
STORAGE=local-zfs

qm disk import ${VMID} aos_server_4.2.0-236.qcow2  ${STORAGE}
qm set ${VMID} -scsi0 ${STORAGE}:vm-${VMID}-disk-1
qm set ${VMID} --boot order=scsi0

Additional disk

According to the documentation, the OS disk is 80GB only and optionally you could add a second 80GB. Given it’s not onerous and I won’t be using additional worker nodes, I’ll add the second disk.

VMNAME=apstra-os-03.ipa.<my TLD>
VMID=$(qm list|grep ${VMNAME}|awk '{ print $1 }')
STORAGE=local-zfs

qm set ${VMID} --scsi1 ${STORAGE}:80
# we start the VM
qm start ${VMID}

ZTP appliance

DHCP & TFTP are not that demanding, you can go with 2 vCPUs and 4GB of RAM.

The network is being served via the default bridge and storage via local ZFS repository. Options are self explanatory, adjust as needed.

VMNAME=apstra-ztp-03.ipa.<my TLD>
VMID=$(pvesh get /cluster/nextid)
STORAGE=local-zfs
VCPU=2
VRAM=4096
VLANLAB=<my vlan>
BRIDGE=vmbr0

# OS doesn't support UEFI
qm create ${VMID} --name ${VMNAME} \
--bios seabios --machine q35 \
--ostype l26 --agent 1 \
--sockets 1 --cores ${VCPU} --cpu cputype=kvm64 \
--scsihw virtio-scsi-single \
--memory ${VRAM} \
--net0 model=virtio,bridge=${BRIDGE},firewall=0,tag=${VLANLAB}

Importing disk

Now, we’ll be importing the QCOW2 file provided by Juniper.

VMNAME=apstra-ztp-03.ipa.<my TLD>
VMID=$(qm list|grep ${VMNAME}|awk '{ print $1 }')
STORAGE=local-zfs

qm disk import ${VMID} apstra-ztp-4.2.0-34.qcow2  ${STORAGE}
qm set ${VMID} -scsi0 ${STORAGE}:vm-${VMID}-disk-1
qm set ${VMID} --boot order=scsi0
qm start ${VMID}

Initial Configuration

AOS setup

  1. Search for the IP of the appliance in your DHCP leases and connect through SSH with admin/admin.
  2. On first login set new password as requested.
  3. On the forced assistant, also set the Apstra UI password as proposed. Cancel at the main menu after setting up the password.
  4. Change hostname to something that makes sense for your environment. With admin user run:
sudo aos_hostname apstra-os-03.ipa.<my TLD>
  1. Adjust timezone to match your environment.
sudo timedatectl set-timezone <my TZ>
  1. As admin, update the operating system. Juniper most probably will tell you that this is not supported. The only issue I had in the past was 4.1.0 breaking because there was a system-wide Juniper library that broke a containerized application (go figure) and required manual dependancies update to fix it. It has been flawless for 4.1.2 and now 4.2.0, pam.d files will be mentioned, keep the locally modified files.
sudo apt update
sudo apt upgrade -y
  1. Install QEMU guest agent
sudo apt install -y qemu-guest-agent
  1. Reboot after full system update sudo shutdown -r now

ZTP appliance setup

  1. Search for the IP of the appliance in your DHCP leases and connect through SSH with admin/admin.
  2. Set hostname
sudo hostnamectl set-hostname apstra-ztp-03.ipa.<my TLD>
sudo sh -c 'echo "127.0.0.1 apstra-ztp-03.ipa.<my TLD>" >> /etc/hosts'
  1. Adjust timezone to match your environment.
sudo timedatectl set-timezone <my TZ>
  1. Clean messed-up config structure: For some reason, the Juniper guys thought it was a good idea to create a directory where a file should be and that breaks Ubuntu upgrade process. We clean that up (what gives Juniper?!)
sudo rmdir /etc/udev/rules.d/70-persistent-net.rules
  1. Update operating system. Again, probably Juniper will tell you it’s not supported.
sudo apt update
sudo apt upgrade -y
  1. At the Apstra Web UI, create a user with device_ztp role. We’ll use it in the next step.
  2. Back to the ZTP appliance, configure the application to report to the AOS server. For some reason, it doesn’t work with FQDN but only with IP.
cat > /containers_data/status/app/aos.conf <<EOF
{
   "ip": "<your-aos-ip>",
   "user": "<your ztp user>",
   "password": "<the super secure password>"
}
EOF
  1. Install QEMU guest agent
sudo apt install -y qemu-guest-agent
  1. Reboot virtual machine
sudo shutdown -r now

Migration from 4.1.2

If you already have a working environment, most probably you would like to migrate all the configuration to your new installation, there is a script you should run in the new Apstra Server.

Take into account that the script will fail if there are any uncommited changes or if devices are not reachable. Make sure the running state of the source environment is clean (no errors).

aos_import_state --ip-address apstra-os-02.ipa.<my TLD> --username admin

Note:

  • The configuration migration would also replace the web UI password with the configuration from the original server.