Skip to content

pivirtd — QEMU/KVM Hypervisor Daemon

pivirtd is the core hypervisor daemon that directly manages QEMU/KVM virtual machines. It exposes a gRPC API for VM lifecycle operations and runs as a systemd service on each hypervisor node.

Property Value
Language Go 1.22+
Protocol gRPC (internal), ConnectRPC (via agent)
Runtime systemd service, runs as root
Dependencies QEMU/KVM, Cloud Hypervisor (optional)
Config /etc/pivirtd/config.yaml
Log journald / /var/log/pivirtd/
flowchart TB
    subgraph Pivirtd["pivirtd Architecture"]
        subgraph Top["Management Layer"]
            API["gRPC API<br/>(Port 9001)"]
            VMM["VM Manager"]
            RES["Resource Tracker<br/>(CPU, Mem, Disk)"]
        end

        subgraph HAL["Hypervisor Abstraction Layer"]
            QEMU["QEMU/KVM<br/>(go-qemu / QMP)"]
            CH["Cloud Hypervisor<br/>(VMM)"]
            FUT[(Future)]
        end

        API --> HAL
        VMM --> HAL
        RES --> HAL
    end
  • Create/Delete: Full VM provisioning from YAML specs
  • Start/Stop/Reboot: Graceful and forceful power control
  • Pause/Resume: Live pause for maintenance
  • Snapshot: Disk + memory snapshots with branching
  • Clone: Linked clones for rapid provisioning
  • Live Migration: Shared-nothing and shared-storage migration
  • Resize Disk: Grow a virtual disk on a running or stopped VM via QMP block_resize
  • Set VNC Password: Rotate the VNC display password at runtime with optional expiry
  • Set Link State: Bring a NIC link up or down without restarting the VM
  • Guest Info: OS, kernel, hostname, and network interface details via the QEMU guest agent
  • Quiesced Snapshots: CreateSnapshot freezes the guest filesystem (fsfreeze) before capture, with a crash-consistent fallback if the agent is unreachable
  • Snapshot Tree: Snapshot chains exposed as a nested parent/child tree
  • CPU: vCPU pinning, NUMA topology, CPU quotas
  • Memory: Hugepages, ballooning, memory overcommit
  • Disk: qcow2, raw, LVM, iSCSI, NFS, RBD backends
  • Network: Bridge, VLAN, VXLAN, SR-IOV, OVS-DPDK

pivirtd manages both local and clustered storage through dedicated storage pools.

The file tree below illustrates how pivirtd structures runtime files, base images, guest disks, and control sockets on the host filesystem:

  • Directoryvar/
    • Directorylib/
      • Directorypivirt/
        • Directorydaemon/
          • pivirtd.sock Unix domain control socket (used by pvcli & local agents)
          • pivirtd.pid Process locking file
        • Directoryimages/ Storage pool: base template images (read-only backing files)
          • debian-12-genericcloud.qcow2
          • ubuntu-24.04-server.qcow2
          • almalinux-9-cloud.qcow2
        • Directoryvms/ Instance working directories (keyed by VM UUID)
          • Directory3e7c81d2-9b54-4a81-bc01-e28a9f4c3b21/
            • vm.json Live VM runtime specification
            • disk-0.qcow2 VM copy-on-write overlay disk (backing: images/debian-12…)
            • nvram.fd Persistent UEFI NVRAM variables template
            • qmp.sock QEMU Monitor Protocol control socket
            • qga.sock QEMU Guest Agent communication channel
            • console.log Serial console output log
        • Directorypools/ Mounted external storage pools
          • Directorynfs-shared/ NFS shared volume mount
            • …
          • Directoryocfs2-cluster/ OCFS2 clustered shared filesystem mount
            • …
    • Directorycache/
      • Directorypivirtd/
        • Directoryimages/ Cached remote images & downloaded artifacts
          • sha256_9c2d1b8… Verified image cache
Backend Type Use Case Performance
dir File Development, single-node ★★☆☆☆
lvm Block Production, thin provisioning ★★★★☆
iscsi Block Clustered, shared storage ★★★★★
nfs File Shared, backup targets ★★★☆☆
rbd Block Ceph clusters ★★★★★

# pivirtd configuration
server:
grpc_addr: "0.0.0.0:9001"
http_addr: "0.0.0.0:9002" # Metrics, health
tls:
cert_file: "/etc/pivirtd/tls/server.crt"
key_file: "/etc/pivirtd/tls/server.key"
ca_file: "/etc/pivirtd/tls/ca.crt"
hypervisor:
type: "qemu" # qemu | cloud-hypervisor
default_machine_type: "pc-q35-8.0"
cpu_model: "host-passthrough"
listen_tcp: "0.0.0.0:8443"
tls_cert_file: "/etc/pivirtd/tls/server.crt"
tls_key_file: "/etc/pivirtd/tls/server.key"
storage:
default_pool: "default"
pools:
- name: "default"
type: "dir"
path: "/var/lib/pivirt/images"
- name: "iscsi-pool"
type: "iscsi"
target_iqn: "iqn.2024-01.hu.pilab:pivirtd"
portal: "10.0.1.10:3260"
network:
default_bridge: "virbr0"
bridges:
- name: "virbr0"
type: "bridge"
interface: "eth0"
- name: "vxlan0"
type: "vxlan"
vni: 1000
remote: "10.0.1.20"
logging:
level: "info"
format: "json"
output: "journald"
metrics:
enabled: true
path: "/metrics"

service VMService {
rpc CreateVM(CreateVMRequest) returns (VM);
rpc GetVM(GetVMRequest) returns (VM);
rpc ListVMs(ListVMsRequest) returns (stream VM);
rpc UpdateVM(UpdateVMRequest) returns (VM);
rpc DeleteVM(DeleteVMRequest) returns (Empty);
rpc StartVM(StartVMRequest) returns (Empty);
rpc StopVM(StopVMRequest) returns (Empty);
rpc RebootVM(RebootVMRequest) returns (Empty);
rpc ShutdownVM(ShutdownVMRequest) returns (Empty);
rpc CreateSnapshot(CreateSnapshotRequest) returns (Snapshot);
rpc ListSnapshots(ListSnapshotsRequest) returns (stream Snapshot);
rpc RestoreSnapshot(RestoreSnapshotRequest) returns (Empty);
rpc DeleteSnapshot(DeleteSnapshotRequest) returns (Empty);
rpc GetConsole(GetConsoleRequest) returns (stream ConsoleOutput);
rpc GetVNC(GetVNCRequest) returns (VNCInfo);
rpc MigrateVM(MigrateVMRequest) returns (MigrationStatus);
}
Terminal window
# Using grpcurl
grpcurl -plaintext -d '{
"spec": {
"name": "web-01",
"cpu": { "cores": 2, "model": "host-passthrough" },
"memory": { "size_mib": 4096 },
"disks": [{ "size_gib": 20, "pool": "default", "bus": "virtio" }],
"nics": [{ "bridge": "virbr0", "model": "virtio" }],
"os": { "type": "linux", "boot_order": ["disk", "network"] }
}
}' localhost:9001 pivirt.vm.v1.VMService/CreateVM

nics:
- bridge: virbr0
model: virtio
vlan: 100 # Optional VLAN tag
nics:
- network: vxlan0
model: virtio
nics:
- pf: eth2
vf: 0
model: vfio-pci

pivirtd is continuously built with GoReleaser and published as native .deb and .rpm packages to the official PiLab Package Repository at https://dl.pilab.hu. The package bundles:

  • pivirtd: The core hypervisor daemon (/usr/bin/pivirtd)
  • pivirtctl: The local host management CLI (/usr/bin/pivirtctl)
  • OVMF Assets: Pre-configured UEFI firmware variables (/usr/share/pivirtd/assets/OVMF_VARS.fd)
  • Systemd Unit: Hardened service unit (/lib/systemd/system/pivirtd.service)
Terminal window
# 1. Add PiLab GPG signing key
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://dl.pilab.hu/pivirt.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/pilab.gpg
# 2. Add PiLab APT repository
echo "deb [signed-by=/etc/apt/keyrings/pilab.gpg] https://dl.pilab.hu/deb stable main" | sudo tee /etc/apt/sources.list.d/pilab.list
# 3. Update & Install pivirtd
sudo apt-get update
sudo apt-get install -y pivirtd

pivirtd is managed via systemd:

Terminal window
# Start the daemon
sudo systemctl start pivirtd
# Stop the daemon
sudo systemctl stop pivirtd
# Restart the daemon
sudo systemctl restart pivirtd
# Check status
sudo systemctl status pivirtd
# Enable on boot
sudo systemctl enable pivirtd
# View logs
sudo journalctl -u pivirtd -f

Terminal window
# gRPC Health Check
grpcurl -plaintext localhost:9001 grpc.health.v1.Health/Check
# HTTP Health Check
curl http://localhost:9002/health
curl http://localhost:9002/ready
curl http://localhost:9002/live
# Prometheus Metrics
curl http://localhost:9002/metrics

Storage Guide

Detailed configuration of directory, LVM, and iSCSI pools.
View Storage Guide