osinfodb — OS Image Database

osinfodb is a libosinfo parser and OS image management tool that provides automated OS image discovery, download, and caching for PiVirt.

Technical Specifications

Property	Value
Language	Go 1.22+
Protocol	HTTP/REST
Database	libosinfo XML
Port	8443 (HTTPS)
Config	/etc/osinfodb/config.yaml

Architecture

flowchart TB
    subgraph Osinfodb["osinfodb Architecture"]
        subgraph Server["HTTP Server (Port 8443)"]
            IMG_API["Image API"]
            OS_API["OS Info API"]
            DL_MGR["Download Manager"]
        end

        subgraph Core["Core Engine"]
            LIB["libosinfo Parser"]
            CACHE_MGR["Image Cache Manager"]
            VER["Signature Verifier<br/>(GPG/SHA256)"]
        end

        Server --> Core
    end

    LIB --> LIB_DB[("libosinfo DB<br/>(XML files)")]
    CACHE_MGR --> CACHE_DIR[("Image Cache<br/>(Local FS)")]
    DL_MGR --> SOURCES[("Image Sources<br/>(HTTP/HTTPS)")]

Key Features

OS Image Discovery

• libosinfo Integration: Parses libosinfo XML database
• Multi-Distro Support: Ubuntu, Debian, CentOS, Fedora, RHEL, etc.
• Version Detection: Automatic OS version identification
• Architecture Support: amd64, arm64, etc.

Image Management

• Download: Automated image download from official sources
• Caching: Local image cache with TTL
• Verification: GPG signature and SHA256 checksum verification
• Metadata: Rich OS metadata (cloud-init support, default user, etc.)

Image Sources

• Official Mirrors: Ubuntu Cloud Images, Debian Cloud, CentOS Cloud
• Cloud Providers: AWS AMI, GCP Images, Azure Images
• Custom Sources: Private image registries

Configuration

Main Config (/etc/osinfodb/config.yaml)

# osinfodb configuration
server:
  http_addr: "0.0.0.0:8443"
  tls:
    cert_file: "/etc/osinfodb/tls/server.crt"
    key_file: "/etc/osinfodb/tls/server.key"
    ca_file: "/etc/osinfodb/tls/ca.crt"

# libosinfo database
libosinfo:
  db_path: "/usr/share/libosinfo/db"
  update_interval: "24h"

# Image cache
cache:
  enabled: true
  path: "/var/cache/osinfodb/images"
  max_size_gb: 100
  ttl_hours: 168  # 7 days

# Image sources
sources:
  # Official distribution mirrors
  - name: "ubuntu"
    type: "http"
    url: "https://cloud-images.ubuntu.com"
    enabled: true

  - name: "debian"
    type: "http"
    url: "https://cloud.debian.org/images/cloud"
    enabled: true

  - name: "centos"
    type: "http"
    url: "https://cloud.centos.org/centos"
    enabled: true

  - name: "fedora"
    type: "http"
    url: "https://download.fedoraproject.org/pub/fedora/linux/releases"
    enabled: true

  - name: "rhel"
    type: "http"
    url: "https://access.redhat.com/downloads/content"
    enabled: true
    auth:
      username_ref: "vault:secret/data/redhat#username"
      password_ref: "vault:secret/data/redhat#password"

  # Cloud provider images
  - name: "aws"
    type: "aws"
    regions:
      - "eu-central-1"
      - "us-east-1"
    enabled: true
    credentials_ref: "vault:secret/data/aws#credentials"

  - name: "gcp"
    type: "gcp"
    project: "centos-cloud"
    enabled: true
    credentials_ref: "vault:secret/data/gcp#credentials"

# Image verification
verification:
  enabled: true
  # GPG signature verification
  gpg:
    enabled: true
    keyring: "/etc/osinfodb/gpg/keyring.gpg"
  # SHA256 checksum verification
  checksum:
    enabled: true

# Logging
logging:
  level: "info"
  format: "json"
  output: "journald"

# Metrics
metrics:
  enabled: true
  path: "/metrics"

API Reference

Image API

GET /api/v1/images
  List all available images
  Query params: os, version, arch, format

GET /api/v1/images/{name}
  Get image details
  Returns: image metadata, download URL, checksum

POST /api/v1/images/{name}/download
  Download image to cache
  Returns: download status, progress

GET /api/v1/images/{name}/info
  Get detailed image information
  Returns: OS metadata, cloud-init support, etc.

DELETE /api/v1/images/{name}
  Remove image from cache

OS Info API

GET /api/v1/os
  List all supported operating systems

GET /api/v1/os/{id}
  Get OS details
  Returns: OS metadata, versions, architectures

GET /api/v1/os/{id}/versions
  List available versions for OS

GET /api/v1/os/{id}/versions/{version}
  Get version-specific information

Example Requests

# List all Ubuntu images
curl -k https://localhost:8443/api/v1/images?os=ubuntu

# Get specific image details
curl -k https://localhost:8443/api/v1/images/ubuntu-24.04

# Download image
curl -k -X POST https://localhost:8443/api/v1/images/ubuntu-24.04/download

# Get image info
curl -k https://localhost:8443/api/v1/images/ubuntu-24.04/info

Image Metadata

Image Information

name: "ubuntu-24.04"
os:
  id: "ubuntu"
  name: "Ubuntu"
  version: "24.04"
  codename: "Noble Numbat"
  family: "linux"
  distro: "ubuntu"
architecture: "amd64"
format: "qcow2"
size_bytes: 2469606144
checksum:
  algorithm: "sha256"
  value: "abc123..."
source:
  url: "https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img"
  signature_url: "https://cloud-images.ubuntu.com/noble/current/SHA256SUMS.gpg"
cloud_init:
  supported: true
  datasource: "NoCloud"
  default_user: "ubuntu"

Running osinfodb

Systemd Service

/etc/systemd/system/osinfodb.service

[Unit]
Description=PiVirt OS Image Database
After=network.target

[Service]
Type=notify
ExecStart=/usr/local/bin/osinfodb --config /etc/osinfodb/config.yaml
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Prerequisites

# Install libosinfo
# Ubuntu/Debian
apt-get install -y libosinfo-bin libosinfo-l10n

# RHEL/Fedora
dnf install -y libosinfo

# Update libosinfo database
osinfo-query os

# Create cache directory
mkdir -p /var/cache/osinfodb/images
chown osinfodb:osinfodb /var/cache/osinfodb/images

Troubleshooting

Image Download Failed

# Check network connectivity
curl -I https://cloud-images.ubuntu.com

# Check source configuration
cat /etc/osinfodb/config.yaml | grep -A10 sources

# Check logs
journalctl -u osinfodb -f | grep -i download

Verification Failed

# Check GPG keys
gpg --list-keys --keyring /etc/osinfodb/gpg/keyring.gpg

# Import missing keys
gpg --keyring /etc/osinfodb/gpg/keyring.gpg --recv-keys <KEY_ID>

# Check checksum
sha256sum /var/cache/osinfodb/images/ubuntu-24.04.qcow2

Cache Issues

# Check cache size
du -sh /var/cache/osinfodb/images

# Clean up cache
osinfodb cache clean

# Rebuild cache
osinfodb cache rebuild

Related Documentation

Custom Images Guide

Building and importing custom OS templates and golden images.
View Custom Images Guide

VM Lifecycle Guide

Provisioning virtual machines with cached OS images.
View VM Guide

CLI Reference

Image pull and search CLI commands.
View CLI Reference