Measured Boot and Remote Attestation in Confidential Containers

Case Study: Measured Boot and Remote Attestation in Confidential Containers Magnus Kulke, swe @Azure Core Linux FOSDEM 25, Image-Based Linux and Boot Integrity Devroom

Containers? Pods? Boot integrity? Confidential Containers (CoCo) is an effort to introduce Confidential Computing (CC) into the container ecosystem. CC today is mostly built on top of Virtualization boundaries K8S Pods: collocated processes in a sandbox, with shared namespaces & resources CoCo wraps Pods into VMs

Typical container launch K8s node K8s RPC K8s API Server Kubelet CRI Calls K8s pod spec Containerd OCI Runtime Calls Sandbox process Runc process

Confidential container launch K8s node K8s RPC K8s API Server Kubelet CRI Calls K8s pod spec OCI Runtime Calls Containerd Kata shim Confidential VM Attestation Service Kata RPC Sandbox Kata Agent process Key Server Guest Comp.

Confidential VM Components

Confidential VM & Measured Boot Minimal Linux system for utility VM Hosting static components to manage container sandbox and facilitate attestation Kata-Agent (container runtime) CoCo Guest Components CC mandates integrity Trusting the Sandbox implies trust in the Guest OS All OS (+ FW) components need to be measured

Options Package FW + Kernel + Kata-Agent as pid1 in initrd (w/o rootfs) Charming b/c simple. Measurements can be precalculated easily Problem: Attestation ceremonies are handled in discrete processes (3) Communicating via ttRPC Processes need to be orchestrated, supervised Kata-Agent s role is diluted with pid1 tasks

Use Systemd to manage components Enables some use cases Apply per-vm configuration prior to kata-agent Simplify agent code Leverage Systemd s measured boot facilities Just got merged

systemd/mkosi Make OS Image (?) Declarative Rootless (runs in docker) Supports many distributions (fedora, debian, arch variants) For multiple architectures

mkosi is fast! Shortens debug cycles

dm-verity setup is trivial PodVM rootfs is immutable

Detour: CVMs and vTPMs Rich(er) userland/kernel support for TPMs Paravisors / SVSMs Application in early boot Use privilege primitives to provide vTPM in CVM Isolated from Host + Guest OS, linked to HW Root-of-Trust Host VM VMPL 0 vTPM VMPL > 0 Guest OS

Using vTPM for Measured Boot Linux TPM PCR Registry | UAPI Group Specifications (pretty crowsed Static OS in PCR11 (UKI) Kernel, initrd, cmdline (verity root-hash) Dynamic CoCo configuration in PCR8 Claimed by grub (which we don t use) Attach vTPM quote to TEE evidence )

Calculate PCRs offline or via sw-tpm

Optimize launch latency VMs should start doing container work ASAP systemd-analyze {plot,criticial-chain} are incredibly helpful

WIP: Encrypted workspace Some container image are large (CUDA) HD Storage is untrusted per-se Tmpfs-backed image + runtime storage is costly Option: use encrypted scratch space Add auto-growing partition Encrypt with random key Add repart + crypttab cfg

Future ideas Leverage Sysext for more opionionated Base OS requirements: Plugin in CoCo sysext w/ kata + attestation tools Have predictable (composite) measurement of base + CoCo bins

Future ideas II Support non-TPM TEE RoT HW in systemd tooling Like libtss for SEV-SNP, TDX, Some UAPI standardization ongoing (configfs) Up now: Confidential GPUs Up next: more attestable devices TEE Device Interface (TDI)

Summing up Using tools from systemd s ecosystem CoCo can quickly build and iterate on an immutuble utility VM. for hosting containers We can leverage Measured Boot facilities where feasible Good template for similar requirements

thx!

References Kata Containers - Open Source Container Runtime Software | Kata Containers Confidential Containers systemd/mkosi: OpenHCL: the new, open source paravisor COCONUT-SVSM Intel WP: Device Attestation Model in Confidential Computing Build Bespoke OS Images