◎01 / IDENTITY
Identity and machine
Distro, family, architecture, virtualisation, container; the hardware (CPU, cores, memory, GPUs, disks) and the toolchain (gcc, clang, rustc, go, python…).
Works on my machine.
Proven on theirs.
Test your binary on real distro kernels.
See why it fails elsewhere.
WHY IT BREAKS ELSEWHERE
Same binary.
Different machine.
A program that works on your machine can break elsewhere: musl instead of glibc, a RHEL 4.18 kernel, no BTF, perf_event_paranoid at 3, lockdown on…
- Probed, never inferred.
- Features are tested directly — BPF with real
bpf()calls — never deduced from a version number, because RHEL kernels backport a lot. - Throwaway by default.
- Images are downloaded once, checked against the distro’s published checksum and cached. Each run works on a disposable qcow2 overlay, deleted at the end — even after a timeout or Ctrl+C.
- Reports that fit CI.
- A table of targets × results in the terminal, and on request
--json,--junitfor CI and--markdownfor a pull-request comment.
ENVPROBE / THE PROBE UNDERNEATH
Everything that can
change a program.
envprobe lists what can change how a userspace or kernel-facing program behaves. It works on its own, read-only, and reports missing or refused values with their reasons.
⌘02 / KERNEL
Kernel
Release, command line, config (CONFIG_BPF*, KPROBES, DEBUG_INFO_BTF, FUNCTION_TRACER and dynamic ftrace…), modules.
⛉03 / SECURITY
Security & privileges
sysctls, LSM, SELinux, AppArmor, lockdown, Secure Boot, capabilities, seccomp, user namespaces and soft/hard RLIMIT_MEMLOCK.
◇04 / BPF · PERF
BPF & perf
Program types, map types and helpers probed with real bpf() calls; software and hardware perf_event_open, PMU; tracefs, kallsyms, BTF.
▤05 / USERSPACE
Userspace
libc and its version, common libraries, FHS (merged-usr, multiarch), /bin/sh, coreutils, init, package manager, cgroups, XDG.
✓06 / STATUS
A status per entry
ok with the value, unavailable with the reason, denied when access was refused — and privileged marks what changes with the process’s rights.
{ }07 / SCHEMA
A versioned report
JSON with a schema_version, and its schema published with the code.
envprobe schema
⧉08 / NO VM
Another distro, without a VM
Compare another distro’s userspace from its extracted rootfs, in user, mount and pid namespaces, without root. The kernel stays the host’s.
scripts/probe-rootfs.sh <rootfs> <envprobe> --json
▣09 / STATIC
Static binaries
Built against musl: envprobe runs as is in any x86_64 VM or container.
FROM A TOML FILE TO A VERDICT
One matrix.
Every target explained.
Describe the targets once; distroproof up boots them in parallel and tells you what differs where it fails.
IN THE TERMINAL
A failure,
with its cause.
Not “it broke on Ubuntu 18.04” — which difference, out of all of them, is the likely reason.
● ● ●
distroproof upLINUX[ubuntu-18.04] failed (exit code 1) compared with ubuntu-22.04: 65 significant differences likely cause: btf.vmlinux: present, 5286152 bytes → unavailable (/sys/kernel/btf/vmlinux missing) also: kernel.CONFIG_DEBUG_INFO_BTF: y → missing from config
A failing target compared to the closest passing one. Example translated into English.
Examples adapted from the distroproof reference, with the verdict translated into English. file_caps grants capabilities to the installed binary after setup; collect brings result files back per target. Setup and command receive DP_TARGET and DP_IMAGE. The TUI adds live logs, report diffs and a console.
TARGETS / THE KERNELS IT BOOTS
Twenty images.
Their own kernels.
Official cloud images on their own kernels. Versions below were observed in the October 2026 case study; rolling and daily images change.
| Image | Kernel observed | Note |
|---|---|---|
debian-10 · 11 · 12 · 13 | 4.19 · 5.10 · 6.1 · 6.12 | 4.19 without BTF; 13 boots in UEFI |
ubuntu-18.04 · 20.04 · 22.04 · 24.04 | 4.15 · 5.4 · 5.15 · 6.8 | 4.15 without BTF |
rocky-8 · 9 | 4.18 · 5.14 | RHEL kernels with backports; 9 boots in UEFI |
debian-sid | 7.2.11 | daily image; UEFI |
fedora-previous · fedora-latest | 6.17 · 6.19 (Fedora 43 · 44) | releases resolved online |
arch | 7.2.9 in the case study | rolling |
gentoo | 6.18 dist-kernel | OpenRC; UEFI |
alpine | 6.18 virt | musl, busybox |
nixos | 6.12 (NixOS 25.05) | Hydra’s Proxmox image; no FHS paths |
aarch64: debian-13 · ubuntu-24.04 · fedora-latest | their own arm64 kernels | emulated on an x86_64 host |
Host needs qemu-system-x86_64, qemu-img, curl, KVM (falls back to TCG without /dev/kvm) and OVMF for the UEFI-only images.
IN PRACTICE / THE EVIDENCE
A problem.
What changed.
Choose a target to read what happened: the problem, the finding and the result. Each story names its evidence and the scope of its validation.
It worked on Arch. The release failed elsewhere.
- Problem
- A kernel observability tool had been developed on one machine. Its release would not even start on a third of the distro images.
- Finding
- Running the installed archives on 17 images and 5 variants, 3 arm64 images and 8 rootless containers exposed 27 defects: 13 in the tool and 14 in its tests. The environment reports explained loader failures, Debian’s perf policy, a 64 KiB memlock limit and Alpine’s missing function tracer.
- Result
- The release gained glibc 2.17 and musl builds. Across the rerun, none of the 158 initially misadvertised source runs still refused at start; refusal reasons agreed with envprobe on all 21 targets. The latest run passed 25 of 25 targets, 33 cells.
The report was missing the facts that explained the failure.
- Problem
- A BPF map failed despite the required capabilities. On another guest, fentry would not attach even though CONFIG_FTRACE was enabled.
- Finding
- The missing facts were the process’s memlock limit and CONFIG_FUNCTION_TRACER. CONFIG_FTRACE alone did not prove that functions had trampoline call sites.
- Result
- envprobe gained soft/hard memlock limits and function-tracer configuration. Comparing it with Omniscient also caught an incorrect “no ftrace” diagnosis on Debian 11.
OPEN SOURCE / BUILT FOR LINUX
Prove it holds
before they find out.
Download the static binaries, then run your first target.
DOWNLOAD · STATIC X86_64 BINARIES
v=v0.1.0 t=x86_64-unknown-linux-musl curl -fLO https://github.com/devutils-org/distroproof/releases/download/$v/distroproof-$v-$t.tar.gz gh attestation verify distroproof-$v-$t.tar.gz --repo devutils-org/distroproof tar xzf distroproof-$v-$t.tar.gz && export PATH="$PWD/distroproof-$v-$t:$PATH" # then distroproof run --target rocky-8 -- envprobe --json > rocky8.json distroproof up # the matrix in distroproof.toml