Skip to content

FAQ

Target audience: All users Goal: Answer common questions about StageX.

Getting Started

What is StageX?

StageX is a minimal, fully bootstrapped, deterministic, multi-party-signed Linux distribution that produces reproducible OCI container images. It builds everything from source — starting from a 181-byte hand-crafted machine code seed (hex0) — with no pre-compiled binaries at any stage. StageX packages are distributed as OCI container images via docker.io/stagex and quay.io/stagex, and each published image must be co-signed by at least two independent maintainers before release.

Persona: evaluator, developer See also: Quick Start, Bootstrapping Journey

What do I need to get started using StageX images?

You need only two things: a container runtime (Podman or Docker) and basic command-line familiarity. No special hardware, no StageX repository clone, no GPG setup is required for the basic consuming workflow. The Quick Start builds a reproducible Rust binary in about 10 minutes using podman build and a Containerfile that pulls stagex/pallet-rust from Docker Hub.

For building StageX packages from source (i.e., running make <package> from the StageX repository), you need Docker Engine 25+ with the containerd image store enabled, Docker Buildx 0.30.1+, ~30 GB free disk, and 8–16 GB RAM. Run make compat to verify your environment.

Persona: developer See also: Dev Environment, Makefile Targets

Can I use Podman with StageX?

Yes, Podman works well for consuming StageX images (pulling, running, building application images on top of pallets). For building StageX packages themselves (running make <package>), Docker with the containerd image store is required — Podman does not support the docker save OCI layout extraction that the build system uses.

Persona: developer See also: Dev Environment, Step 1, OCI Integration

What are pallets?

Pallets are ready-to-use language runtime images published by StageX. Each pallet is a FROM scratch image containing everything needed to build and run applications in a specific language. Available pallets: pallet-rust, pallet-go, pallet-python, pallet-nodejs, pallet-cython (Python with Cython), pallet-cgo (Go with CGO), and pallet-clang. Pallets are the starting point for building your own application images.

Persona: developer See also: Quick Start, Rust App, Package Catalog

How do I pull and verify a StageX image?

Pull by digest (not tag) for production use:

podman pull docker.io/stagex/pallet-rust@sha256:<digest>

Tags like :sx20260501 (release date) or :latest (rolling) are available but can be reassigned. Digests are immutable cryptographic commitments. For automatic signature verification on every pull, configure Podman's policy.json with the StageX keyring and lookaside server at sigs.stagex.tools.

Persona: developer, security-auditor See also: Verify Image, Verify Attestations

Building & Reproducibility

What does "reproducible build" mean in StageX?

A reproducible (deterministic) build means that for fixed inputs, the software always produces the exact same output — bit for bit, every time, on any machine. StageX achieves this through pinned toolchain digests, SOURCE_DATE_EPOCH=1 (normalized timestamps), --network=none (hermetic compilation), and --frozen lockfiles. When two independent maintainers on different hardware get identical digests, the build is proven reproducible.

Persona: developer, security-auditor See also: Reproducible Builds, Reproduce Builds Locally

How do I verify my local build matches the official release?

Build the package from the StageX repo, extract the manifest digest, and compare it against the published digest in digests/<stage>.txt. A match means your build is bit-for-bit identical to the maintainers'. You can also run make verify which automates this comparison for all built packages.

Persona: developer, security-auditor See also: Reproduce Builds, step-by-step, Makefile Targets: make verify

Why does every package use FROM scratch?

FROM scratch starts from an empty filesystem — the final image contains only what you explicitly COPY in. For statically linked binaries (Rust with +crt-static, Go with CGO_ENABLED=0), the image contains just the single binary: no shell, no libraries, no package manager. This minimizes attack surface (nothing to exploit), reduces size (790 KB for a Rust binary vs 100+ MB typical), and ensures the runtime environment is fully determined by your build.

For languages needing shared libraries (Python, Node.js), use the pallet image for both build and runtime stages instead of FROM scratch.

Persona: developer See also: Rust App, Python App

How do I handle external dependencies (crates, npm, Go modules)?

StageX uses a two-step pattern: fetch with network access, then compile with --network=none. For Rust: RUN cargo fetch then RUN --network=none cargo build --frozen. For Go: RUN go mod download then RUN --network=none go build -trimpath. For Node.js: RUN npm ci (with lockfile) then RUN --network=none node --check server.js. Python dependencies must be vendored manually (pip is not available).

Persona: developer See also: Rust App, dependencies, Node.js App, Python App, dependencies

What is make compat and why should I run it?

make compat checks whether your environment meets all StageX tool requirements: Docker 29.1.5+, Buildx 0.30.1+, jq 1.6+, GPG 2.2+, bash 5+, and containerd image store enabled. It is the single most useful diagnostic command — a clean exit means everything is ready; failure reports the specific missing or outdated tool.

Persona: developer, maintainer See also: Dev Environment, Step 5, Troubleshooting

Security & Trust

How does multi-signature verification work?

Every published StageX image is co-signed by at least two independent maintainers. Maintainer A builds the package; Maintainer B independently rebuilds it on different hardware; both compute the SHA-256 digest; if they match, both sign a JSON payload (Container Signature Format) binding the digest to the image name. Signatures are stored in a dedicated repository. The quorum of 2+ means compromising a single maintainer's key is insufficient to publish a malicious image.

Persona: security-auditor See also: Verify Image, Step 3, Multi-Sig

What is the Trusting Trust attack and how does StageX prevent it?

Ken Thompson's 1984 attack showed that a compiler binary can be backdoored to inject vulnerabilities into everything it compiles — and to re-insert that backdoor into its own successor, making it invisible in source code. StageX prevents this through full-source bootstrapping: every compiler is built from source starting from 181 bytes of hand-crafted machine code (hex0). Combined with multi-party verification (at least two maintainers independently rebuild everything), a hidden backdoor would be detected because different machines would produce different outputs.

Persona: security-auditor, evaluator See also: Bootstrapping Journey, introduction, Why Bootstrapping

How do I verify GPG signatures manually?

Import the StageX maintainer keyring, reconstruct the signed JSON payload containing the image reference and digest, download signature files from the signatures repo, and run gpg --verify signature-1 payload.json. A "Good signature" message confirms the payload was signed by that maintainer. Cross-reference the signing key's fingerprint against the MAINTAINERS file.

Persona: security-auditor, developer See also: Verify Image, Steps 4–6, Verify Attestations

What is the hex0 seed?

The hex0 seed is a 181-byte, hand-crafted ELF binary for Intel i386 — no assembler or compiler produced it. It is a hexadecimal-to-binary converter: it reads ASCII hex bytes and writes binary output. Every byte is meaningful and auditable. The seed has been reproduced with the same hash across multiple Linux distributions, proving it contains nothing hidden.

Persona: security-auditor, evaluator See also: Bootstrapping Journey, Stage 0

Technical

What is the bootstrap chain? How many stages are there?

Five stages (0, 1, 2, 3, X): - Stage 0 — 181-byte hex0-seed → hex0/1/2, M2-Planet (C compiler), kaem. ~2 MB, 32-bit only. - Stage 1 — Complete 32-bit Linux userland: mes, tcc, musl, GCC 4→13, bash, python, perl, autotools. ~300 MB. - Stage 2 — Cross-compiler bridge from 32-bit to x86_64 and aarch64. ~700 MB. - Stage 3 — Native 64-bit toolchain: GCC 13.1.0, binutils, cmake, python, busybox. ~1 GB. - Stage X — Everything else: pallets, core packages, 300+ user packages.

Each stage is deterministic and independently verifiable.

Persona: evaluator, security-auditor See also: Bootstrapping Journey, Architecture

What is OCI-native package management?

StageX uses OCI container layers as its package format instead of traditional package managers (APT, DNF, Pacman). Each package is an OCI image composed with others via COPY --from=. This means standard OCI tooling (Podman, Docker, containerd, Kubernetes) can pull, verify, sign, and distribute packages without any special infrastructure. OCI registries provide content-addressable storage, signature verification, and access controls out of the box.

Persona: evaluator, developer See also: OCI-Native, Add a Package

What is the difference between core, pallet, and user packages?

These are the three groups within Stage X (everything built on Stage 3): - core/ — Low-level infrastructure: compilers (LLVM, Go, Rust), libraries (OpenSSL), build tools. ~130 packages. - pallet/ — Batteries-included language runtime images: pallet-rust, pallet-go, pallet-python, etc. ~20 packages. - user/ — Application-level packages: databases, CLI tools, editors, language libraries. ~290+ packages.

Persona: evaluator, maintainer See also: Package Catalog, Add a Package

Does StageX work on ARM64?

Yes. Stage2 builds cross-compilers for aarch64. Pallets are multi-arch. On ARM hosts, use --platform linux/arm64 or let Docker/Podman auto-detect. Cross-architecture builds require QEMU user-mode emulation (docker run --privileged --rm tonistiigi/binfmt --install all).

Persona: developer See also: Cross-Compile, Dev Environment, troubleshooting

What languages does StageX support?

StageX publishes pallets for Rust, Go, Python, Node.js, C/C++ (via pallet-clang), and Cython. Core packages also provide the LLVM toolchain. Languages not yet supported include R, Java, Zig, and OCaml, primarily due to the difficulty of creating verifiable bootstrapped images for these ecosystems.

Persona: developer, evaluator See also: Rust App, Go App, Python App, Node.js App, C/C++ App

Troubleshooting

My build fails at "docker save" or "containerd image store"

Docker Engine must have the containerd image store enabled. Edit /etc/docker/daemon.json to set "features": {"containerd-snapshotter": true}, restart Docker, then run make compat to verify. Podman cannot substitute for Docker when building StageX packages.

Persona: developer, maintainer See also: Dev Environment, Step 1, Troubleshooting, full table

My image digest differs when I rebuild. Why?

Common causes: timestamps not normalized (ensure --timestamp 1 and SOURCE_DATE_EPOCH=1); toolchain mismatch (pin by @sha256: digest, not tag); non-hermetic build (use --network=none during compilation); stale cache (use NOCACHE=1); or different container engines (use the same engine for comparison).

Persona: developer See also: Reproduce Builds, troubleshooting, Quick Start, Step 5

I get "exec: cargo: executable file not found"

Pallet images have their entrypoint set to the language tool (cargo, node, python3, etc.). Override the entrypoint for a shell: podman run -it --rm --entrypoint "" stagex/pallet-rust /bin/sh.

Persona: developer See also: OCI Integration, troubleshooting

Where can I get help?

Matrix: #stagex:matrix.org. IRC: #stagex on OFTC. Issues: codeberg.org/stagex/stagex/issues. Run make compat as your first diagnostic step.

Persona: all users See also: Troubleshooting, Getting Help

See Also