nixos config for self hosting
  • Python 55.5%
  • Nix 44.5%
Find a file
rabbi-gerzi-ci e51a1c4e87
All checks were successful
Check and deploy NixOS / deploy (push) Successful in 1m56s
update rabbi gerzi
2026-11-01 21:01:06 +00:00
.forgejo/workflows Add Forgejo CI and automatic NixOS deployment 2026-10-13 22:14:47 +03:00
checks Add type hints to deployment scripts and tests 2026-10-26 21:01:30 +02:00
ci Add type hints to deployment scripts and tests 2026-10-26 21:01:30 +02:00
home-manager@9ef0dc2036 update home manager submodule 2026-06-03 08:44:03 +03:00
secrets document runner authentication secrets 2026-10-30 08:40:46 +02:00
.envrc add direnv to use flake 2026-05-09 22:21:39 +03:00
.gitignore ignore worktrees 2026-10-24 22:20:04 +03:00
.gitmodules update git submodule for home manager 2026-04-09 22:16:08 +03:00
.sops.yaml configure rabbi gerzi 2026-07-02 11:51:07 +03:00
boot.nix refactor boot info 2026-04-09 10:59:19 +03:00
borgbackup.nix move borg secrets to its own file 2026-06-12 11:20:34 +03:00
ci-deploy.nix Simplify deployment worker package references 2026-10-24 23:32:47 +03:00
configuration.nix move yisroelbaum web stack into module 2026-07-31 11:30:25 +03:00
flake.lock update rabbi gerzi 2026-11-01 21:01:06 +00:00
flake.nix updatee nixpkgs 2026-10-24 23:31:34 +03:00
hardware-configuration.nix initial commit 2026-03-22 22:15:52 +02:00
rabbi-gerzi.nix retire staging domains 2026-08-18 09:58:08 +03:00
README.md document runner authentication secrets 2026-10-30 08:40:46 +02:00
RECOVERY.md add borg recovery details 2026-06-11 22:35:50 +03:00
tide.nix move discourse behind tide module 2026-07-19 10:40:25 +03:00
yisroelbaum-web.nix manage runner credentials with sops 2026-10-30 08:21:04 +02:00

redeploy to home server using this command

nixos-rebuild switch --flake .#nixos \
	--target-host nixos_home \
	--build-host nixos_home \
	--sudo --ask-sudo-password

The main public services under yisroelbaum.com are provided by the yisroelbaum-web flake input. Rabbi Gerzi is imported directly from its own flake so its application module remains independently owned. This repository owns the machine configuration and provisions runtime secrets for both.

Automatic deployment

The server-side Python code is split into ci/dispatcher.py, which handles SSH requests (start, status, and logs), and ci/worker.py, which checks, builds, and activates the system. Both use ci/common.py for shared state and locking helpers. ci-deploy.nix installs these files together and creates a separate executable wrapper for each entry point.

Pushes to master run the Check and deploy NixOS Forgejo workflow on the existing linux runner. The runner requests a deployment over SSH; the host checks out the exact commit (including pinned submodules), runs this flake's checks, builds nixosConfigurations.nixos, and activates the resulting system with nixos-rebuild switch --no-reexec --store-path .... The worker already uses the flake-pinned rebuild executable; --no-reexec avoids a legacy <nixpkgs/nixos> lookup when activating a prebuilt system.

Checks and builds use the committed lock file without updating it, one build job, and two build cores. Imported applications' separate test suites are not run. A failed check or build prevents activation. Deployment uses a host-wide lock and rechecks remote master immediately before switching: superseded commits are reported as skipped, and an activation already underway finishes.

One-time bootstrap

  1. Use the dedicated private key matching ci/deploy-key.pub. If generating a replacement, run the following on your workstation, then include the updated public key in the bootstrap configuration:

    ssh-keygen -t ed25519 -N '' -C home-server-config-ci \
      -f ~/.ssh/home-server-config-ci-deploy
    cp ~/.ssh/home-server-config-ci-deploy.pub ci/deploy-key.pub
    
  2. Add the complete private key as the repository Actions secret DEPLOY_SSH_KEY at https://git.yisroelbaum.com/yisroelbaum/home-server-config/settings/actions/secrets. Keep it scoped to this repository. Never commit the private key. The existing runner authentication token is unrelated and does not need to change.

  3. Stage the new configuration files so Git-based flake evaluation includes them, run nix flake check --no-update-lock-file, and perform the manual rebuild shown above. This installs the ci-deploy account and deployment helpers. Bootstrap must happen before the first workflow can authenticate.

  4. Check access with the matching private key:

    ssh -i ~/.ssh/home-server-config-ci-deploy -o IdentitiesOnly=yes \
      ci-deploy@git.yisroelbaum.com "status $(git rev-parse HEAD)"
    

    An unknown status is expected before the first deployment. Arbitrary commands, interactive sessions, and SSH forwarding are disabled.

  5. Commit and push the change to master. Actions are already enabled for this repository. Confirm that the workflow reports success and that its system value matches readlink -f /run/current-system on the server.

The workflow pins the server's Ed25519 SSH host key, verified through the existing administrative SSH connection. If the server host key changes, verify its replacement through that trusted connection before updating the workflow. Do not disable host-key verification.

Status, logs, and retry

The SSH interface accepts exactly start <sha>, status <sha>, or logs <sha>, with a full 40-character commit hash. It can deploy only this repository's current master; it cannot accept uploaded code, arbitrary repositories, or shell commands. The privileged deployment source and state are root-owned.

ssh -i ~/.ssh/home-server-config-ci-deploy ci-deploy@git.yisroelbaum.com \
  "status COMMIT_SHA"
ssh -i ~/.ssh/home-server-config-ci-deploy ci-deploy@git.yisroelbaum.com \
  "logs COMMIT_SHA"

On the server, administrators can use:

sudo home-server-deploy 'status COMMIT_SHA'
sudo home-server-deploy 'logs COMMIT_SHA'
sudo systemctl status home-server-deploy-COMMIT_SHA.service

Completed transient services disappear; their status and logs remain under /var/lib/home-server-deploy/COMMIT_SHA/. Source checkouts are removed after each attempt; the built system symlink remains as a Nix GC root. CI prints phase changes and retrieves the complete log when the deployment finishes.

The worker runs independently of SSH, Forgejo, Docker, and the Actions runner. If a switch restarts those services, deployment continues, though Forgejo may mark its workflow interrupted. Rerun the workflow after recovery: it attaches to a running attempt, recognizes an already active successful deployment, or retries a failed attempt. CI stops waiting after 50 minutes without stopping the host job. A reboot interrupts the worker and is reported as failure when its status is next queried.

If an older installed worker fails during switching with file 'nixpkgs/nixos' was not found, perform the manual rebuild at the top of this file from the corrected checkout, then rerun the workflow. Pushing the fix alone cannot replace that worker because installing its replacement requires a successful activation.

Disable or roll back

Disable this workflow in Forgejo to stop new automatic requests. This does not cancel a host deployment already running. Removing the deployment public key and manually rebuilding revokes the SSH deployment credential.

There is no automatic rollback. A switch failure can leave partially applied services; inspect its log and use the existing administrative SSH account:

sudo nixos-rebuild switch --rollback

Previous NixOS generations remain available. Disable automatic deployment while investigating, then fix or revert the configuration on master before re-enabling it.