Run GitHub Actions jobs on exe.dev, a fresh VM per job

How it works#

• my-ci-base is an ordinary VM. It never runs jobs; it's what each job's VM is copied from.

• my-ci runs ci-dispatcher.py, a short Python program. GitHub sends it a webhook whenever a job is queued, starts, or ends; it also checks every 30 seconds in case one is missed. Each time, it compares GitHub's queue of exe jobs with the runners it already has, and for each job without one:

  1. cp my-ci-base my-ci-job-<random> --sandbox --cpu=2 --memory=4GB;
  2. asks GitHub for a just-in-time runner config for that VM;
  3. starts run.sh --jitconfig … on it.

  When a job ends, the runner exits, GitHub forgets it, and the dispatcher deletes the VM. Spare runners (the job was cancelled, or another runner took it) are deleted after two minutes.

• The dispatcher holds no exe.dev or GitHub key. It reaches both through two integrations attached to it alone, so neither key is on any VM, the job VMs included.

GitHub's queue is the source of truth, so a lost webhook only delays a job, and a forged one can't start a VM nobody asked for.

Before you start#

• An exe.dev account that can create sandboxes. Personal accounts can. On a Work team, an admin turns them on with ssh exe.dev team settings standalone all-users.
• Admin access to the GitHub repository whose CI you're moving. The steps use you/app; put yours in its place.
• About 20 minutes.

Prepare the base VM#

ssh exe.dev new --name=my-ci-base --tag=ci

ssh my-ci-base.exe.xyz 'curl -fsSL https://exe-staging.dev/deploy/github-actions/base-setup.sh | bash'

The base already has what the default exe.dev image ships: Docker, Go, Python with uv, git, and build tools. Install whatever else your workflows set up, the way you would on any Ubuntu 24.04 machine:

ssh my-ci-base.exe.xyz 'sudo apt-get install -y postgresql-client libvips-dev'

Then warm the caches by building your project there once. Jobs run as the same exedev user, so they find ~/go/pkg/mod, ~/.cache/go-build, ~/.npm, ~/.cache/pip, or ~/.cargo already full. From a checkout on your computer:

rsync -a --exclude=.git ./ my-ci-base.exe.xyz:warm/
ssh my-ci-base.exe.xyz 'cd warm && go test ./... ; cd && rm -rf warm && sync'

(npm ci && npm test, uv sync && uv run pytest, cargo test: whatever your CI runs.) Docker images work the same way: docker pull them on the base, and every job has them.

Don't put secrets on the base. Everything on it is on every job's VM.

Create the dispatcher VM and its keys#

On your computer:

ssh exe.dev new --name=my-ci

exe.dev. Make an API key that can copy, list, reach, and delete VMs tagged ci, and nothing else, and hand it to an HTTP proxy integration that only my-ci can use:

ssh exe.dev ssh-key generate-api-key --label=ci-dispatcher --tag=ci \
    --cmds=cp,ls,rm,ssh --exp=1y --json | jq -r .token \
  | ssh exe.dev integrations add http-proxy --name ci-exe \
    --target https://exe.dev --bearer - --attach vm:my-ci

The copies the key makes are tagged ci too, so it can delete them. ssh exe.dev ssh-key remove ci-dispatcher revokes it.

GitHub. The dispatcher needs to register runners and read the job queue. Create a fine-grained personal access token with access to only the repositories you're moving, and these repository permissions:

• Administration: Read and write, to register just-in-time runners;
• Actions: Read-only, to see queued jobs.

For an organization's repositories, choose the organization as the resource owner; it may have to approve the token. Then put the token in a second integration. This reads it without echoing it or saving it in your shell history:

printf 'GitHub token: '; read -rs T; echo
printf %s "$T" | ssh exe.dev integrations add http-proxy --name ci-github \
  --target https://api.github.com --bearer - --attach vm:my-ci
unset T

Check both from the dispatcher VM:

ssh my-ci.exe.xyz
curl -s -X POST https://ci-exe.int.exe.xyz/exec -d ls | jq -r '.vms[].vm_name'
curl -s https://ci-github.int.exe.xyz/repos/you/app/actions/runners | jq .total_count

The first prints my-ci-base, and nothing that isn't tagged ci. The second prints a number. Resource not accessible by personal access token means the token lacks a permission; an invalid token error from exe.dev means the first integration got something other than the key.

Warning

Attach these two integrations to my-ci only (--attach vm:my-ci), never by the ci tag: every job VM carries that tag, and any workflow could then create VMs or register runners.

Start the dispatcher#

On the VM (ssh my-ci.exe.xyz), fetch the dispatcher and write its settings, with a fresh secret for GitHub's webhooks:

mkdir -p ~/ci
curl -fsSL -o ~/ci/ci-dispatcher.py https://exe-staging.dev/deploy/github-actions/ci-dispatcher.py
(umask 077; cat > ~/ci/env <<EOF
GITHUB_API=https://ci-github.int.exe.xyz
EXE_API=https://ci-exe.int.exe.xyz
REPOS=you/app
BASE_VM=my-ci-base
LABEL=exe
MAX_RUNNERS=5
CP_FLAGS="--cpu=2 --memory=4GB"
WEBHOOK_SECRET=$(openssl rand -hex 32)
EOF
)

• REPOS: the repositories, owner/name, separated by spaces.
• LABEL: jobs with runs-on: [self-hosted, exe] come here.
• MAX_RUNNERS: at most this many job VMs at once; further jobs wait in GitHub's queue. Keep it under your plan's VM limit (50 on Personal) minus the VMs you already have.
• CP_FLAGS: each job VM's size. Leave --cpu and --memory in: a copy otherwise takes the base's memory, which may be more than the 2 GiB per vCPU a sandbox is billed for.
• Optional: IDLE_SECONDS (120) before a spare runner is deleted, MAX_HOURS (6) before any job VM is, POLL_SECONDS (30).

Run it as a service, so it comes back after a crash or a reboot:

sudo tee /etc/systemd/system/ci-dispatcher.service >/dev/null <<'EOF'
[Unit]
Description=GitHub Actions jobs on exe.dev sandboxes
After=network-online.target
Wants=network-online.target

[Service]
User=exedev
EnvironmentFile=/home/exedev/ci/env
ExecStart=/usr/bin/python3 /home/exedev/ci/ci-dispatcher.py
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now ci-dispatcher
journalctl -u ci-dispatcher -n 5 --no-pager -o cat

The log says dispatching exe jobs for you/app to copies of my-ci-base, with no reconcile: errors after it.

Send it GitHub's webhooks#

Without webhooks the dispatcher still finds jobs, up to 30 seconds late. With them, it hears about a job as soon as it's queued. The HTTPS proxy keeps my-ci private until you open port 8000 to the internet, which GitHub needs. On your computer:

ssh exe.dev share set-public my-ci

Only requests signed with the webhook secret do anything; everything else gets 401. Show the secret on the VM:

grep WEBHOOK_SECRET ~/ci/env

In the repository on GitHub, open Settings → Webhooks → Add webhook:

• Payload URL: https://my-ci.exe.xyz/
• Content type: application/json
• Secret: the value after WEBHOOK_SECRET=
• Which events: Let me select individual events, then untick Pushes and tick Workflow jobs.

After Add webhook, its Recent Deliveries tab shows a ping answered with 204.

Move a workflow#

In each job you want on exe.dev, change runs-on:

jobs:
  test:
    runs-on: [self-hosted, exe]
    steps:
      - uses: actions/checkout@v4
      - run: go test ./...

Leave out the setup-* step for a toolchain the base already has: it would download one into the job's VM every time. Keep it where you need an exact version that the base doesn't have.

Push, and watch the dispatcher on the VM:

journalctl -u ci-dispatcher -f -o cat

04:38:37 webhook: test queued
04:38:41 started my-ci-job-c259d116 for you/app in 1.9s
04:38:46 webhook: test in_progress my-ci-job-c259d116
04:38:53 webhook: test completed my-ci-job-c259d116
04:38:54 deleted my-ci-job-c259d116

On GitHub, the job's log names my-ci-job-… as its runner. A matrix of six jobs gets six VMs at once, or MAX_RUNNERS of them with the rest started as the first ones finish. To try it all without pushing, add workflow_dispatch: to the workflow's on: and click Run workflow.

Moving one job at a time is fine: jobs that keep ubuntu-latest stay on GitHub, and needs: works across the two.

Caches: the base or actions/cache#

Two kinds of cache work here, and they combine.

• The base holds whatever was on its disk when a job's VM was copied: toolchains, dependency caches, build caches, Docker images. Nothing a job writes comes back to it. It costs no time per job, but it's only as fresh as your last warm-up. Re-run the warm-up when dependencies change a lot, or weekly.
• actions/cache works as on GitHub's runners: it restores from and saves to GitHub's cache service, which keeps up with every branch. It costs a download per job and counts toward the repository's 10 GB.

Start with the base alone, and add actions/cache for whatever changes faster than you want to refresh it. To update the base, change it (ssh my-ci-base.exe.xyz) and end with sync: new job VMs are copies of its disk, and anything not yet written to disk isn't in them. Jobs already running keep their copy.

Secrets#

GitHub's secrets.* and the job's GITHUB_TOKEN work unchanged, and they reach the job's VM the way they reach a GitHub-hosted runner.

exe.dev can keep some keys off job VMs altogether. An integration attached by the ci tag is usable from every job VM, without a key on it: an HTTP proxy to a private package registry or API, or the LLM gateway for tests that call a model. A workflow calls https://<name>.int.exe.xyz/ and exe.dev adds the credential. Remember that any job can use it, including the code in any pull request your workflows run.

Public repositories#

GitHub's advice is not to use self-hosted runners for public repositories, because a pull request from a fork can run anything on them. Throwaway VMs with no keys answer much of that: nothing survives the job, and there's no other job's data to read. What's left is your bill and your integrations. If you use them for a public repository:

• In Settings → Actions → General, under approval for fork pull request workflows, choose Require approval for all external contributors.

• Keep fork pull requests on GitHub's runners, which are free for public repositories:

  runs-on: ${{ github.event.pull_request.head.repo.fork && 'ubuntu-latest' || fromJSON('["self-hosted", "exe"]') }}
  

• Attach nothing by the ci tag, and keep MAX_RUNNERS and MAX_HOURS low enough to cap what a bad week can cost.

What it costs#

exe.dev bills the sandboxes by the second, from the copy to the delete: $0.105 an hour per 2 vCPU / 4 GiB, so $0.00175 a minute for the default size, $0.0035 for 4 vCPU, $0.007 for 8. Each job VM lives about 15 seconds longer than its job. my-ci and my-ci-base are ordinary VMs that share your plan's capacity with your other VMs and cost nothing extra (on Personal, plans start at $15 a month).

GitHub's hosted runners cost $0.006 a minute for 2-core Linux, $0.012 for 4-core, and $0.022 for 8-core, and round each job up to a whole minute. Private repositories get $12 (Free) or $18 (Pro, Team) of that a month. Self-hosted runners, which these are, are free on GitHub today; GitHub announced a $0.002 a minute charge for them in December 2025, then postponed it.

Some months, on 30 days:

• A team's private repository. 100 jobs a day, 5 minutes each, on 2 cores: 15,000 minutes. GitHub: $90, or $72 after Team's $18. exe.dev: 3,000 VMs × 5¼ minutes ≈ 263 hours × $0.105 ≈ $27.60. If the warm base brings the jobs to 3 minutes, ≈ $17.10.
• Many short jobs. 500 lint and unit-test jobs a day, 20 seconds each. GitHub bills 15,000 minutes: $90. exe.dev: 15,000 VMs × 35 seconds ≈ 146 hours ≈ $15.30.
• Big builds. 2,000 builds a month, 12 minutes each, on 8 cores. GitHub: 24,000 minutes × $0.022 = $528. exe.dev, 8 vCPU / 16 GiB (4 units, $0.42 an hour): ≈ 408 hours ≈ $171.50.

GitHub's runners stay the better deal for public repositories (free), for private ones that fit in the included minutes, and for macOS, Windows, and arm64 jobs, which exe.dev doesn't run. Mind the memory, too: GitHub's 2-core runner for private repositories has 8 GB, and a 2 vCPU sandbox 4 GiB, so a job that needs more takes 4 vCPU here.

Just one runner#

For one small repository whose pull requests all come from you, one always-on runner on one VM may be all you need: no base, no dispatcher, no integrations, and nothing to pay beyond your plan. It runs job after job on the same VM, so each job sees what earlier ones left behind, and any job can read what the runner's user can. Keep it to your own code.

ssh exe.dev new --name=my-runner
ssh my-runner.exe.xyz

cat > /home/exedev/actions-runner/gh-actions-runner.service << 'EOF'
[Unit]
Description=GitHub Actions Runner
After=network.target

[Service]
Type=simple
User=exedev
WorkingDirectory=/home/exedev/actions-runner
ExecStart=/home/exedev/actions-runner/run.sh
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
EOF
sudo cp /home/exedev/actions-runner/gh-actions-runner.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now gh-actions-runner.service
sudo systemctl status gh-actions-runner.service

Or let Shelley do it on a new VM: it asks for the --url and --token from GitHub's page (the token expires after an hour) and sets up the same service.

Update#

• The runner. GitHub releases one every few weeks, and a just-in-time runner that's out of date updates itself before its job, which adds time to every job. Re-run the install on the base when that starts showing in job logs (Runner update in progress): ssh my-ci-base.exe.xyz 'curl -fsSL https://exe-staging.dev/deploy/github-actions/base-setup.sh | bash'.
• The dispatcher. Fetch ci-dispatcher.py again as in Start the dispatcher, then sudo systemctl restart ci-dispatcher. Running jobs are unaffected.
• The keys. The exe.dev key above expires in a year and the GitHub token when you chose. Make a new one, ssh exe.dev integrations remove ci-exe (or ci-github), and add it again as above.

Troubleshooting#

Start with the dispatcher's log on my-ci: journalctl -u ci-dispatcher -n 50 --no-pager -o cat.

• A job says "Waiting for a runner to pick up this job". If the log shows nothing for it, its labels don't match: they must all be among self-hosted, linux, x64, and exe. If MAX_RUNNERS job VMs are running, it waits its turn.
• reconcile: HTTPError 403. The GitHub token lacks Administration (write) or Actions (read) on that repository, or has expired.
• reconcile: HTTPError 401. The exe.dev key in ci-exe is wrong, expired, or revoked.
• launch … failed: HTTP Error 422. exe.dev refused the copy: sandboxes aren't enabled for your team, the VM limit is reached, or BASE_VM isn't tagged ci. ssh exe.dev cp my-ci-base --sandbox on your computer shows the reason; delete that copy afterwards.
• No webhook: lines. The webhook's Recent Deliveries on GitHub show why: 401 is a secret that doesn't match ~/ci/env; a redirect to a login page means my-ci isn't public yet. Jobs still start, within POLL_SECONDS.
• A job can't find a tool or a cached package. It wasn't on the base's disk when the job's VM was copied. Install it, sync, and re-run.
• Leftover job VMs. The dispatcher deletes them while it runs. To see them yourself: ssh exe.dev ls --json | jq -r '.vms[].vm_name | select(startswith("my-ci-job-"))'.