feat: implement auto-deploy webhook for audit endpoint
This commit is contained in:
Sascha
parent
ee8ca1beb7
commit
960943bd60
|
|
@ -16,3 +16,20 @@ SMTP_USER=
|
|||
SMTP_PASS=
|
||||
# Envelope sender. Defaults to SMTP_USER when unset.
|
||||
SMTP_FROM=
|
||||
|
||||
# --- Deploy webhook (backend/webhook/) --------------------------------------
|
||||
# Shared secret, must match the "Secret" field of the webhook configured in
|
||||
# Gitea/Forgejo (repo Settings -> Webhooks). Generate with:
|
||||
# openssl rand -hex 32
|
||||
WEBHOOK_SECRET=
|
||||
|
||||
# Port the webhook listener binds on (127.0.0.1 only - never exposed).
|
||||
WEBHOOK_PORT=3002
|
||||
|
||||
# Only pushes to this ref trigger a deploy. Change if the default branch is
|
||||
# not "master".
|
||||
WEBHOOK_BRANCH=refs/heads/master
|
||||
|
||||
# Optional. Only needed if the checkout lives somewhere other than
|
||||
# /opt/audit-endpoint (deploy.sh's default).
|
||||
# REPO_DIR=/opt/audit-endpoint
|
||||
|
|
|
|||
|
|
@ -216,19 +216,135 @@ The hostname must match in three places or the browser blocks the request:
|
|||
|
||||
---
|
||||
|
||||
## Updating later
|
||||
## 11. Auto-deploy via webhook (optional)
|
||||
|
||||
Without this, updating means SSHing in and running the commands under
|
||||
"Updating later" by hand every time. This section makes a `git push` to
|
||||
your own Gitea/Forgejo trigger a pull + restart automatically.
|
||||
|
||||
Since Git already runs on this same VPS, the webhook listener never needs to
|
||||
be reachable from the internet - it binds to `127.0.0.1` only, and Gitea
|
||||
calls it over loopback. Nothing new is exposed publicly.
|
||||
|
||||
**Check this first:** if Gitea/Forgejo runs inside Docker, `127.0.0.1` inside
|
||||
its container is the container's own loopback, not the host's - the webhook
|
||||
target below would not connect. Run `docker ps` to check. If Gitea is
|
||||
containerized, either add the webhook service to the same Docker network and
|
||||
call it by container name, or expose the webhook through the nginx vhost
|
||||
(same pattern as `/api/audit-request` in step 7) with an `allow`/`deny` block
|
||||
restricting it to the container network's subnet, instead of the loopback
|
||||
target used below.
|
||||
|
||||
### 11.1 Generate a secret and configure it
|
||||
|
||||
```bash
|
||||
openssl rand -hex 32
|
||||
```
|
||||
|
||||
Add the output as `WEBHOOK_SECRET` in `/opt/audit-endpoint/backend/.env`
|
||||
(same file as the SMTP settings - `WEBHOOK_PORT` and `WEBHOOK_BRANCH` already
|
||||
have sane defaults in `.env.example` and normally do not need changing).
|
||||
|
||||
### 11.2 Grant the narrow sudo rule
|
||||
|
||||
The webhook process runs as the unprivileged `auditapi` user but needs to
|
||||
restart a systemd unit, which normally requires root. This grants exactly
|
||||
that, and nothing else:
|
||||
|
||||
```bash
|
||||
sudo visudo -cf /opt/audit-endpoint/backend/deploy/audit-deploy-sudoers
|
||||
# only proceed if that reports "parsed OK"
|
||||
sudo cp /opt/audit-endpoint/backend/deploy/audit-deploy-sudoers \
|
||||
/etc/sudoers.d/audit-deploy
|
||||
sudo chmod 440 /etc/sudoers.d/audit-deploy
|
||||
```
|
||||
|
||||
`visudo -cf` validates the file **before** it takes effect - a broken
|
||||
`/etc/sudoers.d/` file can lock out sudo for the entire machine, so never
|
||||
skip this check.
|
||||
|
||||
### 11.3 systemd service for the webhook
|
||||
|
||||
```bash
|
||||
sudo cp /opt/audit-endpoint/backend/deploy/audit-webhook.service \
|
||||
/etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now audit-webhook
|
||||
|
||||
systemctl status audit-webhook --no-pager
|
||||
```
|
||||
|
||||
### 11.4 Register the webhook in Gitea/Forgejo
|
||||
|
||||
In the repository on `git.sascha-bach.de`: **Settings → Webhooks → Add
|
||||
Webhook → Gitea** (Forgejo: the same menu, still labelled "Gitea" - it is a
|
||||
Gitea fork and uses the same webhook format).
|
||||
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| Target URL | `http://127.0.0.1:3002/deploy-webhook` |
|
||||
| HTTP Method | `POST` |
|
||||
| POST Content Type | `application/json` |
|
||||
| Secret | the value generated in 11.1 |
|
||||
| Trigger On | Push events |
|
||||
| Branch filter | `master` |
|
||||
|
||||
Save, then use Gitea's "Test Delivery" button.
|
||||
|
||||
```bash
|
||||
journalctl -u audit-webhook -f
|
||||
```
|
||||
|
||||
Expect `Deploy started.` followed by `Deployed <short-sha>`. If it instead
|
||||
shows a signature mismatch, the secret in Gitea and in `.env` do not match.
|
||||
|
||||
### 11.5 Try it for real
|
||||
|
||||
```bash
|
||||
# on the workstation, after a real commit
|
||||
git push origin master
|
||||
```
|
||||
|
||||
Within a few seconds, `journalctl -u audit-endpoint -f` should show a
|
||||
restart, and `curl http://127.0.0.1:3001/health` (run locally on the VPS, or
|
||||
`curl https://api.sascha-bach.de/health` from anywhere) keeps answering `200`
|
||||
throughout - restart happens fast enough that a health check running once a
|
||||
minute would not even notice a gap.
|
||||
|
||||
### Known limitation
|
||||
|
||||
Changes to `backend/webhook/` itself (the webhook listener's own code) take
|
||||
effect only after the **next** manual restart:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart audit-webhook
|
||||
```
|
||||
|
||||
It cannot cleanly restart its own systemd unit mid-deploy - see the comment
|
||||
in `backend/webhook/deploy.sh`. This code changes far less often than the
|
||||
audit endpoint's own logic, so it is a fair trade for not adding fragile
|
||||
self-restart handling.
|
||||
|
||||
---
|
||||
|
||||
## Updating later (manual, without the webhook)
|
||||
|
||||
```bash
|
||||
cd /opt/audit-endpoint
|
||||
sudo -u auditapi git pull
|
||||
sudo -u auditapi git fetch origin master
|
||||
sudo -u auditapi git reset --hard origin/master
|
||||
cd backend && sudo -u auditapi npm install --omit=dev
|
||||
sudo systemctl restart audit-endpoint
|
||||
```
|
||||
|
||||
Still useful even with the webhook configured: for a manual redeploy without
|
||||
waiting for the next push, or if the webhook service itself is down.
|
||||
|
||||
## Logs
|
||||
|
||||
```bash
|
||||
journalctl -u audit-endpoint -f
|
||||
journalctl -u audit-webhook -f # only relevant if step 11 is set up
|
||||
```
|
||||
|
||||
Request contents are never logged, only that a request was forwarded — see the
|
||||
|
|
|
|||
|
|
@ -0,0 +1,10 @@
|
|||
# Install with: sudo visudo -cf backend/deploy/audit-deploy-sudoers && \
|
||||
# sudo cp backend/deploy/audit-deploy-sudoers /etc/sudoers.d/audit-deploy
|
||||
#
|
||||
# Grants the unprivileged `auditapi` user exactly two commands, nothing else:
|
||||
# restarting the audit-endpoint unit (from deploy.sh) and its own webhook
|
||||
# unit (for the rare manual restart noted in deploy.sh). No shell, no wildcard
|
||||
# arguments, no other systemctl verb - "restart" only, these two unit names
|
||||
# only.
|
||||
auditapi ALL=(root) NOPASSWD: /usr/bin/systemctl restart audit-endpoint
|
||||
auditapi ALL=(root) NOPASSWD: /usr/bin/systemctl restart audit-webhook
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
[Unit]
|
||||
Description=Deploy webhook for the audit endpoint (git push -> auto pull + restart)
|
||||
Documentation=https://git.sascha-bach.de/saschabach/portfolio-page
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=auditapi
|
||||
Group=auditapi
|
||||
WorkingDirectory=/opt/audit-endpoint/backend/webhook
|
||||
ExecStart=/usr/bin/node server.js
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
# --- Hardening -------------------------------------------------------------
|
||||
# Unlike audit-endpoint.service, this one genuinely needs to write to the
|
||||
# checkout (git reset --hard, npm install), hence ReadWritePaths below -
|
||||
# everything else stays as locked down as the main service.
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
PrivateDevices=true
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths=/opt/audit-endpoint
|
||||
ProtectHome=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
ProtectHostname=true
|
||||
ProtectClock=true
|
||||
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
|
||||
RestrictNamespaces=true
|
||||
RestrictRealtime=true
|
||||
RestrictSUIDSGID=true
|
||||
LockPersonality=true
|
||||
SystemCallArchitectures=native
|
||||
MemoryDenyWriteExecute=false
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
#!/usr/bin/env bash
|
||||
# Invoked by server.js after a verified webhook push to master.
|
||||
#
|
||||
# This checkout is deploy-only: it exists purely so this script can update
|
||||
# it. Never edit files directly in /opt/audit-endpoint - `git reset --hard`
|
||||
# below throws away anything that isn't committed upstream.
|
||||
set -euo pipefail
|
||||
|
||||
REPO_DIR="${REPO_DIR:-/opt/audit-endpoint}"
|
||||
|
||||
cd "$REPO_DIR"
|
||||
git fetch --quiet origin master
|
||||
git reset --hard origin/master
|
||||
|
||||
cd "$REPO_DIR/backend"
|
||||
npm install --omit=dev --no-audit --no-fund
|
||||
|
||||
# Restarting audit-endpoint needs root; the audit-webhook service runs as an
|
||||
# unprivileged user, so this relies on the narrow sudoers grant from
|
||||
# backend/deploy/audit-deploy-sudoers (restart of this exact unit only).
|
||||
sudo /usr/bin/systemctl restart audit-endpoint
|
||||
|
||||
echo "Deployed $(git -C "$REPO_DIR" rev-parse --short HEAD)"
|
||||
|
||||
# Deliberately not restarting audit-webhook itself here: a process cannot
|
||||
# cleanly restart its own systemd unit mid-script (systemd would SIGTERM this
|
||||
# script's parent while it is still running). Changes to backend/webhook/
|
||||
# take effect on the next manual `sudo systemctl restart audit-webhook` -
|
||||
# see DEPLOYMENT.md. That code changes far less often than the audit
|
||||
# endpoint's own logic, so this is a fair trade for the simplicity.
|
||||
|
|
@ -0,0 +1,120 @@
|
|||
import http from 'node:http'
|
||||
import { execFile } from 'node:child_process'
|
||||
import { createHmac, timingSafeEqual } from 'node:crypto'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { dirname, join } from 'node:path'
|
||||
import { config } from 'dotenv'
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url))
|
||||
// Shares backend/.env with the audit endpoint rather than keeping a second
|
||||
// secrets file - one chmod 600 file to manage, not two.
|
||||
config({ path: join(__dirname, '..', '.env') })
|
||||
|
||||
const PORT = Number(process.env.WEBHOOK_PORT ?? 3002)
|
||||
const SECRET = process.env.WEBHOOK_SECRET
|
||||
const BRANCH = process.env.WEBHOOK_BRANCH ?? 'refs/heads/master'
|
||||
|
||||
if (!SECRET) {
|
||||
console.error('WEBHOOK_SECRET is not configured. Refusing to start.')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// Only one deploy runs at a time. Two pushes seconds apart would otherwise
|
||||
// race two concurrent `git reset --hard` + npm install in the same checkout.
|
||||
let deploying = false
|
||||
|
||||
/**
|
||||
* Gitea/Forgejo send `X-Gitea-Signature: <hex hmac-sha256 of body>`.
|
||||
* GitHub-compatible integrations send `X-Hub-Signature-256: sha256=<hex>`.
|
||||
* Accept either so this works regardless of which the instance uses.
|
||||
*/
|
||||
function verifySignature(rawBody, header) {
|
||||
if (!header) return false
|
||||
const provided = header.startsWith('sha256=') ? header.slice(7) : header
|
||||
const expected = createHmac('sha256', SECRET).update(rawBody).digest('hex')
|
||||
|
||||
const a = Buffer.from(provided)
|
||||
const b = Buffer.from(expected)
|
||||
// Length check first: timingSafeEqual throws on mismatched lengths rather
|
||||
// than returning false.
|
||||
return a.length === b.length && timingSafeEqual(a, b)
|
||||
}
|
||||
|
||||
function runDeploy() {
|
||||
if (deploying) {
|
||||
console.log('Deploy already in progress, ignoring this push.')
|
||||
return
|
||||
}
|
||||
deploying = true
|
||||
console.log('Deploy started.')
|
||||
|
||||
// No shell, no string interpolation - argv is a fixed one-element array.
|
||||
execFile(
|
||||
'bash',
|
||||
['deploy.sh'],
|
||||
{ cwd: __dirname, timeout: 5 * 60 * 1000 },
|
||||
(error, stdout, stderr) => {
|
||||
deploying = false
|
||||
if (error) {
|
||||
console.error('Deploy failed:', error.message)
|
||||
if (stderr) console.error(stderr.trim())
|
||||
return
|
||||
}
|
||||
console.log(stdout.trim() || 'Deploy finished.')
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
const server = http.createServer((req, res) => {
|
||||
if (req.method !== 'POST' || req.url !== '/deploy-webhook') {
|
||||
res.writeHead(404).end()
|
||||
return
|
||||
}
|
||||
|
||||
const chunks = []
|
||||
let size = 0
|
||||
|
||||
req.on('data', (chunk) => {
|
||||
size += chunk.length
|
||||
// A webhook payload is a few KB. Anything past 1 MB is not Gitea.
|
||||
if (size > 1_000_000) {
|
||||
req.destroy()
|
||||
return
|
||||
}
|
||||
chunks.push(chunk)
|
||||
})
|
||||
|
||||
req.on('end', () => {
|
||||
const rawBody = Buffer.concat(chunks)
|
||||
const signatureHeader = req.headers['x-gitea-signature'] ?? req.headers['x-hub-signature-256']
|
||||
const signature = Array.isArray(signatureHeader) ? signatureHeader[0] : signatureHeader
|
||||
|
||||
if (!verifySignature(rawBody, signature)) {
|
||||
console.warn('Webhook signature mismatch - ignoring request.')
|
||||
res.writeHead(401).end()
|
||||
return
|
||||
}
|
||||
|
||||
let payload
|
||||
try {
|
||||
payload = JSON.parse(rawBody.toString('utf8'))
|
||||
} catch {
|
||||
res.writeHead(400).end('Invalid JSON.')
|
||||
return
|
||||
}
|
||||
|
||||
if (payload.ref !== BRANCH) {
|
||||
res.writeHead(200).end(`Ignored (push to ${payload.ref}, watching ${BRANCH}).`)
|
||||
return
|
||||
}
|
||||
|
||||
// Respond before deploying: Gitea has its own webhook timeout, and the
|
||||
// deploy (npm install + restart) can take longer than that.
|
||||
res.writeHead(202).end('Deploy queued.')
|
||||
runDeploy()
|
||||
})
|
||||
})
|
||||
|
||||
server.listen(PORT, '127.0.0.1', () => {
|
||||
console.log(`Deploy webhook listening on 127.0.0.1:${PORT}`)
|
||||
})
|
||||
Loading…
Reference in New Issue