Storage and backups
A server keeps everything in two places:
- Postgres: accounts, keys, sites, versions, saved data and lists, domains.
- A folder on disk (
DATA_DIR): every site’s files, one folder per kept version, runtime SQLite databases, file buckets and any SQLite key-value namespaces.
On a small box both are Docker volumes (simple-host_db and simple-host_sites), and
/opt/simple-host/.env holds the admin key and the database password, which only that file
knows.
| Setting | Default | Allowed | What it does |
|---|---|---|---|
VISITOR_AI_WRITES_PER_DAY |
200 |
1–100000 writes | Visitor connector write attempts per person and site per UTC day, shared across apps and restarts. Security-sensitive. |
SITE_STORAGE_FILES_MAX_OBJECTS |
1000 |
1–100000 objects | Maximum committed file objects per bucket, including empty files; overwrites do not add an object. |
SITE_STORAGE_SITE_INFLIGHT |
1 |
1–4 operations | Concurrent SQLite row operations per site (1 to 4); the site slot and global gate share one acquire wait. |
SITE_STORAGE_SQL_CONCURRENCY |
0 |
0–1024 queries | Concurrent SQLite executions across the process; defaults to the CPU count. |
SITE_STORAGE_ACQUIRE_WAIT_MS |
250 |
1–5000 milliseconds | Wait for a SQLite execution slot before returning 503 with Retry-After. |
SITE_STORAGE_WRITE_LOCK_WAIT_MS |
2000 |
1–10000 milliseconds | Wait for the site write/deploy lock before returning 503 with Retry-After. |
SITE_STORAGE_OWNER_TIMEOUT_MS |
5000 |
1–30000 milliseconds | Owner SQLite query, schema and write timeout. |
SITE_STORAGE_VISITOR_WRITE_TIMEOUT_MS |
2000 |
1–5000 milliseconds | Visitor storage write timeout. |
SITE_STORAGE_VISITOR_QUERY_TIMEOUT_MS |
1000 |
1–5000 milliseconds | Visitor SQLite query timeout. |
RATE_LIMIT_STORAGE_IP |
120,100ms |
any (warns past 10× looser) | Visitor storage writes per site and client IP; owner credentials are exempt. |
RATE_LIMIT_STORAGE_VISITOR |
60,200ms |
any (warns past 10× looser) | Storage writes per site and signed-in visitor; limits add-only flooding. |
STORAGE_SWITCH_AT |
none | text | RFC3339 UTC retirement instant for storage presets and older saved-data APIs. Empty means no switch date set. Requires STORAGE_RULES=on. Set the environment (Compose) or values file (Helm) and restart to change it. Security-sensitive. |
STORAGE_RULES |
off |
off / on |
Accept owner-written row rules for SQLite databases and moved key-value namespaces. Saved rules remain effective when this switch is off. Set the environment (Compose) or values file (Helm) and restart to change it. Security-sensitive. |
KV_STORE |
postgres |
postgres / sqlite |
Store for newly created key-value namespaces. Existing namespaces follow their saved marker. Restart after changing the environment or Helm values. |
DB_DSN |
none | secret | The Postgres connection string. install.sh sets it for the bundled database. Required. Security-sensitive. |
DB_MAX_OPEN_CONNS |
25 |
at least 1 connections | Maximum open Postgres connections per server process; excess requests wait. |
DB_MAX_IDLE_CONNS |
5 |
at least 0 connections | Maximum idle Postgres connections per server process; zero keeps none. |
DB_CONN_MAX_LIFETIME_MINUTES |
30 |
at least 1 minutes | Maximum lifetime of each pooled Postgres connection. |
DB_CONN_MAX_IDLE_MINUTES |
5 |
at least 1 minutes | Maximum idle time of each pooled Postgres connection. |
MIGRATE_ON_START |
false |
true / false |
true makes the server build an empty database from the embedded schema.sql and apply every pending migration before it listens (the same as simple-host migrate -init). For Kubernetes, where there is no psql and no install command; two pods starting at once are safe (an advisory lock). |
DATA_DIR |
./data/sites |
text | The folder that holds every site’s files and versions. |
SITE_STORAGE_MAX_BYTES |
10000000 |
1–10737418240 bytes | Durable byte budget per site for KV and SQLite (Hack defaults to 1000000, pooled with files); SQLite WAL and backups need additional temporary disk. |
SITE_STORAGE_FILES_MAX_BYTES |
10000000 |
1–10737418240 bytes | Separate raw-file allowance per website on Simple Host; Hack retains SITE_STORAGE_MAX_BYTES pooled allowance. |
SITE_STORAGE_FILE_MAX_BYTES |
1000000 |
1–10737418240 bytes | Largest individual raw-file upload for a site storage bucket. |
SITE_STORAGE_SQL_RESULT_MAX_BYTES |
1000000 |
1–67108864 bytes | Largest JSON result from one site SQLite query. |
Recipes
Back up a small box (from /opt/simple-host, into the current folder):
sudo docker compose exec -T db pg_dump -U simplehost simplehost | gzip > db-$(date +%F).sql.gz && sudo docker run --rm -v simple-host_sites:/sites -v "$PWD":/out alpine tar czf /out/sites-$(date +%F).tar.gz -C /sites . && sudo cp .env env-$(date +%F).bak
Keep all three together: the .env backup is what lets a restored database be opened.
After a key-value namespace moves to SQLite, its data lives at
runtime/kv/<namespace>.sqlite. The Postgres dump and matching data directory
are one backup: restore them together. Stop the install before taking the pair
for a consistent snapshot. A live SQLite copy must include its -wal file;
copying the main file alone can lose committed writes. Stopping first avoids
copying files while they change. Site exports combine both KV stores into
storage/kv.json and omit the internal runtime/kv/ files.
Keep KV_STORE=postgres until the move and rehearsal are ready. This release
adds export, deletion and authorship support; changing the flag does not move
existing namespaces.
Checking and moving key-value namespaces
Run kv-move with the currently installed program (/usr/local/bin/simple-host
in the standard installs), using that install’s environment,
including DB_DSN and DATA_DIR, and as the user the server runs as. Keep
KV_STORE=postgres until the move and the rollback rehearsal have been approved.
kv-move refuses, changing nothing, when the user running it does not own
DATA_DIR: files created by another user (root, for example) could not be
opened by the server afterwards.
simple-host kv-move --dry-run --scratch /tmp/kv-check --log /tmp/kv-dry-run.jsonl
simple-host kv-move --apply --log /tmp/kv-apply.jsonl
simple-host kv-move --verify --log /tmp/kv-verify.jsonl
simple-host kv-move --rollback --log /tmp/kv-rollback.jsonl
Repeat --site <id> to select sites; omitting it selects every key-value
namespace, including sites in Recently deleted. Other resource kinds are
unaffected.
Dry run can run beside the server. It reads a repeatable Postgres snapshot,
builds and checks temporary files under --scratch, and removes those files
afterwards. Its JSON-lines report gives entry counts, logical bytes, copy
hashes, whether Postgres’s key order equals byte order, and the number of keys
that differ only by case. Keys, values and authors are not included in the log.
A copy difference exits 3; other failures exit 1. A namespace whose Postgres
order differs will list in byte order after moving, so old page cursors may
skip or repeat keys.
Apply requires every server using the data directory to be stopped. Servers
hold a shared lock on DATA_DIR/.maintenance.lock for their lifetime; apply
takes an exclusive lock without waiting and exits 2 if it cannot. A server
also refuses to start while apply holds that lock. The kernel releases the
lock if the process dies; leave the lock file in place. The file is empty and
world-readable, and a server opens it read-only, so a lock file created by
another user does not stop the server. If the server cannot read the file,
its error gives the file’s path, owner and mode.
Apply and rollback also require every server using the database to be stopped, whichever data directory it serves from. A server holds a shared Postgres advisory lock on its database while it runs; apply and rollback take that lock exclusively without waiting and exit 2 if they cannot, and a server refuses to start while a command holds it. Postgres releases the lock when the process’s connection ends.
The database and the data directory must be the same install
Each install has an identity: one random id, saved in the file
DATA_DIR/.install-id and in the single row of the install_identity table.
The server creates both at start when neither exists, and never changes
them. Back up and restore the database and the data directory together and
the pair stays matched.
Apply, rollback and verify refuse, changing nothing and exiting 1, unless both
copies exist and are equal. This stops a command given one install’s DB_DSN
and a different install’s DATA_DIR from moving data into the wrong place.
Dry run is not tied to a data directory and does not check.
The identity does not tell a data directory from a copy of it. A copy
(a rehearsal’s scratch copy, a restored backup in another path) holds the same
.install-id and passes the check. The database lock above stops a command
pointed at a copy only while a server is running on that database; with the
server stopped, nothing does. Before every --apply or --rollback, check
that DATA_DIR in the env file the command reads is the directory the server
serves from, not a copy.
When neither copy exists, start the server once on that database and data directory, stop it, and run the command again.
When exactly one copy exists, or the two differ, the server writes a
WARNING: INSTALL IDENTITY line to its log at each start and keeps serving.
Find out which half was replaced before doing anything else: the usual causes
are a restored database with an older data directory, or an env file that
points at another install. Only when you are certain the two belong together,
with the server stopped, make them agree by hand: copy the id from the
install_identity row into DATA_DIR/.install-id (one line, owned by the
server’s user), or the other way round.
For each namespace, apply builds a file without a WAL, compares every key, normalized JSON value, author and timestamp, checks integrity and the logical byte count, and writes the copy proof to the log. It syncs and renames the file and syncs the containing directories through the site directory, then changes the store marker and deletes the Postgres rows in one transaction. Check the command’s exit status: the copy proof alone does not mean the final transaction committed. An unresolved site directory is reported and left on Postgres. Re-running apply removes interrupted copies for Postgres namespaces and skips namespaces already moved.
Verify is read-only and can run beside the server after later writes. It checks the SQLite file, JSON values, entry and byte totals, and absence of Postgres rows for moved namespaces. It does not compare with the old source content. A leftover file for a Postgres namespace is a warning. A namespace changing during verification is retried once; a second change is reported as a warning, not corruption. Other health failures exit nonzero.
Moving data on a live install is a separate operator step. Take the paired Postgres and data-directory backup while stopped before applying. This command includes rollback; production cutover remains a separate approved step. Once a namespace moves, use a program that understands its marker.
Rollback also requires the exclusive maintenance lock. It checkpoints and checks
each SQLite namespace, inserts every current entry into Postgres and reads those
rows back in the same transaction. It compares values, authors and both timestamps
before changing the marker to postgres. A mismatch or failed JSON cast rolls
the transaction back and leaves the namespace on SQLite. Later writes, edits
and deletes are preserved. Repeat --site to limit the rollback to selected sites.
Two kinds of namespace are left as they are:
- A namespace with saved rules is refused. Rules are enforced only on the SQLite store, so moving it back would stop enforcing them. The error and the log line name the site and the namespace, neither the file nor the database is touched for it, every other namespace is still rolled back, and the command exits 1.
- A namespace that already has rows in Postgres is skipped. Nothing is
changed and it is not a failure: the command exits 0 when that is all it
reports. The log line’s status says which case it is.
skipped: already restoredmeans the namespace is already on Postgres.skipped: still on SQLite; Postgres already holds rows for it (run --verify)means it was not rolled back and is still served from SQLite. Exit 0 therefore does not prove every namespace is back on Postgres: read the log as well as the exit status, and run--verifyafterwards, which reports a namespace still markedsqlitethat has Postgres rows.
After commit the old file is retained as <namespace>.sqlite.rolledback-<UTC time>
and its directory is synced, so a crash cannot bring the old name back.
If the rename fails, the leftover active file is ignored by the Postgres marker;
a later apply removes it and rebuilds from Postgres. A kill between commit and
rename has the same safe result. Repeating rollback skips namespaces already on
Postgres. Set KV_STORE=postgres before restarting if new namespaces should use
Postgres too.
Rehearse the cutover and both ways back
Run scripts/kv-move-rehearsal.sh once for each install, passing its program,
product (host or hack), a custom-format dump from the stopped install and the
matching whole data-directory copy. TEST_PG_URL supplies a bare disposable
Postgres server URL; the script creates and drops only its own database. Inputs
are read only. It uses fresh local credentials, removes copied notification
subscriptions and saved setup hostnames on the scratch database, and inherits no
provider credentials. The scratch environment supplies its local hostname.
The script restores the copies, starts a loopback instance, dry-runs, checks free
space (data plus dump plus 20%), stops, takes the paired backup with sizes and
SHA-256 hashes, applies, verifies and restarts with KV_STORE=sqlite. It compares
all key-value records and live-site owner reads, usage and exported kv.json,
and exercises page and owner-tool writes, reads and deletes on an invented site.
It compares the usage tool’s decoded response with its pre-stop response. Apply
must leave every key-value namespace on SQLite and the Postgres KV table empty;
rollback must return every namespace, including newly created ones, to Postgres.
It then makes new keys, edits, deletes, a new namespace and a namespace
deletion, verifies, stops, rolls back, restarts and compares again. Finally it
restores the paired dump and files onto an emptied scratch directory and compares
with the backup-time state. This last restore deliberately loses the later
changes; it is a last resort, never an overlay onto the current directory.
The output directory retains JSON-lines move logs, per-step wall times, backup
sizes/hashes and key-value read throughput before and after moving (100 reads
with one and four clients on one site, SITE_STORAGE_SITE_INFLIGHT=1). Scratch
databases, file copies and backups are deleted on exit. Keep the output from both
installs for the cutover review. A rehearsal on invented fixtures alone does not
establish that production copies have passed.
The upgrade journal and step list are not available in this release. When they
land, the move registers as OptIn (upgrade --with kv-move), with progress per
namespace, not a remembered install-wide completion. Every attempt must inspect
the current markers and Postgres rows; successful verification must also require
zero Postgres namespaces and rows. Standalone apply/rollback take their own lock;
the future upgrade command holds it across its steps instead.
Converting presets to row rules
migrate-rules is a command for the person who runs the install. It turns the
preset on a SQLite database or key-value namespace into row rules that decide
the same, and can put the preset back. It is not offered to site owners: there
is no API route, AI tool or recipe for it. File buckets keep presets and are
never read or changed by it.
Run it with the installed program and that install’s environment (DB_DSN,
DATA_DIR and the spam-check settings), as the user the server runs as.
simple-host migrate-rules --dry-run --plan rules-plan.json --account <handle>
simple-host migrate-rules --dry-run --plan rules-plan.json --all
simple-host migrate-rules --apply --plan rules-plan.json
simple-host migrate-rules --revert --plan rules-plan.json
simple-host migrate-rules --revert --site <site id> --resource <name>
Repeat --account for several accounts. Sites in Recently deleted are included.
Apply and revert with --plan require a regular file owned by the command’s
user or root, with no group or other write permission. Its directories must
not be writable by others unless the sticky bit is set (as on /tmp). The
plan must record the same commit as the running program. Use the same program
for the dry run, apply and plan-based revert. A refused plan changes no resource.
Dry run. The server may be running. Nothing is changed: Postgres is only read, each SQLite database is opened read-only, no saved row is read, and no request is sent. For every SQLite or key-value resource still on a preset it works out the rules the preset converts to, checks that they fit the tables that exist, and compares, for each kind of caller (not signed in, signed in, the author, the owner signed in on the site, a visitor’s AI, each with and without a passed spam check) and each action, what the preset allows today with what the rules would allow. It prints one line for each answer that would change:
account site resource table caller action old new
A resource with such a line is not converted. Neither is a resource that “needs a person”, which is listed with the reason:
- a database or namespace saved before presets that pages could write freely, where saving rules would clear recorded authors or require an empty database;
- a database that pages still read with their own SQL, which stops when a resource has rules;
- a setting that lets anyone edit or delete, which no rule may say;
- converted rules that the existing tables cannot take, for example a
visitor_idorapproved_atcolumn that is not text.
The owner of such a resource writes its rules by saving them in the ordinary way. The dry run writes the plan file, readable only by the user who ran it. For each resource to convert the plan holds a fingerprint of its settings, a digest of its table columns, its complete present settings, the rules it converts to and a SHA-256 of the entry. Exit codes: 0 when everything in scope can be converted, 3 when something would change or needs a person (the plan still lists what can be converted), 1 when a resource could not be examined (no plan is written).
Apply. Stop the server first: apply takes the data directory’s maintenance lock and exits 2 while a server or another maintenance command holds it. For each resource in the plan it checks again, before changing anything, that the settings and table columns are exactly what the dry run saw and that the rules it would write are the plan’s. A resource that an owner saved, or whose tables changed, since the dry run is skipped as “changed since the dry run” with nothing recorded; run a new dry run for it. A key-value namespace still stored in Postgres is skipped until it has been moved (see above). Then, in order:
- the present settings are recorded on the resource (
rules_converted_from); - the database gets the columns a rules save adds, the author column and its index and the approval stamp where the rules name them, and nothing is cleared or removed;
- in one transaction, the rules replace the preset.
If the command stops after step 1 or step 2 the resource still works under its
preset for everyone; run --apply again with the same plan. Applying a plan
twice changes nothing. Exit 0 when every resource is converted, 1 when any was
skipped or failed. The additive SQLite step is not held to the site’s storage
allowance.
Revert. Also with the server stopped. For each resource the plan names, or
the one named by --site and --resource, the recorded settings and table
presets are restored in one transaction and the record is cleared. A resource
whose rules are no longer the ones apply wrote (its owner saved different
rules, or went back to a preset) is skipped as “rules were changed after
conversion”; its owner decides by an ordinary save. Columns added to SQLite
stay; presets use or ignore them as before.
A program older than the one that first enforces rules must never be installed over converted resources: it would not apply their rules. Revert every converted resource before going back that far.
What fills the disk. At the default allowances, 2,000 fully used hosted sites can add 40 GB of logical KV/SQLite and file storage (10 MB + 10 MB per site), beyond deployed versions. Budget extra for filesystem metadata, Postgres, SQLite WAL, backups and version retention. Storage quotas are shared within a site; visitor rate limits slow filling them but do not reserve space for other visitors.
For deployed versions: set KEEP_VERSIONS (see
Sites and versions). The admin page shows disk use.
Postgres client pool
DB_DSN selects the database. Each server process bounds its own database/sql pool.
Go clamps DB_MAX_IDLE_CONNS to the open limit.
Unset or empty values use these defaults; invalid values stop startup with the variable name. The open limit of 25 is sized for two instances of this binary sharing one 100-connection Postgres role, leaving headroom for other services on the box. These are client settings; they do not change Postgres server configuration. Set them in each instance’s environment and restart that instance to apply them.
Storage execution settings
SITE_STORAGE_SQL_CONCURRENCY=0 uses the CPU count. See the table above for
slot waits, write/deploy-lock waits and owner/visitor execution timeouts.
The durable allowance excludes deployment versions, legacy saved data and transient SQLite WAL.
The website file allowance still applies to each raw-file upload.
Without TRUSTED_PROXY_CIDRS the client address is the last X-Forwarded-For entry from any peer. Keep the app behind a local proxy that appends the transport client address and restrict direct access to the app port; an unproxied install set up that way has no trustworthy IP write limit. With TRUSTED_PROXY_CIDRS set (Kubernetes, a load balancer in front), the header is believed only from those networks and a direct connection from anywhere else is its own client.
Kubernetes sets trusted proxy networks to the cluster pod and node ranges; see Kubernetes. The same client address controls the country gate.
Retirement notices
The operator can recompute the two notice lists without stopping the server:
simple-host migrate-rules --list-recipients --out recipients.json --legacy-log legacy-calls.log
Run with that install’s DB_DSN and DATA_DIR. Run as root for a root-only
report; the output file is replaced with mode 0600 and the terminal prints
only counts. The command is read-only and sends nothing. Omit --legacy-log
when no call log has been collected. Supply only the legacy_storage_call
lines collected since R4-S3 was deployed, including rotated logs. Each running
server logs legacy_storage_call site=<id> at most once per site per UTC day.
This includes sites whose calls store no rows.
presets contains each account holding a site with a SQLite or key-value
resource still using presets, with one description per resource and table
settings included. Conversion of an account’s earlier resources never excludes
its later ones. File buckets are excluded. older_saved_data contains owners
with stored state, collections or declared data, or call-log evidence. It is
empty on Simple Hack, whose older routes already answer 410. Resources held by
an event go in held_by_an_event, with the event label and no mail recipient.
The startup resource total includes these event-held resources too. Recompute
both lists immediately before each send and reminder.
Drafts await the owner’s approval in the notice draft. The site export keeps saved state, declared data and every collection except other people’s Personal records, which it has never contained. Each person’s own Personal records stay in that person’s account export, with every item they submitted on someone else’s site and its data. This does not widen owner access.
STORAGE_SWITCH_AT is an RFC3339 UTC instant. Empty means no date set; startup
refuses a date unless STORAGE_RULES=on. Change the environment in Compose or
storageSwitchAt in Helm values and restart. Setting the date in this release
adds a warning to successful preset saves during the notice period, for SQLite
and key-value resources only. It does not retire routes or change access in
this slice. The server logs the number still using presets once at startup.
Wording comes from JSON templates. To change it without rebuilding, copy
host/app/storage/storage-notice.json from the source into
DATA_DIR/storage-notice.json (outside all site directories), edit it and
restart. The fields are presets, legacy, save_warning, rules_guide and
moving_guide; <DATE>, <RULES_GUIDE_URL> and <MOVING_GUIDE_URL> are filled
from the setting and template data. Site/resource lines in the drafts remain
placeholders for the separately approved send step. Missing override uses the
bundled draft. Invalid JSON stops startup and the recipient command.