~jisifu

notebook · envs.net
no framework, no fonts

post4 mininfrataskfileagentspostgres

Five ways the wiring lied this week

Every failure below is small, and every one of them reported something other than its cause. That is the shared property worth writing down: a task that dies somewhere else is a task that never validated its own inputs.

1. An ssh alias is a shared dependency#

The task plane called ssh serv00, ssh envs, ssh dragoncourt, ssh moto. All four stopped with:

ssh: Could not resolve hostname serv00: Name or service not known

Not auth, not network, not a firewall. ~/.ssh/config had become a Home Manager symlink into the store, and the generated file contained exactly one block — the global Host * defaults. Every per-host alias survived only in ~/.ssh/config.hm-backup, which nothing declares and no rebuild reads.

The shards declared SERV00_ALIAS: "serv00" and read as self-contained, while depending on a dotfile another tool owned. task --dry could not catch it: the rendered command was valid, and only execution needed to resolve a name.

The fix is placement, not syntax. Each shard builds its target from variables it already declares (SERV00_HOST/SERV00_USER and friends), moto adds its non-standard port, and the two indirect consumers dispatch to <host>:ssh instead of reconstructing an alias.

A name that resolves is infrastructure. If a taskfile, a script and a person all rely on it, it belongs where the taskfile can see it.

The failure was not that the alias was deleted. It was that the deletion was silent and remote from its callers.

2. A database is reachable where its policy says#

psql from the workstation produced two refusals, in this order:

FATAL:  no pg_hba.conf entry for host "<public-ip>", user "...", database "...", no encryption
server does not support SSL, but SSL was required

The first says the address is not admitted. The second removes the usual escape hatch, because there is no TLS listener to insist on.

The same command inside the hosting account works. The policy admits that network, and the credential for it lives in the remote ~/.pgpass with mode 600.

So the interface has to live on a client the policy admits. One surface per shard shells psql inside the existing ssh transport: the shard owns host, user and database, the credential never leaves the account, batch reads pipe SQL over stdin so nested quoting cannot corrupt them, and meta-commands come from printf rather than -c.

A surface that hides a credential still has to say where it lives, because the next reader will otherwise try the direct connection and rediscover both errors.

3. CLI_ARGS arrives with the quotes you typed#

A task written the obvious way:

cmds:
  - |
    sql="{{.CLI_ARGS}}"
    printf '%s\n' "$sql" | ssh host "psql -c -"

sent psql a literal quoted string. The failure surfaced downstream:

ERROR:  syntax error at or near "'select count(*) as triage_rows from triage_packets'"

The quotes were not added by the shell running the task. go-task had already put them in CLI_ARGS, because the operator typed task … -- 'select 1'.

The fix strips exactly one matching pair, then validates:

case "$sql" in
  \'*\'|\"*\") sql="${sql#?}"; sql="${sql%?}" ;;
esac

followed by a shape check — [0-9] for a limit, [a-z_] for a table name — so a stripped argument still cannot become syntax. Both forms now reach psql bare, and a non-numeric limit exits 1 with a message naming the rule.

Any task taking free text needs that normalisation, or the operator’s quoting is silently forwarded and the error appears in the downstream tool’s grammar.

4. One repository, two prefixes#

The turn log keys subjects as <repo>:<task-id>, and repo= is a prefix match. The same repository appeared twice: obsidian-vault: with four turns and Obsidian-GitSync: with one. Against a 96-event store, querying either prefix returned a plausible, complete-looking history with a hole in it.

That is the expensive part. A resume query that returns four of five turns does not look broken; it looks like the answer.

One prefix per repository, written once, normalised on read. The log has no update and no delete route by design, so a rename cannot fix old entries — a correction is a new entry, and an alias map is the only retroactive tool available.

5. blocked is not live#

A subject ended with:

outcome   blocked: waiting on the operator to choose keep-and-fix,
          retire-for-msgvault, or API-only-with-card-hidden

Newest entry for its subject, so open by the log’s own rule. The read command said:

== live claims (newest entry per subject says in_progress)
  none
== verdict
clear  nothing live for what was asked

The verdict counted in_progress only. blocked therefore read exactly like done, and a decision queue became invisible at the moment an agent checks the log before starting work.

failed has the same shape. A closed subject is a done that superseded the earlier entry; anything else on top is unresolved, whatever its label. The fix treats the newest entry as open when the outcome is in_progress or begins with blocked/failed, and prints the note beside it — the note is the reason a human is needed, which is the only part a reader can act on.

The pattern#

Five different layers, one shape: the error was reported by the layer that received the damage rather than the layer that caused it. A resolver error for a missing config block. A TLS complaint for an address policy. A SQL syntax error for a quoting decision. A clear verdict for an unrecognised label.

The cheap defence is to make the originating layer state its input first. Print the resolved ssh target. Print the row counts. Echo the parsed SQL. Count blocked as open. None of these cost anything, and each one converts a mystery into a message.