Skip to content

Deploy the keyword archiver: shim, config, systemd instance and a secrets file - #216

Merged
mikelangmayr merged 1 commit into
mainfrom
mike/keygrabber-deployment
Sep 18, 2026
Merged

mikelangmayr merged 1 commit into
mainfrom
mike/keygrabber-deployment

Conversation

@mikelangmayr

@mikelangmayr mikelangmayr commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Deploys libby's keygrabber on the HISPEC host. It records the other daemons' keywords into InfluxDB so Grafana has something to draw; it drives no hardware and owns no device.

  • Add daemons/generic/keygrabber, a four-line shim. The daemon itself lives in libby, since nothing about it is instrument-specific, but the unit template runs daemons/<subdir>/<script>, so it needs a file here to be launched like any other instance
  • Add config/hispec/hispec_keygrabber.yaml covering all 25 deployed peers, including the three Xeryon instances from Add a Xeryon motion daemon for the FAMs and the PIAA deploy stages #217, grouped by how fast they actually change: motion at 5s, filter wheels and thermal at 30s, cryo and vacuum at 60s. That is 158 requests per minute across the fleet, one per peer per tick
  • Every collection uses keywords: ["%"] rather than a hand-listed set. Patterns resolve against the live peer, so this needs no per-daemon keyword knowledge and picks up keywords as daemons gain them. Narrowing later is a config edit plus libby modify hispec.keygrabber.reload=1
  • Add systemd/instances/hispec_keygrabber.env
  • Add a root-only /etc/hispec/secrets.env, created by install.sh at 0600 root:root and read by every unit via an optional second EnvironmentFile. /etc/hispec/instances/*.env is mode 2775 and group-readable by every hispec-ops member, which is correct for configs and wrong for a database write token. systemd reads the file as root before dropping to User=hispec, so it never needs to be readable by anyone else
  • Depend on libby[influxdb], since this repo now ships a daemon that needs it
  • Document both in systemd/README.md: where secrets go, and the archiver's two extra deployment steps, plus that pausing it needs no restart. Note the keygrabber in the README structure table as the one daemon that drives no hardware

Verified against merged libby rather than by eye: the config parses through the real parse_config, all 25 collections satisfy the interval-versus-timeout rule, every referenced peer has a config tracked on main and every deployed peer is covered by a collection, and the default exclusions drop uptime and lasterror while keeping error. The shim runs, and a live run against a real peer collected samples and answered its control keywords over the wire.

That live run also found two libby bugs, both since fixed and merged in CaltechOpticalObservatories/libby#37: isconnected reported healthy while every write failed, and a failing collection logged an error every interval with no backoff.

@mikelangmayr
mikelangmayr force-pushed the mike/keygrabber-deployment branch from 2af36cb to ae695ae Compare September 18, 2026 20:36

@prkrtg prkrtg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mikelangmayr
mikelangmayr merged commit aa78c5d into main Sep 18, 2026
4 checks passed
@mikelangmayr
mikelangmayr deleted the mike/keygrabber-deployment branch September 18, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants