Skip to content

Add the keygrabber control keywords and a testable scheduler - #34

Merged
mikelangmayr merged 1 commit into
mainfrom
mike/keygrabber-control
Sep 17, 2026
Merged

mikelangmayr merged 1 commit into
mainfrom
mike/keygrabber-control

Conversation

@mikelangmayr

Copy link
Copy Markdown
Collaborator

Stacked on #33. Gives the keygrabber a runtime control surface, and extracts the scheduler so its logic is testable and the daemon's attribute-count disable can go.

  • Extract Scheduler: a due-time heap with the in-flight set, deliberately free of threads, pools and sleeping. It answers "what should run now" and nothing else, so the ordering and the skip rule are tested against an injected clock instead of by racing real threads. Twelve unit tests, none of which wait
  • Fix a rescheduling bug the extraction exposed: the old max(due_at + interval, now) always resolved to due_at + interval, so the "never schedule into the past" comment described behaviour the code did not have. Rescheduling now measures from the claim, so a sixty-second stall on a one-second cadence yields one tick rather than sixty that can only skip
  • Add the control keywords: enabled, isconnected, pointswritten, readerrors, writeerrors, queuedepth, skippedticks, droppedbatches, reload and shutdown, plus <collection>.enabled, .interval, .lastsample and .lag
  • No getter blocks. They are answered on the transport's receive thread, the same thread that delivers replies to the reader threads, so isconnected reports a flag the writer thread maintains rather than pinging the database, and writing it only requests a reconnect that the writer thread performs. A getter that blocked would time out every read in flight, which is why there is a test asserting the control keywords answer while ticks are running
  • Writing <collection>.interval re-applies the same headroom rule the config loader enforces, and reschedules only that collection rather than re-seeding every due time and setting the whole fleet reading at once
  • Add reload: re-reads the config file and applies it, forcing a re-resolve so a changed keyword selection is picked up without waiting out the old refresh window. A file that fails to parse leaves the running collections untouched and reports why. It refuses to add or remove collections rather than half-applying, because libby cannot withdraw a keyword and a new collection's control keywords could not appear without a restart
  • Document all of it, including that % does not match across the dot in <collection>.enabled, so listing the control surface needs hispec.keygrabber.%.%
  • 224 tests pass, pylint 10.00/10, -W docs build verified. The control keywords are exercised over both transports, thirteen end-to-end cases each

Separately, these tests surfaced a pre-existing socket leak in ZmqTransport, unrelated to this change. Discovery.stop() only sets a flag and never joins its thread, so up to discover_interval_s after Libby.stop() has closed every socket, that thread can still broadcast, find _dealers empty but _book populated, and create a DEALER nothing will close. Worth its own small fix: have send return early once _stop is set.

@mikelangmayr
mikelangmayr force-pushed the mike/keygrabber-control branch from c145885 to 3eb22bd Compare September 17, 2026 20:57
Base automatically changed from mike/keygrabber-collector to main September 17, 2026 22:29

@prkrtg prkrtg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mikelangmayr
mikelangmayr merged commit 6719d75 into main Sep 17, 2026
3 checks passed
@mikelangmayr
mikelangmayr deleted the mike/keygrabber-control branch September 17, 2026 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants