Skip to content

refactor: ponytail audit cleanup (-836 lines, -2 direct deps) - #98

Merged
JustMaris merged 11 commits into
mainfrom
refactor/ponytail-audit
Sep 27, 2026
Merged

JustMaris merged 11 commits into
mainfrom
refactor/ponytail-audit

Conversation

@JustMaris

@JustMaris JustMaris commented Sep 27, 2026 •

Copy link
Copy Markdown
Member

Applies the ponytail-audit findings, one commit per finding. The branch removes 836 lines net across 28 files and two direct dependencies (goquery with its cascadia, and google/uuid, which stays only as a transitive dependency of modernc sqlite).

Cuts

  • CAPTCHA detection: one detection script instead of eight detector functions. CaptchaInfo keeps only Found and Type; the other fields were never read. Same checks in the same order. The new TestDetectCaptcha has one Chrome fixture per branch. It skips locally without Chrome and runs on CI's Ubuntu runner. I also ran all 9 fixtures in a real browser and they pass.
  • Restart-resume of paused web jobs removed: JobPersistence, PersistentJobState, checkPendingJob, resumePendingJob and saveJobProgress are gone. A paused job's remaining list is exactly what "Send all" or automation picks up next anyway. The Brokers page wording is updated, and leftover pending_job*.json files are deleted at startup.
  • One inbox scan path: Monitor.ScanAndStore and inbox.RecordReply replace the separate implementations in the CLI monitor, the web scan and the web rescan.
  • One send step: email.SendRemoval does render → send → history record for CLI send, the web "Send all" job and the single-broker send.
  • History store: one scanner and one column list for broker responses instead of five copies, and GetBrokerResponses builds its WHERE clause instead of carrying four query variants.
  • Small ones: RateLimiter loses a cleanup goroutine it didn't need (the key set is fixed), email.SequenceKey context plumbing is gone, goquery is replaced by an x/net/html tokenizer, uuid by crypto/rand.Text().

Bugs fixed along the way

  • Web scans could archive the wrong mail. They archived the UIDs of emails fetched from the archive folder too. ArchiveEmails applies UIDs to INBOX, where an archive-folder UID can name an unrelated message. Only INBOX UIDs are archived now.
  • Message-ID: sends now carry a real Message-ID: <random@sender-domain> header, and that's what gets recorded. Before, no header was sent and the recorded ID was a local placeholder (%!d(<nil>) for web sends).
  • Pipeline status: new replies found by the web scan and by monitor --watch now advance the broker's pipeline status, as eraser monitor already did.
  • Unstyled messages: the scan, rescan, reclassify, setup and send-status HTML fragments used Tailwind classes that aren't defined anywhere (there's no Tailwind build), so they rendered unstyled. They now use the layout's .alert, .badge, .btn and text-error/text-warning classes.
  • Rescan body backfill: the "fill the body only if it's empty" check always saw an empty body, because the lookup query didn't select the body. It now does.

Behaviour change to note: with auto_archive on, a rescan now archives INBOX replies, the same as a scan.

Verified: go vet ./... && go test ./... pass, and -race on web, inbox and history. New tests cover SendRemoval (sent, failed, and render error), the Message-ID header matching the recorded ID (fake SMTP server), RecordReply (new / seen / reclassified, pipeline advance, profile attribution), HTML href extraction, and the CAPTCHA fixtures. A dry-run eraser auto --once cycle completes.

BEGIN_COMMIT_OVERRIDE
fix(inbox): web scans no longer archive unrelated INBOX messages
fix(email): send and record a real Message-ID header
fix(inbox): new replies from web scans and monitor --watch advance pipeline status
fix(web): scan, setup and send-status messages render with the app's styles
refactor: ponytail audit cleanup (-836 lines, -2 direct deps)
END_COMMIT_OVERRIDE

The recorded MessageID was a local 'smtp-<to>-<n>' string (and
'%!d(<nil>)' for web sends, which never set the sequence context
value); the actual message had no Message-ID header, so the provider
assigned one we never saw. Sends now carry <random@sender-domain> and
record that, and the email.SequenceKey context plumbing is gone.
CaptchaInfo keeps only Found and Type; Confidence, FrameSrc,
ElementID and the per-detector Description were never read (the
description table overrode it). Same checks, same order - covered by a
new Chrome fixture test per branch.
A job paused by the daily cap used to be saved to pending_job*.json and
resumed on the next 'serve' start. Its remaining list is exactly what
'Send all' (eligible) or an automated cycle picks up next anyway, and it
needed a history re-check to avoid double-sending. Removes
JobPersistence, PersistentJobState, checkPendingJob, resumePendingJob
and saveJobProgress. Leftover pending_job*.json files are ignored.
email.SendRemoval renders the request, sends it and returns the history
record; CLI send, the web Send all job and the single-broker send keep
only their own loop policy (cap, pause, auth cutoff, output).
Monitor.ScanAndStore (fetch, classify, store new, advance pipeline,
archive) and RecordReply (one reply, used by --watch) replace three
copies. Along the way:
- web scans archived the UIDs of archive-folder emails too; ArchiveEmails
  applies them to INBOX, where they can name unrelated messages. Only
  INBOX UIDs are archived now.
- new replies found by the web scan and by --watch now advance the
  broker's pipeline status, as 'eraser monitor' already did.
- scan/rescan/reclassify/setup HTML fragments used Tailwind classes that
  exist nowhere (no Tailwind build); they use the layout's .alert,
  .badge, .btn and text-error/text-warning classes now.
The Brokers page still promised that remaining emails continue when the
app restarts. Also removes leftover pending_job*.json files, and makes
FindBrokerResponseBySubject return the body so the rescan's
'fill body only if empty' check does what it says.
@JustMaris JustMaris changed the title refactor: ponytail audit cleanup (-850 lines, -2 direct deps) refactor: ponytail audit cleanup (-836 lines, -2 direct deps) Sep 27, 2026
@JustMaris
JustMaris merged commit 7251c39 into main Sep 27, 2026
5 checks passed
@JustMaris
JustMaris deleted the refactor/ponytail-audit branch September 27, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant