Skip to content

fix(mysql): add lock_wait_timeout to DSN to stop statement pile-up - #423

Closed
jabbrwcky wants to merge 0 commit into
crossplane-contrib:masterfrom
jabbrwcky:fix/mysql-lock-wait-timeout
Closed

jabbrwcky wants to merge 0 commit into
crossplane-contrib:masterfrom
jabbrwcky:fix/mysql-lock-wait-timeout

Conversation

@jabbrwcky

@jabbrwcky jabbrwcky commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Description of your changes

Under a backlog of pending ALTER USER/GRANT/DDL statements, MySQL/Percona can become unresponsive. go-sql-driver/mysql cancels a context by closing the TCP socket without sending KILL QUERY (intentional; all versions incl. latest v1.10.0). With MySQL's default lock_wait_timeout of 1 year, a statement blocked on a metadata/ACL lock (e.g. behind a backup or long transaction) stays pending server-side even after the 60s reconcile deadline. Every retry adds another; account-management statements serialize on a single global ACL lock, so they pile up until max_connections is exhausted or the server wedges.

This PR appends lock_wait_timeout=30 and dial timeout=10s to the MySQL DSN so blocked statements fail fast server-side and release instead of accumulating. Applies to all MySQL controllers (User, Grant, Database; cluster + namespaced) via the shared client.

Not covered / follow-ups:

See docs/mysql-driver-context-cancellation.md for the full analysis.

Fixes #422

I have:

  • Read and followed Crossplane's contribution process.
  • Run make reviewable to ensure this PR is ready for review.

How has this code been tested

  • Updated DSN assertions; added TestDSNParsesWithLockWaitTimeout validating the DSN parses via the driver and carries the params.
  • go build ./..., go vet, and full MySQL controller test suites pass.
  • make and make reviewable pass.

@jabbrwcky
jabbrwcky force-pushed the fix/mysql-lock-wait-timeout branch 2 times, most recently from a3c8bc3 to b067c97 Compare July 23, 2026 12:58
@jabbrwcky jabbrwcky closed this Aug 18, 2026
@jabbrwcky
jabbrwcky force-pushed the fix/mysql-lock-wait-timeout branch from b067c97 to fe39828 Compare August 18, 2026 12:04
@jabbrwcky

Copy link
Copy Markdown
Contributor Author

Superseded by #435 (same branch got wiped by a bad rebase, GitHub wouldn't let this PR reopen after recovery).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MySQL: pending ALTER USER/GRANT statements pile up and make the server unresponsive

1 participant