Skip to content

fix (DMSHelpers): keep the per-VO singleton through copies and pickles - #8764

Merged
fstagni merged 1 commit into
DIRACGrid:integrationfrom
ryuwd:fix/dmshelpers-copyable
Aug 28, 2026
Merged

fix (DMSHelpers): keep the per-VO singleton through copies and pickles#8764
fstagni merged 1 commit into
DIRACGrid:integrationfrom
ryuwd:fix/dmshelpers-copyable

Conversation

@ryuwd

@ryuwd ryuwd commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Since v9.1.16, copy.deepcopy() or pickle.dumps() of any Request object raises:

TypeError: cannot pickle '_thread.lock' object

Minimal reproduction on v9.1.16:

import copy
from DIRAC.RequestManagementSystem.Client.Request import Request

copy.deepcopy(Request())  # TypeError: cannot pickle '_thread.lock' object

Works fine on v9.1.15.

Cause: b11d7d9 (feat(DMSHelper): cache some methods) added per-instance cachetools caches guarded by real threading.Locks in DMSHelpers.__init__. Request.__init__ stores a DMSHelpers on every instance (self.dmsHelper = DMSHelpers()), so the locks are now reachable from every Request's object graph, and deepcopy/pickle fail via __reduce_ex__. The same applies to anything else holding a DMSHelpers instance.

This is how we found it: LHCbDIRAC's CI installs the latest DIRAC integration tag, and every pipeline started after the v9.1.16 tag now fails Test_Modules_LHCbScript, which deepcopies a workflow_commons fixture containing a Request (nothing LHCbDIRAC-side changed between the green v9.1.15 run and the red v9.1.16 run: https://gitlab.cern.ch/roneil/LHCbDIRAC/-/jobs/81172151).

Fix: since 4c4260f DMSHelpers is a per-VO singleton, so copies should never duplicate it in the first place — __copy__/__deepcopy__ return the shared instance, and __reduce__ pickles as "look up the per-VO singleton again on load" (the vo key is now kept on the instance). Besides restoring copyability, this stops a copy from silently forking a singleton's caches and CS-derived state.

Tests cover copy/deepcopy/pickle identity plus the in-the-wild regression (deepcopy of an object embedding a helper, and of a real Request).

BEGINRELEASENOTES

*DataManagement
FIX: DMSHelpers instances stay per-VO singletons through copy, deepcopy and pickle, restoring deepcopy/pickle of objects that embed one (e.g. RMS Request)

ENDRELEASENOTES

Since the method caches gained instance-level locks, deepcopying or
pickling any object holding a DMSHelpers — e.g. an RMS Request, which
stores one in __init__ — raised TypeError: cannot pickle
'_thread.lock' object.

Copies now return the shared instance and pickles reduce to the
per-VO constructor lookup, which both restores copyability and stops
copies from silently forking a singleton's caches and CS-derived
state.
@ryuwd
ryuwd marked this pull request as ready for review August 28, 2026 13:12
@fstagni
fstagni merged commit bfa74ed into DIRACGrid:integration Aug 28, 2026
21 of 23 checks passed
@DIRACGridBot DIRACGridBot added the sweep:ignore Prevent sweeping from being ran for this PR label Aug 28, 2026
@ryuwd
ryuwd deleted the fix/dmshelpers-copyable branch August 28, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sweep:ignore Prevent sweeping from being ran for this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants