Speed up socketcan capture_message by ~32% - #2097
Open
TheTrueAI wants to merge 1 commit into
Open
Conversation
TheTrueAI
force-pushed
the
perf/socketcan-hoist-msg-dontroute
branch
from
September 1, 2026 01:17
6285ed2 to
90a9123
Compare
`socket.MSG_DONTROUTE` is an `enum.IntFlag`, so masking it built a `MsgFlag` per frame; it is now a module-level int. `Message` is built positionally to skip keyword binding, with a new test pinning the order. Live 500 kbit/s bus (aarch64, CPython 3.13.15): 107.8 -> 73.3 us CPU per frame; output identical over 3000 real and 1995 synthetic frames. Refs hardbyte#1135
TheTrueAI
force-pushed
the
perf/socketcan-hoist-msg-dontroute
branch
from
September 1, 2026 01:21
90a9123 to
48bb561
Compare
TheTrueAI
marked this pull request as ready for review
September 1, 2026 01:26
Contributor
|
Tick the box to add this pull request to the merge queue (same as
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
capture_messageruns once per received frame, and two lines in it cost more thanthe rest of the decode combined. This removes both. No API or behaviour change.
socket.MSG_DONTROUTEto a module-level plainint. It is anenum.IntFlagmember, somsg_flags & socket.MSG_DONTROUTEconstructed aMsgFlaginstance on every frame.Messagepositionally instead of with twelve keyword arguments;binding them per frame is measurable.
SocketCANHotPathTesttotest/test_socketcan.py, pinning theMessage.__init__argument order so a future reordering cannot silently corruptreceived frames, and asserting the hoisted constant still carries the same value.
Net effect on a live 500 kbit/s bus: 107.8 -> 73.3 us of CPU per frame, a ~32%
reduction. Full A/B and correctness evidence under Additional Notes.
Related Issues / Pull Requests
Related to #1135
At the ~10k f/s of a busy 1 Mbit bus, stock
capture_messageneeds 1057 ms of CPUper second of bus time — more than a whole core just to decode, before any user code
runs. That is a concrete mechanism behind the "python can't keep up" reports in
#1135. This brings it to 733 ms/s. It does not close that issue, but it is a real
part of it.
Type of Change
hot path. Behaviour-preserving; verified field-for-field against the previous
implementation (see below).
Checklist
tox).Additional Notes
Measurements
Live 500 kbit/s bus at 1444 frames/s, aarch64 (Cortex-A53) / CPython 3.13.15 /
python-can 4.6.1. CPU time via
time.process_timeso blockingrecvmsgwaits areexcluded. 2500 frames x 3 rounds, median. Each row changes exactly one thing against
a verbatim copy of the current implementation:
capture_messageMSG_DONTROUTEhoisted to intMessage()End to end with this branch applied, same bus: 107.8 -> 73.3 us/frame.
For scale, the bare
recvmsgsyscall is 31 us/frame, so this removes ~45% of thePython-side work above the syscall. At 1444 f/s
capture_messagedrops from 15.3%to 10.4% of one core.
A tight-loop microbenchmark of the
&alone reports only 12.8 us(
13.2 usIntFlag vs0.5 usint), i.e. it understates the in-situ saving of26.3 us. A cache-thrash variant widens the microbenchmark gap (13.2 -> 15.8 us),
so the enum machinery's cold per-frame working set appears to be part of it. I could
not fully account for the remainder; the in-situ A/B is the number I would trust.
Correctness
Compared against the current implementation on identical inputs, all twelve
Messagefields plus their types:
get_channelbothFalseandTrue— 0 mismatches.MSG_DONTROUTEset (798 cases, i.e.is_rx=False)and clear, standard and extended IDs, remote and error frames, classic and FD MTU,
BRS/ESI, every valid DLC and the
len8_dlc9..15 range — 0 mismatches.The live bus only produces standard non-FD
is_rx=Trueframes, so the synthetic setis what actually exercises the changed line.
Things I measured and deliberately did not include
CAN_FD_DLCas afrozensetfor thedata_len not in can.util.CAN_FD_DLCtest in
dissect_can_frame. Looks like a 0.68 us win in isolation (1.03 -> 0.35 us)but measured -0.41 us, i.e. nothing, in situ: classic-CAN
data_lenis 0..8 andhits within the first nine list entries.
asserts on the ancillary data: worth only 1.0 us,not enough to justify the behaviour discussion.
recvmsg_intowith a preallocated buffer: 2.9 us/frame slower.bytes->bytearraycopy:4.6 us/frame slower.