riscv: implement multikernel lifecycle and safe shutdown - #39
Draft
pro-utkarshM wants to merge 8 commits into
Draft
riscv: implement multikernel lifecycle and safe shutdown#39pro-utkarshM wants to merge 8 commits into
pro-utkarshM wants to merge 8 commits into
Conversation
Wire CONFIG_MULTIKERNEL into the 64-bit RISC-V build and add sparse hart ID translations. Reject the invalid hart sentinel before lookup so it cannot alias an unused logical CPU slot. Declare the generic contiguous-allocation and memory-hotplug dependencies so CONFIG_MULTIKERNEL cannot expose an unbuildable configuration. Reserve the architecture control block for the spawn context, DTB and entry stub. Provide safe stubs for the full architecture interface so the functional SBI HSM, Image loader and doorbell work can land incrementally. Signed-off-by: Nikolay Nikolaev <nicknickolaev@gmail.com>
The instance restore and ring handoff paths consume the live OF tree. Without CONFIG_OF, x86 multikernel configurations compile references to OF globals that cannot link and cannot restore a spawn at runtime. Reject that unusable configuration in Kconfig. Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
RISC-V parks CPUs in firmware with SBI HSM, so multikernel must reuse the architecture's HART_START, HART_STOP and HART_STATUS wrappers. Make those helpers available to RISC-V architecture code while retaining the existing SBI-to-Linux errno mapping. Add only the per-instance context and entry-stub bookkeeping required by the lifecycle implementation. Hart IDs remain physical firmware identifiers and are never used as array indexes. Link: multikernel#23 Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
OpenSBI does not invalidate a stopped hart's instruction cache when HART_START restarts it, and an SBI remote fence cannot target a stopped hart. Reusing an Image address can therefore execute stale instructions. Add an immutable, relocation-free entry stub whose first instruction is fence.i and whose target comes from the adjacent data page. Publish its address through the multikernel manifest and route spawn-kernel secondary starts through it as well as the primary start. The stub preserves a0 and a1, so the primary receives its DTB and secondaries receive their normal SBI boot data. Flush the local instruction cache before every HART_STOP so a later start cannot retain an older stub line. Link: multikernel#24 Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Implement the RISC-V multikernel CPU lifecycle with direct SBI HSM calls. A spawn is allowed only after HART_STATUS reaches STOPPED; STARTED, START_PENDING and STOP_PENDING are polled with a bounded timeout. Allocate the immutable entry stub and mutable target context from the instance control block, publish the stub in the manifest, and start the physical hart with the Image entry and DTB boot ABI. Use physical hart IDs for host doorbells and confirm firmware stop state before release or memory reclaim. RISC-V deliberately reports force-stop as unsupported because HSM has no remote HART_STOP operation. Link: multikernel#23 Link: multikernel#24 Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Prime every assigned hart through an immutable host-text fence.i trampoline before it fetches a rewritten instance stub or Image. Preserve distinct HSM state errors for bounded lifecycle handling. Link: multikernel#23 Link: multikernel#24 Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
Reuse the kexec lock for CPU, memory and device transfers so an Image rewrite cannot race with resource mutation. Recover an active instance only after every assigned hart is confirmed stopped. Link: multikernel#23 Link: multikernel#26 Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
A spawned kernel must never invoke SBI SRST or a legacy shutdown because those operations can reset the host. Detect the spawn handoff before reset registration and suppress host-wide reset handlers. Route halt, poweroff, restart, SMP stop and panic shutdown through local HART_STOP. Cache the parent endpoint for a non-allocating panic notification, set a safe spawn panic default, and prevent memory reuse until every assigned hart is confirmed stopped. Link: multikernel#25 Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
physical hart IDs, bounded
HART_STATUSpolling, and parked-state checksfence.ion every start whilepreserving the RISC-V
a0/a1boot ABIHART_STOP, and serialize instance resource changes against teardownAddresses #23, #24, and #25.
Dependency and attribution
Draft: depends on #37 and must not merge first.
The branch temporarily contains
19cc8c6a6fb7, a replay of NikolayNikolaev's #22 foundation commit with his original authorship. It is present
only so this dependent series can build and be reviewed. Once #37 merges, I
will rebase onto updated
master, drop that replay, rerun validation, andforce-update this draft. The foundation commit is not part of my authorship.
Scope
This PR intentionally keeps #23-#25 together because lifecycle, entry-cache
coherency, and safe shutdown share the same HSM handoff invariants.
#26 is maintained as a separate follow-up branch. RISC-V instance-DTB device
filtering (#27) and firmware IPI receive support (#28) are out of scope.
Known pre-merge work
An independent pre-draft review identified two lifecycle races that must be
resolved before this leaves draft status:
out, so a hart that may already be executing the spawn can never be omitted
from parked-state checks
MK_SYS_HALTEDsettlement with CPU-set mutation andkexec teardown before traversing the instance CPU set or changing state
The
HART_STARTinvalid-parameter diagnostic also needs to identify a possibleSBI-domain denial as required by #23. This draft is being opened for early
design review, not as a merge-ready submission.
Validation
vmlinux Imagebuild withCONFIG_MULTIKERNEL=yat7fd9938de88fgit diff --checkand strictcheckpatch.plover every commit (only thegeneric new-file/MAINTAINERS prompt was suppressed)
virt+ OpenSBI multi-hart runtime validation on the stacked riscv: KEXEC_TYPE_MULTIKERNEL Image loader, placement in the grant, no purgatory #26 testbranch, including spawn halt, power-off, reboot, panic, and host survival
both harts reached
HART_STOPbefore the instance returned toreadyriscv: KEXEC_TYPE_MULTIKERNEL Image loader, placement in the grant, no purgatory #26 test branch
AIA runtime validation remains blocked on #27's per-hart IMSIC/APLIC and
device filtering. MKTTY/ring validation remains blocked on #28; neither is
duplicated here.