Skip to content

Stop the child processes of an exec function on timeout - #4800

Open
Nikshay1 wants to merge 2 commits into
kptdev:mainfrom
Nikshay1:fix/exec-fn-kill-process-group-4799
Open

Nikshay1 wants to merge 2 commits into
kptdev:mainfrom
Nikshay1:fix/exec-fn-kill-process-group-4799

Conversation

@Nikshay1

Copy link
Copy Markdown
Contributor

Description

When an exec function times out, kpt now also stops the processes that the function started. Before, kpt killed only the function, and a process that the function had started (for example sleep in a shell script) kept running after kpt had returned.

On Linux and macOS (//go:build unix):

  • An exec function runs in a process group of its own. When the timeout passes, kpt kills the whole group. This is also done when the function itself has already exited and a child still has its output open. Before, kpt waited for such a child with no time limit.
  • The function is no longer in the process group of kpt, so it does not receive the signals of the terminal, such as Ctrl-C. While a function runs, kpt therefore forwards SIGINT, SIGTERM, SIGHUP and SIGQUIT to the group, and then sends the signal to itself, so that the signal ends kpt as before. A signal that was ignored when kpt started, for example SIGHUP under nohup, stays ignored.

On other platforms nothing changes.

In ExecFn.Run only one line changes: cmd.Run() becomes runCmd(ctx, cmd). The rest is in the new files exec_unix.go and exec_other.go.

Motivation

Fixes #4799. The issue came out of the review of #4798.

Notes for reviewers

These are the limits of this approach. I listed the first three in #4799 and asked there if they are acceptable.

  • Functions that read from the terminal. A function that reads from /dev/tty, for example to ask for a passphrase, is now stopped with SIGTTIN, because its process group is not the foreground group of the terminal. It then waits until the timeout. I reproduced this under a pty. Fixing it means giving the terminal to the function the way a shell does, which is much more code.
  • Ctrl-Z suspends kpt, but the function keeps running.
  • A child that leaves the process group, for example with setsid, is still not stopped.
  • kill <kpt pid>. SIGTERM that is sent to kpt alone used to leave the function running. It is now forwarded, so the function is stopped too.
  • Container functions are not changed.
  • Add --timeout flag to kpt fn render and kpt fn eval #4798 changes the same function. The conflict is one line, and I will rebase whichever PR is merged second.

Testing

  • TestExecFn_TimeoutStopsChildProcesses: a function that waits for its child, and a function that has exited and left a child with its output open. In both cases the child is stopped when the timeout passes.
  • TestExecFn_ForwardsSignal: a signal that kpt receives reaches the function, and is then sent to kpt itself.
  • Without the fix, the first case fails, the second case hangs, and the signal test ends the test binary.
  • I ran a built binary of main and of this branch, and sent the signals to the process group of kpt, as a terminal does:
Case Exit status of kpt, main and this branch Function still running on main Function still running on this branch
Ctrl-C while the function runs 130 no no
Ctrl-C after the function has exited and a child has its output open 130 no no
Ctrl-\ while the function runs 2 no no
kill <kpt pid> 143 yes no
  • Locally, with Go 1.26.8: go test for pkg/fn/runtime, thirdparty/kyaml/runfn and commands/fn/..., the new tests with -race, and go build for Linux and Windows all pass.

AI usage

I have used AI in the creation of this PR.

I have used the following AI tools:

  • Claude Code (Claude Opus 5.5) to write the change and the tests
  • Codex (gpt-6-astra) to review the change

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Nikshay <nikshayyadav90@gmail.com>
@Nikshay1
Nikshay1 requested review from a team and a balanced review from Copilot October 12, 2026 17:18
@Nikshay1
Nikshay1 requested a review from liamfallon as a code owner October 12, 2026 17:18
@netlify

netlify Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for kptdocs ready!

Name Link
🔨 Latest commit 588a962
🔍 Latest deploy log https://app.netlify.com/projects/kptdocs/deploys/6acd188a6edefab4295b26aa
😎 Deploy Preview https://deploy-preview-4800--kptdocs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A Unix signal-completion race can swallow a termination signal instead of ending kpt.

1 open finding
What changed in this PR

This PR ensures timed-out exec functions terminate their Unix process groups, preventing child-process leaks.

Changes:

  • Adds Unix process-group cleanup and signal forwarding.
  • Preserves existing behavior on non-Unix platforms.
  • Adds timeout and signal-forwarding tests.
File Description
pkg/​fn/​runtime/​exec.go Delegates command execution to the platform-specific helper.
pkg/​fn/​runtime/​exec_unix.go Implements process-group termination and signal forwarding.
pkg/​fn/​runtime/​exec_unix_test.go Tests child cleanup and signal forwarding.
pkg/​fn/​runtime/​exec_other.go Retains standard execution on non-Unix platforms.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pkg/fn/runtime/exec_unix.go Outdated
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Nikshay <nikshayyadav90@gmail.com>

@mozesl-nokia mozesl-nokia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will be honest, I am not very keen on using os-specific code, but I doubt there is a much better way to do it.

Is there something we could do for Windows/DOS-based as well in a follow up? Even if we don't currently support Windows...

@sonarqubecloud

Copy link
Copy Markdown

@Nikshay1

Nikshay1 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks @mozesl-nokia! On Windows, terminating a Job Object on timeout should stop the child processes too. Opened #4820 for it :)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exec function timeout does not stop the child processes of the function

3 participants