Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ The success body contains `reply`, `session_id`, `profile`, `decision_assist`, `

`POST /api/chat/stream` takes the same body and authorization as `/api/chat` and returns `text/event-stream`. Each event is one `data: <JSON>\n\n` record. Implementations may receive:

An active foreground turn can be cancelled with `POST /api/chat/{session_id}/cancel`. The response reports whether an active turn accepted the cancellation request. Already-dispatched remote or external side effects may still complete; the session event journal records cancellation and that uncertainty.

| `event` | Key data | Meaning |
|---|---|---|
| `content` | `chunk` | Plain response text delta; control/tool prefixes are filtered from user-visible chunks. |
Expand Down
1 change: 1 addition & 0 deletions docs/tool-inventory.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ Parameterized names are part of the contract; clients may substitute a value for
| `POST` | `/api/auth/session` |
| `POST` | `/api/chat` |
| `POST` | `/api/chat/stream` |
| `POST` | `/api/chat/{session_id}/cancel` |
| `GET` | `/api/cost` |
| `POST` | `/api/cost/reset` |
| `GET` | `/api/health` |
Expand Down
5 changes: 4 additions & 1 deletion openkyrozen/agent/delegation_runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -316,12 +316,15 @@ def _invoke_delegated(self, run, *, review, feedback, coordinator, report_repair
"cite actual tool receipt IDs. Do not claim unsupported results.")
)
child.interaction.set_mode("ask" if review or report_repair else "agent")
from openkyrozen.agent.turn import CancellationEvents
context = ExecutionContext(model=model, provider=provider, provider_config=config,
capability_token=issue_capability_token("subagent:" + child_id, capabilities),
child_run_id=run["run_id"], coordinator=coordinator,
delegation_budget=coordinator.budgets.get(run["run_id"]),
delegation_budget_run_id=run.get("budget_id", run["run_id"]),
delegation_tools=frozenset(tools), max_steps=profile.max_steps)
delegation_tools=frozenset(tools), max_steps=profile.max_steps,
cancelled=CancellationEvents(self.execution_context.cancelled,
coordinator.cancelled.get(run["run_id"])))
Comment on lines +326 to +327

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Terminate the parent turn when inherited cancellation fires

When a parent is cancelled while _complete_turn is waiting for delegated work, this combined signal can stop a child's provider/tool operation, but the delegation coordinator determines cancellation solely from its own per-run event. Because that event was not set, it records the child as failed, and _complete_turn treats the terminal failure as an ordinary unresolved result and returns normally, causing the cancelled parent turn to emit completion instead of entering _chat_turn's cancellation cleanup. Propagate inherited cancellation into the coordinator run state and make the waiting parent raise cancellation rather than synthesize a normal result.

Useful? React with 👍 / 👎.

turn_token = self._turn_context.set(context)
state_token = self._active_context_state.set(ContextState(model, config.context_window_tokens))
usage_token = self._active_usage_run_id.set(child_id)
Expand Down
3 changes: 3 additions & 0 deletions openkyrozen/agent/executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -545,6 +545,9 @@ def finish(outcome: ToolResult, authorized: bool = False, raw_result: Any = None
while True:
attempt += 1
with self._delegation_tool_access(action, execution_args):
if self.execution_context.cancelled is not None and self.execution_context.cancelled.is_set():
outcome = tool_failure("cancelled", "Tool call was cancelled before execution.")
Comment on lines 547 to +549

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Observe cancellation while waiting for workspace access

This cancellation check runs only after _delegation_tool_access has been entered. If the foreground action is queued behind a sub-agent's overlapping workspace claim, WorkspaceAccess.acquire() may wait indefinitely relative to the cancellation request; for a parent action its callback is always false, and for a child it observes only the coordinator's event rather than the newly inherited parent event. Check the turn signal before entering and pass it into the access wait so cancellation can interrupt lock contention.

Useful? React with 👍 / 👎.

break
if callable(_pre_execution_check) and not _pre_execution_check():
outcome = tool_failure("cancelled", "Tool retry stopped because its task is no longer active.")
break
Expand Down
32 changes: 28 additions & 4 deletions openkyrozen/agent/loop.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from __future__ import annotations

import json
import asyncio
from typing import Any
from openkyrozen.tasks.engine import is_complete, is_terminal
from openkyrozen.agent.modes import InteractionError, is_plan_acceptance
Expand Down Expand Up @@ -131,9 +132,31 @@ def _chat_turn(self, user_input: str, clear_tasks: bool = False, profile: str |
)
if backend := self._fast_used_backend.get():
reply = f"Applied routing advice from {'Jev' if backend == 'jev' else 'Kev'}.\n\n{reply}"
except Exception as exc:
except (Exception, KeyboardInterrupt, asyncio.CancelledError) as exc:
from openkyrozen.providers.errors import ProviderError, ProviderErrorKind
cancelled = (self.execution_context.cancelled is not None
and self.execution_context.cancelled.is_set()) or isinstance(
exc, (KeyboardInterrupt, asyncio.CancelledError)
) or (isinstance(exc, ProviderError) and exc.kind == ProviderErrorKind.CANCELLED)
if cancelled and active_plan:
for task_id in active_plan.get("working_task_ids", []):
index = next((i for i, task in enumerate(self.tasks.tasks)
if task.get("id") == task_id), None)
if index is not None and self.tasks.tasks[index].get("status") not in {"succeeded", "cancelled"}:
self.tasks.set_status(index, "cancelled")
try:
self._interaction_controller.complete_plan(
active_plan, status="cancelled", task_manager=self.tasks,
)
except Exception:
pass
if cancelled:
self._publish_runtime_event("turn.effect_uncertain", {
"reason": "cancelled_during_execution",
"message": "Remote or already-dispatched side effects may still complete.",
})
Comment on lines +153 to +157

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist cancellation effect uncertainty in the session journal

_publish_runtime_event only notifies the current in-process observers; it does not append to the durable event store read by /api/v2/events. Consequently this newly emitted uncertainty notice disappears after a non-streaming turn or process restart, despite the documented promise that the session event journal records the possibility of already-dispatched side effects completing. Append a scoped durable event or conversation entry in addition to publishing the transient runtime event.

Useful? React with 👍 / 👎.

self._record_conversation_entry("turn.outcome", {
"status": "failed", "error_type": type(exc).__name__,
"status": "cancelled" if cancelled else "failed", "error_type": type(exc).__name__,
})
from openkyrozen.agent.compaction import is_context_digest, retain_context_digests
display = conversation.display_messages()
Expand All @@ -142,9 +165,10 @@ def _chat_turn(self, user_input: str, clear_tasks: bool = False, profile: str |
projected = ([digest] if digest else []) + display
self.short_term_memory = retain_context_digests(projected, limit)
self.current_session.messages_compacted = bool(digest) or len(projected) > limit
if active_plan:
if active_plan and not cancelled:
self._interaction_controller.complete_plan(active_plan, status="failed")
self._track_fix_outcome(original_user_input, f"Turn failed: {type(exc).__name__}: {exc}")
if not cancelled:
self._track_fix_outcome(original_user_input, f"Turn failed: {type(exc).__name__}: {exc}")
runtime.turn_end(reply="", success=False,
error=f"{type(exc).__name__}: {exc}", **context)
raise
Expand Down
33 changes: 32 additions & 1 deletion openkyrozen/agent/runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import asyncio
import json
import uuid
import threading
from contextlib import contextmanager
from contextvars import ContextVar
from .models import AgentSession, WorkspaceState
Expand Down Expand Up @@ -283,6 +284,8 @@ def __init__(self, memory, tasks, interaction, workspace):
self._default_session = AgentSession(memory.session_id or "surface:cli", memory, tasks, interaction, workspace)
self._default_turn = ExecutionContext()
self._turn_context = ContextVar("agent_turn", default=None)
self._active_turns = {}
self._active_turns_lock = threading.Lock()
self._session_context = ContextVar("agent_session", default=None)
self._surface_capability_override = ContextVar("agent_surface_capabilities", default=None)
self._conversation_turn_id = ContextVar("conversation_turn_id", default=None)
Expand Down Expand Up @@ -405,6 +408,14 @@ def chat(self, session, message, *, clear_tasks=False, profile=None, memory_cont
kind, payload if isinstance(payload, dict) else {"data": payload}, task_id=task_id,
)
) if task_event_context is not None else None
cancellation = threading.Event()
turn_context = copy.copy(self.execution_context)
from openkyrozen.agent.turn import CancellationEvents
turn_context.cancelled = CancellationEvents(turn_context.cancelled, cancellation)
turn_context_token = self._turn_context.set(turn_context)
Comment on lines +411 to +415

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify exceptions raised under a set signal as cancelled

The inner _chat_turn treats any exception raised while this cancellation signal is set as cancellation, but chat()'s outer terminal-event handler only recognizes KeyboardInterrupt, asyncio.CancelledError, and ProviderError(CANCELLED). Cancellation-aware paths such as delegation checks can raise an ordinary RuntimeError, producing a persisted cancelled outcome and uncertainty event inside _chat_turn followed by a contradictory turn.failed terminal event outside it. Include the active signal state in the outer classification so every cooperative cancellation has one consistent terminal event.

Useful? React with 👍 / 👎.

session_key = (str(getattr(getattr(session, "memory", None), "user_id", "local")), str(session.session_id))
with self._active_turns_lock:
self._active_turns.setdefault(session_key, set()).add(cancellation)
try:
publisher.publish("turn.started", {})
try:
Expand All @@ -428,6 +439,13 @@ def chat(self, session, message, *, clear_tasks=False, profile=None, memory_cont
publisher.publish("turn.completed", {"status": self._turn_completion_status.get()})
return reply
finally:
with self._active_turns_lock:
active = self._active_turns.get(session_key)
if active is not None:
active.discard(cancellation)
if not active:
self._active_turns.pop(session_key, None)
self._turn_context.reset(turn_context_token)
self._stream_event_callback.reset(events)
self._last_response_is_native.reset(native_response)
self._turn_failed.reset(turn_failed)
Expand All @@ -442,6 +460,16 @@ def chat(self, session, message, *, clear_tasks=False, profile=None, memory_cont
self._context_history_seen.reset(history_seen)
self._context_history_projection_ids.reset(history_projection)

def cancel_turn(self, session_id: str, *, user_id: str = "local") -> bool:
"""Request cancellation of the active foreground turn for one session."""
with self._active_turns_lock:
active = tuple(self._active_turns.get((str(user_id), str(session_id)), ()))
if not active:
return False
for cancellation in active:
cancellation.set()
return True

@property
def conversation_history(self):
from openkyrozen.agent.conversation import ConversationHistory
Expand Down Expand Up @@ -543,7 +571,10 @@ def run_command(self, args):
except (OSError, RuntimeError, ValueError) as exc:
return CommandResult(f"Error starting background process: {exc}", False,
failure="execution_error")
return self.current_session.workspace.adapters.run_command(args)
if self.execution_context.cancelled is not None and self.execution_context.cancelled.is_set():
from openkyrozen.tools.models import CommandResult
return CommandResult("Command cancelled before it started.", False, failure="cancelled")
return self.current_session.workspace.adapters.run_command(args, cancelled=self.execution_context.cancelled)

@legacy_text_api
def _read_process(self, args):
Expand Down
9 changes: 9 additions & 0 deletions openkyrozen/agent/turn.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
"""State shared by the phases of a single turn."""
from dataclasses import dataclass


class CancellationEvents:
def __init__(self, *events):
self.events = tuple(event for event in events if event is not None)

def is_set(self):
return any(event.is_set() for event in self.events)
from typing import Any

@dataclass
Expand Down Expand Up @@ -53,6 +61,7 @@ class ExecutionContext:
delegation_budget_run_id: str | None = None
delegation_tools: frozenset[str] | None = None
max_steps: int | None = None
cancelled: Any = None


def scoped_turn(function):
Expand Down
2 changes: 2 additions & 0 deletions openkyrozen/interfaces/tui/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,8 @@ def stop(self) -> None:
if self._stopping.is_set():
return
self._stopping.set()
session = runtime.current_session
runtime.cancel_turn(session.session_id, user_id=session.memory.user_id)
coordinator = getattr(runtime.subagent_manager, "coordinator", None)
if coordinator:
coordinator.close()
Expand Down
6 changes: 5 additions & 1 deletion openkyrozen/interfaces/tui/commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,11 @@ def _command(self, name: str, args: Any, request_id: str) -> None:
else:
command = raw.lower().replace("-", "_")
arg_text = args if isinstance(args, str) else ""
if command in {"/quit", "/exit", "quit", "exit", "shutdown"}:
if command in {"/cancel", "cancel"}:
session = runtime.current_session
cancelled = runtime.cancel_turn(session.session_id, user_id=session.memory.user_id)
self.emit("response", request_id, text="Cancellation requested." if cancelled else "No active turn to cancel.")
Comment on lines +21 to +24

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Wake pending approvals when cancelling a TUI turn

When a TUI turn is waiting at a high-impact tool approval, _approval() is blocked on its separate waiter for up to APPROVAL_TIMEOUT_SECONDS (15 minutes). This command only sets the turn cancellation event, so even a programmatic /cancel request leaves the worker and UI busy until the user answers the approval or the timeout expires; the cancellation check in _run_tool cannot run until that wait returns. Cancellation should also deny and signal approvals belonging to the active turn, or the approval wait should observe the turn signal.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep the TUI busy until the cancelled turn actually exits

Once /cancel is allowed during a busy turn, emitting its acknowledgement as a normal response makes the Go client's response handler replace the current streaming assistant placeholder and immediately set m.busy=false (tui/events.go:138-162). The cancelled worker may still be unwinding, especially for cooperative or external operations, so the UI can accept another submission that the backend rejects as busy and the eventual terminal event can be attributed to an already-finalized placeholder. Send a nonterminal status/ack event here and let the original turn's terminal event clear the busy state.

Useful? React with 👍 / 👎.

elif command in {"/quit", "/exit", "quit", "exit", "shutdown"}:
self.stop()
elif command in {"/provider", "provider"}:
if isinstance(args, Mapping) and args.get("provider"):
Expand Down
8 changes: 8 additions & 0 deletions openkyrozen/interfaces/web/chat.py
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,14 @@ async def api_chat(self, request: Request):
"interaction": session["interaction"], "context": session.get("context")}


async def api_cancel_chat(self, session_id: str, request: Request):
service = self
session_id = service._normalise_session_id(session_id)
user_id = service._actor_for_request(request)
return {"session_id": session_id,
"cancelled": service._agent.cancel_turn(session_id, user_id=user_id)}
Comment on lines +437 to +442

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Run non-streaming chat off the ASGI event loop

For a turn started through POST /api/chat, api_chat is asynchronous but calls the synchronous _run_session_chat directly, blocking the process's event loop for the entire turn. Consequently this new cancellation endpoint cannot be scheduled on the same worker until the turn has already finished; sending it to another multi-process worker does not help because _active_turns is process-local. Dispatch the non-streaming chat work to a thread, as the streaming endpoint already does, so cancellation requests can run concurrently.

Useful? React with 👍 / 👎.



async def api_chat_stream(self, request: Request):
service = self
"""SSE streaming chat endpoint."""
Expand Down
3 changes: 2 additions & 1 deletion openkyrozen/interfaces/web/service.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ def __init__(self, application=None):
self.app.get('/api/v2/decision-assist', dependencies=[Depends(self.require_api_access)])(self.decision_assist_status)
self.app.post('/api/chat', dependencies=[Depends(self.require_api_access)])(self.api_chat)
self.app.post('/api/chat/stream', dependencies=[Depends(self.require_api_access)])(self.api_chat_stream)
self.app.post('/api/chat/{session_id}/cancel', dependencies=[Depends(self.require_api_access)])(self.api_cancel_chat)
self.app.get('/api/memory', dependencies=[Depends(self.require_api_access)])(self.api_memory)
self.app.get('/api/v2/memory', dependencies=[Depends(self.require_api_access)])(self.api_v2_memory)
self.app.get('/api/v2/memory/claims', dependencies=[Depends(self.require_api_access)])(self.api_v2_memory_claims)
Expand Down Expand Up @@ -147,7 +148,7 @@ def CHAT_HTML(self):

from openkyrozen.interfaces.web.tasks import (_execute_durable_task, _task_manager, _task_worker, _task_scopes, _recover_task_scopes, _run_task_worker_cycle, _run_scheduled_job, api_v2_tasks, api_v2_create_task, api_v2_resume_task, api_v2_schedules, api_v2_create_schedule, api_v2_disable_schedule)

from openkyrozen.interfaces.web.chat import (_normalise_session_id, _normalise_profile, _normalise_memory_actor, _normalise_memory_context, _initialise_session_defaults, _set_memory_context, _actor_for_request, _get_or_create_session, _interaction_for_session, _history_for_session, _ensure_history_baseline, _public_history_node, _apply_chat_controls, _run_session_chat, _validate_message, _server_capabilities, _allowed_server_tools, _cost_summary, _cost_report, _json_object, api_chat, api_chat_stream)
from openkyrozen.interfaces.web.chat import (_normalise_session_id, _normalise_profile, _normalise_memory_actor, _normalise_memory_context, _initialise_session_defaults, _set_memory_context, _actor_for_request, _get_or_create_session, _interaction_for_session, _history_for_session, _ensure_history_baseline, _public_history_node, _apply_chat_controls, _run_session_chat, _validate_message, _server_capabilities, _allowed_server_tools, _cost_summary, _cost_report, _json_object, api_chat, api_chat_stream, api_cancel_chat)

from openkyrozen.interfaces.web.pages import (chat_page, fast_diagnostics, decision_assist_status, pwa_manifest)

Expand Down
11 changes: 9 additions & 2 deletions openkyrozen/providers/calls.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,16 @@ def _provider_timeout_seconds(self) -> float:

def _child_cancellation(self):
context = self.execution_context
cancelled = getattr(context, "cancelled", None)
child = None
if context.child_run_id and context.coordinator:
return context.coordinator.cancelled.get(context.child_run_id)
return None
child = context.coordinator.cancelled.get(context.child_run_id)
if cancelled is not None and child is not None:
from openkyrozen.agent.turn import CancellationEvents
return CancellationEvents(cancelled, child)
if child is not None:
return child
return cancelled


def _bounded_provider_call(self, callback: Any) -> Any:
Expand Down
2 changes: 1 addition & 1 deletion openkyrozen/tools/ports.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
class ToolPort(Protocol):
tool_registry: ToolRegistry
AVAILABLE_TOOLS: MutableMapping[str, Callable[[str], str]]
def run_command(self, args: str | dict) -> CommandResult: ...
def run_command(self, args: str | dict, *, cancelled=None) -> CommandResult: ...
def new_process_manager(self, owner_id: str, workspace_id: str, session_id: str): ...
def set_workspace_root(self, root: str) -> None: ...
def set_project_graph(self, graph) -> None: ...
Expand Down
35 changes: 33 additions & 2 deletions openkyrozen/tools/shell.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def _validated_shell_options(self, args: str | dict):
return cmd, timeout, str(working_directory), env, background


def run_command(self, args: str | dict) -> CommandResult:
def run_command(self, args: str | dict, *, cancelled=None) -> CommandResult:
"""
Execute a shell command and retain its machine-readable outcome.
"""
Expand All @@ -87,7 +87,38 @@ def run_command(self, args: str | dict) -> CommandResult:
start_new_session=(os.name != "nt"),
)
try:
out, err = process.communicate(timeout=timeout)
deadline = started + timeout
while True:
if cancelled is not None and cancelled.is_set():
raise InterruptedError
try:
out, err = process.communicate(timeout=min(0.1, max(0.01, deadline - time.monotonic())))
break
except subprocess.TimeoutExpired:
if time.monotonic() >= deadline:
raise
except InterruptedError:
if os.name == "nt":
subprocess.run(["taskkill", "/PID", str(process.pid), "/T", "/F"],
capture_output=True, timeout=5, check=False)
else:
try:
os.killpg(process.pid, signal.SIGTERM)
except ProcessLookupError:
pass
try:
out, err = process.communicate(timeout=2)
except subprocess.TimeoutExpired:
if os.name != "nt":
try:
os.killpg(process.pid, signal.SIGKILL)
except ProcessLookupError:
pass
out, err = process.communicate()
return CommandResult("Error: command cancelled", False, process.returncode,
"cancelled", out, err, time.monotonic() - started, True,
sandbox_backend=invocation.backend,
Comment on lines +118 to +120

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not mark cancelled commands as timed out

The positional True passed here populates CommandResult.timed_out, so every command stopped by cancellation is serialized with both failure="cancelled" and timed_out=true. Consumers and receipts therefore cannot distinguish a user cancellation from deadline expiry even though the command was terminated before its timeout. Leave timed_out false for this path, preferably by using named arguments.

Useful? React with 👍 / 👎.

sandbox_reason=invocation.unavailable_reason)
except subprocess.TimeoutExpired:
if os.name == "nt":
subprocess.run(["taskkill", "/PID", str(process.pid), "/T", "/F"],
Expand Down
18 changes: 18 additions & 0 deletions tests/test_process_lifecycle.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,27 @@
import unittest
import shlex
import tempfile
import threading


class ProcessLifecycleTests(unittest.TestCase):
def test_foreground_shell_cancellation_terminates_process_group(self):
from openkyrozen.tools import ToolAdapters
with tempfile.TemporaryDirectory() as root:
adapters = ToolAdapters(root)
cancelled = threading.Event()
result = []
worker = threading.Thread(target=lambda: result.append(adapters.run_command({
"command": shlex.join([sys.executable, "-c", "import time; time.sleep(30)"]),
"timeout_seconds": 30,
}, cancelled=cancelled)))
worker.start()
time.sleep(.2)
cancelled.set()
worker.join(5)
self.assertFalse(worker.is_alive())
self.assertEqual(result[0].failure, "cancelled")

def test_runtime_exposes_session_process_tools_and_application_close_stops_process(self):
from pathlib import Path
from unittest.mock import patch
Expand Down
Loading
Loading