regorus/docs/limits/execution_time.md

Execution Time Limiting Design

Goal

Introduce configurable execution-time limits so every evaluation stays bounded. Policy service can configure these limits to ensure that a single runaway policy never degrades overall availability.

Design Goals

• Support execution limits per engine. Also support configuring a global fallback execution limit.
• Use best time sources automatically in std builds but provide ability to set the time source in no-std builds and for deterministic testing.
• Since computing elapsed time is relatively expensive, provide ability to control how often it is computed during evaluation.

Prevalent Approaches

The following approaches are used by various execution engines.

Cooperative Tick Budgets

Evaluations burn through a configurable “fuel” counter one work unit at a time. When fuel hits zero the engine raises an error, forcing callers to top up or abandon the run.

• Examples: Wasmtime fuel, Wasmer metering, Lua debug hooks
• Pros: Precise control over work units; compatible with no_std targets
• Cons: Requires instrumentation at every checkpoint; budget choice affects responsiveness

Wall-Clock Watchdogs

Hosts schedule a real-time deadline alongside the policy evaluation. When a timer fires, the engine aborts or interrupts, guaranteeing a hard wall-clock cap.

• Examples: V8 termination handler, SpiderMonkey interrupt callback, PostgreSQL statement_timeout
• Pros: Tracks real elapsed time; easy to reason about deadlines
• Cons: Needs host timers or threads; harder to support in no_std environments, and Rust threads cannot be force-cancelled so watchdogs must coordinate cooperative shutdown

Scheduler Time-Slicing

Policies run inside a cooperative scheduler that yields after fixed work slices. The host can deprioritize or cancel long tasks while allowing others to continue.

• Examples: Erlang BEAM reductions, .NET ThreadPool throttling
• Pros: Isolates runaway workloads by design; integrates with host schedulers
• Cons: Significant state bookkeeping; higher overhead for frequent yields, and feasible for VM control loops but practically impossible for the interpreter without a deep rewrite

Regorus Approach

Regorus adopts the cooperative tick-budget model. In std builds, the best time source is used without any special configuration. In no_std builds (there is no standard time source) and test builds (which need to control the time source to avoid flakiness), the time source can be configured via a trait instance.

Instead of checking elapsed time at each Evaluations accrue work units via tick, only reading the clock once the configured check_interval elapses, which balances responsiveness with low overhead when limits stay off.

• Closest Prevalent Approach: Cooperative tick budgets
• Rationale: Runs on std and no_std targets by swapping time sources, keeps tests deterministic, avoids watchdog threads or complex schedulers, and applies uniformly across evaluation modes without deep architectural changes.

Configuration Model

ExecutionTimerConfig

ExecutionTimerConfig lives in utils::limits::time and contains:

• limit: Duration — enforced wall-clock budget.
• check_interval: NonZeroU32 — number of work units between timer checks.

Work Units

The timer does not prescribe what constitutes a single “work unit.” Instead, each caller chooses a granularity that matches its execution model:

• The interpreter treats each scheduling step (e.g., evaluating a single statement or expression in a rule) as a unit.
• The VM typically reports one instruction per unit.

This abstraction keeps the timer flexible while still guaranteeing that, regardless of the unit definition, the timer observes elapsed wall-clock time at predictable checkpoints controlled by check_interval.

Interpreters treat the absence of an ExecutionTimerConfig as "no limit". When a configuration is present, callers can increase check_interval to amortize the cost of frequent checks.

Global Fallback vs Engine Overrides

• set_fallback_execution_timer_config installs a process-wide fallback stored behind a spin mutex. Engines without an explicit override consult this value before every evaluation.
• Each engine holds an optional execution_timer_config. When Engine::set_execution_timer_config is invoked, the engine stores the provided configuration and applies it to the interpreter immediately. Clearing the override via Engine::clear_execution_timer_config restores reliance on the global fallback.
• Engines default to no time limit. Newly created engines apply the effective configuration (engine override or global fallback or default) during construction so that any first evaluation honors the expected budget.

Effective Configuration Lifecycle

Before any evaluation entry point (query, rule, compilation), the engine:

1. Computes the effective configuration via execution_timer_config.or_else(fallback_execution_timer_config).
2. Applies it to the interpreter using apply_effective_execution_timer_config, which resets the timer to ensure a fresh window.
3. Proceeds with evaluation, relying on interpreter checkpoints to enforce the deadline.

This approach guarantees that changing the global fallback impacts both new and existing engines on their next evaluation, while engine overrides remain isolated.

ExecutionTimer Behavior

ExecutionTimer maintains four fields:

• config: the active ExecutionTimerConfig, if any.
• start: optional start instant.
• accumulated_units: tracks work units until the next check.
• last_elapsed: caches the most recent elapsed duration.

Key Operations

• start(now) records the baseline instant and clears accumulated counters.
• tick(work_units, now) increments the accumulator and triggers check_now when the accumulator reaches check_interval. If no configuration is installed, the function returns early with Ok(()).
• check_now(now) computes elapsed time, updates last_elapsed, and returns LimitError::TimeLimitExceeded when elapsed > limit.
• elapsed(now) reports elapsed time without mutating state, enabling diagnostics and tests.

Because ticks only perform the expensive comparison after the configured interval, callers can tune check_interval to their workloads.

Time Sources

To avoid direct dependencies on Instant, the timer expects callers to supply a monotonic Duration via monotonic_now() or custom sources.

• On std builds, StdTimeSource captures a single Instant per process (via OnceLock) and reports elapsed durations. This keeps time monotonic and stable across threads.
• Tests and no_std builds can install overrides through set_time_source, which stores an &'static dyn TimeSource in a spin mutex. YAML tests use this hook to provide deterministic timestamps, ensuring repeatable limit violations.

If no source is available (e.g., no_std without an override), monotonic_now returns None; the interpreter treats this as “time limiting unavailable,” effectively disabling checks.

Interpreter Integration

The interpreter carries an ExecutionTimer. Evaluation steps integrate with the timer as follows:

1. prepare_for_eval applies the effective configuration and calls reset on internal state.
2. At key checkpoints (rule scheduling, loop iterations, query evaluation steps) the interpreter:
   • Calls monotonic_now to fetch the current time (when available).
   • Invokes tick(1, now) to check for limit violations.
3. When LimitError::TimeLimitExceeded is returned, the interpreter converts it into an error surface consistent with existing APIs (e.g., anyhow::Error on Rust, host-specific exceptions on bindings).

Because the interpreter amortizes clock reads via check_interval, the overhead remains low even with many evaluation steps.

Compiled Policy Integration

Compiled policies (VM paths) share the interpreter’s timer via apply_effective_execution_timer_config. Before VM execution begins, the engine ensures the VM’s interpreter state reflects the current timer configuration and resets any per-evaluation state.

• VM loops call tick with the number of instructions executed since the last check (commonly 1).
• Helper functions responsible for longer-running host interactions (e.g., print gathering) may call check_now to enforce the deadline before crossing the FFI boundary.

This shared timer model avoids duplicate configuration state and maintains consistent semantics across evaluation modes.

Public API Surface

The design exposes these primary methods:

• Engine::set_execution_timer_config(config: ExecutionTimerConfig) stores a per-engine override and reapplies it immediately, ensuring the next evaluation enforces the new limits.
• Engine::clear_execution_timer_config() removes the override and reverts to the global fallback.
• set_fallback_execution_timer_config(config: Option<ExecutionTimerConfig>) installs an optional global fallback. Passing None clears it.
• fallback_execution_timer_config() -> Option<ExecutionTimerConfig> returns the currently active fallback for diagnostics.

Documentation highlights that engines default to no time limit, global settings provide a quick way to protect all engines, and overrides preempt the global value until cleared.

Testing Strategy

• Unit Tests in utils::limits::time verify timer configuration handling, tick behavior, limit enforcement, and custom time sources.
• Integration Tests (YAML) configure deterministic time sources and assert that engine-level and global configurations interact correctly (override precedence, clearing behavior, fresh windows per evaluation).
• Binding Tests (planned) will demonstrate that FFI surfaces propagate timer errors.

Each test resets global configuration and time sources via RAII guards to avoid cross-test interference.

Operational Guidance

• Choose conservative check_interval values (e.g., 1–10) for latency-sensitive workloads to catch runaway loops quickly. Larger intervals reduce overhead but increase the window between checks.
• When applying global limits in multi-tenant services, consider setting per-engine overrides for trusted workloads that need higher budgets.
• Combine with monitoring of last_elapsed to understand how close evaluations come to their deadlines.

Pros and Cons of the Current Design

Pros

• Low overhead via amortized checks: check_interval keeps clock reads cheap while bounding elapsed time.
• Works on std + no_std: TimeSource abstraction enables std Instant or user-provided clocks.
• Simple API surface: One config struct, plus global fallback and per-engine override.
• Suspend-aware VM: suspendable execution snapshots elapsed time and resumes from that value, so suspended time is not counted.

Cons

• Best-effort when no time source: If monotonic_now() returns None, time limits effectively disable for that run.
• Granularity depends on check_interval: Large intervals can overshoot the limit before the next check fires.
• Cooperative gaps: Any long-running host work outside evaluation loops is not accounted for.
• Global fallback is process-wide: Requires coordination in tests or multi-tenant hosts.

Binding Surface (C# / FFI)

Regorus exposes execution timers in bindings through a thin FFI layer:

• FFI struct: RegorusExecutionTimerConfig uses limit_ns + check_interval and validates non-zero intervals.
• Engine methods: per-engine SetExecutionTimerConfig / ClearExecutionTimerConfig map to regorus_engine_set_execution_timer_config / regorus_engine_clear_execution_timer_config.
• Fallback: static Engine.SetFallbackExecutionTimerConfig / ClearFallbackExecutionTimerConfig map to regorus_set_fallback_execution_timer_config and regorus_clear_fallback_execution_timer_config.

These bindings preserve Rust semantics and propagate the LimitError message (“execution exceeded time limit”) up to host exceptions.

Comparison: Cancellation Token Approach

A cancellation token design would add an explicit “stop now” signal that evaluators check at the same checkpoints used for time limiting. This has value for host‑initiated cancellation, but it is not a replacement for the current time‑based approach.

• Semantics: Tokens require the host to decide when to cancel; the current design enforces wall‑clock budgets inside Regorus.
• Portability: Tokens are portable, but time‑based cancellation would require a watchdog or scheduler to flip the token at a deadline.
• Threads: A watchdog typically implies background threads or an async runtime. We explicitly want to avoid introducing threads inside Regorus for simplicity, no_std compatibility, and predictability.
• Performance: Frequent timer‑thread ticks or per‑evaluation scheduling can add overhead for microsecond‑scale evaluations. The current cooperative checks amortize clock reads and avoid extra threads, keeping steady‑state costs low.

For these reasons, Regorus keeps the cooperative time‑limit checks as the primary mechanism and treats cancellation tokens (if added) as an optional, host‑driven complement rather than a replacement.

Future Work

• Expose per-evaluation overrides (e.g., request-scoped budgets) to complement global and engine-level configuration.
• Surface telemetry events whenever limits are hit, providing elapsed time at breach for observability pipelines.
• Investigate dynamic adjustment of check_interval based on observed evaluation patterns to balance overhead and responsiveness.