* feat(compiler): support registered host-await builtins
Allow hosts to register function names at compile time so that calls to
those names emit HostAwait instructions directly, enabling natural syntax
like fetch(x) instead of __builtin_host_await(x, "fetch").
- Add host_await_builtins map and register_host_await_builtin() to Compiler
- Validate arg_count == 1 and reject reserved __builtin_host_await name
- Extend determine_call_target() resolution: explicit > registered > user > builtin
- Both explicit and registered paths emit identical HostAwait bytecode
- Add compile_from_policy_with_host_await() entry point in rules.rs
- Extended test harness with HostAwaitBuiltinSpec and args assertion
- 9 YAML test cases: suspend/resume, run-to-completion, multiple names,
queue, shadowing, object packing, arg_count rejection, reserved name
rejection, standard builtin override
- Documentation: instruction-set.md, architecture.md
* Update src/languages/rego/compiler/function_calls.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Mark Birger <birgerm@yandex.ru>
* fix(compiler): address PR #667 review feedback on host-await registration
- Compiler::register_host_await_builtin now rejects duplicate, empty,
and whitespace-only names. Previously a duplicate registration would
silently overwrite the existing entry, which could mask the host's
own registration mistakes.
- YAML test cases added: empty registration list as no-op, duplicate
name rejection, empty/whitespace name rejection, out-param (a, out)
calling syntax with a single-arg registered builtin, and mixed
__builtin_host_await + registered builtins in the same policy
consuming from their respective identifier queues.
- Test harness: replace assert_eq! on HostAwait argument mismatch with
anyhow::Error so mismatches propagate through the case reporter
instead of panicking and skipping the harness's normal error path.
- YAML comment fix: "Registration panics" -> "Registration fails with
an error" (registration returns Err, never panics).
Addresses anakrish + Copilot inline review comments on PR #667.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* compiler: split CallTarget::HostAwait into explicit and registered variants
Addresses PR #667 review item #8: at the emit site in
`compile_function_call`, the discrimination between explicit
`__builtin_host_await(arg, id)` and a registered host-awaitable
builtin was being recovered by string-comparing `original_fcn_path`
against `"__builtin_host_await"`. The information was already known
in `determine_call_target` and was being thrown away.
Replace the single `CallTarget::HostAwait` variant with two:
* `ExplicitHostAwait` (unit) — the two-argument call form. The
identifier register comes from the user's second argument.
* `RegisteredHostAwait { identifier: String }` — the one-argument
call form for registered builtins. The identifier is the registered
name and is captured in the variant at recognition time, so the
emit site never re-derives it from the function path.
This removes the magic-string comparison at the emit site (the source
of truth is now `determine_call_target`) and makes both match sites
in `compile_function_call` exhaustive over the two forms — adding a
third host-await form in the future would force a compile error at
every match site instead of silently falling through.
Arities are now hardcoded in the `expected_args` extraction
(`Some(2)` for explicit, `Some(1)` for registered) rather than
carried in the variant; registered builtins are constrained to
`arg_count == 1` at registration time, so there is no per-call
variability to carry.
Bytecode output is unchanged; the full RVM test suite (97 cases) and
the registered_host_await suite (15 cases) pass without modification.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs(compiler): clarify registered host-await intercepts unqualified calls only
PR #667 review (Medium): the docs implied registered host-await names
shadow user functions and builtins unconditionally, but
determine_call_target matches only the bare original_fcn_path. A
package-qualified call such as data.demo.resolve(x) is therefore not
intercepted -- it resolves through the normal path like any other call.
Rather than expand registration to qualified paths (which would let a
registered name leak into every package exposing a same-named rule),
document the unqualified-only behavior and pin it with tests.
- register_host_await_builtin: doc now states only the unqualified call
form is intercepted; qualified calls resolve normally.
- determine_call_target: inline comment explaining the deliberate
original_fcn_path-only match.
- docs/rvm/instruction-set.md: describe qualified-call resolution,
including that builtins have no qualified form.
- tests: cross-package and same-package qualified calls resolve to the
rule; bare-name shadowing of a standard builtin; Unknown-function
outcome when no rule exists at the qualified path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(tests): compare host-await argument without re-running process_value
PR #667 review (Low): the suspendable test harness compared the
host-await argument via process_value(argument), but argument is already
a runtime Value. process_value is a YAML-fixture decoder -- it rewrites
"#undefined" to Undefined, {set!: [...]} to a set, and errors on a
runtime Value::Set. Re-running it on the runtime argument could coerce a
legitimate payload into a fixture sentinel (passing for the wrong
reason) or error outright on sets.
Compare the runtime argument directly against the expected value, which
is already decoded once at YAML load time.
Add a regression case (registered_builtin_suspendable_set_argument) that
passes a set payload: it fails under the old double-processing
("unexpected set in value read from json/yaml") and passes with the fix.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(tests): reject `args:` payload expectations in run-to-completion mode
PR #667 review (Low): a run-to-completion host-await response could carry
an `args:` payload expectation, but RTC execution pre-loads responses and
never surfaces the call argument to the harness, so the expectation was
parsed and silently dropped. A case with `args: "WRONG"` passed as long as
the result matched -- asserting a payload that was never checked.
Reject `args:` for run-to-completion fixtures at load time, directing the
author to suspendable mode where arguments are validated. Also only build
the run-to-completion response vector when the case actually runs in RTC
mode, so a suspendable case using the shared host_await_responses field
with `args:` is not wrongly rejected.
Route the fixture-load error through the same want_error handling used for
compilation errors, and add registered_builtin_run_to_completion_rejects_args
which now fails loudly instead of passing silently.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(compiler): reject host-await builtin names with surrounding whitespace
PR #667 review (Low): register_host_await_builtin rejected all-whitespace
names via name.trim().is_empty(), but accepted padded names like " lookup"
or "lookup ". Those were inserted into host_await_builtins, but Rego
function-call paths produce the trimmed identifier, so a padded
registration could never match -- a silent dead registration.
Reject any name that is not already trimmed (name != name.trim()) in
addition to empty names, and update the error message accordingly. Add
test cases for leading and trailing whitespace.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Signed-off-by: Mark Birger <birgerm@yandex.ru>
Co-authored-by: Mark Birger <markbirger@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add an opaque `Set` newtype paralleling `Object`, living under
`src/value/set/` with the same module structure (`mod.rs` /
`iter.rs` / `serde.rs`). `Set` wraps `BTreeSet<Value>` today but
exposes only a curated surface: `contains`, `insert`, `remove`,
`iter`, `iter_sorted`, `cursor` (resumable), `is_subset`,
`intersection`, `difference`, serde, and a hand-written `Ord`.
The cursor types are re-exported behind the `rvm` feature so the
follow-up `IterationState::Set` swap can land additively.
To free the `Set` name for the new public type, the crate-internal
`BTreeSet as Set` / `HashSet as Set` aliases in `lib.rs` are
renamed to `MapSet`. All in-tree consumers of the old alias are
updated in lockstep.
`Value::Set` is unchanged in this commit (still wraps
`Rc<BTreeSet<Value>>`); the payload swap and call-site migration
ship in the next PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add an opaque Object type for the key→value storage backing
Value::Object. It exposes a small set of methods (get, insert, remove,
iter, iter_sorted, cursor, serde) and keeps the backing store private,
so future representations -- inline small-map, hash-backed, lazy,
arena, FFI-callback -- can plug in without touching the call sites
that name this type.
Nothing in the engine uses Object yet. Value::Object still wraps
Rc<BTreeMap<Value, Value>>; the payload swap and call-site migration
come in the next PR. Object stands on its own unit tests in the
meantime.
docs/value/object.md walks through the design, the precedents it
follows (serde_json::Map, toml::Table, simdjson DOM), and the
concrete workloads the abstraction is meant to unlock.
A matching Set abstraction follows in a separate PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extend the Azure Policy parser to handle complete policyRule and
policyDefinition JSON structures, not just standalone constraints.
Policy rule parser (policy_rule.rs):
- Parse top-level { "if": ..., "then": ... } objects
- Extract effect kind (deny, audit, append, modify, etc.) into typed AST
- Parse "details" structurally when it is an object to pull out
existenceCondition as a first-class Constraint; fall back to opaque
JSON for non-object details (e.g. append array form)
- Detect duplicate/missing keys for "if", "then", "effect", "details"
Policy definition parser (policy_definition.rs):
- Handle both wrapped ARM envelope ({ "properties": { ... } }) and
unwrapped (properties-level keys at top level) forms
- Type-extract displayName, description, mode, metadata, parameters,
and policyRule; everything else goes into extra
- Parse parameter definitions with type, defaultValue, allowedValues,
and metadata; detect duplicate parameter names
- Duplicate key detection throughout
Grammar documentation (docs/azure-policy/azurepolicy.ebnf):
- Add formal EBNF grammar covering policy-rule, then-block,
constraints, conditions, all 19 operators, count expressions,
JSON values, and ARM template expressions
Test harness changes:
- Add parse_level field to YAML test cases: "constraint" (default),
"policy_rule", or "policy_definition"
- Un-skip three parse_errors cases that needed policy_rule-level parsing
- Add policy_rule.yaml with 12 cases covering all 9 effect kinds,
existenceCondition, parameterized effects, complex conditions, and
extra key handling
- Add policy_definition.yaml with wrapped, unwrapped, parameterized,
missing-policyRule, and duplicate-key error cases
Move RVM binary encoding from bincode to postcard and bump the format version. Update test helpers, docs, changelog, and refresh lockfiles after the swap.
Closes#575
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Introduce ExecutionTimer/ExecutionTimerConfig to allow limiting evaluating time.
- To amortize time checking costs, checking interval can be configured via the notion of work units
- A global fallback time limit can be set to universally limit all evaluation in addition to engine level limit setting.
- Implement limnits in interpreter and RVM. In RVM, also handle suspend/resume so that time during pause is not counted.
- Add engine-level APIs to set/clear per-engine timer configuration and apply global fallback defaults.
- Surface execution-time limits through FFI and C# bindings
- Add C# tests and example usage to validate engine overrides, global fallback behavior, and compiled policy enforcement.
- Expand docs for execution-time limit
- Add interpreter YAML cases and VM unit tests for time-limit behavior and deterministic time sources.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat!: add Rego Virtual Machine (RVM) implementation
This commit introduces a register-based virtual machine for executing Rego
policies with bytecode-style instructions. Unlike the existing tree-walking
interpreter, the RVM compiles policies into instruction sequences that operate
on virtual registers, offering better performance and optimization potential.
Core Components:
Instruction Set Architecture:
- Define instruction types for data operations, control flow, and builtins
- Implement instruction parameter encoding and display formatting
- Add instruction parser with comprehensive test coverage
Virtual Machine Engine:
- Register-based execution model with program counter management
- Loop execution supporting iterators, comprehensions, and quantifiers
- Function call handling with argument evaluation and context management
- Rule evaluation with default value resolution and virtual data support
- Arithmetic and comparison operation implementations
Program Representation:
- Program listing builder with instruction sequencing
- Rule tree construction for organizing policy rules
- Binary and JSON serialization for compiled programs
- Recompilation support for program modification
Testing Infrastructure:
- Extensive YAML test suites covering all VM features
- Rust unit tests for VM execution and instruction parsing
- Test suites for loops, comprehensions, builtins, and control flow
BREAKING CHANGE: Introduces new VM execution path alongside interpreter
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* docs: add detailed RVM architecture references
Introduce architecture.md explaining program artifacts, serialization, and runtime subsystems.
Document the full opcode catalog in instruction-set.md, including operands, parameter tables, and outcomes.
Walk through execution flow, stacks, and operational guidance in vm-runtime.md, tying the runtime to the new architecture docs.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- add a dedicated `compiler/destructuring_planner` feature that precomputes binding plans for assignments, parameters, and `some in` expressions
- enrich `ScopeContext` with same-scope tracking, local scheduling hints, and module globals so the planner enforces := shadowing rules without blocking parent scopes
- wire the planner through compiler, hoist, interpreter, and engine paths while updating binding plan variants and adding query traversal helpers for dependency analysis
- document the new planner architecture and ship interpreter regressions that exercise nested destructuring, shadowing, and error reporting
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Add `or` operator to Rego languages. Available via `rego-extensions`
Cargo feature.
If the evaluated lhs value is not false, null or undefined it is returned.
Otherwise rhs is evaluated and returned.
or operator has least precedence, and is left-associative.
closes#314
