mirror of
https://github.com/microsoft/regorus.git
synced 2026-08-05 02:16:11 +00:00
f0acc64195ac3cc92f33c1dd107380400c3d2a8c
23 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Mark Birger
|
f0acc64195 |
feat(compiler): support registered host-await builtins for natural function call syntax (#667)
* feat(compiler): support registered host-await builtins Allow hosts to register function names at compile time so that calls to those names emit HostAwait instructions directly, enabling natural syntax like fetch(x) instead of __builtin_host_await(x, "fetch"). - Add host_await_builtins map and register_host_await_builtin() to Compiler - Validate arg_count == 1 and reject reserved __builtin_host_await name - Extend determine_call_target() resolution: explicit > registered > user > builtin - Both explicit and registered paths emit identical HostAwait bytecode - Add compile_from_policy_with_host_await() entry point in rules.rs - Extended test harness with HostAwaitBuiltinSpec and args assertion - 9 YAML test cases: suspend/resume, run-to-completion, multiple names, queue, shadowing, object packing, arg_count rejection, reserved name rejection, standard builtin override - Documentation: instruction-set.md, architecture.md * Update src/languages/rego/compiler/function_calls.rs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Mark Birger <birgerm@yandex.ru> * fix(compiler): address PR #667 review feedback on host-await registration - Compiler::register_host_await_builtin now rejects duplicate, empty, and whitespace-only names. Previously a duplicate registration would silently overwrite the existing entry, which could mask the host's own registration mistakes. - YAML test cases added: empty registration list as no-op, duplicate name rejection, empty/whitespace name rejection, out-param (a, out) calling syntax with a single-arg registered builtin, and mixed __builtin_host_await + registered builtins in the same policy consuming from their respective identifier queues. - Test harness: replace assert_eq! on HostAwait argument mismatch with anyhow::Error so mismatches propagate through the case reporter instead of panicking and skipping the harness's normal error path. - YAML comment fix: "Registration panics" -> "Registration fails with an error" (registration returns Err, never panics). Addresses anakrish + Copilot inline review comments on PR #667. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * compiler: split CallTarget::HostAwait into explicit and registered variants Addresses PR #667 review item #8: at the emit site in `compile_function_call`, the discrimination between explicit `__builtin_host_await(arg, id)` and a registered host-awaitable builtin was being recovered by string-comparing `original_fcn_path` against `"__builtin_host_await"`. The information was already known in `determine_call_target` and was being thrown away. Replace the single `CallTarget::HostAwait` variant with two: * `ExplicitHostAwait` (unit) — the two-argument call form. The identifier register comes from the user's second argument. * `RegisteredHostAwait { identifier: String }` — the one-argument call form for registered builtins. The identifier is the registered name and is captured in the variant at recognition time, so the emit site never re-derives it from the function path. This removes the magic-string comparison at the emit site (the source of truth is now `determine_call_target`) and makes both match sites in `compile_function_call` exhaustive over the two forms — adding a third host-await form in the future would force a compile error at every match site instead of silently falling through. Arities are now hardcoded in the `expected_args` extraction (`Some(2)` for explicit, `Some(1)` for registered) rather than carried in the variant; registered builtins are constrained to `arg_count == 1` at registration time, so there is no per-call variability to carry. Bytecode output is unchanged; the full RVM test suite (97 cases) and the registered_host_await suite (15 cases) pass without modification. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(compiler): clarify registered host-await intercepts unqualified calls only PR #667 review (Medium): the docs implied registered host-await names shadow user functions and builtins unconditionally, but determine_call_target matches only the bare original_fcn_path. A package-qualified call such as data.demo.resolve(x) is therefore not intercepted -- it resolves through the normal path like any other call. Rather than expand registration to qualified paths (which would let a registered name leak into every package exposing a same-named rule), document the unqualified-only behavior and pin it with tests. - register_host_await_builtin: doc now states only the unqualified call form is intercepted; qualified calls resolve normally. - determine_call_target: inline comment explaining the deliberate original_fcn_path-only match. - docs/rvm/instruction-set.md: describe qualified-call resolution, including that builtins have no qualified form. - tests: cross-package and same-package qualified calls resolve to the rule; bare-name shadowing of a standard builtin; Unknown-function outcome when no rule exists at the qualified path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(tests): compare host-await argument without re-running process_value PR #667 review (Low): the suspendable test harness compared the host-await argument via process_value(argument), but argument is already a runtime Value. process_value is a YAML-fixture decoder -- it rewrites "#undefined" to Undefined, {set!: [...]} to a set, and errors on a runtime Value::Set. Re-running it on the runtime argument could coerce a legitimate payload into a fixture sentinel (passing for the wrong reason) or error outright on sets. Compare the runtime argument directly against the expected value, which is already decoded once at YAML load time. Add a regression case (registered_builtin_suspendable_set_argument) that passes a set payload: it fails under the old double-processing ("unexpected set in value read from json/yaml") and passes with the fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(tests): reject `args:` payload expectations in run-to-completion mode PR #667 review (Low): a run-to-completion host-await response could carry an `args:` payload expectation, but RTC execution pre-loads responses and never surfaces the call argument to the harness, so the expectation was parsed and silently dropped. A case with `args: "WRONG"` passed as long as the result matched -- asserting a payload that was never checked. Reject `args:` for run-to-completion fixtures at load time, directing the author to suspendable mode where arguments are validated. Also only build the run-to-completion response vector when the case actually runs in RTC mode, so a suspendable case using the shared host_await_responses field with `args:` is not wrongly rejected. Route the fixture-load error through the same want_error handling used for compilation errors, and add registered_builtin_run_to_completion_rejects_args which now fails loudly instead of passing silently. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compiler): reject host-await builtin names with surrounding whitespace PR #667 review (Low): register_host_await_builtin rejected all-whitespace names via name.trim().is_empty(), but accepted padded names like " lookup" or "lookup ". Those were inserted into host_await_builtins, but Rego function-call paths produce the trimmed identifier, so a padded registration could never match -- a silent dead registration. Reject any name that is not already trimmed (name != name.trim()) in addition to empty names, and update the error message accordingly. Add test cases for leading and trailing whitespace. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: Mark Birger <birgerm@yandex.ru> Co-authored-by: Mark Birger <markbirger@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Copilot
|
9b42239327 |
Expand keyword-in-ref coverage for complex parser edge cases (interpreter + RVM) (#744)
* Initial plan * Add keywords_in_refs: allow reserved keywords as dot-notation field names * Address review feedback: improve parse_ref_field doc comment and clean up test comment * Add complex keyword-in-ref test cases * Polish keyword-ref test expectations and validate coverage --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> |
||
|
Anand Krishnamoorthi
|
ed6ae465b0 |
refactor(value): migrate Value::Object to Object storage abstraction (#736)
Builds on #57. Swap Value::Object's payload from Rc<BTreeMap<Value, Value>> to Rc<Object> and migrate all call sites to the Object API. as_object / as_object_mut keep their names but return &Object / &mut Object. The mutable accessor handles Rc::make_mut internally, so callers no longer do it themselves. Object grows into_value() and From<Object> for Value. Value's serializer now delegates to Object::serialize, dropping a duplicate non-string-key stringification path. RVM IterationState::Object is rewritten around ObjectCursor: O(log n) steps over a shared Rc<Object>, no eager pair snapshot. Snapshot independence is preserved by Rc copy-on-write; setup_next_iteration advances the cursor inline and advance() becomes a no-op for this variant. A new iteration_state_object_is_snapshot_independent_of_source test covers CoW against a mutated alias. Value::Set still wraps Rc<BTreeSet<Value>>; the matching Set abstraction and its swap ship in follow-up PRs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Copilot
|
dae3052781 |
fix(interpreter,rvm): correct partial object rule iteration and classification (#718)
Partial object rules with dynamic keys (e.g. `violations[k] if { ... }`)
only produced a single entry instead of collecting all bindings. Two
independent bugs caused this:
1. Interpreter: the early-return optimization in eval_output_expr_in_loop
checked whether the rule_ref was constant but never verified whether
the key expression was also constant. A variable key like `k` was
treated as constant output, causing the loop to exit after the first
iteration. Fixed by gating early-return on key_expr constness.
2. RVM: compute_rule_type incorrectly classified `p[k] if { ... }` as
PartialSet instead of PartialObject. OPA v1 semantics define this
form as a partial object (key -> true). Fixed the classification and
added compiler error guards for patterns the RVM codegen cannot yet
handle (constant keys, nested bracket keys), ensuring graceful
fallback to the interpreter.
The OPA test harness now skips RVM validation per-case when partial
object compiler errors are raised, rather than blanket-skipping entire
folders. This preserves RVM coverage for unrelated tests in the same
folders.
Closes #712
Co-authored-by: Anand Krishnamoorthi <anakrish@microsoft.com>
|
||
|
Anand Krishnamoorthi
|
87f22a79ca |
fix: harden regex builtins with compiled-size limit (#705)
Add a 100KB cap on compiled regex NFA size via RegexBuilder::size_limit() to block patterns that blow up in memory or CPU. Regex compilation now goes through a single helper (compile_regex_for_builtin) so the limit is enforced consistently across all regex builtins. While doing this, found and fixed a pre-existing bug: resource-limit errors (time, memory, instruction count) raised inside builtins were quietly swallowed to Undefined when strict_builtin_errors was off (the default). This is a problem because `not regex.match(...)` would see Undefined and flip to true -- silently wrong. The same issue now applies to the new regex size limit. Fixed by teaching the three error-absorption paths (interpreter builtin call, RVM builtin dispatch, and RVM rule-execution loop) to recognize LimitError and let it propagate instead of eating it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Anand Krishnamoorthi
|
4d35744c4f |
feat(rvm): implement Azure Policy condition evaluation (#661)
Add VM support for Azure Policy's condition operators and allOf/anyOf short-circuit logic, gated behind cfg(feature = "azure_policy"). Policy conditions (equals, contains, like, match, exists, and their negations — 21 total) are encoded as a single PolicyCondition instruction with a PolicyOp sub-opcode rather than bloating the Instruction enum with 21 variants. The dispatch handles Azure Policy's quirky comparison semantics: case-insensitive string comparison, string↔number coercion, null vs undefined distinction, and element-wise collection membership. allOf/anyOf blocks use four instructions — LogicalBlockStart, AllOfNext/AnyOfNext, and LogicalBlockEnd — that wire up a result register and short-circuit on the first failing (allOf) or passing (anyOf) child. Helper functions for case-folded comparison, wildcard/glob matching, and type coercion live in builtins::azure_policy::helpers. Two YAML test suites (~2200 lines) exercise the full operator matrix and the allOf/anyOf control flow. |
||
|
Mark Birger
|
83ce8c3580 |
Fix RVM evaluation of default-only rules (#664)
Default-only rules (e.g., `default deny := true` with no conditional body) returned Undefined in the RVM instead of the default value. Compiler: - compute_rule_type: return Complete when rule exists only in default_rules map - compile_worklist_rule: emit register slots and data-tree entries for default-only rules (else branch) VM: - execute_call_rule_common + execute_call_rule_suspendable: check default_literal_index before returning Undefined when definitions is empty Tests: - 3 new RVM cases (default_rules.yaml): bool, object, entry-point - 3 new interpreter cases (default/basic.yaml): matching coverage Co-authored-by: Mark Birger <markbirger@microsoft.com> |
||
|
Anand Krishnamoorthi
|
e5ac9a2734 |
feat(rvm): new instructions and loop semantics for Azure Policy support (#659)
The Rego VM was designed around Rego's semantics, but Azure Policy needs a few things Rego doesn't: host-supplied context alongside input/data, undefined-to-null coercion for missing fields, skip-undefined collection behavior for wildcard aliases, and non-vacuous iteration over non-array values. This commit adds five new instructions to bridge those gaps: LoadContext / LoadMetadata — give programs access to host-supplied evaluation context and cached program metadata at runtime. ArrayPushDefined — like ArrayPush but silently drops undefined values, so wildcard alias collection (field[*].property) excludes absent nested properties instead of leaking undefined entries into the array. ReturnUndefinedIfNotTrue — early return with Undefined when a guard condition isn't satisfied, without tripping a VM assertion failure. This models "condition doesn't match" cleanly. CoalesceUndefinedToNull — turns Undefined into Null in-place so that downstream builtins see null rather than short-circuiting on undefined. The loop engine also gains an Azure Policy mode: when the source language is "azure_policy", an Every loop over a non-array value (scalars, null, objects) iterates once over a virtual Null element instead of being vacuously true. This matches how field[*] behaves on non-array fields in Azure Policy — the condition body runs once against Null, which typically evaluates to false. On the plumbing side: the VM gets a context field with set_context(), metadata is cached as a Value on program load, and map_limit_error is inlined into memory_check since it had only one call site. Four new YAML test suites (~880 lines) cover the new instructions and context/metadata loading, along with instruction parser, display, and assembly listing support for everything added here. |
||
|
Anand Krishnamoorthi
|
126cc12eb5 |
refactor: consolidate RVM instruction variants and clean up VM internals (#651)
Merge the three separate Assert* instructions (AssertNot, AssertCondition,
AssertNotUndefined) into a single `Guard { register, mode }` instruction
with a GuardMode enum. This cuts duplicated match arms across display,
listing, parser, dispatch, and all compiler emit sites.
Drop the unnecessary `#[repr(C)]` from the Instruction enum. It was never
exposed across FFI, so the C-compatible 4-byte discriminant was pure waste.
Without it Rust picks a 1-byte discriminant, shrinking every instruction
from 8 bytes to 6. A new `instruction_size` unit test locks this at 6.
While touching these files, also clean up several long-standing issues:
- Deduplicate the iteration-state setup in loops.rs by extracting a shared
resolve_iteration_state() helper -- the stack-based and stackless paths
had near-identical 40-line blocks.
- Collapse the ExitWithSuccess / ExitWithFailure match arms into one.
- In rules.rs, stop cloning Arc<Program> just to borrow a RuleInfo -- clone
the small RuleInfo struct directly and extract a get_rule_info() helper.
- Move the memory check into dispatch (runs per instruction) and remove the
now-dead enforce_memory_check() entry-point calls.
- Apply map_or_else style throughout listing.rs for consistency.
|
||
|
Anand Krishnamoorthi
|
83891d7782 |
RVM compiler & runtime optimizations: caching, instruction fusion, constant hoisting, and correctness fixes (#626)
* perf!: add LRU caches for compiled regex and glob patterns
Add bounded LRU caches for compiled regex and glob patterns used by
Rego builtins, avoiding repeated recompilation of the same patterns
during policy evaluation.
New `cache` feature (included in `full-opa` and `opa-no-std`) backed by
the `lru` crate (no_std compatible) with `spin::Mutex` for thread safety.
- `src/cache.rs`: generic `LruCache<V>` wrapper, global `REGEX_CACHE`
(default capacity 256) and `GLOB_CACHE` (default capacity 128)
- `src/builtins/regex.rs`: all regex builtins route through the cache
- `src/builtins/glob.rs`: glob.match routes through the cache
- Public API: `regorus::cache::{Config, configure, clear}`
Compilation costs avoided per cache hit:
regex 10-55 µs (simple to complex patterns)
glob 10-12 µs
LRU hit ~10 ns
BREAKING CHANGE: new `cache` Cargo feature added to `full-opa` and
`opa-no-std` feature sets; adds `lru` as a dependency.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(vm): amortize per-instruction memory and time limit checks
Deduplicate per-instruction memory_check calls by hoisting them to the
main dispatch loop, and amortize monotonic_now() syscalls in the
execution timer by checking elapsed time every N instructions instead
of on every tick.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix(vm): correct object membership to check values only, not keys
The Contains instruction for objects was checking both keys and values:
object_fields.contains_key(v) || object_fields.values().any(|v| ...)
Per the Rego specification, `x in obj` tests whether x is a VALUE of
the object, not a key. The two-argument form `k, v in obj` is needed
to access keys. The interpreter already implemented this correctly
(values-only scan), but the RVM had the extra contains_key() check
which would incorrectly return true when the search value happened to
match a key name.
Remove the contains_key() branch so the behavior matches the interpreter
and the Rego spec. Add two regression tests:
- object_membership_checks_values_not_keys: "foo" in {"foo": "bar"}
must be false (key, not a value)
- object_membership_finds_value: "bar" in {"foo": "bar"} must be true
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(compiler): hoist all-constant collection literals to the literal table
When an array, set, or object literal consists entirely of compile-time
constant expressions (numbers, strings, bools, null, and nested constant
collections), the compiler now evaluates them at compile time and emits a
single Load instruction from the literal table instead of generating
per-element instructions at runtime.
Previously, a Rego expression like `x in [1, 2, 3]` would emit
ArrayCreate + three Load + three ArrayAppend instructions, allocating a
new Vec and Rc on every evaluation. With this change, the entire array
is built once during compilation and loaded as a single constant.
This optimization applies to all three collection types:
- Array literals: avoids ArrayCreate + N x (Load + ArrayAppend)
- Set literals: avoids SetCreate + N x (Load + SetAdd)
- Object literals: avoids ObjectCreate + N x (Load + Load + ObjectInsert)
The implementation adds a try_eval_const() helper that recursively
evaluates an AST expression as a constant Value, returning None if any
sub-expression is non-constant. Each compile method for collection
literals attempts the all-constant fast path first and falls through to
the existing instruction-by-instruction codegen otherwise.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(compiler): fuse Eq + AssertCondition into AssertEq instruction
Add a new `AssertEq { left, right }` instruction that combines equality
comparison and condition assertion into a single operation. This replaces
the previous two-instruction pattern of `Eq { dest, left, right }` followed
by `AssertCondition { condition: dest }`, saving one instruction and one
register per equality assertion.
The fused instruction checks two registers for equality and directly calls
handle_condition with the result, avoiding the intermediate boolean
register entirely. If either operand is undefined or the values differ,
the condition fails and the rule/loop backtracks.
The optimization applies to four destructuring sites:
- EqualityCheck (assignment re-binding with `x = expr; x = expr`)
- EqualityExpr (destructuring against an expression)
- EqualityValue (destructuring against a literal value)
- assert_array_length (array length validation in destructuring)
In soft_assert_mode the compiler still emits the original Eq instruction
since the boolean result register is needed by callers.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(compiler): fuse Not + AssertCondition into AssertNot instruction
Add a new `AssertNot { operand }` instruction that combines logical
negation and condition assertion into a single operation. This replaces
the previous two-instruction pattern of `Not { dest, operand }` followed
by `AssertCondition { condition: dest }`, saving one instruction and one
register allocation.
The fused instruction checks the operand register and passes the
condition if the value is false or undefined (per Rego semantics where
`not expr` succeeds when the expression has no results or is false),
and fails the condition if the value is true or any non-boolean truthy
value.
This was the only emission site for the Not+AssertCondition pair,
occurring in the compilation of `Literal::NotExpr` statements.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(vm): early exit for same-value multi-definition rules
When a rule has multiple definitions that all produce the same value
(e.g. implicit true, or identical literal), set early_exit_on_first_success
on RuleInfo so the VM can stop after the first successful definition.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat!: expose cache configuration API to all language bindings
Add `set_cache_config` and `clear_cache` functions to every binding
so callers can tune or reset the global regex/glob pattern caches
introduced in the cache feature.
Bindings updated:
- FFI (C): `regorus_set_cache_config`, `regorus_clear_cache`
- C++ header: free functions `regorus::set_cache_config`, `regorus::clear_cache`
- Python: module-level `set_cache_config(*, regex, glob)`, `clear_cache()`
- Java: static methods on new `CacheConfig` class
- Go: package-level `SetCacheConfig`, `ClearCache`
- Ruby: module functions `Regorus.set_cache_config`, `Regorus.clear_cache`
- WASM: free functions `setCacheConfig`, `clearCache`
- C#: static methods `Engine.SetCacheConfig`, `Engine.ClearCache`
BREAKING CHANGE: Bump SERIALIZATION_VERSION from 4 to 5 due to new
AssertEq and AssertNot instruction variants added in the instruction
fusion commits. Programs serialized with version 5 cannot be loaded
by older versions of regorus.
* fix: address PR review feedback
Cache subsystem:
- Gate REGEX_CACHE and related imports behind #[cfg(feature = "regex")]
so that building with --features cache without regex compiles correctly.
- Gate LruCache struct behind #[cfg(any(feature = "regex", feature = "glob"))].
- Add Config::MAX_CAPACITY (2^16) hard upper bound; clamp values in
configure() to prevent unbounded cache growth.
- Use parking_lot::Mutex for std builds and spin::Mutex for no_std to
avoid CPU spinning under contention in tight regex/glob eval loops.
- Narrow lock scopes in regex/glob builtins: release the mutex before
compiling a pattern, then re-acquire to insert.
Java JNI binding:
- Fix cache config overflow: negative jlong values now saturate to 0
and positive overflow saturates to usize::MAX (then clamped by
MAX_CAPACITY) instead of silently disabling the cache.
- Gate JNI cache config/clear functions behind #[cfg(feature = "cache")].
Compiler:
- Refactor static_value_of_expr to delegate to try_eval_const,
gaining support for negated numbers and constant collections.
- Make try_eval_const pub(in crate::languages::rego::compiler) and
re-export through expressions.rs.
- Handle Expr::UnaryExpr with numeric literals in try_eval_const so
collections containing negated numbers (e.g. [-1, 2]) are hoisted.
VM correctness:
- Fix Not instruction to follow Rego semantics: not expr yields
true when expr is undefined or false, false for any other defined
value (including non-booleans) -- no longer errors on non-boolean
operands.
- Add enforce_memory_check() call at execute_suspendable_entry to
ensure memory limits are checked before the first instruction.
- Update AssertNot listing comment to "exit if any defined truthy
value" to match actual VM behaviour.
- Add doc comment on Not instruction clarifying Rego negation
semantics.
Bindings:
- Fix C++ header indentation for set_cache_config / clear_cache.
- Propagate Cargo.lock parking_lot addition across ffi, java, python,
and wasm binding lockfiles.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
|
||
|
Anand Krishnamoorthi
|
3f7a5496dc |
feat(bindings)!: add RVM/Program support across FFI and language bindings (#565)
- FFI: add RVM/Program APIs, execution state accessors, HostAwait handling, and buffer/result helpers in rvm.rs, common.rs, engine.rs. - Compiler: emit HostAwait for __builtin_host_await in function_calls.rs. - RVM tests: add HostAwait regression cases and extend harness for suspend/resume responses in host_await.yaml and mod.rs. - C/C++: add RVM tests/examples and wrapper updates in rvm_tests.c, rvm_tests.cpp, regorus.hpp, plus CMake wiring. - C#: add Program/Rvm bindings, SafeHandle/PInvoke, tests, and example usage in Regorus, RvmProgramTests.cs, Program.cs, and README updates. - Go: add Program/Rvm bindings, tests, and examples in rvm.go, rvm_test.go, main.go. - Java: add Program/Rvm bindings, JNI glue, and examples in lib.rs, regorus, Test.java. - Python: add Program/Rvm bindings and examples in lib.rs, test.py. - WASM: add Program/Rvm bindings and examples in lib.rs, test.js. - Tooling: wire binding tests in xtask and ignore generated Java artifacts in .gitignore. Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
9426b2ec02 |
fix: Imports without a name binding (#543)
Handle imports that don't use the `as` clause to create a binding. These imports are bound to the last identifier in the imported path. Fix both interpreter and compiler. Add tests. fixes #541 Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
9fa8036ce4 |
feat: Implement Rego else block compilation
- teach the Rego compiler to compile else chains correctly - test suite Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
a232b13e50 |
feat: Else blocks in definitions
- ensure both run-to-completion and suspendable rule execution stop evaluating bodies once one succeeds so later else branches are skipped - test cases Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
a514e8da83 |
fix: Implement RVM set ops correctly
- treat set subtraction in RVM the same as the interpreter by supporting Value::Set operands in sub_values - emit internal-only builtin names for set union/intersection and register handlers so compiled bytecode resolves without exposing new Rego builtins - add regression coverage for literal set difference/intersection (x/y from failure.rego) in tests/rvm/rego/cases/sets.yaml Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
bedf667adc |
feat: Handle literal comparisons that use = and comprehensions without loops
- emit AssertCondition for equality-only assignment plans (outside soft-assert mode) so rules like `0 = 1` fail under the VM just like the interpreter - let comprehension bodies consume assertion failures by advancing or exiting their iteration context, both in run-to-completion and suspendable execution Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
8269968c4a |
feat: Handle computed reference roots in RVM compiler
Allow compile_chained_ref to fall back to “evaluate root expression → chain access” so literal arrays, comprehensions, and other computed roots no longer raise NotSimpleReferenceChain. Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
e3d23766ae |
feat: Ensure RVM caches deterministic builtins
Mirror interpreter implementation: - use builtins::must_cache to determine whether builtin must be cached. - reuse cached value when applicable - clear the VM’s builtin cache whenever execution state resets to avoid leaking values across runs - add a YAML regression for rand.intn set comprehensions and re-enable the rand cases in the OPA test suite Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
b7b3d3ec87 |
feat: Soft-assert mode for builtin out-params under not
- Add a scoped soft_assert_mode to the compiler so `not` statements compile their subexpressions without emitting hard AssertCondition/AssertNotUndefined instructions. - Teach binding-plan application to return an optional result register; equality plans now yield a boolean in soft mode, allowing not abs(-5 , 3) to succeed instead of aborting. - Update function-call, loop, and rule plumbing to consume the new binding-plan outcome, including copying the produced register when an out-parameter equality is used. - Trim the OPA TODO list to the remaining troublesome folders. Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
30bd134a0b |
fix: Handle builtin out-parameter calls in RVM compiler
- Teach the hoister/destructuring planner to respect parent scope when building binding plans for extra arguments, so already-bound vars yield equality checks. - Update the compiler’s function-call path to drop the trailing out-argument, run its binding plan after the call, and share call-target resolution logic. - Add regression suites for builtin and user-defined out-parameter scenarios plus align the CLI example output when RVM returns undefined. Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
a8a3a9809b |
feat!: Use num-bigint for large numbers (#500)
- Supply chain: Use the popular num-bigint crate for handling large integers - Optimization: Handle f64, i64, u64 directly. These will be the most common instances of a number. OPA number semantics isn't clear. https://github.com/open-policy-agent/opa/issues/6281 As part of this change, we update the following failing tests: - A local test that relies on what 15.3/3 evaluates to. With our current change, we round in a different direction than what OPA does, but consistent with Rust. We produce 5.1000000000000005 where as the OPA test expects 5.1. There is no clear definition in Rego of what the right answer is. Moreover, policies should not rely on exact floating point value comparison. Therefore this deviations is justified. The test is patched to pass. - Another local vm test that exercised 1.1 + 2.2 - Another local vm test that exercises 5.5 - 2.2 - An OPA test that expects that a large integer number say 10e308 is printed in exponent notation. num-bigint does not print using scientific notation and instead prints all the digits. The benefit of preserving this compatibility is not clear. We skip this test. - Doc tests that exercised handling floating point numbers with more than 15 (what f64 supports) digits of precision. There is no usecase for this scenario. The tests are updated to reflect the behavior. Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
a3a20a1235 |
feat!: Rego -> RVM Compiler and extensive testsuite (#506)
# RVM compiler test cases Coverage: - arithmetic - arrays - chained lookups - comparisons - comprehensions - default rules - destructuring - function rules - loops/quantifiers - multiple entrypoints - objects/sets - variables - negative/edge scenarios such as data/rule conflicts - virtual data lookups - etc # Modify interpreter and compiled policy for RVM Compilation - Interpreter::eval_default_rule_for_compiler: evaluates a named default rule in isolation - allows compiler to emit a constant value instead of instructions for the default value # feat: Rego Compiler Scaffolding - Introduce the rego::compiler module surface and entry point wiring - Add the core compiler concepts: - register allocator - scope tracking - literal/builtin tables - rule worklists - instruction emit helpers - compiler-specific error types - context structs for rules, comprehensions, and loops to support later lowering passes. # feat: Compile Rules/Queries - add compiler::compile_from_policy workflow plus rule worklist, entry-point wiring, and recursion checks - implement query lowering: - scheduling-aware statement ordering - loop hoisting - “every/some” semantics - context yields - literal assertions - finalize Program construction # feat: Expression Lowering - add compile_rego_expr and helpers to translate every AST expression into RVM instructions, - interop with binding plans, comprehensions, and membership checks. - implement collection literal builders (ArrayCreate, SetCreate, ObjectCreate) - dedupe literal keys and handle mixed literal/dynamic fields via instruction data blocks. - operations: - arithmetic/boolean/bin operators - membership - unary minus - set unions/intersections - etc - user-defined and builtin function calls - reference handling - analyse chained refs - distinguishe data/input/local roots - perform rule dispatch or virtual document lookups - emits optimized Index/ChainedIndex instructions. # feat: Comprehensions & Loops - shared comprehension emitter - wraps array/set/object comprehensions with ComprehensionBegin/End - context management - loop lowering utilities - read hoisting metadata - emit LoopStart/LoopNext - some in lowering - every quantifiers - index iteration - propagate binding plans into stored registers so downstream statements see bound variables. # feat: Destructuring Lowering - destructuring planner integration - assignment/parameter/loop bindings use hoisted plans instead of re-walking ASTs. - handle :=, =, wildcard matches, and equality - evaluate RHS - applying destructuring plans - emit assert condition as needed - support nested array/object destructuring, dynamic keys, and some ... in forms # test: Shared Testing + RVM Suites - move YAML test helpers into test_utils.rs and re-export via common.rs for use by interpreter and vm test suites - comprehensive compiler test suite - compiles policies with the new Rego→RVM compiler - runs them through RegoVM - compares against interpreter behavior - supports multiple entry points - provides assembly listings - filterable YAML suites. Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |
||
|
Anand Krishnamoorthi
|
49bd3c22f3 |
feat!: add Rego Virtual Machine (RVM) implementation (#495)
* feat!: add Rego Virtual Machine (RVM) implementation This commit introduces a register-based virtual machine for executing Rego policies with bytecode-style instructions. Unlike the existing tree-walking interpreter, the RVM compiles policies into instruction sequences that operate on virtual registers, offering better performance and optimization potential. Core Components: Instruction Set Architecture: - Define instruction types for data operations, control flow, and builtins - Implement instruction parameter encoding and display formatting - Add instruction parser with comprehensive test coverage Virtual Machine Engine: - Register-based execution model with program counter management - Loop execution supporting iterators, comprehensions, and quantifiers - Function call handling with argument evaluation and context management - Rule evaluation with default value resolution and virtual data support - Arithmetic and comparison operation implementations Program Representation: - Program listing builder with instruction sequencing - Rule tree construction for organizing policy rules - Binary and JSON serialization for compiled programs - Recompilation support for program modification Testing Infrastructure: - Extensive YAML test suites covering all VM features - Rust unit tests for VM execution and instruction parsing - Test suites for loops, comprehensions, builtins, and control flow BREAKING CHANGE: Introduces new VM execution path alongside interpreter Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> * docs: add detailed RVM architecture references Introduce architecture.md explaining program artifacts, serialization, and runtime subsystems. Document the full opcode catalog in instruction-set.md, including operands, parameter tables, and outcomes. Walk through execution flow, stacks, and operational guidance in vm-runtime.md, tying the runtime to the new architecture docs. Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> --------- Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> |