The RVM was silently succeeding on `every` quantifiers (and loops nested
inside an `every` body) that should have failed. In each case the loop
computed a pass/fail into a register that the surrounding query then
ignored, so the RVM disagreed with the interpreter.
Four related fixes:
- compile_every_quantifier: guard the loop result so a failing `every`
body makes the rule undefined instead of always succeeding.
- resolve_iteration_state: `every` over a non-iterable scalar (number,
string, bool, null, undefined) is now undefined, not vacuously true.
Only genuinely empty collections stay true; any/forEach are untouched.
- a `some ... in` inside an `every` body now guards its loop result, so
a `some` that matches nothing fails the current iteration. Top-level
rule bodies still rely on context yields and are unaffected.
- a hoisted index iteration (`some i` / `arr[i]`) inside an `every` body
gets the same guard.
Also drop `every` from OPA_TODO_FOLDERS so the interpreter-vs-RVM
differential suite covers it, add an OPA_UNSKIP_FOLDERS env override for
auditing other still-skipped folders, and add regression cases for every
variant above.
* Deep-merge nested data documents in Engine::add_data
add_data previously performed a shallow merge: adding a nested object under a key that already existed either replaced the whole subtree or errored on a spurious conflict, instead of merging the trees. This makes Engine::add_data (and the shared Value::merge) recurse into nested objects so keys from both sides are preserved, matching OPA's data-document merge semantics. Nested sets are unioned as a regorus extension (OPA data is JSON and has no sets). Genuine leaf conflicts (same path, two different scalar values) still error; equal values remain a no-op, which the shared rule-evaluation path relies on. Adds tests for object deep-merge, set union, leaf/type conflicts, and interaction with the 'with data.x' modifier.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* docs(value): clarify Value::merge conflict wording
Copilot review on #760 noted the doc comment called non-mergeable variants 'non-container values', which is misleading since arrays are containers yet still conflict unless equal. Reword to describe a conflict as any differing pair that is not both objects or both sets (e.g. unequal scalars or arrays).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* perf(value): avoid deep-cloning RHS set during merge union
When unioning sets in Value::merge, the RHS set is often shared: the object arm recurses via existing.merge(v.clone()), which bumps the incoming set's Rc refcount. The old Rc::make_mut(new) then structurally deep-cloned the entire RHS BTreeSet just to drain it via append and immediately discard the copy.
Move the elements out when the RHS set is uniquely owned, and otherwise clone only the per-element Rc handles into the destination. The union result is identical (BTreeSet dedups), but no throwaway set is allocated on the nested-merge path exercised by add_data deep-merge.
Addresses a Copilot review comment on #760.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* fix(engine): make add_data atomic on merge conflict
Now that Value::merge recurses, a conflict in a later nested key was reported only after earlier keys of the same document had already been written into the live init_data, leaving the engine partially mutated on a rejected add_data.
Add a read-only Value::check_mergeable that mirrors merge's conflict rule (objects deep-merge, sets union, equal values no-op, anything else conflicts) and run it in add_data before merging. On conflict nothing is mutated, so add_data is all-or-nothing. The check allocates nothing and never copies the data spine, preserving merge's in-place uniquely-owned fast path (no candidate copy of the data document).
Adds regression tests for a partial object-leaf conflict and a partial set-union conflict. Reported by a maintainer on #760.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* test(engine): add array atomicity regression for add_data
Arrays are atomic leaves, so a differing array at a shared path is a
conflict. The new key sorts before the conflicting array key, so a naive
in-place merge would leak the new key before hitting the conflict. This
test locks in that add_data rejects the whole call and leaves data
untouched.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* fix: make add_data atomic under allocator memory limits
On llocator-memory-limits builds, Value::merge runs the limit check
*after* inserting each key, so an add_data whose merge trips the limit
mid-way left the data document partially mutated. check_mergeable only
models semantic conflicts, not limit failures, so the validate-then-merge
precheck couldn't cover this failure mode.
Use a build-split strategy in dd_data:
- default builds: keep the zero-copy validate-then-merge fast path
(a conflict is the only way the merge can fail).
- allocator-memory-limits builds: merge into a candidate copy and commit
only on success, making both conflict and limit failures transactional.
Value is Rc/copy-on-write, so only touched subtrees are cloned.
check_mergeable is now cfg-gated to the default build to avoid dead code.
Tests (allocator-memory-limits build): add a partial-merge atomicity test
(limit trips mid-merge, data must be untouched) and a candidate-copy
conflict-atomicity test.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* fix: separate strict rule-output merge from data-document deep-merge
#760 made Value::merge recursive so Engine::add_data deep-merges nested
data documents. But that same method also backs rule materialization,
where recursion is wrong: two rule definitions producing different
outputs for one path must conflict (OPA complete-rule semantics), not
silently combine.
Split the two behaviors:
- Value::merge is strict and shallow again (as pre-#760): a key on both
sides must be equal or it conflicts; used for rule outputs.
- Value::deep_merge is the recursive data-document merge behind add_data;
check_mergeable validates it up front without allocating, so the
default build merges in place instead of cloning a candidate.
Also fix zero-arg functions (f() := ...): route their materialization
through strict equality via a new RuleValueMerge selector, so disjoint
outputs ({a:1} vs {b:2}) conflict as OPA does while prefix scaffolding
(a.foo + a.bar) still combines.
Add a 14-case interpreter conformance matrix (multiple_outputs.yaml)
covering functions, static/dynamic partial objects, and ref-heads,
matched against OPA v1.2.0.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* perf(value): make deep_merge acquire mutable access lazily
deep_merge's object arm called Rc::make_mut on the target map up front,
cloning a shared map's spine even when the merge changed nothing (a
no-op subset re-add) or conflicted before any mutation. Decide each
incoming key from a read-only probe (skip / insert / recurse / conflict)
and take Rc::make_mut only when a key actually mutates, so no-op and
conflict merges leave shared maps untouched.
Behavior is unchanged: the equality short-circuit that previously ran
inside the recursive call now runs in the probe, and conflicts bail with
the same message. Add value tests asserting Rc::ptr_eq is preserved
across no-op subset, equal-nested-object, and first-key-conflict merges.
OPA conformance unchanged (3021 pass / 651 fail, byte-identical).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* feat(value): bound deep_merge recursion depth to prevent stack-overflow DoS
deep_merge and check_mergeable recursed unbounded on object/set nesting.
A Value built without serde_json's parse-time recursion limit (the Python
and Ruby native bindings, or programmatic construction) could therefore
drive add_data into a stack overflow -- an uncatchable abort that poisons
every engine in an FFI process.
Thread a depth counter through both functions and bail past MAX_MERGE_DEPTH
(128, matching serde_json's default) so over-deep data fails with a clean
Err. In the default build check_mergeable trips first, keeping add_data
atomic; the guard in deep_merge covers the allocator-memory-limits build
and any disjoint-then-overlapping merge.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* docs(changelog): note strict zero-arg function conflict and add_data depth limit
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Mark Birger <markbirger@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* feat(compiler): support registered host-await builtins
Allow hosts to register function names at compile time so that calls to
those names emit HostAwait instructions directly, enabling natural syntax
like fetch(x) instead of __builtin_host_await(x, "fetch").
- Add host_await_builtins map and register_host_await_builtin() to Compiler
- Validate arg_count == 1 and reject reserved __builtin_host_await name
- Extend determine_call_target() resolution: explicit > registered > user > builtin
- Both explicit and registered paths emit identical HostAwait bytecode
- Add compile_from_policy_with_host_await() entry point in rules.rs
- Extended test harness with HostAwaitBuiltinSpec and args assertion
- 9 YAML test cases: suspend/resume, run-to-completion, multiple names,
queue, shadowing, object packing, arg_count rejection, reserved name
rejection, standard builtin override
- Documentation: instruction-set.md, architecture.md
* Update src/languages/rego/compiler/function_calls.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Mark Birger <birgerm@yandex.ru>
* fix(compiler): address PR #667 review feedback on host-await registration
- Compiler::register_host_await_builtin now rejects duplicate, empty,
and whitespace-only names. Previously a duplicate registration would
silently overwrite the existing entry, which could mask the host's
own registration mistakes.
- YAML test cases added: empty registration list as no-op, duplicate
name rejection, empty/whitespace name rejection, out-param (a, out)
calling syntax with a single-arg registered builtin, and mixed
__builtin_host_await + registered builtins in the same policy
consuming from their respective identifier queues.
- Test harness: replace assert_eq! on HostAwait argument mismatch with
anyhow::Error so mismatches propagate through the case reporter
instead of panicking and skipping the harness's normal error path.
- YAML comment fix: "Registration panics" -> "Registration fails with
an error" (registration returns Err, never panics).
Addresses anakrish + Copilot inline review comments on PR #667.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* compiler: split CallTarget::HostAwait into explicit and registered variants
Addresses PR #667 review item #8: at the emit site in
`compile_function_call`, the discrimination between explicit
`__builtin_host_await(arg, id)` and a registered host-awaitable
builtin was being recovered by string-comparing `original_fcn_path`
against `"__builtin_host_await"`. The information was already known
in `determine_call_target` and was being thrown away.
Replace the single `CallTarget::HostAwait` variant with two:
* `ExplicitHostAwait` (unit) — the two-argument call form. The
identifier register comes from the user's second argument.
* `RegisteredHostAwait { identifier: String }` — the one-argument
call form for registered builtins. The identifier is the registered
name and is captured in the variant at recognition time, so the
emit site never re-derives it from the function path.
This removes the magic-string comparison at the emit site (the source
of truth is now `determine_call_target`) and makes both match sites
in `compile_function_call` exhaustive over the two forms — adding a
third host-await form in the future would force a compile error at
every match site instead of silently falling through.
Arities are now hardcoded in the `expected_args` extraction
(`Some(2)` for explicit, `Some(1)` for registered) rather than
carried in the variant; registered builtins are constrained to
`arg_count == 1` at registration time, so there is no per-call
variability to carry.
Bytecode output is unchanged; the full RVM test suite (97 cases) and
the registered_host_await suite (15 cases) pass without modification.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs(compiler): clarify registered host-await intercepts unqualified calls only
PR #667 review (Medium): the docs implied registered host-await names
shadow user functions and builtins unconditionally, but
determine_call_target matches only the bare original_fcn_path. A
package-qualified call such as data.demo.resolve(x) is therefore not
intercepted -- it resolves through the normal path like any other call.
Rather than expand registration to qualified paths (which would let a
registered name leak into every package exposing a same-named rule),
document the unqualified-only behavior and pin it with tests.
- register_host_await_builtin: doc now states only the unqualified call
form is intercepted; qualified calls resolve normally.
- determine_call_target: inline comment explaining the deliberate
original_fcn_path-only match.
- docs/rvm/instruction-set.md: describe qualified-call resolution,
including that builtins have no qualified form.
- tests: cross-package and same-package qualified calls resolve to the
rule; bare-name shadowing of a standard builtin; Unknown-function
outcome when no rule exists at the qualified path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(tests): compare host-await argument without re-running process_value
PR #667 review (Low): the suspendable test harness compared the
host-await argument via process_value(argument), but argument is already
a runtime Value. process_value is a YAML-fixture decoder -- it rewrites
"#undefined" to Undefined, {set!: [...]} to a set, and errors on a
runtime Value::Set. Re-running it on the runtime argument could coerce a
legitimate payload into a fixture sentinel (passing for the wrong
reason) or error outright on sets.
Compare the runtime argument directly against the expected value, which
is already decoded once at YAML load time.
Add a regression case (registered_builtin_suspendable_set_argument) that
passes a set payload: it fails under the old double-processing
("unexpected set in value read from json/yaml") and passes with the fix.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(tests): reject `args:` payload expectations in run-to-completion mode
PR #667 review (Low): a run-to-completion host-await response could carry
an `args:` payload expectation, but RTC execution pre-loads responses and
never surfaces the call argument to the harness, so the expectation was
parsed and silently dropped. A case with `args: "WRONG"` passed as long as
the result matched -- asserting a payload that was never checked.
Reject `args:` for run-to-completion fixtures at load time, directing the
author to suspendable mode where arguments are validated. Also only build
the run-to-completion response vector when the case actually runs in RTC
mode, so a suspendable case using the shared host_await_responses field
with `args:` is not wrongly rejected.
Route the fixture-load error through the same want_error handling used for
compilation errors, and add registered_builtin_run_to_completion_rejects_args
which now fails loudly instead of passing silently.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(compiler): reject host-await builtin names with surrounding whitespace
PR #667 review (Low): register_host_await_builtin rejected all-whitespace
names via name.trim().is_empty(), but accepted padded names like " lookup"
or "lookup ". Those were inserted into host_await_builtins, but Rego
function-call paths produce the trimmed identifier, so a padded
registration could never match -- a silent dead registration.
Reject any name that is not already trimmed (name != name.trim()) in
addition to empty names, and update the error message accordingly. Add
test cases for leading and trailing whitespace.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Signed-off-by: Mark Birger <birgerm@yandex.ru>
Co-authored-by: Mark Birger <markbirger@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add an opaque `Set` newtype paralleling `Object`, living under
`src/value/set/` with the same module structure (`mod.rs` /
`iter.rs` / `serde.rs`). `Set` wraps `BTreeSet<Value>` today but
exposes only a curated surface: `contains`, `insert`, `remove`,
`iter`, `iter_sorted`, `cursor` (resumable), `is_subset`,
`intersection`, `difference`, serde, and a hand-written `Ord`.
The cursor types are re-exported behind the `rvm` feature so the
follow-up `IterationState::Set` swap can land additively.
To free the `Set` name for the new public type, the crate-internal
`BTreeSet as Set` / `HashSet as Set` aliases in `lib.rs` are
renamed to `MapSet`. All in-tree consumers of the old alias are
updated in lockstep.
`Value::Set` is unchanged in this commit (still wraps
`Rc<BTreeSet<Value>>`); the payload swap and call-site migration
ship in the next PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Initial plan
* Add keywords_in_refs: allow reserved keywords as dot-notation field names
* Address review feedback: improve parse_ref_field doc comment and clean up test comment
* Add complex keyword-in-ref test cases
* Polish keyword-ref test expectations and validate coverage
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Builds on #57. Swap Value::Object's payload from Rc<BTreeMap<Value, Value>>
to Rc<Object> and migrate all call sites to the Object API.
as_object / as_object_mut keep their names but return &Object / &mut Object.
The mutable accessor handles Rc::make_mut internally, so callers no longer
do it themselves. Object grows into_value() and From<Object> for Value.
Value's serializer now delegates to Object::serialize, dropping a duplicate
non-string-key stringification path.
RVM IterationState::Object is rewritten around ObjectCursor: O(log n)
steps over a shared Rc<Object>, no eager pair snapshot. Snapshot
independence is preserved by Rc copy-on-write; setup_next_iteration
advances the cursor inline and advance() becomes a no-op for this variant.
A new iteration_state_object_is_snapshot_independent_of_source test
covers CoW against a mutated alias.
Value::Set still wraps Rc<BTreeSet<Value>>; the matching Set abstraction
and its swap ship in follow-up PRs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add an opaque Object type for the key→value storage backing
Value::Object. It exposes a small set of methods (get, insert, remove,
iter, iter_sorted, cursor, serde) and keeps the backing store private,
so future representations -- inline small-map, hash-backed, lazy,
arena, FFI-callback -- can plug in without touching the call sites
that name this type.
Nothing in the engine uses Object yet. Value::Object still wraps
Rc<BTreeMap<Value, Value>>; the payload swap and call-site migration
come in the next PR. Object stands on its own unit tests in the
meantime.
docs/value/object.md walks through the design, the precedents it
follows (serde_json::Map, toml::Table, simdjson DOM), and the
concrete workloads the abstraction is meant to unlock.
A matching Set abstraction follows in a separate PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Encode VM stack/context/register lifecycle invariants as
debug_assert!s. Zero cost in release; surfaces violations during
debug-mode tests and CI.
Invariants covered:
- reset_execution_state postcondition: all stacks empty, registers
resized to base and Undefined, rule_cache reset, pc/executed
counters zeroed, builtins_cache cleared, execution_state Ready.
- Per-opcode invariant check (assert_vm_invariants) invoked at the
top of run_stackless_loop and jump_to iterations: state is
Ready/Running, registers non-empty, rule_cache sized to program,
execution stack bounded by a debug-only sanity ceiling
(DEBUG_MAX_EXECUTION_STACK_DEPTH = 4096; not a production limit).
- resume() precondition: execution_state is Suspended.
- execute_suspendable_entry precondition: clean state (callers reset
immediately before).
- Rule finalize: call_rule_stack pop matches the finalized rule_index.
- IterationState::advance: Single iterator not advanced past
consumption, Array index not at usize::MAX before saturating_add.
All assertions are gated by #[cfg(debug_assertions)] (directly or via
debug_assert!) so release builds are unaffected.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(ffi): eliminate aliasing UB via to_shared_ref migration
Add to_shared_ref() helper that creates &T (shared reference) from raw
pointers instead of &mut T. This eliminates undefined behavior caused by
violating Rust's aliasing invariant when C# SafeHandle permits concurrent
FFI calls on the same handle.
With &mut T, the compiler may assume exclusive (noalias) access and
reorder or elide reads/writes — a miscompilation risk when another thread
holds a reference to the same object. Switching to &T removes that
assumption; actual mutation is mediated by the interior RwLock inside
Handle<T>, which is the sole synchronization mechanism.
Migrated sites:
- rvm.rs: 20 non-drop call sites
- engine.rs: 30 non-drop call sites + with_unwind_guard for timer fns
- compiled_policy.rs: 2 call sites
- Fix null-data UB in regorus_program_deserialize_binary
Drop paths retain to_ref() where exclusive access is guaranteed by the
caller contract (preventing use-after-free).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(ffi): add Azure Policy JSON compilation FFI and C# bindings
- AliasRegistry builder pattern: RegorusAliasRegistryBuilder (mutable,
single-threaded) + RegorusAliasRegistry (immutable, Arc-wrapped)
- Azure Policy JSON compilation: regorus_compile_azure_policy_rule and
regorus_compile_azure_policy_definition with alias registry support
- regorus_rvm_set_context for host-supplied ambient data
- C# AliasRegistryBuilder and AliasRegistry classes with convenience
factories (FromJson, FromManifest, Empty)
- C# AzurePolicyCompiler static class for policy rule/definition compilation
- Compile functions take *const RegorusAliasRegistry (read-only via
to_shared_ref for concurrent compilation safety)
- Fix pre-existing clippy warnings across multiple crates
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the compiler's two cloned BTreeMap fields (alias_map and
alias_modifiable) with a single Option<Rc<AliasRegistry>>. This
eliminates cloning two 73K-entry maps on every compilation by sharing
the registry through a reference-counted pointer.
Additional improvements:
- alias_map()/alias_modifiable_map() return &BTreeMap (zero-copy)
- Safe .get() indexing in ingest_alias_entries and build_object_from_keys
- Null-tolerant deserialization for AliasEntry.paths (~97% of real
Azure catalog entries emit "paths": null)
All changes are within the azure_policy feature gate.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Partial object rules with dynamic keys (e.g. `violations[k] if { ... }`)
only produced a single entry instead of collecting all bindings. Two
independent bugs caused this:
1. Interpreter: the early-return optimization in eval_output_expr_in_loop
checked whether the rule_ref was constant but never verified whether
the key expression was also constant. A variable key like `k` was
treated as constant output, causing the loop to exit after the first
iteration. Fixed by gating early-return on key_expr constness.
2. RVM: compute_rule_type incorrectly classified `p[k] if { ... }` as
PartialSet instead of PartialObject. OPA v1 semantics define this
form as a partial object (key -> true). Fixed the classification and
added compiler error guards for patterns the RVM codegen cannot yet
handle (constant keys, nested bracket keys), ensuring graceful
fallback to the interpreter.
The OPA test harness now skips RVM validation per-case when partial
object compiler errors are raised, rather than blanket-skipping entire
folders. This preserves RVM coverage for unrelated tests in the same
folders.
Closes#712
Co-authored-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Replace per-alias heap allocations with reference-counted string interning
throughout the normalizer's alias resolution pipeline:
- Store alias ARM path segments as Vec<Rc<str>> instead of Vec<String>,
enabling zero-alloc BTreeMap lookups via Rc::clone (refcount bump)
rather than Value::from() (heap allocation per lookup)
- Pre-compute lowercased short name (short_name_lc: Rc<str>) at
registry-load time, enabling allocation-free insertion for the
common case of non-dotted, non-collision alias short names
- Add rc_lowercase() fast-path that skips allocation when the input
string is already lowercase ASCII (common for ARM property names)
- Update set_nested_lowercased/set_nested_inner/set_nested_in_btree to
thread Rc<str> through intermediate object construction, avoiding
String temporaries at every nesting level
- Replace to_ascii_lowercase().starts_with() in is_root_field_collision
with a zero-alloc byte-level comparison
Measured 27-49% improvement in normalization time across a range of
Azure Policy alias-heavy resource types (293-608 aliases per type).
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Add a 100KB cap on compiled regex NFA size via RegexBuilder::size_limit()
to block patterns that blow up in memory or CPU. Regex compilation now
goes through a single helper (compile_regex_for_builtin) so the limit
is enforced consistently across all regex builtins.
While doing this, found and fixed a pre-existing bug: resource-limit
errors (time, memory, instruction count) raised inside builtins were
quietly swallowed to Undefined when strict_builtin_errors was off
(the default). This is a problem because `not regex.match(...)` would
see Undefined and flip to true -- silently wrong. The same issue now
applies to the new regex size limit.
Fixed by teaching the three error-absorption paths (interpreter builtin
call, RVM builtin dispatch, and RVM rule-execution loop) to recognize
LimitError and let it propagate instead of eating it.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the YAML test runner that exercises the companion test data PRs, plus
several compiler fixes surfaced during testing:
- Removed parameter register caching that produced wrong results inside
short-circuiting allOf/anyOf blocks; added literal-index caching for
parameter defaults to avoid repeated O(n) literal-table scans
- Simplified cross-resource effect details to only emit roleDefinitionIds
and type (deployment templates are not evaluated for compliance)
- Replaced guid/uniqueString builtins with clear "unsupported" errors
- Normalized datetime output to ISO 8601 with Z suffix
- Added azure_policy parser MAX_COL constant (8192) for long template
expressions, keeping the global DEFAULT_MAX_COL at 1024
- Added rvm to azure_policy feature dependencies since the compiler
targets RVM bytecode
Also restructures the example binary into examples/regorus/ with new
azure-policy-eval and azure-policy-aliases subcommands, adds C# alias
normalization tests, and documents Azure Policy support in the README.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the stub implementations in effects.rs, effects_modify_append.rs,
and metadata.rs with full working code.
Effect compilation dispatches all effect kinds (Deny, Audit, Modify, Append,
AuditIfNotExists, DeployIfNotExists, etc.) including parameterized effects
that resolve at runtime via [parameters('effect')]. Cross-resource effects
(AINE/DINE) emit a HostAwait to fetch the related resource and evaluate an
optional existenceCondition against it. Modify and Append effects compile
their operation/detail arrays, including template expressions in values.
Metadata recording tracks which policy features are used during compilation
(field kinds, aliases, operators, resource types, count, wildcards) and
writes them into the program annotations so the runtime can inspect
capabilities without re-analyzing the AST. Definition-level metadata
(display name, category, version, parameter names, etc.) is also extracted.
Detail field values in AINE/DINE (type, name, resourceGroupName) are compiled
as expressions rather than frozen as literals, so template expressions like
[field('name')] are properly evaluated at runtime.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Implement the full count loop compiler, replacing the stubs in count.rs,
count_any.rs, and count_bindings.rs with a single consolidated module.
Handles both field-based and value-based count nodes. Field counts walk
the resource via resolve_alias_path then iterate the wildcard array;
value counts operate on an arbitrary collection expression.
For nested wildcard paths like A[*].B[*].C, the compiler emits recursive
ForEach loops, drilling one wildcard level at a time. When an outer
count binding already covers a prefix, the inner loop starts from the
bound element register instead of re-walking from the resource root.
Existence patterns (count > 0, count == 0) are recognized and lowered
to LoopMode::Any, which exits on the first match rather than counting
every element.
Count-binding resolution threads the current-element register through
inner field references and current() calls so that nested conditions
can address fields relative to the loop variable.
Also fixes the bound_len arithmetic in conditions_wildcard.rs with a
cleaner strip_prefix call, and removes the nested-wildcard bail in
split_count_wildcard_path since the compiler now handles them.
Fill in the compiler stubs for the evaluation layer.
Condition and wildcard compilation:
- Compile allOf/anyOf/not constraints, operator conditions with
value-condition guards, and implicit allOf for unbound [*] fields
via recursive Every loops.
- Defensively lowercase prefix/suffix path segments in wildcard
handling for consistency with the collect path.
Expression and field compilation:
- Parse ARM template expressions and dispatch calls to parameters,
field, current, resourceGroup, subscription, and others.
- Compile all FieldKind variants (type, id, name, location, tags,
aliases, dynamic if/concat), resolve resource paths, and collect
wildcard values via ForEach loops.
Template function dispatch:
- Wire up 50+ ARM template functions covering string, numeric,
encoding, collection, date/time, logical, and comparison categories.
Compiler infrastructure (core.rs):
- Add emit helpers: load_literal, emit_builtin_call,
emit_chained_index_literal_path, load_input, load_context,
emit_coalesce_undefined_to_null, add_literal_u16, and
get_or_add_builtin_index.
- Add alias resolution via resolve_alias_path and strip_fq_prefix.
Misc cleanup:
- Handle ARM template `[[` escape sequences in json_value_to_runtime
and add a test for it.
- Tighten module visibility (pub -> pub(crate)/pub(super)) where
appropriate.
- Add span context to bail errors in stubs so diagnostics carry
source locations.
- Take CountBinding by reference in compile_from_binding.
- Suppress clippy warnings on the no-op memory_check stub.
Some Azure Policy JSON documents contain very long lines — ARM template
expressions with deeply nested if()/concat() calls can easily exceed
the default 1024-column lexer limit.
Add Parser::new_with_max_col() and corresponding parse_policy_rule_with_max_col()
/ parse_policy_definition_with_max_col() entry points so callers can raise
the limit when needed. Also bump ExprParser's own default to 65536 since
template expressions are routinely thousands of characters wide.
Add VM support for Azure Policy's condition operators and allOf/anyOf
short-circuit logic, gated behind cfg(feature = "azure_policy").
Policy conditions (equals, contains, like, match, exists, and their
negations — 21 total) are encoded as a single PolicyCondition
instruction with a PolicyOp sub-opcode rather than bloating the
Instruction enum with 21 variants. The dispatch handles Azure Policy's
quirky comparison semantics: case-insensitive string comparison,
string↔number coercion, null vs undefined distinction, and element-wise
collection membership.
allOf/anyOf blocks use four instructions — LogicalBlockStart,
AllOfNext/AnyOfNext, and LogicalBlockEnd — that wire up a result
register and short-circuit on the first failing (allOf) or passing
(anyOf) child.
Helper functions for case-folded comparison, wildcard/glob matching, and
type coercion live in builtins::azure_policy::helpers.
Two YAML test suites (~2200 lines) exercise the full operator matrix and
the allOf/anyOf control flow.
Default-only rules (e.g., `default deny := true` with no conditional body)
returned Undefined in the RVM instead of the default value.
Compiler:
- compute_rule_type: return Complete when rule exists only in default_rules map
- compile_worklist_rule: emit register slots and data-tree entries for
default-only rules (else branch)
VM:
- execute_call_rule_common + execute_call_rule_suspendable: check
default_literal_index before returning Undefined when definitions is empty
Tests:
- 3 new RVM cases (default_rules.yaml): bool, object, entry-point
- 3 new interpreter cases (default/basic.yaml): matching coverage
Co-authored-by: Mark Birger <markbirger@microsoft.com>
The Rego VM was designed around Rego's semantics, but Azure Policy needs
a few things Rego doesn't: host-supplied context alongside input/data,
undefined-to-null coercion for missing fields, skip-undefined collection
behavior for wildcard aliases, and non-vacuous iteration over non-array
values.
This commit adds five new instructions to bridge those gaps:
LoadContext / LoadMetadata — give programs access to host-supplied
evaluation context and cached program metadata at runtime.
ArrayPushDefined — like ArrayPush but silently drops undefined values,
so wildcard alias collection (field[*].property) excludes absent
nested properties instead of leaking undefined entries into the array.
ReturnUndefinedIfNotTrue — early return with Undefined when a guard
condition isn't satisfied, without tripping a VM assertion failure.
This models "condition doesn't match" cleanly.
CoalesceUndefinedToNull — turns Undefined into Null in-place so that
downstream builtins see null rather than short-circuiting on undefined.
The loop engine also gains an Azure Policy mode: when the source language
is "azure_policy", an Every loop over a non-array value (scalars, null,
objects) iterates once over a virtual Null element instead of being
vacuously true. This matches how field[*] behaves on non-array fields
in Azure Policy — the condition body runs once against Null, which
typically evaluates to false.
On the plumbing side: the VM gets a context field with set_context(),
metadata is cached as a Value on program load, and map_limit_error is
inlined into memory_check since it had only one call site.
Four new YAML test suites (~880 lines) cover the new instructions and
context/metadata loading, along with instruction parser, display, and
assembly listing support for everything added here.
Extend the Azure Policy parser to handle complete policyRule and
policyDefinition JSON structures, not just standalone constraints.
Policy rule parser (policy_rule.rs):
- Parse top-level { "if": ..., "then": ... } objects
- Extract effect kind (deny, audit, append, modify, etc.) into typed AST
- Parse "details" structurally when it is an object to pull out
existenceCondition as a first-class Constraint; fall back to opaque
JSON for non-object details (e.g. append array form)
- Detect duplicate/missing keys for "if", "then", "effect", "details"
Policy definition parser (policy_definition.rs):
- Handle both wrapped ARM envelope ({ "properties": { ... } }) and
unwrapped (properties-level keys at top level) forms
- Type-extract displayName, description, mode, metadata, parameters,
and policyRule; everything else goes into extra
- Parse parameter definitions with type, defaultValue, allowedValues,
and metadata; detect duplicate parameter names
- Duplicate key detection throughout
Grammar documentation (docs/azure-policy/azurepolicy.ebnf):
- Add formal EBNF grammar covering policy-rule, then-block,
constraints, conditions, all 19 operators, count expressions,
JSON values, and ARM template expressions
Test harness changes:
- Add parse_level field to YAML test cases: "constraint" (default),
"policy_rule", or "policy_definition"
- Un-skip three parse_errors cases that needed policy_rule-level parsing
- Add policy_rule.yaml with 12 cases covering all 9 effect kinds,
existenceCondition, parameterized effects, complex conditions, and
extra key handling
- Add policy_definition.yaml with wrapped, unwrapped, parameterized,
missing-policyRule, and duplicate-key error cases
Add constraint.rs module that parses Azure Policy JSON constraints
into span-annotated AST nodes:
- Logical combinators: allOf, anyOf, not
- Leaf conditions: field/value with all 19 operators
- Count blocks: field-count and value-count with where clauses
Public API: parse_constraint() parses a standalone constraint from JSON.
Includes YAML-driven test suite with 6 test files covering operators,
fields, expressions, logical combinators, count, and parse errors.
Add typed metadata support to RVM programs so that language frontends
can store language identity and arbitrary annotations alongside the
compiled bytecode.
Program metadata:
- Add `language` field to identify the source language (e.g. "rego",
"azure_policy") so the VM can adjust semantics at runtime
- Add `annotations` map (BTreeMap<String, MetadataValue>) for
frontend-specific key-value metadata
- Add MetadataValue enum with String, Bool, Integer, Float, Array,
and Object variants, plus full serde support
- Add to_value() conversion for runtime access from VM instructions
- Add has_host_await flag with recompute_host_await_presence()
Serialization:
- Bump binary format version from 5 to 6
- Add JSON serialization for the new metadata fields
Assembly listing:
- Display language and annotations in the program header
Compiler:
- Track has_host_await during Rego compilation
Add the foundational parsing infrastructure for Azure Policy JSON:
- ExprParser: ARM template expression parser for "[...]" strings,
supporting function calls, dot access, index access, and literals
- Parser (core): recursive-descent JSON tokenizer-to-AST parser that
reads directly from Lexer tokens with no intermediate serde_json step
- ParseError: structured error types with span context for diagnostics
- Helper functions: field classification, operator kind parsing, and
ARM template expression detection
These components are consumed by the policy-aware parsing modules
(constraint, policy_rule, policy_definition) in a subsequent PR.
Add span-annotated AST types for Azure Policy conditions and rules.
- PolicyDefinition, PolicyRule with if/then/details structure
- Condition enum: field conditions, value conditions, logical
combinators (allOf, anyOf, not), and count expressions
- Operator enums for all 19 Azure Policy constraint operators
(equals, contains, greater, matchInsensitively, etc.)
- Expr enum for field references, literal values (number, string,
bool), template function calls, and policy function invocations
- Value types with span tracking for error reporting
Merge the three separate Assert* instructions (AssertNot, AssertCondition,
AssertNotUndefined) into a single `Guard { register, mode }` instruction
with a GuardMode enum. This cuts duplicated match arms across display,
listing, parser, dispatch, and all compiler emit sites.
Drop the unnecessary `#[repr(C)]` from the Instruction enum. It was never
exposed across FFI, so the C-compatible 4-byte discriminant was pure waste.
Without it Rust picks a 1-byte discriminant, shrinking every instruction
from 8 bytes to 6. A new `instruction_size` unit test locks this at 6.
While touching these files, also clean up several long-standing issues:
- Deduplicate the iteration-state setup in loops.rs by extracting a shared
resolve_iteration_state() helper -- the stack-based and stackless paths
had near-identical 40-line blocks.
- Collapse the ExitWithSuccess / ExitWithFailure match arms into one.
- In rules.rs, stop cloning Arc<Program> just to borrow a RuleInfo -- clone
the small RuleInfo struct directly and extract a get_rule_info() helper.
- Move the memory check into dispatch (runs per instruction) and remove the
now-dead enforce_memory_check() entry-point calls.
- Apply map_or_else style throughout listing.rs for consistency.
* feat: add Azure Policy alias normalization/denormalization
Add normalizer and denormalizer for ARM JSON resources, enabling Azure
Policy alias short names to become direct paths into a flat structure.
- Normalizer: flattens properties wrappers, lowercases keys, resolves
per-alias versioned ARM paths, handles sub-resource array flattening,
element-level field remaps, and array base renames
- Denormalizer: reverses all transformations with casing restoration
- AliasRegistry: loads production alias catalogs and data policy manifests
- Types: serde deserialization for ARM provider alias formats
- YAML test suite: 13 test files covering normalize, denormalize, round-trip,
data-plane, edge cases, malformed input, sub-resources, and registry API
- Benchmark suite for normalization performance
* feat: add FFI and C# bindings for alias normalization
- FFI: alias_registry.rs with C-compatible API for loading catalogs,
normalizing resources, and denormalizing back to ARM JSON
- C#: AliasRegistry wrapper class with NativeMethods P/Invoke bindings
and integration tests
- Updated Cargo.lock files for new serde_json dependency
* fix: update bindings and builtins for breaking dependency upgrades
- Update rand 0.10 API: use RngExt trait instead of removed Rng trait
- Update jsonschema 0.45 API: replace removed BasicOutput/apply with
iter_errors for schema validation
- Update jni 0.22 API: migrate from deprecated JNIEnv to EnvUnowned
with_env pattern, replace deprecated get_string/new_string/throw
methods with their modern equivalents
- Update pyo3 0.28 API: replace removed PyObject with Py<PyAny>,
deprecated downcast with cast, and removed with_gil with attach
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* ci(dependabot): create per-dependency PRs for Cargo updates
Remove the groups.rust-dependencies catch-all group so Dependabot
opens a separate PR for each Cargo dependency update instead of
bundling them all into a single PR.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: enable getrandom wasm_js feature for wasm32-unknown-unknown builds
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address PR review comments
- Stream iter_errors directly into BTreeSet without intermediate Vec
- Use JNI_TRUE/JNI_FALSE for jboolean instead of bool coercion
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat: add Azure Policy builtins with YAML test suite
Implement ARM template functions for Azure Policy evaluation:
Builtins:
- String: indexOf, lastIndexOf, trim, format, split, startsWith, endsWith,
padLeft, concat, replace, toLower, toUpper, substring, guid, uniqueString
- DateTime: dateTimeAdd, dateTimeFromEpoch, dateTimeToEpoch, addDays
- Collection: intersection, union, take, skip, first, last, min, max,
range, items, tryGet, tryIndexFromEnd, empty, array, createObject
- Encoding: base64, base64ToString, base64ToJson, uri, uriComponent,
uriComponentToString, dataUri, dataUriToString
- Numeric: int, float, intDiv, intMod
- Misc: json, join, bool, string, coalesce, if, getParameter, resolveField
- Logic: logicAll, logicAny
Key implementation details:
- Unicode case-insensitive search via ICU4X case folding with single-pass
fold_with_char_map() for indexOf/lastIndexOf
- .NET composite formatting (System.String.Format) with alignment, standard
and custom datetime format specifiers, numeric format specifiers
- DateTime round-trip preserves input shape (Z vs +00:00, T vs space,
fractional seconds) when no explicit output format is supplied
- Zero-cost as_str() helper borrows directly from Value::String(Rc<str>)
- BTreeSet<&Value> in array union avoids redundant cloning
Test suite:
- 53 YAML test files exercising all builtins via direct BUILTINS registry
- Coverage for edge cases: empty inputs, Unicode, fractional seconds,
invalid alignment, unknown format specifiers, RFC3339 offset shapes
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address PR review comments
- Fix percent_encode to only uppercase hex digits, not entire string
- Remove guid/uniqueString (unsupported); delete custom SHA-1 impl
- Replace unwrap_or(0) with proper error in format placeholder parsing
- Hoist CaseMapper into static CaseMapperBorrowed for zero per-call overhead
- Pre-allocate Vec in range() with_capacity
- Update bindings/ffi and bindings/ruby Cargo.lock
- Fix uri_component test expectations for correct case preservation
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address second round of PR review comments
- float(): return Undefined when as_f64() fails instead of leaking
the original non-f64 representation
- createObject(): reject odd number of arguments with an error
(ARM-template parity)
- format(): error on unknown numeric format specifiers instead of
silently passing through (matches .NET FormatException behavior)
- format(): cap alignment width at 10,000 to prevent DoS from
user-controlled format strings like {0,1000000000}
- Add YAML test cases for all new error behaviors
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address third round of PR review comments
- percent_decode: reject incomplete % escapes (e.g. "%", "%2") instead
of treating them as literal characters
- parse_iso8601_duration: reject leftover digits without a unit designator
at T boundary and end-of-input (e.g. "P1", "P1T2H")
- yaml_to_value: panic on unsupported YAML numeric representations instead
of silently mapping to Null
- Revert unused src/languages/mod.rs changes (module is defined inline in
lib.rs)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: add missing edge-case tests and fix empty-delimiter panic
- fn_split: return input as single-element array for empty string
delimiter instead of panicking (Rust's str::split("") panics)
- format: add test for F3 higher precision ({0:F3} + 1.23456 → 1.235)
- format: add test for N2 float with thousands separator
- format: add test for negative index error ({-1})
- split: add test for empty-string delimiter
- uri: add tests for query string and fragment in relative URI
- createObject: add test for non-string (numeric) keys
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address fourth round of PR review comments
- Add MAX_VARIADIC_ARGS (64) constant for variadic builtin arity
instead of registering with 0 (logic_all, logic_any, min, max,
format, intersection, union, coalesce, createObject); set
dateTimeAdd to exact arity 3
- Switch indexOf/lastIndexOf to UTF-16 code-unit indices to match
.NET String.IndexOf semantics (track ch.len_utf16() in
fold_with_char_map, use encode_utf16().count() for empty-needle
lastIndexOf)
- Use DateTime::<Utc>::from_timestamp for explicit timezone type
- Remove stale docs/azure-policy/casing.md link from module doc
- Fix misleading comment in want_error test branch (code bails on
Undefined, not accepts it)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf!: add LRU caches for compiled regex and glob patterns
Add bounded LRU caches for compiled regex and glob patterns used by
Rego builtins, avoiding repeated recompilation of the same patterns
during policy evaluation.
New `cache` feature (included in `full-opa` and `opa-no-std`) backed by
the `lru` crate (no_std compatible) with `spin::Mutex` for thread safety.
- `src/cache.rs`: generic `LruCache<V>` wrapper, global `REGEX_CACHE`
(default capacity 256) and `GLOB_CACHE` (default capacity 128)
- `src/builtins/regex.rs`: all regex builtins route through the cache
- `src/builtins/glob.rs`: glob.match routes through the cache
- Public API: `regorus::cache::{Config, configure, clear}`
Compilation costs avoided per cache hit:
regex 10-55 µs (simple to complex patterns)
glob 10-12 µs
LRU hit ~10 ns
BREAKING CHANGE: new `cache` Cargo feature added to `full-opa` and
`opa-no-std` feature sets; adds `lru` as a dependency.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(vm): amortize per-instruction memory and time limit checks
Deduplicate per-instruction memory_check calls by hoisting them to the
main dispatch loop, and amortize monotonic_now() syscalls in the
execution timer by checking elapsed time every N instructions instead
of on every tick.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix(vm): correct object membership to check values only, not keys
The Contains instruction for objects was checking both keys and values:
object_fields.contains_key(v) || object_fields.values().any(|v| ...)
Per the Rego specification, `x in obj` tests whether x is a VALUE of
the object, not a key. The two-argument form `k, v in obj` is needed
to access keys. The interpreter already implemented this correctly
(values-only scan), but the RVM had the extra contains_key() check
which would incorrectly return true when the search value happened to
match a key name.
Remove the contains_key() branch so the behavior matches the interpreter
and the Rego spec. Add two regression tests:
- object_membership_checks_values_not_keys: "foo" in {"foo": "bar"}
must be false (key, not a value)
- object_membership_finds_value: "bar" in {"foo": "bar"} must be true
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(compiler): hoist all-constant collection literals to the literal table
When an array, set, or object literal consists entirely of compile-time
constant expressions (numbers, strings, bools, null, and nested constant
collections), the compiler now evaluates them at compile time and emits a
single Load instruction from the literal table instead of generating
per-element instructions at runtime.
Previously, a Rego expression like `x in [1, 2, 3]` would emit
ArrayCreate + three Load + three ArrayAppend instructions, allocating a
new Vec and Rc on every evaluation. With this change, the entire array
is built once during compilation and loaded as a single constant.
This optimization applies to all three collection types:
- Array literals: avoids ArrayCreate + N x (Load + ArrayAppend)
- Set literals: avoids SetCreate + N x (Load + SetAdd)
- Object literals: avoids ObjectCreate + N x (Load + Load + ObjectInsert)
The implementation adds a try_eval_const() helper that recursively
evaluates an AST expression as a constant Value, returning None if any
sub-expression is non-constant. Each compile method for collection
literals attempts the all-constant fast path first and falls through to
the existing instruction-by-instruction codegen otherwise.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(compiler): fuse Eq + AssertCondition into AssertEq instruction
Add a new `AssertEq { left, right }` instruction that combines equality
comparison and condition assertion into a single operation. This replaces
the previous two-instruction pattern of `Eq { dest, left, right }` followed
by `AssertCondition { condition: dest }`, saving one instruction and one
register per equality assertion.
The fused instruction checks two registers for equality and directly calls
handle_condition with the result, avoiding the intermediate boolean
register entirely. If either operand is undefined or the values differ,
the condition fails and the rule/loop backtracks.
The optimization applies to four destructuring sites:
- EqualityCheck (assignment re-binding with `x = expr; x = expr`)
- EqualityExpr (destructuring against an expression)
- EqualityValue (destructuring against a literal value)
- assert_array_length (array length validation in destructuring)
In soft_assert_mode the compiler still emits the original Eq instruction
since the boolean result register is needed by callers.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(compiler): fuse Not + AssertCondition into AssertNot instruction
Add a new `AssertNot { operand }` instruction that combines logical
negation and condition assertion into a single operation. This replaces
the previous two-instruction pattern of `Not { dest, operand }` followed
by `AssertCondition { condition: dest }`, saving one instruction and one
register allocation.
The fused instruction checks the operand register and passes the
condition if the value is false or undefined (per Rego semantics where
`not expr` succeeds when the expression has no results or is false),
and fails the condition if the value is true or any non-boolean truthy
value.
This was the only emission site for the Not+AssertCondition pair,
occurring in the compilation of `Literal::NotExpr` statements.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(vm): early exit for same-value multi-definition rules
When a rule has multiple definitions that all produce the same value
(e.g. implicit true, or identical literal), set early_exit_on_first_success
on RuleInfo so the VM can stop after the first successful definition.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat!: expose cache configuration API to all language bindings
Add `set_cache_config` and `clear_cache` functions to every binding
so callers can tune or reset the global regex/glob pattern caches
introduced in the cache feature.
Bindings updated:
- FFI (C): `regorus_set_cache_config`, `regorus_clear_cache`
- C++ header: free functions `regorus::set_cache_config`, `regorus::clear_cache`
- Python: module-level `set_cache_config(*, regex, glob)`, `clear_cache()`
- Java: static methods on new `CacheConfig` class
- Go: package-level `SetCacheConfig`, `ClearCache`
- Ruby: module functions `Regorus.set_cache_config`, `Regorus.clear_cache`
- WASM: free functions `setCacheConfig`, `clearCache`
- C#: static methods `Engine.SetCacheConfig`, `Engine.ClearCache`
BREAKING CHANGE: Bump SERIALIZATION_VERSION from 4 to 5 due to new
AssertEq and AssertNot instruction variants added in the instruction
fusion commits. Programs serialized with version 5 cannot be loaded
by older versions of regorus.
* fix: address PR review feedback
Cache subsystem:
- Gate REGEX_CACHE and related imports behind #[cfg(feature = "regex")]
so that building with --features cache without regex compiles correctly.
- Gate LruCache struct behind #[cfg(any(feature = "regex", feature = "glob"))].
- Add Config::MAX_CAPACITY (2^16) hard upper bound; clamp values in
configure() to prevent unbounded cache growth.
- Use parking_lot::Mutex for std builds and spin::Mutex for no_std to
avoid CPU spinning under contention in tight regex/glob eval loops.
- Narrow lock scopes in regex/glob builtins: release the mutex before
compiling a pattern, then re-acquire to insert.
Java JNI binding:
- Fix cache config overflow: negative jlong values now saturate to 0
and positive overflow saturates to usize::MAX (then clamped by
MAX_CAPACITY) instead of silently disabling the cache.
- Gate JNI cache config/clear functions behind #[cfg(feature = "cache")].
Compiler:
- Refactor static_value_of_expr to delegate to try_eval_const,
gaining support for negated numbers and constant collections.
- Make try_eval_const pub(in crate::languages::rego::compiler) and
re-export through expressions.rs.
- Handle Expr::UnaryExpr with numeric literals in try_eval_const so
collections containing negated numbers (e.g. [-1, 2]) are hoisted.
VM correctness:
- Fix Not instruction to follow Rego semantics: not expr yields
true when expr is undefined or false, false for any other defined
value (including non-booleans) -- no longer errors on non-boolean
operands.
- Add enforce_memory_check() call at execute_suspendable_entry to
ensure memory limits are checked before the first instruction.
- Update AssertNot listing comment to "exit if any defined truthy
value" to match actual VM behaviour.
- Add doc comment on Not instruction clarifying Rego negation
semantics.
Bindings:
- Fix C++ header indentation for set_cache_config / clear_cache.
- Propagate Cargo.lock parking_lot addition across ffi, java, python,
and wasm binding lockfiles.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Add PolicyLengthConfig struct with max_col, max_file_bytes, and
max_lines fields, replacing hardcoded constants in the lexer.
- Add Engine::set_policy_length_config and clear_policy_length_config
to allow callers to override the default limits.
- Add Source::from_contents_with_limits and from_file_with_limits for
direct Source construction with custom limits; existing from_contents
and from_file signatures are preserved using defaults.
- Add tests for default rejection, custom limits, and engine plumbing.
- Add bindings for C, C++, Python, WASM/JS, Java, Ruby, C#, Go
* perf(rvm): fix O(n²) comprehension yield by mutating in-place
Instead of cloning the entire accumulator collection on every yield
iteration, use take_register + Rc::make_mut to get exclusive ownership
and mutate in-place. This reduces comprehension yield from O(n²) to O(n)
for both run-to-completion and suspendable execution modes.
- Add RegoVM::take_register() helper that swaps register with Undefined
- Comprehension yield now takes the accumulator, mutates via Rc::make_mut,
and writes back — avoiding deep clones when refcount == 1
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(rvm): use take_register for ObjectSet, ArrayPush, SetAdd
These instructions were cloning the container register (bumping Rc to 2),
then calling as_object_mut/as_array_mut/as_set_mut which invokes
Rc::make_mut — deep-cloning the entire collection since refcount > 1.
Use take_register instead so the Rc refcount stays at 1, making
Rc::make_mut a no-op and allowing in-place mutation.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(rvm): remove unnecessary clones in rule caching
- execute_call_rule_common: move final_value into cache instead of
cloning, since it is not used afterwards
- finalize_rule_frame_data: add comment clarifying the clone is needed
because the value is both cached and returned
- Remove unnecessary .clone() on result_from_rule when setting register
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* rvm: avoid RuleInfo clone per rule call
Replace RuleInfo.clone() (which heap-allocates name, destructuring_blocks, and
potentially function_info) with a cheap Arc<Program> clone (atomic refcount
bump) followed by borrowing &RuleInfo from the local Arc. This eliminates
per-rule-call heap allocations.
Sites changed:
- execute_call_rule_common: Arc clone + borrow
- execute_call_rule_suspendable: Arc clone + borrow
- finalize_rule_frame_data: Arc clone + borrow
- handle_rule_break_event: inline Arc clone + borrow (was get_rule_info)
- handle_rule_error_event: inline Arc clone + borrow (was get_rule_info)
- Removed now-unused get_rule_info method
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* rvm: replace bincode with postcard for serialization
Remove unlinked bincode dependency. Use postcard (already a dep for rvm feature)
for all binary serialization/deserialization in program serialization and tests.
Also adds rvm_benchmark benchmark.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(rvm): cache dummy Span/Expr for builtin calls
Every builtin call was allocating a Source (via from_contents), a Span, and
N Ref<Expr> wrappers just to satisfy the builtin function signature. These
dummy values are only used for error reporting context.
Cache the dummy Span and Vec<Ref<Expr>> on the RegoVM struct. The Source and
Span are created once on first builtin call; dummy Expr entries grow as
needed and are reused across calls via mem::take/put-back pattern.
This eliminates per-builtin-call heap allocations for Source (Rc + String +
Vec<lines>), Span clones, and Rc<Expr> wrappers.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(rvm): round 2 allocation reduction in builtins, entry points, virtual data
- Cache builtin args Vec on RegoVM (mem::take/clear/put-back pattern)
- Restructure builtins_cache as two-level map for clone-free lookup
- Use IndexMap::get_index() in execute_entry_point_by_index
- Use mutable Vec path stack in traverse_rule_tree_subobject (push/pop)
- Walk data tree and rule-result paths by reference, clone only leaf
- Use mem::replace in resume() instead of cloning ExecutionState
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix(rvm): address PR review feedback
- Restore cached_builtin_args on all error/early-return paths in
execute_builtin_call to preserve allocation reuse
- Use 1-based line/col and \"<builtin>\" filename in dummy span for
clearer diagnostics
- Restore result register before returning errors in comprehension
mode-mismatch branches (both run-to-completion and suspendable)
- Avoid clone in resume() invalid-state error path by formatting
debug string before moving state back
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Add cfg(not(miri)) guards to mimalloc module, global allocator, and
allocator-memory-limits code paths so Miri falls back to the default
system allocator instead of calling unsupported FFI functions.
- Set MIRIFLAGS="-Zmiri-disable-isolation" in the workflow so tests
that perform file I/O can run under Miri.
- Skip units/parse tests under Miri due to Float-vs-BigInt Number
representation mismatch with Miri's soft-float emulation.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Move RVM binary encoding from bincode to postcard and bump the format version. Update test helpers, docs, changelog, and refresh lockfiles after the swap.
Closes#575
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- add Azure RBAC condition interpreter and builtin evaluation in core (expressions, parser updates, evaluator, and test harness)
- introduce comprehensive RBAC YAML test suites and coverage for i
- action/suboperation
- strings
- numbers
- bools
- IP
- GUID
- dates
- times
- lists
- quantifiers (ForAnyOfAnyValues, ForAllOfAllValues)
- expose RBAC evaluation through FFI with an `rbac` feature flag enabled by default
- add C# `RbacEngine` wrapper + P/Invoke entrypoint and document usage in C# README
- expand C# tests to execute all RBAC YAML cases with per-case logging
- wire test assets into C# test output and centralize YAML dependency versions
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Having a separate integration test allows the execution tests to freely
change the global fallback limits without affecting other tests.
also ask release-plz to ignore xtask package
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Introduce ExecutionTimer/ExecutionTimerConfig to allow limiting evaluating time.
- To amortize time checking costs, checking interval can be configured via the notion of work units
- A global fallback time limit can be set to universally limit all evaluation in addition to engine level limit setting.
- Implement limnits in interpreter and RVM. In RVM, also handle suspend/resume so that time during pause is not counted.
- Add engine-level APIs to set/clear per-engine timer configuration and apply global fallback defaults.
- Surface execution-time limits through FFI and C# bindings
- Add C# tests and example usage to validate engine overrides, global fallback behavior, and compiled policy enforcement.
- Expand docs for execution-time limit
- Add interpreter YAML cases and VM unit tests for time-limit behavior and deterministic time sources.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Policy evaluation at scale needs to be able to set memory limits
so that a bad policy does not hog memory or to ensure that
policy evaluation itself does not use too much memory which could
cause other components to suffer.
This PR introduces capability to set and enforce global memory limits.
It also lays the groundwork for enabling per evaluation limits in future.
Once a global memory limit is set, Regorus maintains per thread counters
to track memory activity (allocation, deallocation) of a thread.
These counters are periodically flushed to global memory counters.
Per thread counters avoid the contention that updating global counters
on each alloc/free would cause.
Policy evaluation periodically checks these counters and raises errors
if allocated memory has exceeded the configured limit.
Currently memory limit capability is exposed only to FFI and C#.
Also update mimalloc to v2.2.6
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Handle imports that don't use the `as` clause to create a binding.
These imports are bound to the last identifier in the imported path.
Fix both interpreter and compiler.
Add tests.
fixes#541
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Jay Lorch