Builds on #57. Swap Value::Object's payload from Rc<BTreeMap<Value, Value>>
to Rc<Object> and migrate all call sites to the Object API.
as_object / as_object_mut keep their names but return &Object / &mut Object.
The mutable accessor handles Rc::make_mut internally, so callers no longer
do it themselves. Object grows into_value() and From<Object> for Value.
Value's serializer now delegates to Object::serialize, dropping a duplicate
non-string-key stringification path.
RVM IterationState::Object is rewritten around ObjectCursor: O(log n)
steps over a shared Rc<Object>, no eager pair snapshot. Snapshot
independence is preserved by Rc copy-on-write; setup_next_iteration
advances the cursor inline and advance() becomes a no-op for this variant.
A new iteration_state_object_is_snapshot_independent_of_source test
covers CoW against a mutated alias.
Value::Set still wraps Rc<BTreeSet<Value>>; the matching Set abstraction
and its swap ship in follow-up PRs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`run_opa_tests` builds `path_dir_str` from `path.strip_prefix(...).to_string_lossy()`,
which on Windows yields strings with backslash separators (e.g.
`v0\aggregates`). The folder filter then does an exact-string
comparison against the CLI arguments:
let run_test = folders.is_empty()
|| folders.iter().any(|f| &path_dir_str == f);
CLI arguments use forward slashes (`v0/aggregates`), so on Windows
the comparison never matches, no tests are selected, and the function
bails with `"no matching tests found"`. This blocks the
`cargo xtask pre-push` hook for any Windows contributor.
Normalize `path_dir_str` to use forward slashes at construction
time. Reproduces before the fix as `cargo test ... --test opa --
v1/aggregates` exiting 1 with `no matching tests found`; after the
fix the same command runs 72 cases and the full hook command runs
2861 / 0 across 188 folders.
The duplicate platform check at the `is_rego_v0_test` site
(`path_dir_str.starts_with("v0/") || path_dir.starts_with("v0\\")`)
is left intact to keep the change minimal — the backslash branch
becomes redundant but is harmless.
Co-authored-by: Mark Birger <markbirger@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add an opaque Object type for the key→value storage backing
Value::Object. It exposes a small set of methods (get, insert, remove,
iter, iter_sorted, cursor, serde) and keeps the backing store private,
so future representations -- inline small-map, hash-backed, lazy,
arena, FFI-callback -- can plug in without touching the call sites
that name this type.
Nothing in the engine uses Object yet. Value::Object still wraps
Rc<BTreeMap<Value, Value>>; the payload swap and call-site migration
come in the next PR. Object stands on its own unit tests in the
meantime.
docs/value/object.md walks through the design, the precedents it
follows (serde_json::Map, toml::Table, simdjson DOM), and the
concrete workloads the abstraction is meant to unlock.
A matching Set abstraction follows in a separate PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Encode VM stack/context/register lifecycle invariants as
debug_assert!s. Zero cost in release; surfaces violations during
debug-mode tests and CI.
Invariants covered:
- reset_execution_state postcondition: all stacks empty, registers
resized to base and Undefined, rule_cache reset, pc/executed
counters zeroed, builtins_cache cleared, execution_state Ready.
- Per-opcode invariant check (assert_vm_invariants) invoked at the
top of run_stackless_loop and jump_to iterations: state is
Ready/Running, registers non-empty, rule_cache sized to program,
execution stack bounded by a debug-only sanity ceiling
(DEBUG_MAX_EXECUTION_STACK_DEPTH = 4096; not a production limit).
- resume() precondition: execution_state is Suspended.
- execute_suspendable_entry precondition: clean state (callers reset
immediately before).
- Rule finalize: call_rule_stack pop matches the finalized rule_index.
- IterationState::advance: Single iterator not advanced past
consumption, Array index not at usize::MAX before saturating_add.
All assertions are gated by #[cfg(debug_assertions)] (directly or via
debug_assert!) so release builds are unaffected.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(ffi): eliminate aliasing UB via to_shared_ref migration
Add to_shared_ref() helper that creates &T (shared reference) from raw
pointers instead of &mut T. This eliminates undefined behavior caused by
violating Rust's aliasing invariant when C# SafeHandle permits concurrent
FFI calls on the same handle.
With &mut T, the compiler may assume exclusive (noalias) access and
reorder or elide reads/writes — a miscompilation risk when another thread
holds a reference to the same object. Switching to &T removes that
assumption; actual mutation is mediated by the interior RwLock inside
Handle<T>, which is the sole synchronization mechanism.
Migrated sites:
- rvm.rs: 20 non-drop call sites
- engine.rs: 30 non-drop call sites + with_unwind_guard for timer fns
- compiled_policy.rs: 2 call sites
- Fix null-data UB in regorus_program_deserialize_binary
Drop paths retain to_ref() where exclusive access is guaranteed by the
caller contract (preventing use-after-free).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(ffi): add Azure Policy JSON compilation FFI and C# bindings
- AliasRegistry builder pattern: RegorusAliasRegistryBuilder (mutable,
single-threaded) + RegorusAliasRegistry (immutable, Arc-wrapped)
- Azure Policy JSON compilation: regorus_compile_azure_policy_rule and
regorus_compile_azure_policy_definition with alias registry support
- regorus_rvm_set_context for host-supplied ambient data
- C# AliasRegistryBuilder and AliasRegistry classes with convenience
factories (FromJson, FromManifest, Empty)
- C# AzurePolicyCompiler static class for policy rule/definition compilation
- Compile functions take *const RegorusAliasRegistry (read-only via
to_shared_ref for concurrent compilation safety)
- Fix pre-existing clippy warnings across multiple crates
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the compiler's two cloned BTreeMap fields (alias_map and
alias_modifiable) with a single Option<Rc<AliasRegistry>>. This
eliminates cloning two 73K-entry maps on every compilation by sharing
the registry through a reference-counted pointer.
Additional improvements:
- alias_map()/alias_modifiable_map() return &BTreeMap (zero-copy)
- Safe .get() indexing in ingest_alias_entries and build_object_from_keys
- Null-tolerant deserialization for AliasEntry.paths (~97% of real
Azure catalog entries emit "paths": null)
All changes are within the azure_policy feature gate.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Partial object rules with dynamic keys (e.g. `violations[k] if { ... }`)
only produced a single entry instead of collecting all bindings. Two
independent bugs caused this:
1. Interpreter: the early-return optimization in eval_output_expr_in_loop
checked whether the rule_ref was constant but never verified whether
the key expression was also constant. A variable key like `k` was
treated as constant output, causing the loop to exit after the first
iteration. Fixed by gating early-return on key_expr constness.
2. RVM: compute_rule_type incorrectly classified `p[k] if { ... }` as
PartialSet instead of PartialObject. OPA v1 semantics define this
form as a partial object (key -> true). Fixed the classification and
added compiler error guards for patterns the RVM codegen cannot yet
handle (constant keys, nested bracket keys), ensuring graceful
fallback to the interpreter.
The OPA test harness now skips RVM validation per-case when partial
object compiler errors are raised, rather than blanket-skipping entire
folders. This preserves RVM coverage for unrelated tests in the same
folders.
Closes#712
Co-authored-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Replace per-alias heap allocations with reference-counted string interning
throughout the normalizer's alias resolution pipeline:
- Store alias ARM path segments as Vec<Rc<str>> instead of Vec<String>,
enabling zero-alloc BTreeMap lookups via Rc::clone (refcount bump)
rather than Value::from() (heap allocation per lookup)
- Pre-compute lowercased short name (short_name_lc: Rc<str>) at
registry-load time, enabling allocation-free insertion for the
common case of non-dotted, non-collision alias short names
- Add rc_lowercase() fast-path that skips allocation when the input
string is already lowercase ASCII (common for ARM property names)
- Update set_nested_lowercased/set_nested_inner/set_nested_in_btree to
thread Rc<str> through intermediate object construction, avoiding
String temporaries at every nesting level
- Replace to_ascii_lowercase().starts_with() in is_root_field_collision
with a zero-alloc byte-level comparison
Measured 27-49% improvement in normalization time across a range of
Azure Policy alias-heavy resource types (293-608 aliases per type).
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
The cloud agent checks out a branch like copilot/review-pr-NNN which
may not have upstream/main or origin/main refs available for merge-base.
Changes:
- Use gh pr diff as primary method (always works in PR context)
- Fall back to git merge-base for local non-PR usage
- Remove path filters (*.rs *.toml examples/) — review full diff
- Remove head -2000 truncation — let agents see everything
- Explicitly fetch origin/main in copilot-setup-steps.yml as backup
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update regorus core crate and all language bindings (ffi, java, python,
wasm, ruby, csharp) to version 0.10.0.
- Centralize C# package version via Directory.Packages.props
- Remove redundant C# version suffix properties from project files
- Regenerate all binding Cargo.lock files including Ruby
- Fix xtask to read/write RegorusPackageVersion from Directory.Packages.props
instead of parsing VersionPrefix from the csproj (which now uses an MSBuild
property indirection)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add Copilot review skills, project instructions, and coding agent setup
for automated code review on regorus PRs.
Files added:
- .github/copilot-instructions.md — project context (no_std, 9 bindings,
dual execution paths, deny lints, security-critical evaluation)
- .github/skills/code-review/SKILL.md — fast single-agent review (~2 min)
- .github/skills/deep-review/SKILL.md — multi-agent deep review (~12 min)
- .github/copilot-setup-steps.yml — minimal coding agent environment
Development and testing methodology:
The skills were developed iteratively (v3 through v11.4) against a
460-line SARIF output module on the feature/sarif-output branch, which
served as a controlled test bed with 25 known issues of varying severity
(correctness, safety, API design, platform, security, performance).
Each version was tested by running the skill via the Copilot CLI, then
mapping discovered findings against the ground truth set to measure
recall and precision. Key iterations:
- v3: baseline single-agent (8/25 recall, 32%)
- v7: 3 parallel agents + verification (14/25, 56%)
- v10c: model diversity + adversarial pass (10/25, 40%)
- v11.3: merged adversarial-verifier architecture (12/25 + 2 novel, 0 noise)
- v11.4: domain expertise prompting (12/25 + 2 novel, 0 noise, full report)
The final architecture uses 3 parallel discovery agents (with cross-model
diversity and context asymmetry), risk-triggered micro-passes, and a
single adversarial verifier that both validates candidates via disproval
and hunts blind spots. Agents are prompted to reason from policy-author
perspective across Rego/OPA, Azure Policy, and RVM workloads.
Combined CR+DR catches 16-17/25 ground truth with zero false positives
and produces verified findings with confidence levels, test gap analysis,
and agent performance metrics.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a 100KB cap on compiled regex NFA size via RegexBuilder::size_limit()
to block patterns that blow up in memory or CPU. Regex compilation now
goes through a single helper (compile_regex_for_builtin) so the limit
is enforced consistently across all regex builtins.
While doing this, found and fixed a pre-existing bug: resource-limit
errors (time, memory, instruction count) raised inside builtins were
quietly swallowed to Undefined when strict_builtin_errors was off
(the default). This is a problem because `not regex.match(...)` would
see Undefined and flip to true -- silently wrong. The same issue now
applies to the new regex size limit.
Fixed by teaching the three error-absorption paths (interpreter builtin
call, RVM builtin dispatch, and RVM rule-execution loop) to recognize
LimitError and let it propagate instead of eating it.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* build(deps): update all Rust dependencies to latest versions
Bulk-update all Cargo.lock files across the workspace and bindings
to their latest compatible versions. This supersedes the individual
per-directory dependabot PRs (#678-#682) that fail CI due to version
skew when only one lockfile is updated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: refresh ALL Cargo lockfiles on dependabot PRs
Dependabot security updates bypass the grouped-updates config and
create per-directory PRs (one per Cargo.lock). This causes version
skew — e.g. rand gets bumped in bindings/ruby but stays old elsewhere,
breaking the build.
Fix by unconditionally refreshing all lockfiles whenever any Cargo
manifest or lockfile changes, rather than only the affected directory.
Also harden the workflow against expression injection:
- Move head.ref and base_ref to env vars (not inline ${{ }})
- Validate refs via git check-ref-format --branch
- Validate SHA format (hex, 40 chars) before use
- Fetch base branch by ref (not bare SHA) for reliable diffing
- Add security boundary comment on untrusted code checkout
- Add version comment on pinned checkout action SHA
Ref: https://github.com/dependabot/dependabot-core/issues/7547
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the YAML test runner that exercises the companion test data PRs, plus
several compiler fixes surfaced during testing:
- Removed parameter register caching that produced wrong results inside
short-circuiting allOf/anyOf blocks; added literal-index caching for
parameter defaults to avoid repeated O(n) literal-table scans
- Simplified cross-resource effect details to only emit roleDefinitionIds
and type (deployment templates are not evaluated for compliance)
- Replaced guid/uniqueString builtins with clear "unsupported" errors
- Normalized datetime output to ISO 8601 with Z suffix
- Added azure_policy parser MAX_COL constant (8192) for long template
expressions, keeping the global DEFAULT_MAX_COL at 1024
- Added rvm to azure_policy feature dependencies since the compiler
targets RVM bytecode
Also restructures the example binary into examples/regorus/ with new
azure-policy-eval and azure-policy-aliases subcommands, adds C# alias
normalization tests, and documents Azure Policy support in the README.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
YAML-driven test cases for the core Azure Policy compiler. These cover
alias resolution, field conditions, logical operators, type coercion,
count expressions, template functions, effect compilation, and policy
definition parsing. 24 files, each a self-contained scenario exercised
by the test runner in the companion code PR.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the stub implementations in effects.rs, effects_modify_append.rs,
and metadata.rs with full working code.
Effect compilation dispatches all effect kinds (Deny, Audit, Modify, Append,
AuditIfNotExists, DeployIfNotExists, etc.) including parameterized effects
that resolve at runtime via [parameters('effect')]. Cross-resource effects
(AINE/DINE) emit a HostAwait to fetch the related resource and evaluate an
optional existenceCondition against it. Modify and Append effects compile
their operation/detail arrays, including template expressions in values.
Metadata recording tracks which policy features are used during compilation
(field kinds, aliases, operators, resource types, count, wildcards) and
writes them into the program annotations so the runtime can inspect
capabilities without re-analyzing the AST. Definition-level metadata
(display name, category, version, parameter names, etc.) is also extracted.
Detail field values in AINE/DINE (type, name, resourceGroupName) are compiled
as expressions rather than frozen as literals, so template expressions like
[field('name')] are properly evaluated at runtime.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Implement the full count loop compiler, replacing the stubs in count.rs,
count_any.rs, and count_bindings.rs with a single consolidated module.
Handles both field-based and value-based count nodes. Field counts walk
the resource via resolve_alias_path then iterate the wildcard array;
value counts operate on an arbitrary collection expression.
For nested wildcard paths like A[*].B[*].C, the compiler emits recursive
ForEach loops, drilling one wildcard level at a time. When an outer
count binding already covers a prefix, the inner loop starts from the
bound element register instead of re-walking from the resource root.
Existence patterns (count > 0, count == 0) are recognized and lowered
to LoopMode::Any, which exits on the first match rather than counting
every element.
Count-binding resolution threads the current-element register through
inner field references and current() calls so that nested conditions
can address fields relative to the loop variable.
Also fixes the bound_len arithmetic in conditions_wildcard.rs with a
cleaner strip_prefix call, and removes the nested-wildcard bail in
split_count_wildcard_path since the compiler now handles them.
Fill in the compiler stubs for the evaluation layer.
Condition and wildcard compilation:
- Compile allOf/anyOf/not constraints, operator conditions with
value-condition guards, and implicit allOf for unbound [*] fields
via recursive Every loops.
- Defensively lowercase prefix/suffix path segments in wildcard
handling for consistency with the collect path.
Expression and field compilation:
- Parse ARM template expressions and dispatch calls to parameters,
field, current, resourceGroup, subscription, and others.
- Compile all FieldKind variants (type, id, name, location, tags,
aliases, dynamic if/concat), resolve resource paths, and collect
wildcard values via ForEach loops.
Template function dispatch:
- Wire up 50+ ARM template functions covering string, numeric,
encoding, collection, date/time, logical, and comparison categories.
Compiler infrastructure (core.rs):
- Add emit helpers: load_literal, emit_builtin_call,
emit_chained_index_literal_path, load_input, load_context,
emit_coalesce_undefined_to_null, add_literal_u16, and
get_or_add_builtin_index.
- Add alias resolution via resolve_alias_path and strip_fq_prefix.
Misc cleanup:
- Handle ARM template `[[` escape sequences in json_value_to_runtime
and add a test for it.
- Tighten module visibility (pub -> pub(crate)/pub(super)) where
appropriate.
- Add span context to bail errors in stubs so diagnostics carry
source locations.
- Take CountBinding by reference in compile_from_binding.
- Suppress clippy warnings on the no-op memory_check stub.
Update Cargo.lock to move rand to 0.10.1 so cargo-deny stops failing on RUSTSEC-2026-0097.
Also tighten the Python workflow cache boundaries by keying rust-cache to pinned runner images. The workflow now keeps Ubuntu 22.04, Ubuntu 24.04, and Windows 2022 caches separate, which avoids reusing host build artifacts across runner image changes. That is the class of issue behind the intermittent GLIBC mismatch seen in CI.
Some Azure Policy JSON documents contain very long lines — ARM template
expressions with deeply nested if()/concat() calls can easily exceed
the default 1024-column lexer limit.
Add Parser::new_with_max_col() and corresponding parse_policy_rule_with_max_col()
/ parse_policy_definition_with_max_col() entry points so callers can raise
the limit when needed. Also bump ExprParser's own default to 65536 since
template expressions are routinely thousands of characters wide.
* build(csharp): rename NuGet package to Microsoft.Regorus
Align the NuGet package identity with the Microsoft.Regorus
namespace and the Microsoft.* reserved prefix on nuget.org
in preparation for publishing the package.
- Add explicit <PackageId>Microsoft.Regorus</PackageId>
- Update PackageVersion in Directory.Packages.props
- Update version bump regex for new package name
* build(csharp): default to ProjectReference for in-repo consumers
Use ProjectReference for Regorus.Tests, Benchmarks, TestApp,
and TargetExampleApp so that local development does not require
a pre-built .nupkg. PackageReference mode remains available via
/p:UsePackageReference=true for validating the packaged NuGet.
* build(csharp): add nuget.config with source mapping and xtask integration
Add explicit NuGet configuration to ensure in-repo builds always
resolve Microsoft.Regorus from the locally built package, even
after the package is published on nuget.org.
- Add nuget.config with <clear/> and packageSourceMapping
- Update xtask to copy .nupkg into local-packages/ directory
- Pass /p:UsePackageReference=true from xtask for CI testing
- Add restore validation step to the xtask test flow
- Add .gitignore for the local-packages directory
Add VM support for Azure Policy's condition operators and allOf/anyOf
short-circuit logic, gated behind cfg(feature = "azure_policy").
Policy conditions (equals, contains, like, match, exists, and their
negations — 21 total) are encoded as a single PolicyCondition
instruction with a PolicyOp sub-opcode rather than bloating the
Instruction enum with 21 variants. The dispatch handles Azure Policy's
quirky comparison semantics: case-insensitive string comparison,
string↔number coercion, null vs undefined distinction, and element-wise
collection membership.
allOf/anyOf blocks use four instructions — LogicalBlockStart,
AllOfNext/AnyOfNext, and LogicalBlockEnd — that wire up a result
register and short-circuit on the first failing (allOf) or passing
(anyOf) child.
Helper functions for case-folded comparison, wildcard/glob matching, and
type coercion live in builtins::azure_policy::helpers.
Two YAML test suites (~2200 lines) exercise the full operator matrix and
the allOf/anyOf control flow.
Default-only rules (e.g., `default deny := true` with no conditional body)
returned Undefined in the RVM instead of the default value.
Compiler:
- compute_rule_type: return Complete when rule exists only in default_rules map
- compile_worklist_rule: emit register slots and data-tree entries for
default-only rules (else branch)
VM:
- execute_call_rule_common + execute_call_rule_suspendable: check
default_literal_index before returning Undefined when definitions is empty
Tests:
- 3 new RVM cases (default_rules.yaml): bool, object, entry-point
- 3 new interpreter cases (default/basic.yaml): matching coverage
Co-authored-by: Mark Birger <markbirger@microsoft.com>
The Rego VM was designed around Rego's semantics, but Azure Policy needs
a few things Rego doesn't: host-supplied context alongside input/data,
undefined-to-null coercion for missing fields, skip-undefined collection
behavior for wildcard aliases, and non-vacuous iteration over non-array
values.
This commit adds five new instructions to bridge those gaps:
LoadContext / LoadMetadata — give programs access to host-supplied
evaluation context and cached program metadata at runtime.
ArrayPushDefined — like ArrayPush but silently drops undefined values,
so wildcard alias collection (field[*].property) excludes absent
nested properties instead of leaking undefined entries into the array.
ReturnUndefinedIfNotTrue — early return with Undefined when a guard
condition isn't satisfied, without tripping a VM assertion failure.
This models "condition doesn't match" cleanly.
CoalesceUndefinedToNull — turns Undefined into Null in-place so that
downstream builtins see null rather than short-circuiting on undefined.
The loop engine also gains an Azure Policy mode: when the source language
is "azure_policy", an Every loop over a non-array value (scalars, null,
objects) iterates once over a virtual Null element instead of being
vacuously true. This matches how field[*] behaves on non-array fields
in Azure Policy — the condition body runs once against Null, which
typically evaluates to false.
On the plumbing side: the VM gets a context field with set_context(),
metadata is cached as a Value on program load, and map_limit_error is
inlined into memory_check since it had only one call site.
Four new YAML test suites (~880 lines) cover the new instructions and
context/metadata loading, along with instruction parser, display, and
assembly listing support for everything added here.
Extend the Azure Policy parser to handle complete policyRule and
policyDefinition JSON structures, not just standalone constraints.
Policy rule parser (policy_rule.rs):
- Parse top-level { "if": ..., "then": ... } objects
- Extract effect kind (deny, audit, append, modify, etc.) into typed AST
- Parse "details" structurally when it is an object to pull out
existenceCondition as a first-class Constraint; fall back to opaque
JSON for non-object details (e.g. append array form)
- Detect duplicate/missing keys for "if", "then", "effect", "details"
Policy definition parser (policy_definition.rs):
- Handle both wrapped ARM envelope ({ "properties": { ... } }) and
unwrapped (properties-level keys at top level) forms
- Type-extract displayName, description, mode, metadata, parameters,
and policyRule; everything else goes into extra
- Parse parameter definitions with type, defaultValue, allowedValues,
and metadata; detect duplicate parameter names
- Duplicate key detection throughout
Grammar documentation (docs/azure-policy/azurepolicy.ebnf):
- Add formal EBNF grammar covering policy-rule, then-block,
constraints, conditions, all 19 operators, count expressions,
JSON values, and ARM template expressions
Test harness changes:
- Add parse_level field to YAML test cases: "constraint" (default),
"policy_rule", or "policy_definition"
- Un-skip three parse_errors cases that needed policy_rule-level parsing
- Add policy_rule.yaml with 12 cases covering all 9 effect kinds,
existenceCondition, parameterized effects, complex conditions, and
extra key handling
- Add policy_definition.yaml with wrapped, unwrapped, parameterized,
missing-policyRule, and duplicate-key error cases
Add constraint.rs module that parses Azure Policy JSON constraints
into span-annotated AST nodes:
- Logical combinators: allOf, anyOf, not
- Leaf conditions: field/value with all 19 operators
- Count blocks: field-count and value-count with where clauses
Public API: parse_constraint() parses a standalone constraint from JSON.
Includes YAML-driven test suite with 6 test files covering operators,
fields, expressions, logical combinators, count, and parse errors.
Add typed metadata support to RVM programs so that language frontends
can store language identity and arbitrary annotations alongside the
compiled bytecode.
Program metadata:
- Add `language` field to identify the source language (e.g. "rego",
"azure_policy") so the VM can adjust semantics at runtime
- Add `annotations` map (BTreeMap<String, MetadataValue>) for
frontend-specific key-value metadata
- Add MetadataValue enum with String, Bool, Integer, Float, Array,
and Object variants, plus full serde support
- Add to_value() conversion for runtime access from VM instructions
- Add has_host_await flag with recompute_host_await_presence()
Serialization:
- Bump binary format version from 5 to 6
- Add JSON serialization for the new metadata fields
Assembly listing:
- Display language and annotations in the program header
Compiler:
- Track has_host_await during Rego compilation
Add the foundational parsing infrastructure for Azure Policy JSON:
- ExprParser: ARM template expression parser for "[...]" strings,
supporting function calls, dot access, index access, and literals
- Parser (core): recursive-descent JSON tokenizer-to-AST parser that
reads directly from Lexer tokens with no intermediate serde_json step
- ParseError: structured error types with span context for diagnostics
- Helper functions: field classification, operator kind parsing, and
ARM template expression detection
These components are consumed by the policy-aware parsing modules
(constraint, policy_rule, policy_definition) in a subsequent PR.
Add span-annotated AST types for Azure Policy conditions and rules.
- PolicyDefinition, PolicyRule with if/then/details structure
- Condition enum: field conditions, value conditions, logical
combinators (allOf, anyOf, not), and count expressions
- Operator enums for all 19 Azure Policy constraint operators
(equals, contains, greater, matchInsensitively, etc.)
- Expr enum for field references, literal values (number, string,
bool), template function calls, and policy function invocations
- Value types with span tracking for error reporting
Merge the three separate Assert* instructions (AssertNot, AssertCondition,
AssertNotUndefined) into a single `Guard { register, mode }` instruction
with a GuardMode enum. This cuts duplicated match arms across display,
listing, parser, dispatch, and all compiler emit sites.
Drop the unnecessary `#[repr(C)]` from the Instruction enum. It was never
exposed across FFI, so the C-compatible 4-byte discriminant was pure waste.
Without it Rust picks a 1-byte discriminant, shrinking every instruction
from 8 bytes to 6. A new `instruction_size` unit test locks this at 6.
While touching these files, also clean up several long-standing issues:
- Deduplicate the iteration-state setup in loops.rs by extracting a shared
resolve_iteration_state() helper -- the stack-based and stackless paths
had near-identical 40-line blocks.
- Collapse the ExitWithSuccess / ExitWithFailure match arms into one.
- In rules.rs, stop cloning Arc<Program> just to borrow a RuleInfo -- clone
the small RuleInfo struct directly and extract a get_rule_info() helper.
- Move the memory check into dispatch (runs per instruction) and remove the
now-dead enforce_memory_check() entry-point calls.
- Apply map_or_else style throughout listing.rs for consistency.
Without grouping, dependabot creates a separate PR per directory for the
same dependency. Each individual PR fails to build due to version skew
across the root workspace and binding crates.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat: add Azure Policy alias normalization/denormalization
Add normalizer and denormalizer for ARM JSON resources, enabling Azure
Policy alias short names to become direct paths into a flat structure.
- Normalizer: flattens properties wrappers, lowercases keys, resolves
per-alias versioned ARM paths, handles sub-resource array flattening,
element-level field remaps, and array base renames
- Denormalizer: reverses all transformations with casing restoration
- AliasRegistry: loads production alias catalogs and data policy manifests
- Types: serde deserialization for ARM provider alias formats
- YAML test suite: 13 test files covering normalize, denormalize, round-trip,
data-plane, edge cases, malformed input, sub-resources, and registry API
- Benchmark suite for normalization performance
* feat: add FFI and C# bindings for alias normalization
- FFI: alias_registry.rs with C-compatible API for loading catalogs,
normalizing resources, and denormalizing back to ARM JSON
- C#: AliasRegistry wrapper class with NativeMethods P/Invoke bindings
and integration tests
- Updated Cargo.lock files for new serde_json dependency
* fix: update bindings and builtins for breaking dependency upgrades
- Update rand 0.10 API: use RngExt trait instead of removed Rng trait
- Update jsonschema 0.45 API: replace removed BasicOutput/apply with
iter_errors for schema validation
- Update jni 0.22 API: migrate from deprecated JNIEnv to EnvUnowned
with_env pattern, replace deprecated get_string/new_string/throw
methods with their modern equivalents
- Update pyo3 0.28 API: replace removed PyObject with Py<PyAny>,
deprecated downcast with cast, and removed with_gil with attach
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* ci(dependabot): create per-dependency PRs for Cargo updates
Remove the groups.rust-dependencies catch-all group so Dependabot
opens a separate PR for each Cargo dependency update instead of
bundling them all into a single PR.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: enable getrandom wasm_js feature for wasm32-unknown-unknown builds
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address PR review comments
- Stream iter_errors directly into BTreeSet without intermediate Vec
- Use JNI_TRUE/JNI_FALSE for jboolean instead of bool coercion
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat: add Azure Policy builtins with YAML test suite
Implement ARM template functions for Azure Policy evaluation:
Builtins:
- String: indexOf, lastIndexOf, trim, format, split, startsWith, endsWith,
padLeft, concat, replace, toLower, toUpper, substring, guid, uniqueString
- DateTime: dateTimeAdd, dateTimeFromEpoch, dateTimeToEpoch, addDays
- Collection: intersection, union, take, skip, first, last, min, max,
range, items, tryGet, tryIndexFromEnd, empty, array, createObject
- Encoding: base64, base64ToString, base64ToJson, uri, uriComponent,
uriComponentToString, dataUri, dataUriToString
- Numeric: int, float, intDiv, intMod
- Misc: json, join, bool, string, coalesce, if, getParameter, resolveField
- Logic: logicAll, logicAny
Key implementation details:
- Unicode case-insensitive search via ICU4X case folding with single-pass
fold_with_char_map() for indexOf/lastIndexOf
- .NET composite formatting (System.String.Format) with alignment, standard
and custom datetime format specifiers, numeric format specifiers
- DateTime round-trip preserves input shape (Z vs +00:00, T vs space,
fractional seconds) when no explicit output format is supplied
- Zero-cost as_str() helper borrows directly from Value::String(Rc<str>)
- BTreeSet<&Value> in array union avoids redundant cloning
Test suite:
- 53 YAML test files exercising all builtins via direct BUILTINS registry
- Coverage for edge cases: empty inputs, Unicode, fractional seconds,
invalid alignment, unknown format specifiers, RFC3339 offset shapes
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address PR review comments
- Fix percent_encode to only uppercase hex digits, not entire string
- Remove guid/uniqueString (unsupported); delete custom SHA-1 impl
- Replace unwrap_or(0) with proper error in format placeholder parsing
- Hoist CaseMapper into static CaseMapperBorrowed for zero per-call overhead
- Pre-allocate Vec in range() with_capacity
- Update bindings/ffi and bindings/ruby Cargo.lock
- Fix uri_component test expectations for correct case preservation
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address second round of PR review comments
- float(): return Undefined when as_f64() fails instead of leaking
the original non-f64 representation
- createObject(): reject odd number of arguments with an error
(ARM-template parity)
- format(): error on unknown numeric format specifiers instead of
silently passing through (matches .NET FormatException behavior)
- format(): cap alignment width at 10,000 to prevent DoS from
user-controlled format strings like {0,1000000000}
- Add YAML test cases for all new error behaviors
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address third round of PR review comments
- percent_decode: reject incomplete % escapes (e.g. "%", "%2") instead
of treating them as literal characters
- parse_iso8601_duration: reject leftover digits without a unit designator
at T boundary and end-of-input (e.g. "P1", "P1T2H")
- yaml_to_value: panic on unsupported YAML numeric representations instead
of silently mapping to Null
- Revert unused src/languages/mod.rs changes (module is defined inline in
lib.rs)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: add missing edge-case tests and fix empty-delimiter panic
- fn_split: return input as single-element array for empty string
delimiter instead of panicking (Rust's str::split("") panics)
- format: add test for F3 higher precision ({0:F3} + 1.23456 → 1.235)
- format: add test for N2 float with thousands separator
- format: add test for negative index error ({-1})
- split: add test for empty-string delimiter
- uri: add tests for query string and fragment in relative URI
- createObject: add test for non-string (numeric) keys
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: address fourth round of PR review comments
- Add MAX_VARIADIC_ARGS (64) constant for variadic builtin arity
instead of registering with 0 (logic_all, logic_any, min, max,
format, intersection, union, coalesce, createObject); set
dateTimeAdd to exact arity 3
- Switch indexOf/lastIndexOf to UTF-16 code-unit indices to match
.NET String.IndexOf semantics (track ch.len_utf16() in
fold_with_char_map, use encode_utf16().count() for empty-needle
lastIndexOf)
- Use DateTime::<Utc>::from_timestamp for explicit timezone type
- Remove stale docs/azure-policy/casing.md link from module doc
- Fix misleading comment in want_error test branch (code bails on
Undefined, not accepts it)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* build: consolidate dependabot cargo entries and add commit prefixes
Consolidate all 9 separate cargo ecosystem entries into a single entry
using the 'directories' key. This ensures Dependabot creates one PR per
dependency update across the root workspace and all bindings, preventing
version skew that caused build failures.
Also add semantic commit-message prefixes to all ecosystem entries:
- build(deps) for cargo, gomod, maven, nuget, pip, bundler
- ci(deps) for github-actions
Rename the cargo group to 'rust-dependencies' and the github-actions
group to 'github-actions' for clarity.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: remove mimalloc from default features, fix indexmap/std propagation
Address #595: the vendored mimalloc allocator should not be imposed on
library consumers. Remove allocator-memory-limits and mimalloc from the
full-opa feature so that users of regorus as a library can choose their
own global allocator.
Bindings (ffi, java, python, ruby) that ship as standalone artifacts
continue to opt in to regorus/allocator-memory-limits explicitly so they
retain the performant allocator.
Also propagate indexmap/std via the std feature (using the indexmap?/std
weak-dependency syntax) so that users enabling std + rvm without default
features no longer hit 'IndexMap takes 3 generic arguments' errors.
Closes#595
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* ci: add feature-combination checks to PR CI and weekly matrix
PR CI (xtask): add cargo check for 5 non-default feature combos in
run_ci_suite(). These run on every PR and catch compile failures from
feature-gating issues (e.g. #595) with near-zero overhead.
Weekly workflow: new feature-matrix.yml runs cargo build + cargo test
across 9 feature combinations every Monday. Uses a GitHub Actions matrix
with fail-fast: false so all combos are tested even if one fails.
Combinations tested weekly:
- std,arc (minimal library)
- std,arc,rvm (common library usage)
- std,arc,full-opa (full-opa without mimalloc)
- std,arc,full-opa,allocator-memory-limits (binding-style)
- std,arc,rvm,regex,time,semver,cache (cherry-picked builtins)
- std,arc,rvm,coverage,cache (observability)
- std,arc,full-opa,azure_policy (Azure Policy)
- std,arc,full-opa,azure-rbac (Azure RBAC)
- arc,opa-no-std (no_std codepath)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: gate benchmark memory-limit calls behind allocator-memory-limits feature
The set_global_memory_limit function is only available when the
allocator-memory-limits feature is enabled. After removing mimalloc
from the default feature set, the rvm_benchmark failed to compile.
Add #[cfg(feature = "allocator-memory-limits")] guards around the
call sites and the MEMORY_LIMIT_BYTES constant.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf!: add LRU caches for compiled regex and glob patterns
Add bounded LRU caches for compiled regex and glob patterns used by
Rego builtins, avoiding repeated recompilation of the same patterns
during policy evaluation.
New `cache` feature (included in `full-opa` and `opa-no-std`) backed by
the `lru` crate (no_std compatible) with `spin::Mutex` for thread safety.
- `src/cache.rs`: generic `LruCache<V>` wrapper, global `REGEX_CACHE`
(default capacity 256) and `GLOB_CACHE` (default capacity 128)
- `src/builtins/regex.rs`: all regex builtins route through the cache
- `src/builtins/glob.rs`: glob.match routes through the cache
- Public API: `regorus::cache::{Config, configure, clear}`
Compilation costs avoided per cache hit:
regex 10-55 µs (simple to complex patterns)
glob 10-12 µs
LRU hit ~10 ns
BREAKING CHANGE: new `cache` Cargo feature added to `full-opa` and
`opa-no-std` feature sets; adds `lru` as a dependency.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(vm): amortize per-instruction memory and time limit checks
Deduplicate per-instruction memory_check calls by hoisting them to the
main dispatch loop, and amortize monotonic_now() syscalls in the
execution timer by checking elapsed time every N instructions instead
of on every tick.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix(vm): correct object membership to check values only, not keys
The Contains instruction for objects was checking both keys and values:
object_fields.contains_key(v) || object_fields.values().any(|v| ...)
Per the Rego specification, `x in obj` tests whether x is a VALUE of
the object, not a key. The two-argument form `k, v in obj` is needed
to access keys. The interpreter already implemented this correctly
(values-only scan), but the RVM had the extra contains_key() check
which would incorrectly return true when the search value happened to
match a key name.
Remove the contains_key() branch so the behavior matches the interpreter
and the Rego spec. Add two regression tests:
- object_membership_checks_values_not_keys: "foo" in {"foo": "bar"}
must be false (key, not a value)
- object_membership_finds_value: "bar" in {"foo": "bar"} must be true
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(compiler): hoist all-constant collection literals to the literal table
When an array, set, or object literal consists entirely of compile-time
constant expressions (numbers, strings, bools, null, and nested constant
collections), the compiler now evaluates them at compile time and emits a
single Load instruction from the literal table instead of generating
per-element instructions at runtime.
Previously, a Rego expression like `x in [1, 2, 3]` would emit
ArrayCreate + three Load + three ArrayAppend instructions, allocating a
new Vec and Rc on every evaluation. With this change, the entire array
is built once during compilation and loaded as a single constant.
This optimization applies to all three collection types:
- Array literals: avoids ArrayCreate + N x (Load + ArrayAppend)
- Set literals: avoids SetCreate + N x (Load + SetAdd)
- Object literals: avoids ObjectCreate + N x (Load + Load + ObjectInsert)
The implementation adds a try_eval_const() helper that recursively
evaluates an AST expression as a constant Value, returning None if any
sub-expression is non-constant. Each compile method for collection
literals attempts the all-constant fast path first and falls through to
the existing instruction-by-instruction codegen otherwise.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(compiler): fuse Eq + AssertCondition into AssertEq instruction
Add a new `AssertEq { left, right }` instruction that combines equality
comparison and condition assertion into a single operation. This replaces
the previous two-instruction pattern of `Eq { dest, left, right }` followed
by `AssertCondition { condition: dest }`, saving one instruction and one
register per equality assertion.
The fused instruction checks two registers for equality and directly calls
handle_condition with the result, avoiding the intermediate boolean
register entirely. If either operand is undefined or the values differ,
the condition fails and the rule/loop backtracks.
The optimization applies to four destructuring sites:
- EqualityCheck (assignment re-binding with `x = expr; x = expr`)
- EqualityExpr (destructuring against an expression)
- EqualityValue (destructuring against a literal value)
- assert_array_length (array length validation in destructuring)
In soft_assert_mode the compiler still emits the original Eq instruction
since the boolean result register is needed by callers.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(compiler): fuse Not + AssertCondition into AssertNot instruction
Add a new `AssertNot { operand }` instruction that combines logical
negation and condition assertion into a single operation. This replaces
the previous two-instruction pattern of `Not { dest, operand }` followed
by `AssertCondition { condition: dest }`, saving one instruction and one
register allocation.
The fused instruction checks the operand register and passes the
condition if the value is false or undefined (per Rego semantics where
`not expr` succeeds when the expression has no results or is false),
and fails the condition if the value is true or any non-boolean truthy
value.
This was the only emission site for the Not+AssertCondition pair,
occurring in the compilation of `Literal::NotExpr` statements.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(vm): early exit for same-value multi-definition rules
When a rule has multiple definitions that all produce the same value
(e.g. implicit true, or identical literal), set early_exit_on_first_success
on RuleInfo so the VM can stop after the first successful definition.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat!: expose cache configuration API to all language bindings
Add `set_cache_config` and `clear_cache` functions to every binding
so callers can tune or reset the global regex/glob pattern caches
introduced in the cache feature.
Bindings updated:
- FFI (C): `regorus_set_cache_config`, `regorus_clear_cache`
- C++ header: free functions `regorus::set_cache_config`, `regorus::clear_cache`
- Python: module-level `set_cache_config(*, regex, glob)`, `clear_cache()`
- Java: static methods on new `CacheConfig` class
- Go: package-level `SetCacheConfig`, `ClearCache`
- Ruby: module functions `Regorus.set_cache_config`, `Regorus.clear_cache`
- WASM: free functions `setCacheConfig`, `clearCache`
- C#: static methods `Engine.SetCacheConfig`, `Engine.ClearCache`
BREAKING CHANGE: Bump SERIALIZATION_VERSION from 4 to 5 due to new
AssertEq and AssertNot instruction variants added in the instruction
fusion commits. Programs serialized with version 5 cannot be loaded
by older versions of regorus.
* fix: address PR review feedback
Cache subsystem:
- Gate REGEX_CACHE and related imports behind #[cfg(feature = "regex")]
so that building with --features cache without regex compiles correctly.
- Gate LruCache struct behind #[cfg(any(feature = "regex", feature = "glob"))].
- Add Config::MAX_CAPACITY (2^16) hard upper bound; clamp values in
configure() to prevent unbounded cache growth.
- Use parking_lot::Mutex for std builds and spin::Mutex for no_std to
avoid CPU spinning under contention in tight regex/glob eval loops.
- Narrow lock scopes in regex/glob builtins: release the mutex before
compiling a pattern, then re-acquire to insert.
Java JNI binding:
- Fix cache config overflow: negative jlong values now saturate to 0
and positive overflow saturates to usize::MAX (then clamped by
MAX_CAPACITY) instead of silently disabling the cache.
- Gate JNI cache config/clear functions behind #[cfg(feature = "cache")].
Compiler:
- Refactor static_value_of_expr to delegate to try_eval_const,
gaining support for negated numbers and constant collections.
- Make try_eval_const pub(in crate::languages::rego::compiler) and
re-export through expressions.rs.
- Handle Expr::UnaryExpr with numeric literals in try_eval_const so
collections containing negated numbers (e.g. [-1, 2]) are hoisted.
VM correctness:
- Fix Not instruction to follow Rego semantics: not expr yields
true when expr is undefined or false, false for any other defined
value (including non-booleans) -- no longer errors on non-boolean
operands.
- Add enforce_memory_check() call at execute_suspendable_entry to
ensure memory limits are checked before the first instruction.
- Update AssertNot listing comment to "exit if any defined truthy
value" to match actual VM behaviour.
- Add doc comment on Not instruction clarifying Rego negation
semantics.
Bindings:
- Fix C++ header indentation for set_cache_config / clear_cache.
- Propagate Cargo.lock parking_lot addition across ffi, java, python,
and wasm binding lockfiles.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Add PolicyLengthConfig struct with max_col, max_file_bytes, and
max_lines fields, replacing hardcoded constants in the lexer.
- Add Engine::set_policy_length_config and clear_policy_length_config
to allow callers to override the default limits.
- Add Source::from_contents_with_limits and from_file_with_limits for
direct Source construction with custom limits; existing from_contents
and from_file signatures are preserved using defaults.
- Add tests for default rejection, custom limits, and engine plumbing.
- Add bindings for C, C++, Python, WASM/JS, Java, Ruby, C#, Go
* perf(rvm): fix O(n²) comprehension yield by mutating in-place
Instead of cloning the entire accumulator collection on every yield
iteration, use take_register + Rc::make_mut to get exclusive ownership
and mutate in-place. This reduces comprehension yield from O(n²) to O(n)
for both run-to-completion and suspendable execution modes.
- Add RegoVM::take_register() helper that swaps register with Undefined
- Comprehension yield now takes the accumulator, mutates via Rc::make_mut,
and writes back — avoiding deep clones when refcount == 1
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(rvm): use take_register for ObjectSet, ArrayPush, SetAdd
These instructions were cloning the container register (bumping Rc to 2),
then calling as_object_mut/as_array_mut/as_set_mut which invokes
Rc::make_mut — deep-cloning the entire collection since refcount > 1.
Use take_register instead so the Rc refcount stays at 1, making
Rc::make_mut a no-op and allowing in-place mutation.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(rvm): remove unnecessary clones in rule caching
- execute_call_rule_common: move final_value into cache instead of
cloning, since it is not used afterwards
- finalize_rule_frame_data: add comment clarifying the clone is needed
because the value is both cached and returned
- Remove unnecessary .clone() on result_from_rule when setting register
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* rvm: avoid RuleInfo clone per rule call
Replace RuleInfo.clone() (which heap-allocates name, destructuring_blocks, and
potentially function_info) with a cheap Arc<Program> clone (atomic refcount
bump) followed by borrowing &RuleInfo from the local Arc. This eliminates
per-rule-call heap allocations.
Sites changed:
- execute_call_rule_common: Arc clone + borrow
- execute_call_rule_suspendable: Arc clone + borrow
- finalize_rule_frame_data: Arc clone + borrow
- handle_rule_break_event: inline Arc clone + borrow (was get_rule_info)
- handle_rule_error_event: inline Arc clone + borrow (was get_rule_info)
- Removed now-unused get_rule_info method
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* rvm: replace bincode with postcard for serialization
Remove unlinked bincode dependency. Use postcard (already a dep for rvm feature)
for all binary serialization/deserialization in program serialization and tests.
Also adds rvm_benchmark benchmark.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(rvm): cache dummy Span/Expr for builtin calls
Every builtin call was allocating a Source (via from_contents), a Span, and
N Ref<Expr> wrappers just to satisfy the builtin function signature. These
dummy values are only used for error reporting context.
Cache the dummy Span and Vec<Ref<Expr>> on the RegoVM struct. The Source and
Span are created once on first builtin call; dummy Expr entries grow as
needed and are reused across calls via mem::take/put-back pattern.
This eliminates per-builtin-call heap allocations for Source (Rc + String +
Vec<lines>), Span clones, and Rc<Expr> wrappers.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* perf(rvm): round 2 allocation reduction in builtins, entry points, virtual data
- Cache builtin args Vec on RegoVM (mem::take/clear/put-back pattern)
- Restructure builtins_cache as two-level map for clone-free lookup
- Use IndexMap::get_index() in execute_entry_point_by_index
- Use mutable Vec path stack in traverse_rule_tree_subobject (push/pop)
- Walk data tree and rule-result paths by reference, clone only leaf
- Use mem::replace in resume() instead of cloning ExecutionState
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix(rvm): address PR review feedback
- Restore cached_builtin_args on all error/early-return paths in
execute_builtin_call to preserve allocation reuse
- Use 1-based line/col and \"<builtin>\" filename in dummy span for
clearer diagnostics
- Restore result register before returning errors in comprehension
mode-mismatch branches (both run-to-completion and suspendable)
- Avoid clone in resume() invalid-state error path by formatting
debug string before moving state back
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Add cfg(not(miri)) guards to mimalloc module, global allocator, and
allocator-memory-limits code paths so Miri falls back to the default
system allocator instead of calling unsupported FFI functions.
- Set MIRIFLAGS="-Zmiri-disable-isolation" in the workflow so tests
that perform file I/O can run under Miri.
- Skip units/parse tests under Miri due to Float-vs-BigInt Number
representation mismatch with Miri's soft-float emulation.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Move RVM binary encoding from bincode to postcard and bump the format version. Update test helpers, docs, changelog, and refresh lockfiles after the swap.
Closes#575
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Add support for registering custom Python functions as Rego extensions,
allowing users to call Python callables directly from Rego policies.
The implementation:
- Converts Rego values to Python types on call, and back on return
- Validates that the extension is callable at registration time
- Wraps errors with the extension name for easier debugging
- Documents clone semantics (shared callable reference across clones)
Tests cover: basic execution, type conversions (int, float, bool, None,
list, dict, set), zero-arg extensions, wrong arity, exception
propagation, non-callable rejection, and duplicate registration.
Contributed by @paulolieuthier
- add Azure RBAC condition interpreter and builtin evaluation in core (expressions, parser updates, evaluator, and test harness)
- introduce comprehensive RBAC YAML test suites and coverage for i
- action/suboperation
- strings
- numbers
- bools
- IP
- GUID
- dates
- times
- lists
- quantifiers (ForAnyOfAnyValues, ForAllOfAllValues)
- expose RBAC evaluation through FFI with an `rbac` feature flag enabled by default
- add C# `RbacEngine` wrapper + P/Invoke entrypoint and document usage in C# README
- expand C# tests to execute all RBAC YAML cases with per-case logging
- wire test assets into C# test output and centralize YAML dependency versions
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Expand dependabot coverage across Rust subcrates and other ecosystems.
- Group updates per dependency and ignore vendored mimalloc crates.
- Pin GitHub Actions to exact SHAs in existing workflows.
Additionally
- Include more metadata in nuget package
- Also generate snupkg for native symbols.
We intentionally don't add the symbols for native rust shared library
to the nuget package since that could increase the size of the nuget.
We will revisit that later.
- update licenses of all the bindings.
closes#551
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Prefix regorus- to mimalloc crates and add MIT licenses
- alias dependencies to avoid code changes
- add versions and release-plz publish entries
- update Cargo.lock files for new crate names
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Having a separate integration test allows the execution tests to freely
change the global fallback limits without affecting other tests.
also ask release-plz to ignore xtask package
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Bump up the versions to 0.9.0 to match the C# binding version.
Also use central version management for C# projects
Also fix clippy lint errors
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Introduce ExecutionTimer/ExecutionTimerConfig to allow limiting evaluating time.
- To amortize time checking costs, checking interval can be configured via the notion of work units
- A global fallback time limit can be set to universally limit all evaluation in addition to engine level limit setting.
- Implement limnits in interpreter and RVM. In RVM, also handle suspend/resume so that time during pause is not counted.
- Add engine-level APIs to set/clear per-engine timer configuration and apply global fallback defaults.
- Surface execution-time limits through FFI and C# bindings
- Add C# tests and example usage to validate engine overrides, global fallback behavior, and compiled policy enforcement.
- Expand docs for execution-time limit
- Add interpreter YAML cases and VM unit tests for time-limit behavior and deterministic time sources.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- split the xtask crate into structured modules for
- bindings
- ci
- dev
- util
- no-std
- Adding commands for
- ci-release/ci-debug
- MUSL/no-std
- per- binding language smoke tests
- developer tasks (fmt, clippy, pre-commit, pre-push)
- refresh Cargo manifests/locks, binding readmes, and shared FFI helpers so every binding reuses the same preparation steps
- refactor GitHub Actions (release/debug, extensions, CodeQL, clippy, bindings) to call the new xtask commands
- Use rust-cache in ci workflows (microsoft qdk also does this)
- extend README with a contributor workflow section describing how xtask mirrors CI expectations
- update pre-commit and pre-push hooks to use the xtask dev commands
WORKAROUND:
When dotnet is run from an xtask, codeql tracer intercepts it an routes to a nonexistent binary.
Therefore in codeql workflow, xtask is not used for c# and instead dotnet is directly invoked.
Tracked by #545closes#475
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Ruby binstubs were checked in, but ignore via .gitignore **bin pattern.
This causes release-plz to think that the source tree is dirty.
The binstubs are deleted from source repo since they are always regenerated by bundler.
Also simplizy release-plz to focus only on the core crate.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Policy evaluation at scale needs to be able to set memory limits
so that a bad policy does not hog memory or to ensure that
policy evaluation itself does not use too much memory which could
cause other components to suffer.
This PR introduces capability to set and enforce global memory limits.
It also lays the groundwork for enabling per evaluation limits in future.
Once a global memory limit is set, Regorus maintains per thread counters
to track memory activity (allocation, deallocation) of a thread.
These counters are periodically flushed to global memory counters.
Per thread counters avoid the contention that updating global counters
on each alloc/free would cause.
Policy evaluation periodically checks these counters and raises errors
if allocated memory has exceeded the configured limit.
Currently memory limit capability is exposed only to FFI and C#.
Also update mimalloc to v2.2.6
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
This PR implements widely accepted Rust programming practices for
dealing with panics across ABI (programming language) boundaries.
- Add panic_guard.rs to wrap FFI calls and prevent panic across FFI/ABI boundary (undefined behavior).
- Capture per-thread backtraces via a temporary panic hook
- After a panic, subsequent invocations are poisoned.
- Integrate with_unwind_guard across the engine, schema registry, and target registry exportis
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Handle imports that don't use the `as` clause to create a binding.
These imports are bound to the last identifier in the imported path.
Fix both interpreter and compiler.
Add tests.
fixes#541
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Change the SPDX license expression from `MIT` to
`MIT AND Apache-2.0 AND BSD-3-Clause` to reflect all of the licenses
that apply to the crate’s sources.
Add license text for `Apache-2.0` and the `BSD-3-Clause` license from
Go’s `time` module to `LICENSE`. Like `MIT`, both of these licenses
require the license text to be distributed with source and/or binaries.
- Document arithmetic safety assumptions and add explicit lexer limits for columns, file size (1 MiB), and line count.
Realistic policies will be well within these bounds.
- Use checked arithmetic to prevent overflow underflow.
- Avoid var name shadowing.
- Misc clippy lints
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Made the lookup module crate-visible to address clippy’s redundant visibility lint.
- Replaced unchecked as casts with a fallible usize_from_u32 helper and propagate conversion errors in lookup accessors.
- Switched LookupIndexError to implement core::error::Error for no_std correctness.
- Fixed the pattern type mismatch by matching on the value in the Display impl.
- Promoted trivial helpers to const fn (new, module_len) per clippy suggestions.
- Centralized bounds-checked slot access via slot_ref/slot_mut to keep getters/clearers lint-clean and avoid unchecked indexing.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Lints are added (deny) at crate level.
In each offending file, the failing lints are explicitly allowed.
Each file will be fixed in subsequent PRs.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
In case all the statements of a query don't execute,
skip reordering the result expressions to match the
source order. Doing so requires maintaining additional
data structures not worth the complexity for now.
Additionally we want to discourage queries and encourage
evaluating rules. Queries are inherently less performant
than rules which can be precompiled.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- ensure both run-to-completion and suspendable rule execution stop evaluating
bodies once one succeeds so later else branches are skipped
- test cases
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
RVM does not plan to support the `with` keyword which is mainly used
for testing.
- introduce CompilerError::WithKeywordUnsupported and fail query compilation
when any literal carries with_mods
- skip OPA test cases that hit the error
The "withkeyword" folder is retained in the TODO list to indicate its
lack of support.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- treat set subtraction in RVM the same as the interpreter by supporting
Value::Set operands in sub_values
- emit internal-only builtin names for set union/intersection and register
handlers so compiled bytecode resolves without exposing new Rego builtins
- add regression coverage for literal set difference/intersection
(x/y from failure.rego) in tests/rvm/rego/cases/sets.yaml
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Release mode uses LTO optimization for binaries.
This can take up a lot of time especially for doc tests which
create a separate binary for each test.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- emit AssertCondition for equality-only assignment plans (outside soft-assert mode) so rules like `0 = 1` fail under the VM just like the interpreter
- let comprehension bodies consume assertion failures by advancing or exiting their iteration context, both in run-to-completion and suspendable execution
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Allow compile_chained_ref to fall back to “evaluate root expression → chain access”
so literal arrays, comprehensions, and other computed roots no longer raise NotSimpleReferenceChain.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Mirror interpreter implementation:
- use builtins::must_cache to determine whether builtin must be cached.
- reuse cached value when applicable
- clear the VM’s builtin cache whenever execution state resets to avoid leaking values across runs
- add a YAML regression for rand.intn set comprehensions and re-enable the rand cases in the OPA test suite
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Add a scoped soft_assert_mode to the compiler so `not` statements compile their subexpressions without emitting hard AssertCondition/AssertNotUndefined instructions.
- Teach binding-plan application to return an optional result register; equality plans now yield a boolean in soft mode, allowing not abs(-5 , 3) to succeed instead of aborting.
- Update function-call, loop, and rule plumbing to consume the new binding-plan outcome, including copying the produced register when an out-parameter equality is used.
- Trim the OPA TODO list to the remaining troublesome folders.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Teach the hoister/destructuring planner to respect parent scope when building binding plans for extra arguments, so already-bound vars yield equality checks.
- Update the compiler’s function-call path to drop the trailing out-argument, run its binding plan after the call, and share call-target resolution logic.
- Add regression suites for builtin and user-defined out-parameter scenarios plus align the CLI example output when RVM returns undefined.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- fixes:
- ensure loop hoist lookups reserve query capacity and keep loop-var tables sized when compiling default rules
- rebuild hoisting tables with the analyzer’s schedule when available so statement order matches evaluation
- OPA test
- Also test using RVM workflow in OPA suite
- Maintain a list of test folders that don't yet pass and skip them
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
code fixes:
- compiler: add `is_var_bound_in_current_scope` and use it in destructuring so
only the innermost scope blocks rebinding while still catching duplicates
within that block.
- rvm: treat `not` over undefined operands as a successful negation to match
interpreter semantics.
tests/aci:
migrate YAML cases to `data.policy.rule` queries with `{x: …}`
bindings, expand the harness to run interpreter plus RVM (with optional
skipping), align results to the binding format, add readable timing output,
and support a `--filter` flag for targeting cases.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Supply chain: Use the popular num-bigint crate for handling large integers
- Optimization: Handle f64, i64, u64 directly. These will be the most common instances of a number.
OPA number semantics isn't clear.
https://github.com/open-policy-agent/opa/issues/6281
As part of this change, we update the following failing tests:
- A local test that relies on what 15.3/3 evaluates to.
With our current change, we round in a different direction than what OPA does, but consistent
with Rust. We produce 5.1000000000000005 where as the OPA test expects 5.1.
There is no clear definition in Rego of what the right answer is. Moreover, policies should not
rely on exact floating point value comparison. Therefore this deviations is justified.
The test is patched to pass.
- Another local vm test that exercised 1.1 + 2.2
- Another local vm test that exercises 5.5 - 2.2
- An OPA test that expects that a large integer number say 10e308 is printed in exponent notation.
num-bigint does not print using scientific notation and instead prints all the digits.
The benefit of preserving this compatibility is not clear. We skip this test.
- Doc tests that exercised handling floating point numbers with more than 15 (what f64 supports)
digits of precision. There is no usecase for this scenario. The tests are updated to reflect
the behavior.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Add runtime detection for shared handle misuse
wrap the FFI engine handle with parking_lot::RwLock when the new
contention_checks feature is enabled, surfacing a clear “handle is already
in use” error instead of allowing undefined behavior
keep the feature optional so no_std builds or environments that supply
their own synchronization can opt out
caution users that this guards the handle itself but does not make the
engine’s operations globally thread-safe on its own
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat: Add Azure RBAC condition parser
- declare an `azure-rbac` feature and expose the Azure RBAC module with parser, AST, and YAML-driven tests
- extend the shared lexer with RBAC-specific tokens, single-quoted strings, and corrected raw-string spans
- verify the parser via comprehensive test cases covering every operator and complex chaining
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat!: add Rego Virtual Machine (RVM) implementation
This commit introduces a register-based virtual machine for executing Rego
policies with bytecode-style instructions. Unlike the existing tree-walking
interpreter, the RVM compiles policies into instruction sequences that operate
on virtual registers, offering better performance and optimization potential.
Core Components:
Instruction Set Architecture:
- Define instruction types for data operations, control flow, and builtins
- Implement instruction parameter encoding and display formatting
- Add instruction parser with comprehensive test coverage
Virtual Machine Engine:
- Register-based execution model with program counter management
- Loop execution supporting iterators, comprehensions, and quantifiers
- Function call handling with argument evaluation and context management
- Rule evaluation with default value resolution and virtual data support
- Arithmetic and comparison operation implementations
Program Representation:
- Program listing builder with instruction sequencing
- Rule tree construction for organizing policy rules
- Binary and JSON serialization for compiled programs
- Recompilation support for program modification
Testing Infrastructure:
- Extensive YAML test suites covering all VM features
- Rust unit tests for VM execution and instruction parsing
- Test suites for loops, comprehensions, builtins, and control flow
BREAKING CHANGE: Introduces new VM execution path alongside interpreter
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* docs: add detailed RVM architecture references
Introduce architecture.md explaining program artifacts, serialization, and runtime subsystems.
Document the full opcode catalog in instruction-set.md, including operands, parameter tables, and outcomes.
Walk through execution flow, stacks, and operational guidance in vm-runtime.md, tying the runtime to the new architecture docs.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* build: Add xtask automation for binding version management
Introduces a dedicated xtask crate that keeps language binding versions
in sync with the core regorus crate, following the workflow pattern used
by rust-analyzer, gitoxide, and ripgrep.
Key features:
- Git-based change detection: compares binding source files against a
base ref (merge-base with origin/main by default) plus unstaged/
untracked files to identify which bindings have been modified
- SemVer-aware bumping: binding edits trigger a minor version increment
(e.g. 0.5.1 → 0.6.0) under pre-1.0 semantics, signaling potential
breaking changes; clean bindings simply align to the root version
- Multi-language support: updates Cargo manifests (Rust FFI, Java,
Python, WASM, Ruby), Maven pom.xml (Java), Ruby version constants,
and C# project files in a single pass
- CI integration: --check mode fails fast when manifests are out of
sync, ensuring pre-commit and release-plz workflows catch stale
versions before merge
Integration points:
- release-plz.toml: runs cargo xtask bindings --base-ref origin/main
after bumping the root crate, so binding versions are updated
atomically during the release process
- scripts/pre-commit: invokes cargo xtask bindings --check to block
commits that would leave bindings out of sync
- .cargo/config.toml: defines cargo xtask alias for convenience
Documentation includes inline examples showing how version bumps behave
when bindings are ahead/behind the root, and notes that the minor
field acts as the major version under SemVer 0.y.z initial development
phase.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* build: refresh xtask tooling, workflows, and locks
- cargo xtask bindings: keep the binding version-sync pipeline intact
- cargo xtask update-deps: new helper to regenerate workspace/binding Cargo.lock files
- workflows: auto-detect the Java jar version in CI and temporarily disable the Ruby workflow
- lock files: refresh root + binding snapshots after the dependency sweep
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Introduce Utf8Marshaller helpers and SafeHandle wrappers so the managed API centralizes UTF-8 conversions and lifetime management for native pointers.
- Update Engine, Compiler, CompiledPolicy, SchemaRegistry, and TargetRegistry to rely on the new marshaller/safe handles, tightening disposal and reducing transient allocations during interop calls.
- Add allocation guard coverage in Regorus.Tests and report bytes/op in the compiled policy benchmark to surface future regressions.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- add a dedicated `compiler/destructuring_planner` feature that precomputes binding plans for assignments, parameters, and `some in` expressions
- enrich `ScopeContext` with same-scope tracking, local scheduling hints, and module globals so the planner enforces := shadowing rules without blocking parent scopes
- wire the planner through compiler, hoist, interpreter, and engine paths while updating binding plan variants and adding query traversal helpers for dependency analysis
- document the new planner architecture and ship interpreter regressions that exercise nested destructuring, shadowing, and error reporting
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Introduce a compiler pass that analyzes and pre-computes loop hoisting information
during policy compilation. This hoisted metadata is stored in lookup tables and made
available to downstream consumers:
- interpreter: use HoistedLoop entries during evaluation (replaces runtime scanning)
- type inference: can leverage pre-computed loop structure for type propagation
- RVM compiler: will consume hoisting metadata for optimized bytecode generation
Changes:
- populate loop hoisting tables during engine preparation and query snippet execution
- refactor eval_stmts_in_loop and eval_output_expr_in_loop to consume HoistedLoop directly
- add helper methods for accessing loop expressions, collections, and indices from HoistedLoop
- extend Lookup with get_checked and into_slots for safe query context access and merging
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Major changes:
- Implement the `net.cidr_contains` builtin
- Enable the v0 and v1 test for `net.cidr_contains`
- Add the `netip` crate to standardize CIDR searching and other
operations
Key Concept:
- Allow users to leverage the `net.cidr_contains` builtin to check
whether an IPv4 or IPv6 CIDR contains a specified IP address or
subnet.
Testing:
- All tests passing.
Signed-off-by: tjons <tylerschade99@gmail.com>
Major Changes:
- Add generic Lookup<T> structure for efficient O(1) module-level data access
- Combine separate scope and order lookups into unified QuerySchedule structure
- Add query_schedule field to Interpreter for dedicated user query scheduling
- Refactor loop hoising to separate module
- Use efficient lookup for loop vars
- Also added more tests for loops
Key Concept:
- Ensure module context and indexing stay synchronized during function calls
Testing:
- All scheduler and interpreter tests passing
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Security Improvements:
- Pin all GitHub Actions to specific commit hashes instead of version tags
- Update actions/checkout from v4 to commit 08eba0b27e820071cde6df949e0beb9ba4906955
- Update actions/setup-python from v5 to commit a26af69be951a213d495a4c3e4e4022e16d87065 (v5.6.0)
- Update actions/setup-java from v4 to commit dded0888837ed1f317902acf8a20df0ad188d165 (v5.0.0)
- Update actions/setup-node from v4 to commit 1e60f620b9541d16bece96c5465dc8ee9832be0b (v4.4.0)
- Update actions/setup-go from v5 to commit 41dfa10bad2bb2ae585af6ee5bb4d7d973ad74ed (v5.1.0)
- Update actions/setup-dotnet from v4 to commit 3e891b0cb619bf60e2c25674b222b8940e2c1c25 (v4.1.0)
- Update actions/upload-artifact from v4 to commit ea165f8d65b6e75b540449e92b4886f43607fa02 (v4.6.2)
- Update actions/download-artifact from v4 to commit 634f93cb2916e3fdff6788551b99b062d0335ce0 (v5.0.0)
- Update github/codeql-action from v3 to commit 01fe2e8c43536ad5e1085bad5e7cd6fbc8a30988 (v3.29.11)
Rust Toolchain Consolidation:
- Create custom composite action .github/actions/toolchains/rust/action.yml
- Standardize on Rust 1.89.0 (latest stable) with clippy and rustfmt components
- Add optional targets parameter for cross-compilation support
- Replace dtolnay/rust-toolchain@stable across 16 workflows
This creates a more secure, maintainable, and consistent CI/CD pipeline
with centralized Rust toolchain management across all workflows.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Disable publishing for `ensure_no_std` test crate
- Add write content permissions to release-plz job expliclity
Signed-off-by: Burak Varlı <burakvar@amazon.co.uk>
This commit introduces a complete multi-threaded evaluation benchmark suite for both Rust and C# implementations of Regorus.
- Implemented engine evaluation benchmark with input and engine cloning strategies
- Implemented compiled policy evaluation benchmark with input cloning and shared compiled policy strategies.
- Created EngineEvaluationBenchmark.cs and CompiledPolicyEvaluationBenchmark.cs with time-based execution (3s warmup + 3s evaluation)
- Implemented configuration options matching Rust implementation (useClonedEngines, useSharedPolicies parameters)
- Created markdown analysis documentation with cross-platform performance analysis
- C# seems to achieve 58-89% of Rust performance on test machine.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Since Regorus is MIT-licensed, it would be better to specify `license = "MIT"` rather than using `licene-file` to point to the MIT license file.
Some tools do not support parsing of `license-file`, for example crates.io classifies Regorus' license as "non-standard":
$ curl -s https://crates.io/api/v1/crates/regorus/0.4.0 | jq .version.license
"non-standard"
Using `license` would provide better compatability with various tools.
Signed-off-by: Burak Varlı <burakvar@amazon.co.uk>
Details:
- Implement complete Type enum with 12 variants: Any, Integer, Number, Boolean,
Null, String, Array, Set, Object, Enum, Const, AnyOf
- Add Schema wrapper struct with reference counting for efficient sharing
- Support JSON Schema-compatible deserialization with serde
- Implement discriminated subobjects for polymorphic type definitions
- Add comprehensive test suite covering all type variants
- Include Azure resource schema examples (Storage, VM, Key Vault, App Service)
- Create meta-schema validation system with lazy static validator
- Add extensive edge case and corner case test coverage
- Implement custom deserializers for complex schema patterns
This establishes the foundation for type checking and validation of Rego
policies, particularly useful for cloud resource schemas and policy validation.
Regorus's type system is a first of many features intended to
enable type checking and various other constraints on Rego policies.
The type system is inspired from:
- JSON schema
- Bicep
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
The clone optimization PR didn't have the latest changes for "azure_policy".
Integration resulted in compile errors.
Also fix errors due to updated clippy lints.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Introduce the notion of CompiledPolicy to hold stuff that
remains immutable during evaluation - e.g. rules, function,
schedules etc
Cloning takes about 60 nano seconds for an engine loaded with
ACI policies. Earlier it used to take 40 microseconds.
Thus there is easily more than 100x speedup.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
When using a clean build dir the install failed as it referenced
a non-existing `regorus_ffiCorrosion.cmake` file. I believe I used
the shorter `regorus_ffi` as my `EXPORT` in the `corrosion_install`
at some point and then later didn't notice that I referenced a stale
generated file when I initially handed in this PR. We must make sure
that the same identifier is used here too. See also the documentation
from `corrosion_install`:
> * **EXPORT**: Creates an export that can be installed with `install(EXPORT)`. <export-name> must be globally unique.
> Also creates a file at ${CMAKE_BINARY_DIR}/corrosion/<export-name>Corrosion.cmake that must be included in the installed config file.
bindings/ruby/bin/console and bindings/ruby/bin/setup show up
as dirty to release-plz and causes it to fail to update.
As a workaround, set allow_dirty to true to enable update.
However ensure that no dirty files are published by setting
publish_allow_dirty to false.
Also default to not publishing any packages except regorus.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Also update binding versions and lock files.
Note:
- Ruby binding is not updated
- C# binding is v0.7.0. We will make it match Regorus version later.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: use early exit in 'some in' statements
Update kata tests:
Since 'early return' now works with 'some in' statement, interpreter
does not do any evaluation after it found match for rule, therefore
we don't have other rule checks after interpreter found match
Indexes allow associating extra data with nodes in the AST
using an array and then quickly looking up the array to fetch
the extra data.
- Index eidx for expressions
- Index sidx for statements
- Index qidx for queries.
AST nodes are not cloneable. Therefore once a module is created,
it is not possible to accidentally create two nodes with the same
index inadvertently via clone.
Also added IndexChecker in debug builds. When a module is parsed,
it will assert that indexes have been constructed correctly.
AST Cleanup
- Make literal expressions (null, val, number, string etc) also structs
to match all other expressions
- Merge True and False nodes into a single Bool node.
Also update dependencies.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
This leverages the still-undocumented `corrosion_install` to get
an installable ffi binding that can be consumed by other projects,
e.g. from yocto:
```
$ ninja install
[4/5] Install the project...
-- Install configuration: "Debug"
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/libregorus_ffi.so
-- Up-to-date: /home/milian/projects/compiled/regorus-test/include/regorus_ffi/regorus.hpp
-- Up-to-date: /home/milian/projects/compiled/regorus-test/include/regorus_ffi/regorus.ffi.hpp
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/cmake/regorus_ffi/regorus_ffi_targets.cmake
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/cmake/regorus_ffi/regorus_ffiConfig.cmake
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/cmake/regorus_ffi/regorus_ffiCorrosion.cmake
```
This can then be consumed as such:
```
find_package(regorus_ffi CONFIG REQUIRED)
add_executable(test test.cpp)
target_link_libraries(test PRIVATE regorus_ffi::regorus_ffi)
```
- Documentation
- Regorus Engine is intended to be used from a single thread
- Clone the engine after adding policies and data to use from another thread
- Builtin errors strictness:
- default to less strict for OPA compatibility
- Provide API to change strictness
- Expose GetAstAsJson to C#,
This can allow writing policy validations in C#.
- Use spectre mitigated msvc crt libs (binskim compliance)
- Update dependencies
fixes#404
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
else blocks following contains and old-style sets will raise
a parse error. Consistent with OPA.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Cryptographic builtins are removed due to various reasons like FIPS
compliance. Users needing crypto builtins are encouraged to use
extensions.
Deprecated functions are also removed.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Specify compatibility = linux in pyproject.toml to ensure
manylinux compatibility.
- Use abi-py310 pyo3 feature to ensure compatibility with python
3.10 and later.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Organize C# binding example into separate Regorus nuget package and
a test app.
The nuget package targets netstandard 2.0 and 2.1.
The test app is tested for netframework 8.0.
Implement IDisposable for Engine cleanup.
Also remove net40 example. Can be added back later if needed.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Remove dependency on jsonwebtoken which brings in the ring crate.
Ring crate triggers governance violations.
Support for JWT will be implemented in future using a more governance
compliant crate.
BREAKING CHANGE
Prior to this PR, support for jwt builtins was minimially implemented.
Only io.jwt.decode and io.jwt.decode_verify was implemented.
With this PR, those builtins will no longer be available. They are
planned to be implemented in the future. In the meantime, they can be
brought back in via Engine::add_extension.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Removed cryptographically insecure sha1. This existed only for OPA
compatibility.
Also exclude bindings from main workspace
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Regorus now defaults to rego v1. `import rego.v1` is no longer needed.
Additionally, `future` keywords are automatically imported.
See
https://www.openpolicyagent.org/docs/latest/v0-upgrade/#changes-to-rego-in-opa-v10
to understand the differences between rego v1 and v0.
BREAKING CHANGE:
v0 style policies will error out by default. To enable v0 behavior, call engine.set_rego_v0(true) before
loading policies.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Specify `js` feature for `uuid` when building wasm by
specifying it as a non-optional dependency in wasm binding's Cargo.toml.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Add `or` operator to Rego languages. Available via `rego-extensions`
Cargo feature.
If the evaluated lhs value is not false, null or undefined it is returned.
Otherwise rhs is evaluated and returned.
or operator has least precedence, and is left-associative.
closes#314
Also add test to lock down example policy path.
Also Fix clippy warning by using unwrap_or_default
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
A block with a single or expression needs to be treated as a comprehension instead of a
set/array with 1 item. e.g.: {1 | 1 }, [2 | foo]
Allow successfully parsing object comprehensions as rule body
x if { 1:2 | 1 }
fixes#306, fixes#307
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Handle undefined values correctly in ordered-else. Previously an undefined value
in one of the blocks could cause the entire rule to evaluate to undefined.
Handle undefined values correctly in generic rule refs to prevent them from
propagating to output.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
In case of empty delimiter, Rust's split returns leading and trailing
empty strings whereas Golang's doesn't.
Change behavior to match Golang/OPA.
fixes#291
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Init document is the aggregated data documen that the user has
specified using multiple `add_data` calls. Each query evaluation
starts of by initializing the current data to the init document.
Previously `add_data` was incorrectly added to the current document,
causing the added data to be lost if the addition happened after query
evaluation.
With this fix, scenarios where data addition may be interspersed with
query evaluation calls are supported.
Also provide a get_data method to obtain the (init) data document.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Earlier scheduler only recognized rules and would raise an
`unsafe var` error on alias.
Register alias var names to fix this.
fixes#284
Also fix clippy warning treated as error
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Created an example of extension policy
- Added C# binding support of .NET framework 4.0 and created a Nuget
spec for it.
- Added a pytest in python bindings to test the extension policy and the
python binding
- Restructured the example and Csharp binding directories due to above
changes.
- Added copyrights.
- Added a Windows workflow for .NET 4.0 build and test.
- Fix warning due to use of deprecated function.
This was causing a build issue in the hava and csharp bindings
- Lock use of csbindgen@1.9.0
The newer version 1.9.2 causes a "type of namespace C could not be fond" error
In the generated code, struct inherits from C instead of uint
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Use regorus_ffi in target_link_libraries instead of regorus-ffi.
Something seems to have changed in corrosion-rs to need this.
- Workaround for cmake issue where FFI header may not be generated in time
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- c, cpp
- csharp
- ffi
- go
- Java
- Python
- WASM
`arc` feature is turned on for all bindings
Use pretty string instead of colored string.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Disable default features in dependencies
- Use anyhow::Error::msg to map errors. Note: anyhow will itself be removed later.
- lazy_static/spin_no_std used in no_std environments
- ensure_no_std binary is built to target thumbv7m-none-eabi to ensure that
there are no std dependencies. thumbv7m-none-eabi target has no std support.
- The opa-no-std feature enables only those Regorus features that work with no_std.
- Enable tests with no_std
- Update sizes of regorus binary in README.md
- Ensure that regorus example can be built with only std
- Ensure that regorus example can be built with no_std
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- `std` feature is enabled by default
- By default enable #![no_std] compilation
- Import std create if `std` feature is enabled or if testing
- Use core, alloc types
- Make it clear where std types are being used
- In no std, use BTreeMap in place of HashMap.
HashMap is not available in no std due to lack of a
secure random number generator
Note: The project does not yet compile without std feature being specified.
But it's really close to being able to do so.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Replace std with alloc, core in most places in src
Tests, bindings aren't changed.
- Introduce BuiltinsMap type alias inplace of HashMap.
In no_std case, this could be aliases to BTreeMap
- Fix clippy warnings
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-07 18:41:09 -07:00
950 changed files with 244962 additions and 5040 deletions
- Azure RBAC condition interpreter with builtin evaluation coverage and YAML test suite, including quantifier (ForAnyOfAnyValues/ForAllOfAllValues), datetime (DateTimeEquals), IP (IpInRange), GUID (GuidEquals), list (ListContains), and string (StringEquals) semantics.
- FFI surface for Azure RBAC condition evaluation (see bindings changelog for language-specific wrappers).
### Fixed
- harden regex builtins with compiled-size limit (#705)
- *(ci)* skip mimalloc FFI and disable isolation for Miri (#621)
### Other
- bump version to 0.10.0 across all bindings
- *(deps)* update all Rust dependencies and fix lockfile refresh workflow (#704)
- *(deps)* bump com.google.code.gson:gson (#702)
- *(deps)* bump the github-actions group across 1 directory with 5 updates (#690)
- *(deps)* bump the per-dependency group across 1 directory with 5 updates (#703)
- Make `git rev-parse` in `build.rs` optional with graceful fallback (#701)
- *(azure_policy)* add foundation test cases (#698)
- *(azure_policy)* add end-to-end policy test cases (#699)
- fix rand advisory and harden python CI caching (#675)
- azure-policy parser: allow overriding the column-width limit (#673)
- *(deps)* bump the rust-dependencies group across 5 directories with 6 updates (#671)
- *(deps)* bump ruby/setup-ruby in the github-actions group (#670)
- *(csharp)* prepare NuGet package for nuget.org publishing (#668)
- Fix RVM evaluation of default-only rules (#664)
- *(deps)* bump minitest in /bindings/ruby in the per-dependency group (#656)
- *(deps)* bump the rust-dependencies group across 2 directories with 3 updates (#657)
- consolidate RVM instruction variants and clean up VM internals (#651)
- *(deps)* bump wasm-bindgen-test (#650)
- *(deps)* bump rb_sys in /bindings/ruby in the per-dependency group (#649)
- *(deps)* bump the rust-dependencies group across 3 directories with 4 updates (#647)
- *(deps)* bump the github-actions group across 1 directory with 3 updates (#646)
- Centralize C# handle gating with a short dispose wait and deferred release to avoid leaks while blocking new calls ([#571](https://github.com/microsoft/regorus/pull/571)).
### Added
- Manual C# memory growth tests for both `using` and finalizer paths ([#571](https://github.com/microsoft/regorus/pull/571)).
- C# test runner options for filtered tests, console logging, and skipping sample apps ([#571](https://github.com/microsoft/regorus/pull/571)).
- Update binding versions for next release ([#270](https://github.com/microsoft/regorus/pull/270))
- rename method from 'Clone' to 'clone' in 'Engine' class to match the java naming convention and definiont in the of java.lang.Object. ([#268](https://github.com/microsoft/regorus/pull/268))
- *Rego*-*Rus(t)* - A fast, light-weight [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/)
interpreter written in Rust.
interpreter written in Rust.
- *Rigorous* - A rigorous enforcer of well-defined Rego semantics.
Regorus is also
- *cross-platform* - Written in platform-agnostic Rust.
- *current* - We strive to keep Regorus up to date with latest OPA release. Regorus supports `import rego.v1`.
- *compliant* - Regorus is mostly compliant with the latest [OPA release v0.64.0](https://github.com/open-policy-agent/opa/releases/tag/v0.64.0). See [OPA Conformance](#opa-conformance) for details. Note that while we behaviorally produce the same results, we don't yet support all the builtins.
- *no_std compatible* - Regorus can be used in `no_std` environments too. Most of the builtins are supported.
- *current* - We strive to keep Regorus up to date with latest OPA release. Regorus defaults to `v1` of the Rego language.
- *compliant* - Regorus is mostly compliant with the latest [OPA release v1.2.0](https://github.com/open-policy-agent/opa/releases/tag/v1.2.0). See [OPA Conformance](#opa-conformance) for details. Note that while we behaviorally produce the same results, we don't yet support all the builtins.
- *extensible* - Extend the Rego language by implementing custom stateful builtins in Rust.
See [add_extension](https://github.com/microsoft/regorus/blob/fc68bf9c8bea36427dae9401a7d1f6ada771f7ab/src/engine.rs#L352).
Support for extensibility using other languages coming soon.
Regorus passes the [OPA v0.64.0 test-suite](https://www.openpolicyagent.org/docs/latest/ir/#test-suite) barring a few
Regorus passes the [OPA v1.2.0 test-suite](https://www.openpolicyagent.org/docs/latest/ir/#test-suite) barring a few
builtins. See [OPA Conformance](#opa-conformance) below.
## Bindings
@@ -107,10 +107,11 @@ Regorus can be used from a variety of languages:
- *C*: C binding is generated using [cbindgen](https://github.com/mozilla/cbindgen).
[corrosion-rs](https://github.com/corrosion-rs/corrosion) can be used to seamlessly use Regorous
in your CMake based projects. See [bindings/c](https://github.com/microsoft/regorus/tree/main/bindings/c).
in your CMake based projects. See [bindings/c](https://github.com/microsoft/regorus/tree/main/bindings/c).
- *C freestanding*: [bindings/c_no_std](https://github.com/microsoft/regorus/tree/main/bindings/c_no_std) shows how to use Regorus from C environments without a libc.
- *C++*: C++ binding is generated using [cbindgen](https://github.com/mozilla/cbindgen).
[corrosion-rs](https://github.com/corrosion-rs/corrosion) can be used to seamlessly use Regorous
in your CMake based projects. See [bindings/cpp](https://github.com/microsoft/regorus/tree/main/bindings/cpp).
in your CMake based projects. See [bindings/cpp](https://github.com/microsoft/regorus/tree/main/bindings/cpp).
- *C#*: C# binding is generated using [csbindgen](https://github.com/Cysharp/csbindgen). See [bindings/csharp](https://github.com/microsoft/regorus/tree/main/bindings/csharp) for an example of how to build and use Regorus in your C# projects.
- *Golang*: The C bindings are exposed to Golang via [CGo](https://pkg.go.dev/cmd/cgo). See [bindings/go](https://github.com/microsoft/regorus/tree/main/bindings/go) for an example of how to build and use Regorus in your Go projects.
- *Python*: Python bindings are generated using [pyo3](https://github.com/PyO3/pyo3). Wheels are created using [maturin](https://github.com/PyO3/maturin). See [bindings/python](https://github.com/microsoft/regorus/tree/main/bindings/python).
@@ -128,7 +129,7 @@ It is straight-forward to build these bindings yourself.
## Getting Started
[examples/regorus](https://github.com/microsoft/regorus/blob/main/examples/regorus.rs) is an example program that
[examples/regorus](https://github.com/microsoft/regorus/blob/main/examples/regorus/main.rs) is an example program that
shows how to integrate Regorus into your project and evaluate Rego policies.
To build and install it, do
@@ -144,6 +145,7 @@ $ regorus
Usage: regorus <COMMAND>
Commands:
ast Parse a Rego policy and dump AST
eval Evaluate a Rego Query
lex Tokenize a Rego policy
parse Parse a Rego policy
@@ -182,11 +184,11 @@ This produces the following output
}
```
Next, evaluate a sample [policy](https://github.com/microsoft/regorus/blob/main/examples/example.rego) and [input](https://github.com/microsoft/regorus/blob/main/examples/input.json)
Next, evaluate a sample [policy](https://github.com/microsoft/regorus/blob/main/examples/server/allowed_server.rego) and [input](https://github.com/microsoft/regorus/blob/main/examples/server/input.json)
(borrowed from [Rego tutorial](https://www.openpolicyagent.org/docs/latest/#2-try-opa-eval)):
Regorus uses a small companion CLI under the `xtask` package to keep CI and local development in sync.
The commands mirror our GitHub Actions jobs, making it easy to dry-run CI steps before sending a pull request.
- Run the full release pipeline with `cargo xtask ci-release` and the debug checks with `cargo xtask ci-debug`.
- Exercise language bindings through focused helpers such as `cargo xtask test-java --release --frozen` or `cargo xtask test-go`.
- Use `cargo xtask test-musl --release --frozen` for the cross-compilation matrix and `cargo xtask test-no-std` for embedded targets.
- Formatting (`cargo xtask fmt`) and linting (`cargo xtask clippy --sarif`) wrap the usual Cargo tooling while matching CI defaults.
The workflows in `.github/workflows` invoke the same commands, so keeping local runs green is usually enough to satisfy the checks enforced on `main`.
## OPA Conformance
Regorus has been verified to be compliant with [OPA v0.64.0](https://github.com/open-policy-agent/opa/releases/tag/v0.64.0)
Regorus has been verified to be compliant with [OPA v1.2.0](https://github.com/open-policy-agent/opa/releases/tag/v1.2.0)
using a [test driver](https://github.com/microsoft/regorus/blob/main/tests/opa.rs) that loads and runs the OPA testsuite using Regorus, and verifies that expected outputs are produced.
Currently, Regorus passes all the non-builtin specific tests.
See [passing tests suites](https://github.com/microsoft/regorus/blob/main/tests/opa.passing).
The following test suites don't pass fully due to mising builtins:
-`cryptoparsersaprivatekeys`
-`cryptox509parseandverifycertificates`
-`cryptox509parsecertificaterequest`
-`cryptox509parsecertificates`
-`cryptox509parsekeypair`
-`cryptox509parsersaprivatekey`
The following test suites don't pass fully due to missing builtins:
-`globsmatch`
-`graphql`
-`invalidkeyerror`
-`jsonpatch`
-`jwtbuiltins`
-`jwtdecodeverify`
-`jwtencodesign`
-`jwtencodesignheadererrors`
-`jwtencodesignpayloaderrors`
-`jwtencodesignraw`
-`jwtverifyhs256`
-`jwtverifyhs384`
-`jwtverifyhs512`
-`jwtverifyrsa`
-`netcidrcontains`
-`netcidrcontainsmatches`
-`netcidrexpand`
-`netcidrintersects`
-`netcidrisvalid`
-`netcidrmerge`
-`netcidroverlap`
-`netlookupipaddr`
@@ -320,6 +375,7 @@ The following test suites don't pass fully due to mising builtins:
They are captured in the following [github issues](https://github.com/microsoft/regorus/issues?q=is%3Aopen+is%3Aissue+label%3Alib).
Cryptographic builtins are not supported by design. Users that need cryptographic builtins are encouraged to use [extensions](https://docs.rs/regorus/latest/regorus/struct.Engine.html#method.add_extension).
A benchmark suite for measuring the multi-threaded performance of the Regorus policy evaluation engine.
## Overview
This benchmark evaluates the throughput and scalability of Regorus policy evaluation across different thread counts and configuration strategies. It measures performance variations between fresh and cloned engine instances, as well as fresh and cloned input data.
## Features
- **Multi-threaded evaluation** testing from 1 to `num_cpus * 2` threads
- **Configurable engine strategies**: Fresh vs. cloned engine instances
- **Configurable input strategies**: Fresh parsing vs. cloned input data
- **Complex policy evaluation** using realistic RBAC and data sensitivity policies
- **Criterion-based benchmarking** with statistical analysis
- **Performance metrics** including throughput and timing
## Benchmark Structure
### Test Configurations
The benchmark tests four different configuration combinations:
1.**Cloned Engines + Cloned Inputs**: Pre-instantiated engines with pre-parsed input data
- **Test Data**: 20,000 inputs per evaluation (1000 per thread)
- **Policy**: Complex authorization policy with nested rules
## Benchmark Overview
The compiled policy evaluation benchmark tests Regorus compiled policy performance across multiple thread configurations (1-32 threads). It measures throughput (thousands of evaluations per second) for different combinations of compiled policy and input data reuse strategies.
## Configuration Combinations
1.**Compiled Shared Policies, Cloned Inputs**: Each thread uses shared compiled policies and clones of parsed input data - optimal for performance
2.**Compiled Shared Policies, Fresh Inputs**: Each thread uses shared compiled policies but parses new inputs each time
3.**Compiled Per Iteration, Cloned Inputs**: Each thread compiles the policy each iteration but reuses input data
4.**Compiled Per Iteration, Fresh Inputs**: Each thread compiles new policies and parses new inputs for each iteration
The compiled policy evaluation shows performance characteristics that are generally comparable to engine evaluation, though with some notable differences. While single-threaded performance is very close between the systems, there are observable impacts from the compilation approach that become more apparent under different threading scenarios.
**Key Observations:**
- **Single-threaded performance**: Very close parity between systems, though results may vary between runs
- **Multi-threaded impact**: Compiled policies show more pronounced performance degradation under thread contention in shared policy configurations
- **Contention resistance**: Per-iteration compilation shows more consistent (though lower absolute) performance across thread counts
- **Optimal usage**: Both systems achieve best results with minimal threading (1-4 threads), though engine evaluation maintains better performance at higher thread counts for shared configurations
- **Test Data**: 20,000 inputs per evaluation (1000 per thread)
- **Policy**: Complex authorization policy with nested rules
## Benchmark Overview
The engine evaluation benchmark tests Regorus policy evaluation performance across multiple thread configurations (1-32 threads). It measures throughput (thousands of evaluations per second) for different combinations of engine and input data reuse strategies.
## Configuration Combinations
1.**Cloned Engines, Cloned Inputs**: Each thread uses its own engine and clones of parsed input data - optimal for performance
2.**Cloned Engines, Fresh Inputs**: Each thread uses its own engine but parses new inputs each time
3.**Fresh Engines, Cloned Inputs**: Each thread creates a new engine each iteration but reuses input data
4.**Fresh Engines, Fresh Inputs**: Each thread creates new engines and parses new inputs for each iteration
6. **Thread Contention**: Performance degradation occurs with higher thread counts across all configurations, though mimalloc helps mitigate some allocation-related contention
This document describes the C# API for Regorus, focusing on the compiled policy approach for high-performance policy evaluation.
## Overview
The Regorus C# bindings provide a modern, thread-safe API for compiling and evaluating Open Policy Agent (OPA) Rego policies. The API is designed around pre-compiled policies that can be evaluated efficiently multiple times with different inputs.
- **Pre-compiled Policies**: Compile once, evaluate many times for optimal performance
- **Target System Support**: Built-in support for Azure Policy targets with resource type inference
- **Thread Safety**: All operations are thread-safe without external synchronization
- **Registry Management**: Centralized management of targets and schemas
- **Policy Introspection**: Rich metadata about compiled policies
## Core Classes
### CompiledPolicy
The `CompiledPolicy` class represents a pre-compiled Rego policy that can be evaluated efficiently.
```csharp
publicsealedclassCompiledPolicy:IDisposable
{
// Evaluate the policy with input data
publicstring?EvalWithInput(stringinputJson);
// Get comprehensive policy metadata
publicPolicyInfoGetPolicyInfo();
// Dispose of unmanaged resources
publicvoidDispose();
}
```
**Thread Safety**: All methods are thread-safe. Multiple threads can call `EvalWithInput()` concurrently, and `Dispose()` will safely wait for active evaluations to complete.
### Compiler
The `Compiler` class provides static methods for compiling policies.
```csharp
publicstaticclassCompiler
{
// Compile a policy with a specific entrypoint rule
// Compile a target-aware policy (requires azure_policy feature)
publicstaticCompiledPolicyCompilePolicyForTarget(
stringdataJson,
IEnumerable<PolicyModule>modules);
}
```
### PolicyModule
Represents a single policy module to be compiled. Each PolicyModule corresponds to a Rego file (.rego), and each Rego file defines a Rego package using the `package` declaration at the top of the file.
```csharp
publicstructPolicyModule
{
publicstringId{get;set;}
publicstringContent{get;set;}
publicPolicyModule(stringid,stringcontent);
}
```
**Properties:**
-`Id`: A unique identifier for the module, typically the filename (e.g., "policy.rego", "rules/storage.rego")
-`Content`: The complete Rego policy content, including the `package` declaration and all rules
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.