9 Commits

Author SHA1 Message Date
Anand Krishnamoorthi
d36f952133 feat(azure-policy): add alias normalization and denormalization (#635)
* feat: add Azure Policy alias normalization/denormalization

Add normalizer and denormalizer for ARM JSON resources, enabling Azure
Policy alias short names to become direct paths into a flat structure.

- Normalizer: flattens properties wrappers, lowercases keys, resolves
  per-alias versioned ARM paths, handles sub-resource array flattening,
  element-level field remaps, and array base renames
- Denormalizer: reverses all transformations with casing restoration
- AliasRegistry: loads production alias catalogs and data policy manifests
- Types: serde deserialization for ARM provider alias formats
- YAML test suite: 13 test files covering normalize, denormalize, round-trip,
  data-plane, edge cases, malformed input, sub-resources, and registry API
- Benchmark suite for normalization performance

* feat: add FFI and C# bindings for alias normalization

- FFI: alias_registry.rs with C-compatible API for loading catalogs,
  normalizing resources, and denormalizing back to ARM JSON
- C#: AliasRegistry wrapper class with NativeMethods P/Invoke bindings
  and integration tests
- Updated Cargo.lock files for new serde_json dependency
2026-03-30 18:44:36 -05:00
Anand Krishnamoorthi
86088d2049 Consolidate Dependabot, fix #595 (mimalloc + indexmap), add feature-matrix CI (#627)
* build: consolidate dependabot cargo entries and add commit prefixes

Consolidate all 9 separate cargo ecosystem entries into a single entry
using the 'directories' key. This ensures Dependabot creates one PR per
dependency update across the root workspace and all bindings, preventing
version skew that caused build failures.

Also add semantic commit-message prefixes to all ecosystem entries:
- build(deps) for cargo, gomod, maven, nuget, pip, bundler
- ci(deps) for github-actions

Rename the cargo group to 'rust-dependencies' and the github-actions
group to 'github-actions' for clarity.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* fix: remove mimalloc from default features, fix indexmap/std propagation

Address #595: the vendored mimalloc allocator should not be imposed on
library consumers. Remove allocator-memory-limits and mimalloc from the
full-opa feature so that users of regorus as a library can choose their
own global allocator.

Bindings (ffi, java, python, ruby) that ship as standalone artifacts
continue to opt in to regorus/allocator-memory-limits explicitly so they
retain the performant allocator.

Also propagate indexmap/std via the std feature (using the indexmap?/std
weak-dependency syntax) so that users enabling std + rvm without default
features no longer hit 'IndexMap takes 3 generic arguments' errors.

Closes #595

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* ci: add feature-combination checks to PR CI and weekly matrix

PR CI (xtask): add cargo check for 5 non-default feature combos in
run_ci_suite(). These run on every PR and catch compile failures from
feature-gating issues (e.g. #595) with near-zero overhead.

Weekly workflow: new feature-matrix.yml runs cargo build + cargo test
across 9 feature combinations every Monday. Uses a GitHub Actions matrix
with fail-fast: false so all combos are tested even if one fails.

Combinations tested weekly:
- std,arc (minimal library)
- std,arc,rvm (common library usage)
- std,arc,full-opa (full-opa without mimalloc)
- std,arc,full-opa,allocator-memory-limits (binding-style)
- std,arc,rvm,regex,time,semver,cache (cherry-picked builtins)
- std,arc,rvm,coverage,cache (observability)
- std,arc,full-opa,azure_policy (Azure Policy)
- std,arc,full-opa,azure-rbac (Azure RBAC)
- arc,opa-no-std (no_std codepath)

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* fix: gate benchmark memory-limit calls behind allocator-memory-limits feature

The set_global_memory_limit function is only available when the
allocator-memory-limits feature is enabled. After removing mimalloc
from the default feature set, the rvm_benchmark failed to compile.

Add #[cfg(feature = "allocator-memory-limits")] guards around the
call sites and the MEMORY_LIMIT_BYTES constant.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2026-03-24 11:54:01 -05:00
Anand Krishnamoorthi
50c0215fdb Rvm optimizations (#620)
* perf(rvm): fix O(n²) comprehension yield by mutating in-place

Instead of cloning the entire accumulator collection on every yield
iteration, use take_register + Rc::make_mut to get exclusive ownership
and mutate in-place. This reduces comprehension yield from O(n²) to O(n)
for both run-to-completion and suspendable execution modes.

- Add RegoVM::take_register() helper that swaps register with Undefined
- Comprehension yield now takes the accumulator, mutates via Rc::make_mut,
  and writes back — avoiding deep clones when refcount == 1

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* perf(rvm): use take_register for ObjectSet, ArrayPush, SetAdd

These instructions were cloning the container register (bumping Rc to 2),
then calling as_object_mut/as_array_mut/as_set_mut which invokes
Rc::make_mut — deep-cloning the entire collection since refcount > 1.

Use take_register instead so the Rc refcount stays at 1, making
Rc::make_mut a no-op and allowing in-place mutation.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* perf(rvm): remove unnecessary clones in rule caching

- execute_call_rule_common: move final_value into cache instead of
  cloning, since it is not used afterwards
- finalize_rule_frame_data: add comment clarifying the clone is needed
  because the value is both cached and returned
- Remove unnecessary .clone() on result_from_rule when setting register

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* rvm: avoid RuleInfo clone per rule call

Replace RuleInfo.clone() (which heap-allocates name, destructuring_blocks, and
potentially function_info) with a cheap Arc<Program> clone (atomic refcount
bump) followed by borrowing &RuleInfo from the local Arc. This eliminates
per-rule-call heap allocations.

Sites changed:
- execute_call_rule_common: Arc clone + borrow
- execute_call_rule_suspendable: Arc clone + borrow
- finalize_rule_frame_data: Arc clone + borrow
- handle_rule_break_event: inline Arc clone + borrow (was get_rule_info)
- handle_rule_error_event: inline Arc clone + borrow (was get_rule_info)
- Removed now-unused get_rule_info method

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* rvm: replace bincode with postcard for serialization

Remove unlinked bincode dependency. Use postcard (already a dep for rvm feature)
for all binary serialization/deserialization in program serialization and tests.

Also adds rvm_benchmark benchmark.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* perf(rvm): cache dummy Span/Expr for builtin calls

Every builtin call was allocating a Source (via from_contents), a Span, and
N Ref<Expr> wrappers just to satisfy the builtin function signature. These
dummy values are only used for error reporting context.

Cache the dummy Span and Vec<Ref<Expr>> on the RegoVM struct. The Source and
Span are created once on first builtin call; dummy Expr entries grow as
needed and are reused across calls via mem::take/put-back pattern.

This eliminates per-builtin-call heap allocations for Source (Rc + String +
Vec<lines>), Span clones, and Rc<Expr> wrappers.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* perf(rvm): round 2 allocation reduction in builtins, entry points, virtual data

- Cache builtin args Vec on RegoVM (mem::take/clear/put-back pattern)
- Restructure builtins_cache as two-level map for clone-free lookup
- Use IndexMap::get_index() in execute_entry_point_by_index
- Use mutable Vec path stack in traverse_rule_tree_subobject (push/pop)
- Walk data tree and rule-result paths by reference, clone only leaf
- Use mem::replace in resume() instead of cloning ExecutionState

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* fix(rvm): address PR review feedback

- Restore cached_builtin_args on all error/early-return paths in
  execute_builtin_call to preserve allocation reuse
- Use 1-based line/col and \"<builtin>\" filename in dummy span for
  clearer diagnostics
- Restore result register before returning errors in comprehension
  mode-mismatch branches (both run-to-completion and suspendable)
- Avoid clone in resume() invalid-state error path by formatting
  debug string before moving state back

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2026-03-11 21:39:58 -05:00
Anand Krishnamoorthi
5d8387f4d9 feat(hoist): pre-compute loop hoisting metadata at compilation time (#483)
Introduce a compiler pass that analyzes and pre-computes loop hoisting information
during policy compilation. This hoisted metadata is stored in lookup tables and made
available to downstream consumers:

- interpreter: use HoistedLoop entries during evaluation (replaces runtime scanning)
- type inference: can leverage pre-computed loop structure for type propagation
- RVM compiler: will consume hoisting metadata for optimized bytecode generation

Changes:
- populate loop hoisting tables during engine preparation and query snippet execution
- refactor eval_stmts_in_loop and eval_output_expr_in_loop to consume HoistedLoop directly
- add helper methods for accessing loop expressions, collections, and indices from HoistedLoop
- extend Lookup with get_checked and into_slots for safe query context access and merging

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-10-08 11:11:30 -05:00
Anand Krishnamoorthi
2a0b4ae6b5 feat! Mimalloc as the default allocator (#434)
This change integrates mimalloc as the default memory allocator for Regorus,
delivering significant performance improvements across all evaluation modes
and language bindings.

Technical Implementation:
- Build mimalloc in vendored mode from C sources (following QSharp approach)
- Implement GlobalAlloc trait for seamless Rust integration
- Add optional 'mimalloc' feature flag for conditional compilation
- Add comprehensive ACI benchmarks to measure evaluation performance

Performance Impact:

Rust Engine Evaluation:
- Single-threaded: ~29% improvement (423 vs 328 Kelem/s)
- Multi-threaded: Better scaling with reduced thread contention
- Fresh engines: ~24% improvement (56 vs 45 Kelem/s)

Rust Compiled Policy Evaluation:
- Single-threaded: ~41% improvement (426 vs 303 Kelem/s)
- Multi-threaded: Improved allocation efficiency under contention
- Fresh compilation: ~26% improvement (53 vs 42 Kelem/s)

C# FFI Bindings:
- Engine evaluation: ~27% improvement (279 vs 219 Kelem/s)
- Compiled policies: ~29% improvement (273 vs 211 Kelem/s)
- Better threading characteristics through improved underlying allocation

Key Benefits:
- Reduced allocation-related contention in multi-threaded scenarios
- More consistent performance across different thread counts
- Improved memory allocation efficiency for both native Rust and FFI workloads
- Better scaling characteristics for production deployments

The mimalloc integration provides substantial performance gains while
maintaining full compatibility with existing code through feature flags.

Reference: QSharp allocator implementation
(https://github.com/microsoft/qsharp/tree/main/source/allocator)

Fixes #297

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-25 15:01:38 -05:00
Anand Krishnamoorthi
d561531613 feat: add multi-threaded evaluation benchmark suite with comprehensive C# implementation (#457)
This commit introduces a complete multi-threaded evaluation benchmark suite for both Rust and C# implementations of Regorus.

- Implemented engine evaluation benchmark with input and engine cloning strategies
- Implemented compiled policy evaluation benchmark with input cloning and shared compiled policy strategies.

- Created EngineEvaluationBenchmark.cs and CompiledPolicyEvaluationBenchmark.cs with time-based execution (3s warmup + 3s evaluation)
- Implemented configuration options matching Rust implementation (useClonedEngines, useSharedPolicies parameters)

- Created markdown analysis documentation with cross-platform performance analysis
- C# seems to achieve 58-89% of Rust performance on test machine.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-22 11:40:39 -05:00
Anand Krishnamoorthi
db718654b5 feat: Add Schema Registry and Validation Framework (#456)
* feat: Add Schema Registry and Validation Framework

This commit introduces a comprehensive schema registry and validation framework, providing schema-based validation of resources and policy effects.

- Thread-safe, in-memory registry for schema storage and management
- Global registry patterns for effects and resources
- Concurrent access with proper error handling
- Unicode schema names support

- JSON Schema-compliant validation for all primitive types
- Advanced constraint validation (patterns, ranges, length limits)
- Discriminated union support with anyOf schemas
- Detailed error reporting with nested validation paths
- Discriminated subobject validation for polymorphic schemas

- **Registry Tests**: All registry operations
- **Effect Tests**: Policy effect validation
- **Resource Tests**: Resource validation
- **Validation Tests**: Core validation engine
- Thread-safety, error handling, integration scenarios, edge cases

- **Dependencies**: dashmap, once_cell, regex
- **Thread Safety**: Minimal locking with Rc<Schema> sharing
- **Error Types**: TypeMismatch, OutOfRange, PatternMismatch, etc.

- Complete schema registry and validation subsystem
- Comprehensive test coverage
- Foundation for policy validation in Regorus

Benchmarks:

- Criterion benchmarks for basic types, effects and Azure resources
- Performance range: 3.22ns (string) to 34.74µs (Azure VM resource schema validation)
- String withs patterns validation: 30.2µs. Need to explore whether regex caching helps
  bring this down.
- Azure policy effects: 188ns-1.4µs

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* Address PR feedback

- move error to a separate file
- use meaningful var names

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* Refactor

- Reusable Registry struct
- Split and simplify tests

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-14 15:59:30 -05:00
Anand Krishnamoorthi
dbba57f499 perf: Optimize clone (#443)
Introduce the notion of CompiledPolicy to hold stuff that
remains immutable during evaluation - e.g. rules, function,
schedules etc

Cloning takes about 60 nano seconds for an engine loaded with
ACI policies. Earlier it used to take 40 microseconds.
Thus there is easily more than 100x speedup.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-06 20:50:47 -05:00
Burak Varlı
60ac4a7a7c Basic benchmarking setup with Criterion (#408)
* Basic benchmarking setup with Criterion
* Fix Clippy warnings

Signed-off-by: Burak Varlı <burakvar@amazon.co.uk>
2025-05-14 10:41:06 -05:00