* feat!: add Rego Virtual Machine (RVM) implementation
This commit introduces a register-based virtual machine for executing Rego
policies with bytecode-style instructions. Unlike the existing tree-walking
interpreter, the RVM compiles policies into instruction sequences that operate
on virtual registers, offering better performance and optimization potential.
Core Components:
Instruction Set Architecture:
- Define instruction types for data operations, control flow, and builtins
- Implement instruction parameter encoding and display formatting
- Add instruction parser with comprehensive test coverage
Virtual Machine Engine:
- Register-based execution model with program counter management
- Loop execution supporting iterators, comprehensions, and quantifiers
- Function call handling with argument evaluation and context management
- Rule evaluation with default value resolution and virtual data support
- Arithmetic and comparison operation implementations
Program Representation:
- Program listing builder with instruction sequencing
- Rule tree construction for organizing policy rules
- Binary and JSON serialization for compiled programs
- Recompilation support for program modification
Testing Infrastructure:
- Extensive YAML test suites covering all VM features
- Rust unit tests for VM execution and instruction parsing
- Test suites for loops, comprehensions, builtins, and control flow
BREAKING CHANGE: Introduces new VM execution path alongside interpreter
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* docs: add detailed RVM architecture references
Introduce architecture.md explaining program artifacts, serialization, and runtime subsystems.
Document the full opcode catalog in instruction-set.md, including operands, parameter tables, and outcomes.
Walk through execution flow, stacks, and operational guidance in vm-runtime.md, tying the runtime to the new architecture docs.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* build: Add xtask automation for binding version management
Introduces a dedicated xtask crate that keeps language binding versions
in sync with the core regorus crate, following the workflow pattern used
by rust-analyzer, gitoxide, and ripgrep.
Key features:
- Git-based change detection: compares binding source files against a
base ref (merge-base with origin/main by default) plus unstaged/
untracked files to identify which bindings have been modified
- SemVer-aware bumping: binding edits trigger a minor version increment
(e.g. 0.5.1 → 0.6.0) under pre-1.0 semantics, signaling potential
breaking changes; clean bindings simply align to the root version
- Multi-language support: updates Cargo manifests (Rust FFI, Java,
Python, WASM, Ruby), Maven pom.xml (Java), Ruby version constants,
and C# project files in a single pass
- CI integration: --check mode fails fast when manifests are out of
sync, ensuring pre-commit and release-plz workflows catch stale
versions before merge
Integration points:
- release-plz.toml: runs cargo xtask bindings --base-ref origin/main
after bumping the root crate, so binding versions are updated
atomically during the release process
- scripts/pre-commit: invokes cargo xtask bindings --check to block
commits that would leave bindings out of sync
- .cargo/config.toml: defines cargo xtask alias for convenience
Documentation includes inline examples showing how version bumps behave
when bindings are ahead/behind the root, and notes that the minor
field acts as the major version under SemVer 0.y.z initial development
phase.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* build: refresh xtask tooling, workflows, and locks
- cargo xtask bindings: keep the binding version-sync pipeline intact
- cargo xtask update-deps: new helper to regenerate workspace/binding Cargo.lock files
- workflows: auto-detect the Java jar version in CI and temporarily disable the Ruby workflow
- lock files: refresh root + binding snapshots after the dependency sweep
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Introduce Utf8Marshaller helpers and SafeHandle wrappers so the managed API centralizes UTF-8 conversions and lifetime management for native pointers.
- Update Engine, Compiler, CompiledPolicy, SchemaRegistry, and TargetRegistry to rely on the new marshaller/safe handles, tightening disposal and reducing transient allocations during interop calls.
- Add allocation guard coverage in Regorus.Tests and report bytes/op in the compiled policy benchmark to surface future regressions.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- add a dedicated `compiler/destructuring_planner` feature that precomputes binding plans for assignments, parameters, and `some in` expressions
- enrich `ScopeContext` with same-scope tracking, local scheduling hints, and module globals so the planner enforces := shadowing rules without blocking parent scopes
- wire the planner through compiler, hoist, interpreter, and engine paths while updating binding plan variants and adding query traversal helpers for dependency analysis
- document the new planner architecture and ship interpreter regressions that exercise nested destructuring, shadowing, and error reporting
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Introduce a compiler pass that analyzes and pre-computes loop hoisting information
during policy compilation. This hoisted metadata is stored in lookup tables and made
available to downstream consumers:
- interpreter: use HoistedLoop entries during evaluation (replaces runtime scanning)
- type inference: can leverage pre-computed loop structure for type propagation
- RVM compiler: will consume hoisting metadata for optimized bytecode generation
Changes:
- populate loop hoisting tables during engine preparation and query snippet execution
- refactor eval_stmts_in_loop and eval_output_expr_in_loop to consume HoistedLoop directly
- add helper methods for accessing loop expressions, collections, and indices from HoistedLoop
- extend Lookup with get_checked and into_slots for safe query context access and merging
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Major changes:
- Implement the `net.cidr_contains` builtin
- Enable the v0 and v1 test for `net.cidr_contains`
- Add the `netip` crate to standardize CIDR searching and other
operations
Key Concept:
- Allow users to leverage the `net.cidr_contains` builtin to check
whether an IPv4 or IPv6 CIDR contains a specified IP address or
subnet.
Testing:
- All tests passing.
Signed-off-by: tjons <tylerschade99@gmail.com>
Major Changes:
- Add generic Lookup<T> structure for efficient O(1) module-level data access
- Combine separate scope and order lookups into unified QuerySchedule structure
- Add query_schedule field to Interpreter for dedicated user query scheduling
- Refactor loop hoising to separate module
- Use efficient lookup for loop vars
- Also added more tests for loops
Key Concept:
- Ensure module context and indexing stay synchronized during function calls
Testing:
- All scheduler and interpreter tests passing
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Security Improvements:
- Pin all GitHub Actions to specific commit hashes instead of version tags
- Update actions/checkout from v4 to commit 08eba0b27e820071cde6df949e0beb9ba4906955
- Update actions/setup-python from v5 to commit a26af69be951a213d495a4c3e4e4022e16d87065 (v5.6.0)
- Update actions/setup-java from v4 to commit dded0888837ed1f317902acf8a20df0ad188d165 (v5.0.0)
- Update actions/setup-node from v4 to commit 1e60f620b9541d16bece96c5465dc8ee9832be0b (v4.4.0)
- Update actions/setup-go from v5 to commit 41dfa10bad2bb2ae585af6ee5bb4d7d973ad74ed (v5.1.0)
- Update actions/setup-dotnet from v4 to commit 3e891b0cb619bf60e2c25674b222b8940e2c1c25 (v4.1.0)
- Update actions/upload-artifact from v4 to commit ea165f8d65b6e75b540449e92b4886f43607fa02 (v4.6.2)
- Update actions/download-artifact from v4 to commit 634f93cb2916e3fdff6788551b99b062d0335ce0 (v5.0.0)
- Update github/codeql-action from v3 to commit 01fe2e8c43536ad5e1085bad5e7cd6fbc8a30988 (v3.29.11)
Rust Toolchain Consolidation:
- Create custom composite action .github/actions/toolchains/rust/action.yml
- Standardize on Rust 1.89.0 (latest stable) with clippy and rustfmt components
- Add optional targets parameter for cross-compilation support
- Replace dtolnay/rust-toolchain@stable across 16 workflows
This creates a more secure, maintainable, and consistent CI/CD pipeline
with centralized Rust toolchain management across all workflows.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Disable publishing for `ensure_no_std` test crate
- Add write content permissions to release-plz job expliclity
Signed-off-by: Burak Varlı <burakvar@amazon.co.uk>
This commit introduces a complete multi-threaded evaluation benchmark suite for both Rust and C# implementations of Regorus.
- Implemented engine evaluation benchmark with input and engine cloning strategies
- Implemented compiled policy evaluation benchmark with input cloning and shared compiled policy strategies.
- Created EngineEvaluationBenchmark.cs and CompiledPolicyEvaluationBenchmark.cs with time-based execution (3s warmup + 3s evaluation)
- Implemented configuration options matching Rust implementation (useClonedEngines, useSharedPolicies parameters)
- Created markdown analysis documentation with cross-platform performance analysis
- C# seems to achieve 58-89% of Rust performance on test machine.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Since Regorus is MIT-licensed, it would be better to specify `license = "MIT"` rather than using `licene-file` to point to the MIT license file.
Some tools do not support parsing of `license-file`, for example crates.io classifies Regorus' license as "non-standard":
$ curl -s https://crates.io/api/v1/crates/regorus/0.4.0 | jq .version.license
"non-standard"
Using `license` would provide better compatability with various tools.
Signed-off-by: Burak Varlı <burakvar@amazon.co.uk>
Details:
- Implement complete Type enum with 12 variants: Any, Integer, Number, Boolean,
Null, String, Array, Set, Object, Enum, Const, AnyOf
- Add Schema wrapper struct with reference counting for efficient sharing
- Support JSON Schema-compatible deserialization with serde
- Implement discriminated subobjects for polymorphic type definitions
- Add comprehensive test suite covering all type variants
- Include Azure resource schema examples (Storage, VM, Key Vault, App Service)
- Create meta-schema validation system with lazy static validator
- Add extensive edge case and corner case test coverage
- Implement custom deserializers for complex schema patterns
This establishes the foundation for type checking and validation of Rego
policies, particularly useful for cloud resource schemas and policy validation.
Regorus's type system is a first of many features intended to
enable type checking and various other constraints on Rego policies.
The type system is inspired from:
- JSON schema
- Bicep
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
The clone optimization PR didn't have the latest changes for "azure_policy".
Integration resulted in compile errors.
Also fix errors due to updated clippy lints.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Introduce the notion of CompiledPolicy to hold stuff that
remains immutable during evaluation - e.g. rules, function,
schedules etc
Cloning takes about 60 nano seconds for an engine loaded with
ACI policies. Earlier it used to take 40 microseconds.
Thus there is easily more than 100x speedup.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
When using a clean build dir the install failed as it referenced
a non-existing `regorus_ffiCorrosion.cmake` file. I believe I used
the shorter `regorus_ffi` as my `EXPORT` in the `corrosion_install`
at some point and then later didn't notice that I referenced a stale
generated file when I initially handed in this PR. We must make sure
that the same identifier is used here too. See also the documentation
from `corrosion_install`:
> * **EXPORT**: Creates an export that can be installed with `install(EXPORT)`. <export-name> must be globally unique.
> Also creates a file at ${CMAKE_BINARY_DIR}/corrosion/<export-name>Corrosion.cmake that must be included in the installed config file.
bindings/ruby/bin/console and bindings/ruby/bin/setup show up
as dirty to release-plz and causes it to fail to update.
As a workaround, set allow_dirty to true to enable update.
However ensure that no dirty files are published by setting
publish_allow_dirty to false.
Also default to not publishing any packages except regorus.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Also update binding versions and lock files.
Note:
- Ruby binding is not updated
- C# binding is v0.7.0. We will make it match Regorus version later.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* fix: use early exit in 'some in' statements
Update kata tests:
Since 'early return' now works with 'some in' statement, interpreter
does not do any evaluation after it found match for rule, therefore
we don't have other rule checks after interpreter found match
Indexes allow associating extra data with nodes in the AST
using an array and then quickly looking up the array to fetch
the extra data.
- Index eidx for expressions
- Index sidx for statements
- Index qidx for queries.
AST nodes are not cloneable. Therefore once a module is created,
it is not possible to accidentally create two nodes with the same
index inadvertently via clone.
Also added IndexChecker in debug builds. When a module is parsed,
it will assert that indexes have been constructed correctly.
AST Cleanup
- Make literal expressions (null, val, number, string etc) also structs
to match all other expressions
- Merge True and False nodes into a single Bool node.
Also update dependencies.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
This leverages the still-undocumented `corrosion_install` to get
an installable ffi binding that can be consumed by other projects,
e.g. from yocto:
```
$ ninja install
[4/5] Install the project...
-- Install configuration: "Debug"
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/libregorus_ffi.so
-- Up-to-date: /home/milian/projects/compiled/regorus-test/include/regorus_ffi/regorus.hpp
-- Up-to-date: /home/milian/projects/compiled/regorus-test/include/regorus_ffi/regorus.ffi.hpp
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/cmake/regorus_ffi/regorus_ffi_targets.cmake
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/cmake/regorus_ffi/regorus_ffiConfig.cmake
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/cmake/regorus_ffi/regorus_ffiCorrosion.cmake
```
This can then be consumed as such:
```
find_package(regorus_ffi CONFIG REQUIRED)
add_executable(test test.cpp)
target_link_libraries(test PRIVATE regorus_ffi::regorus_ffi)
```
- Documentation
- Regorus Engine is intended to be used from a single thread
- Clone the engine after adding policies and data to use from another thread
- Builtin errors strictness:
- default to less strict for OPA compatibility
- Provide API to change strictness
- Expose GetAstAsJson to C#,
This can allow writing policy validations in C#.
- Use spectre mitigated msvc crt libs (binskim compliance)
- Update dependencies
fixes#404
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
else blocks following contains and old-style sets will raise
a parse error. Consistent with OPA.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
