Additionally
- Include more metadata in nuget package
- Also generate snupkg for native symbols.
We intentionally don't add the symbols for native rust shared library
to the nuget package since that could increase the size of the nuget.
We will revisit that later.
- update licenses of all the bindings.
closes#551
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Prefix regorus- to mimalloc crates and add MIT licenses
- alias dependencies to avoid code changes
- add versions and release-plz publish entries
- update Cargo.lock files for new crate names
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Bump up the versions to 0.9.0 to match the C# binding version.
Also use central version management for C# projects
Also fix clippy lint errors
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Introduce ExecutionTimer/ExecutionTimerConfig to allow limiting evaluating time.
- To amortize time checking costs, checking interval can be configured via the notion of work units
- A global fallback time limit can be set to universally limit all evaluation in addition to engine level limit setting.
- Implement limnits in interpreter and RVM. In RVM, also handle suspend/resume so that time during pause is not counted.
- Add engine-level APIs to set/clear per-engine timer configuration and apply global fallback defaults.
- Surface execution-time limits through FFI and C# bindings
- Add C# tests and example usage to validate engine overrides, global fallback behavior, and compiled policy enforcement.
- Expand docs for execution-time limit
- Add interpreter YAML cases and VM unit tests for time-limit behavior and deterministic time sources.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- split the xtask crate into structured modules for
- bindings
- ci
- dev
- util
- no-std
- Adding commands for
- ci-release/ci-debug
- MUSL/no-std
- per- binding language smoke tests
- developer tasks (fmt, clippy, pre-commit, pre-push)
- refresh Cargo manifests/locks, binding readmes, and shared FFI helpers so every binding reuses the same preparation steps
- refactor GitHub Actions (release/debug, extensions, CodeQL, clippy, bindings) to call the new xtask commands
- Use rust-cache in ci workflows (microsoft qdk also does this)
- extend README with a contributor workflow section describing how xtask mirrors CI expectations
- update pre-commit and pre-push hooks to use the xtask dev commands
WORKAROUND:
When dotnet is run from an xtask, codeql tracer intercepts it an routes to a nonexistent binary.
Therefore in codeql workflow, xtask is not used for c# and instead dotnet is directly invoked.
Tracked by #545closes#475
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Ruby binstubs were checked in, but ignore via .gitignore **bin pattern.
This causes release-plz to think that the source tree is dirty.
The binstubs are deleted from source repo since they are always regenerated by bundler.
Also simplizy release-plz to focus only on the core crate.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Policy evaluation at scale needs to be able to set memory limits
so that a bad policy does not hog memory or to ensure that
policy evaluation itself does not use too much memory which could
cause other components to suffer.
This PR introduces capability to set and enforce global memory limits.
It also lays the groundwork for enabling per evaluation limits in future.
Once a global memory limit is set, Regorus maintains per thread counters
to track memory activity (allocation, deallocation) of a thread.
These counters are periodically flushed to global memory counters.
Per thread counters avoid the contention that updating global counters
on each alloc/free would cause.
Policy evaluation periodically checks these counters and raises errors
if allocated memory has exceeded the configured limit.
Currently memory limit capability is exposed only to FFI and C#.
Also update mimalloc to v2.2.6
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
This PR implements widely accepted Rust programming practices for
dealing with panics across ABI (programming language) boundaries.
- Add panic_guard.rs to wrap FFI calls and prevent panic across FFI/ABI boundary (undefined behavior).
- Capture per-thread backtraces via a temporary panic hook
- After a panic, subsequent invocations are poisoned.
- Integrate with_unwind_guard across the engine, schema registry, and target registry exportis
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Supply chain: Use the popular num-bigint crate for handling large integers
- Optimization: Handle f64, i64, u64 directly. These will be the most common instances of a number.
OPA number semantics isn't clear.
https://github.com/open-policy-agent/opa/issues/6281
As part of this change, we update the following failing tests:
- A local test that relies on what 15.3/3 evaluates to.
With our current change, we round in a different direction than what OPA does, but consistent
with Rust. We produce 5.1000000000000005 where as the OPA test expects 5.1.
There is no clear definition in Rego of what the right answer is. Moreover, policies should not
rely on exact floating point value comparison. Therefore this deviations is justified.
The test is patched to pass.
- Another local vm test that exercised 1.1 + 2.2
- Another local vm test that exercises 5.5 - 2.2
- An OPA test that expects that a large integer number say 10e308 is printed in exponent notation.
num-bigint does not print using scientific notation and instead prints all the digits.
The benefit of preserving this compatibility is not clear. We skip this test.
- Doc tests that exercised handling floating point numbers with more than 15 (what f64 supports)
digits of precision. There is no usecase for this scenario. The tests are updated to reflect
the behavior.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Add runtime detection for shared handle misuse
wrap the FFI engine handle with parking_lot::RwLock when the new
contention_checks feature is enabled, surfacing a clear “handle is already
in use” error instead of allowing undefined behavior
keep the feature optional so no_std builds or environments that supply
their own synchronization can opt out
caution users that this guards the handle itself but does not make the
engine’s operations globally thread-safe on its own
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* feat!: add Rego Virtual Machine (RVM) implementation
This commit introduces a register-based virtual machine for executing Rego
policies with bytecode-style instructions. Unlike the existing tree-walking
interpreter, the RVM compiles policies into instruction sequences that operate
on virtual registers, offering better performance and optimization potential.
Core Components:
Instruction Set Architecture:
- Define instruction types for data operations, control flow, and builtins
- Implement instruction parameter encoding and display formatting
- Add instruction parser with comprehensive test coverage
Virtual Machine Engine:
- Register-based execution model with program counter management
- Loop execution supporting iterators, comprehensions, and quantifiers
- Function call handling with argument evaluation and context management
- Rule evaluation with default value resolution and virtual data support
- Arithmetic and comparison operation implementations
Program Representation:
- Program listing builder with instruction sequencing
- Rule tree construction for organizing policy rules
- Binary and JSON serialization for compiled programs
- Recompilation support for program modification
Testing Infrastructure:
- Extensive YAML test suites covering all VM features
- Rust unit tests for VM execution and instruction parsing
- Test suites for loops, comprehensions, builtins, and control flow
BREAKING CHANGE: Introduces new VM execution path alongside interpreter
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* docs: add detailed RVM architecture references
Introduce architecture.md explaining program artifacts, serialization, and runtime subsystems.
Document the full opcode catalog in instruction-set.md, including operands, parameter tables, and outcomes.
Walk through execution flow, stacks, and operational guidance in vm-runtime.md, tying the runtime to the new architecture docs.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* build: Add xtask automation for binding version management
Introduces a dedicated xtask crate that keeps language binding versions
in sync with the core regorus crate, following the workflow pattern used
by rust-analyzer, gitoxide, and ripgrep.
Key features:
- Git-based change detection: compares binding source files against a
base ref (merge-base with origin/main by default) plus unstaged/
untracked files to identify which bindings have been modified
- SemVer-aware bumping: binding edits trigger a minor version increment
(e.g. 0.5.1 → 0.6.0) under pre-1.0 semantics, signaling potential
breaking changes; clean bindings simply align to the root version
- Multi-language support: updates Cargo manifests (Rust FFI, Java,
Python, WASM, Ruby), Maven pom.xml (Java), Ruby version constants,
and C# project files in a single pass
- CI integration: --check mode fails fast when manifests are out of
sync, ensuring pre-commit and release-plz workflows catch stale
versions before merge
Integration points:
- release-plz.toml: runs cargo xtask bindings --base-ref origin/main
after bumping the root crate, so binding versions are updated
atomically during the release process
- scripts/pre-commit: invokes cargo xtask bindings --check to block
commits that would leave bindings out of sync
- .cargo/config.toml: defines cargo xtask alias for convenience
Documentation includes inline examples showing how version bumps behave
when bindings are ahead/behind the root, and notes that the minor
field acts as the major version under SemVer 0.y.z initial development
phase.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* build: refresh xtask tooling, workflows, and locks
- cargo xtask bindings: keep the binding version-sync pipeline intact
- cargo xtask update-deps: new helper to regenerate workspace/binding Cargo.lock files
- workflows: auto-detect the Java jar version in CI and temporarily disable the Ruby workflow
- lock files: refresh root + binding snapshots after the dependency sweep
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Introduce Utf8Marshaller helpers and SafeHandle wrappers so the managed API centralizes UTF-8 conversions and lifetime management for native pointers.
- Update Engine, Compiler, CompiledPolicy, SchemaRegistry, and TargetRegistry to rely on the new marshaller/safe handles, tightening disposal and reducing transient allocations during interop calls.
- Add allocation guard coverage in Regorus.Tests and report bytes/op in the compiled policy benchmark to surface future regressions.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Major changes:
- Implement the `net.cidr_contains` builtin
- Enable the v0 and v1 test for `net.cidr_contains`
- Add the `netip` crate to standardize CIDR searching and other
operations
Key Concept:
- Allow users to leverage the `net.cidr_contains` builtin to check
whether an IPv4 or IPv6 CIDR contains a specified IP address or
subnet.
Testing:
- All tests passing.
Signed-off-by: tjons <tylerschade99@gmail.com>
This commit introduces a complete multi-threaded evaluation benchmark suite for both Rust and C# implementations of Regorus.
- Implemented engine evaluation benchmark with input and engine cloning strategies
- Implemented compiled policy evaluation benchmark with input cloning and shared compiled policy strategies.
- Created EngineEvaluationBenchmark.cs and CompiledPolicyEvaluationBenchmark.cs with time-based execution (3s warmup + 3s evaluation)
- Implemented configuration options matching Rust implementation (useClonedEngines, useSharedPolicies parameters)
- Created markdown analysis documentation with cross-platform performance analysis
- C# seems to achieve 58-89% of Rust performance on test machine.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
When using a clean build dir the install failed as it referenced
a non-existing `regorus_ffiCorrosion.cmake` file. I believe I used
the shorter `regorus_ffi` as my `EXPORT` in the `corrosion_install`
at some point and then later didn't notice that I referenced a stale
generated file when I initially handed in this PR. We must make sure
that the same identifier is used here too. See also the documentation
from `corrosion_install`:
> * **EXPORT**: Creates an export that can be installed with `install(EXPORT)`. <export-name> must be globally unique.
> Also creates a file at ${CMAKE_BINARY_DIR}/corrosion/<export-name>Corrosion.cmake that must be included in the installed config file.
Also update binding versions and lock files.
Note:
- Ruby binding is not updated
- C# binding is v0.7.0. We will make it match Regorus version later.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Indexes allow associating extra data with nodes in the AST
using an array and then quickly looking up the array to fetch
the extra data.
- Index eidx for expressions
- Index sidx for statements
- Index qidx for queries.
AST nodes are not cloneable. Therefore once a module is created,
it is not possible to accidentally create two nodes with the same
index inadvertently via clone.
Also added IndexChecker in debug builds. When a module is parsed,
it will assert that indexes have been constructed correctly.
AST Cleanup
- Make literal expressions (null, val, number, string etc) also structs
to match all other expressions
- Merge True and False nodes into a single Bool node.
Also update dependencies.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
This leverages the still-undocumented `corrosion_install` to get
an installable ffi binding that can be consumed by other projects,
e.g. from yocto:
```
$ ninja install
[4/5] Install the project...
-- Install configuration: "Debug"
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/libregorus_ffi.so
-- Up-to-date: /home/milian/projects/compiled/regorus-test/include/regorus_ffi/regorus.hpp
-- Up-to-date: /home/milian/projects/compiled/regorus-test/include/regorus_ffi/regorus.ffi.hpp
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/cmake/regorus_ffi/regorus_ffi_targets.cmake
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/cmake/regorus_ffi/regorus_ffiConfig.cmake
-- Up-to-date: /home/milian/projects/compiled/regorus-test/lib/cmake/regorus_ffi/regorus_ffiCorrosion.cmake
```
This can then be consumed as such:
```
find_package(regorus_ffi CONFIG REQUIRED)
add_executable(test test.cpp)
target_link_libraries(test PRIVATE regorus_ffi::regorus_ffi)
```
- Documentation
- Regorus Engine is intended to be used from a single thread
- Clone the engine after adding policies and data to use from another thread
- Builtin errors strictness:
- default to less strict for OPA compatibility
- Provide API to change strictness
- Expose GetAstAsJson to C#,
This can allow writing policy validations in C#.
- Use spectre mitigated msvc crt libs (binskim compliance)
- Update dependencies
fixes#404
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Cryptographic builtins are removed due to various reasons like FIPS
compliance. Users needing crypto builtins are encouraged to use
extensions.
Deprecated functions are also removed.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Specify compatibility = linux in pyproject.toml to ensure
manylinux compatibility.
- Use abi-py310 pyo3 feature to ensure compatibility with python
3.10 and later.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Organize C# binding example into separate Regorus nuget package and
a test app.
The nuget package targets netstandard 2.0 and 2.1.
The test app is tested for netframework 8.0.
Implement IDisposable for Engine cleanup.
Also remove net40 example. Can be added back later if needed.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Remove dependency on jsonwebtoken which brings in the ring crate.
Ring crate triggers governance violations.
Support for JWT will be implemented in future using a more governance
compliant crate.
BREAKING CHANGE
Prior to this PR, support for jwt builtins was minimially implemented.
Only io.jwt.decode and io.jwt.decode_verify was implemented.
With this PR, those builtins will no longer be available. They are
planned to be implemented in the future. In the meantime, they can be
brought back in via Engine::add_extension.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Removed cryptographically insecure sha1. This existed only for OPA
compatibility.
Also exclude bindings from main workspace
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
