Mark Birger 9a486c79bf fix: Deep-merge nested data documents in Engine::add_data (#760)
* Deep-merge nested data documents in Engine::add_data

add_data previously performed a shallow merge: adding a nested object under a key that already existed either replaced the whole subtree or errored on a spurious conflict, instead of merging the trees. This makes Engine::add_data (and the shared Value::merge) recurse into nested objects so keys from both sides are preserved, matching OPA's data-document merge semantics. Nested sets are unioned as a regorus extension (OPA data is JSON and has no sets). Genuine leaf conflicts (same path, two different scalar values) still error; equal values remain a no-op, which the shared rule-evaluation path relies on. Adds tests for object deep-merge, set union, leaf/type conflicts, and interaction with the 'with data.x' modifier.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* docs(value): clarify Value::merge conflict wording

Copilot review on #760 noted the doc comment called non-mergeable variants 'non-container values', which is misleading since arrays are containers yet still conflict unless equal. Reword to describe a conflict as any differing pair that is not both objects or both sets (e.g. unequal scalars or arrays).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* perf(value): avoid deep-cloning RHS set during merge union

When unioning sets in Value::merge, the RHS set is often shared: the object arm recurses via existing.merge(v.clone()), which bumps the incoming set's Rc refcount. The old Rc::make_mut(new) then structurally deep-cloned the entire RHS BTreeSet just to drain it via append and immediately discard the copy.

Move the elements out when the RHS set is uniquely owned, and otherwise clone only the per-element Rc handles into the destination. The union result is identical (BTreeSet dedups), but no throwaway set is allocated on the nested-merge path exercised by add_data deep-merge.

Addresses a Copilot review comment on #760.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(engine): make add_data atomic on merge conflict

Now that Value::merge recurses, a conflict in a later nested key was reported only after earlier keys of the same document had already been written into the live init_data, leaving the engine partially mutated on a rejected add_data.

Add a read-only Value::check_mergeable that mirrors merge's conflict rule (objects deep-merge, sets union, equal values no-op, anything else conflicts) and run it in add_data before merging. On conflict nothing is mutated, so add_data is all-or-nothing. The check allocates nothing and never copies the data spine, preserving merge's in-place uniquely-owned fast path (no candidate copy of the data document).

Adds regression tests for a partial object-leaf conflict and a partial set-union conflict. Reported by a maintainer on #760.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* test(engine): add array atomicity regression for add_data

Arrays are atomic leaves, so a differing array at a shared path is a
conflict. The new key sorts before the conflicting array key, so a naive
in-place merge would leak the new key before hitting the conflict. This
test locks in that add_data rejects the whole call and leaves data
untouched.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix: make add_data atomic under allocator memory limits

On llocator-memory-limits builds, Value::merge runs the limit check
*after* inserting each key, so an add_data whose merge trips the limit
mid-way left the data document partially mutated. check_mergeable only
models semantic conflicts, not limit failures, so the validate-then-merge
precheck couldn't cover this failure mode.

Use a build-split strategy in dd_data:
- default builds: keep the zero-copy validate-then-merge fast path
  (a conflict is the only way the merge can fail).
- allocator-memory-limits builds: merge into a candidate copy and commit
  only on success, making both conflict and limit failures transactional.
  Value is Rc/copy-on-write, so only touched subtrees are cloned.

check_mergeable is now cfg-gated to the default build to avoid dead code.

Tests (allocator-memory-limits build): add a partial-merge atomicity test
(limit trips mid-merge, data must be untouched) and a candidate-copy
conflict-atomicity test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix: separate strict rule-output merge from data-document deep-merge

#760 made Value::merge recursive so Engine::add_data deep-merges nested
data documents. But that same method also backs rule materialization,
where recursion is wrong: two rule definitions producing different
outputs for one path must conflict (OPA complete-rule semantics), not
silently combine.

Split the two behaviors:
- Value::merge is strict and shallow again (as pre-#760): a key on both
  sides must be equal or it conflicts; used for rule outputs.
- Value::deep_merge is the recursive data-document merge behind add_data;
  check_mergeable validates it up front without allocating, so the
  default build merges in place instead of cloning a candidate.

Also fix zero-arg functions (f() := ...): route their materialization
through strict equality via a new RuleValueMerge selector, so disjoint
outputs ({a:1} vs {b:2}) conflict as OPA does while prefix scaffolding
(a.foo + a.bar) still combines.

Add a 14-case interpreter conformance matrix (multiple_outputs.yaml)
covering functions, static/dynamic partial objects, and ref-heads,
matched against OPA v1.2.0.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* perf(value): make deep_merge acquire mutable access lazily

deep_merge's object arm called Rc::make_mut on the target map up front,
cloning a shared map's spine even when the merge changed nothing (a
no-op subset re-add) or conflicted before any mutation. Decide each
incoming key from a read-only probe (skip / insert / recurse / conflict)
and take Rc::make_mut only when a key actually mutates, so no-op and
conflict merges leave shared maps untouched.

Behavior is unchanged: the equality short-circuit that previously ran
inside the recursive call now runs in the probe, and conflicts bail with
the same message. Add value tests asserting Rc::ptr_eq is preserved
across no-op subset, equal-nested-object, and first-key-conflict merges.

OPA conformance unchanged (3021 pass / 651 fail, byte-identical).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* feat(value): bound deep_merge recursion depth to prevent stack-overflow DoS

deep_merge and check_mergeable recursed unbounded on object/set nesting.
A Value built without serde_json's parse-time recursion limit (the Python
and Ruby native bindings, or programmatic construction) could therefore
drive add_data into a stack overflow -- an uncatchable abort that poisons
every engine in an FFI process.

Thread a depth counter through both functions and bail past MAX_MERGE_DEPTH
(128, matching serde_json's default) so over-deep data fails with a clean
Err. In the default build check_mergeable trips first, keeping add_data
atomic; the guard in deep_merge covers the allocator-memory-limits build
and any disjoint-then-overlapping merge.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* docs(changelog): note strict zero-arg function conflict and add_data depth limit

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Mark Birger <markbirger@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-07-21 15:18:00 -05:00
2025-04-23 08:01:51 -05:00
2026-05-22 15:50:53 -05:00
2023-02-09 10:46:50 -08:00
2023-02-28 09:38:40 +05:30

Regorus

Regorus is

  • Rego-Rus(t) - A fast, light-weight Rego interpreter written in Rust.
  • Rigorous - A rigorous enforcer of well-defined Rego semantics.

Regorus is also

  • cross-platform - Written in platform-agnostic Rust.

  • no_std compatible - Regorus can be used in no_std environments too. Most of the builtins are supported.

  • current - We strive to keep Regorus up to date with latest OPA release. Regorus defaults to v1 of the Rego language.

  • compliant - Regorus is mostly compliant with the latest OPA release v1.2.0. See OPA Conformance for details. Note that while we behaviorally produce the same results, we don't yet support all the builtins.

  • extensible - Extend the Rego language by implementing custom stateful builtins in Rust. See add_extension. Support for extensibility using other languages coming soon.

  • polyglot - In addition to Rust, Regorus can be used from C, C++, C#, Golang, Java, Javascript, Python, and Ruby. This is made possible by the excellent FFI tools available in the Rust ecosystem. See bindings for information on how to use Regorus from different languages.

    To try out a Javascript(WASM) compiled version of Regorus from your browser, visit Regorus Playground.

Regorus is available as a library that can be easily integrated into your Rust projects. Here is an example of evaluating a simple Rego policy:

fn main() -> anyhow::Result<()> {
    // Create an engine for evaluating Rego policies.
    let mut engine = regorus::Engine::new();

    let policy = String::from(
        r#"
       package example

       allow if {
          ## All actions are allowed for admins.
          input.principal == "admin"
       } else if {
          ## Check if action is allowed for given user.
          input.action in data.allowed_actions[input.principal]
       }
	"#,
    );

    // Add policy to the engine.
    engine.add_policy(String::from("policy.rego"), policy)?;

    // Add data to engine.
    engine.add_data(regorus::Value::from_json_str(
        r#"{
     "allowed_actions": {
        "user1" : ["read", "write"],
        "user2" : ["read"]
     }}"#,
    )?)?;

    // Set input and evaluate whether user1 can write.
    engine.set_input(regorus::Value::from_json_str(
        r#"{
      "principal": "user1",
      "action": "write"
    }"#,
    )?);

    let r = engine.eval_rule(String::from("data.example.allow"))?;
    assert_eq!(r, regorus::Value::from(true));

    // Set input and evaluate whether user2 can write.
    engine.set_input(regorus::Value::from_json_str(
        r#"{
      "principal": "user2",
      "action": "write"
    }"#,
    )?);

    let r = engine.eval_rule(String::from("data.example.allow"))?;
    assert_eq!(r, regorus::Value::Undefined);

    Ok(())
}

Regorus is designed with Confidential Computing in mind. In Confidential Computing environments, it is important to be able to control exactly what is being run. Regorus allows enabling and disabling various components using cargo features. By default all features are enabled.

The default build of regorus example program is 6.3M:

$ cargo build -r --example regorus; strip target/release/examples/regorus; ls -lh target/release/examples/regorus
-rwxr-xr-x  1 anand  staff   6.3M May 11 22:03 target/release/examples/regorus*

When all default features are disabled, the binary size drops down to 1.9M.

$ cargo build -r --example regorus --no-default-features; strip target/release/examples/regorus; ls -lh target/release/examples/regorus
-rwxr-xr-x  1 anand  staff   1.9M May 11 22:04 target/release/examples/regorus*

Regorus passes the OPA v1.2.0 test-suite barring a few builtins. See OPA Conformance below.

Bindings

Regorus can be used from a variety of languages:

  • C: C binding is generated using cbindgen. corrosion-rs can be used to seamlessly use Regorous in your CMake based projects. See bindings/c.
  • C freestanding: bindings/c_no_std shows how to use Regorus from C environments without a libc.
  • C++: C++ binding is generated using cbindgen. corrosion-rs can be used to seamlessly use Regorous in your CMake based projects. See bindings/cpp.
  • C#: C# binding is generated using csbindgen. See bindings/csharp for an example of how to build and use Regorus in your C# projects.
  • Golang: The C bindings are exposed to Golang via CGo. See bindings/go for an example of how to build and use Regorus in your Go projects.
  • Python: Python bindings are generated using pyo3. Wheels are created using maturin. See bindings/python.
  • Java: Java bindings are developed using jni-rs. See bindings/java.
  • Javascript: Regorus is compiled to WASM using wasmpack. See bindings/wasm for an example of using Regorus from nodejs. To try out a Javascript(WASM) compiled version of Regorus from your browser, visit Regorus Playground.
  • Ruby: Ruby bindings are developed using magnus. See bindings/ruby.

To avoid operational overhead, we currently don't publish these bindings to various repositories. It is straight-forward to build these bindings yourself.

Getting Started

examples/regorus is an example program that shows how to integrate Regorus into your project and evaluate Rego policies.

To build and install it, do

$ cargo install --example regorus --path .

Check that the regorus example program is working

$ regorus
Usage: regorus <COMMAND>

Commands:
  ast    Parse a Rego policy and dump AST
  eval   Evaluate a Rego Query
  lex    Tokenize a Rego policy
  parse  Parse a Rego policy
  help   Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

First, let's evaluate a simple Rego expression 1*2+3

$ regorus eval "1*2+3"

This produces the following output

{
  "result": [
    {
      "expressions": [
        {
           "value": 5,
           "text": "1*2+3",
           "location": {
              "row": 1,
              "col": 1
            }
        }
      ]
    }
  ]
}

Next, evaluate a sample policy and input (borrowed from Rego tutorial):

$ regorus eval -d examples/server/allowed_server.rego -i examples/server/input.json data.example

Finally, evaluate real-world policies used in Azure Container Instances (ACI)

$ regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.policy.mount_overlay=x

Policy coverage

Regorus allows determining which lines of a policy have been executed using the coverage feature (enabled by default).

We can try it out using the regorus example program by passing in the --coverage flag.

$ regorus eval -d examples/server/allowed_server.rego -i examples/server/input.json data.example --coverage

It produces the following coverage report which shows that all lines are executed except the line that sets allow to true.

coverage.png

See Engine::get_coverage_report for details. Policy coverage information is useful for debugging your policy as well as to write tests for your policy so that all lines of the policy are exercised by the tests.

ACI Policies

Regorus successfully passes the ACI policy test-suite. It is fast and can run each of the tests in a few milliseconds.

$ cargo test -r --test aci
    Finished release [optimized + debuginfo] target(s) in 0.05s
    Running tests/aci/main.rs (target/release/deps/aci-2cd8d21a893a2450)
aci/mount_device                                  passed    3.863292ms
aci/mount_overlay                                 passed    3.6905ms
aci/scratch_mount                                 passed    3.643041ms
aci/create_container                              passed    5.046333ms
aci/shutdown_container                            passed    3.632ms
aci/scratch_unmount                               passed    3.631333ms
aci/unmount_overlay                               passed    3.609916ms
aci/unmount_device                                passed    3.626875ms
aci/load_fragment                                 passed    4.045167ms

Run the ACI policies in the tests/aci directory, using data tests/aci/data.json and input tests/aci/input.json:

$ regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.policy.mount_overlay=x

Verify that OPA produces the same output

$ diff <(regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x) \
       <(opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x)

Azure Policy (Preview)

Regorus can evaluate Azure Policy definitions natively. A dedicated compiler translates Azure Policy JSON directly into RVM (Regorus Virtual Machine) bytecode — the same VM that powers Rego evaluation — so you don't have to rewrite policies in Rego. Enable it with the azure_policy cargo feature.

Most of the policy language is supported: conditions with field, count, and value; logical connectives (allOf, anyOf, not); comparison operators; template expressions like parameters(), concat(), dateTimeAdd(), and utcNow(); and effects including Deny, Audit, Modify, Append, AuditIfNotExists, and DeployIfNotExists. An alias registry handles the translation from fully-qualified alias names to the flattened ARM resource shape expected by the engine.

Quick start

cargo install --example regorus --features azure_policy --path .

# Evaluate a policy against a non-compliant storage account (→ Deny)
regorus azure-policy-eval \
    --policy-definition examples/regorus/azure_policy_data/require_https_storage.json \
    --resource examples/regorus/azure_policy_data/non_compliant_storage.json \
    --aliases tests/azure_policy/aliases/test_aliases.json

# Same policy against a compliant resource (→ undefined, no effect)
regorus azure-policy-eval \
    --policy-definition examples/regorus/azure_policy_data/require_https_storage.json \
    --resource examples/regorus/azure_policy_data/compliant_storage.json \
    --aliases tests/azure_policy/aliases/test_aliases.json

# List aliases for a resource type
regorus azure-policy-aliases \
    --aliases tests/azure_policy/aliases/test_aliases.json \
    --resource-type Microsoft.Storage

The test suite covers conditions, effects, template functions, alias resolution, and end-to-end scenarios across YAML-driven test files:

cargo test --features azure_policy -- azure_policy

Performance

To check how fast Regorus runs on your system, first install a tool like hyperfine.

$ cargo install hyperfine

Then benchmark evaluation of the ACI policies,

$ hyperfine "regorus eval -b tests/aci -d tests/aci/data.json -i   tests/aci/input.json data.framework.mount_overlay=x"
Benchmark 1: regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x
  Time (mean ± σ):       4.6 ms ±   0.2 ms    [User: 4.1 ms, System: 0.4 ms]
  Range (min … max):     4.4 ms …   6.0 ms    422 runs

Compare it with OPA

$ hyperfine "opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x"
Benchmark 1: opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x
  Time (mean ± σ):      45.2 ms ±   0.6 ms    [User: 68.8 ms, System: 5.1 ms]
  Range (min … max):    43.8 ms …  46.7 ms    62 runs

Contributor Workflow

Regorus uses a small companion CLI under the xtask package to keep CI and local development in sync. The commands mirror our GitHub Actions jobs, making it easy to dry-run CI steps before sending a pull request.

  • Run the full release pipeline with cargo xtask ci-release and the debug checks with cargo xtask ci-debug.
  • Exercise language bindings through focused helpers such as cargo xtask test-java --release --frozen or cargo xtask test-go.
  • Use cargo xtask test-musl --release --frozen for the cross-compilation matrix and cargo xtask test-no-std for embedded targets.
  • Formatting (cargo xtask fmt) and linting (cargo xtask clippy --sarif) wrap the usual Cargo tooling while matching CI defaults.

The workflows in .github/workflows invoke the same commands, so keeping local runs green is usually enough to satisfy the checks enforced on main.

OPA Conformance

Regorus has been verified to be compliant with OPA v1.2.0 using a test driver that loads and runs the OPA testsuite using Regorus, and verifies that expected outputs are produced.

The test driver can be invoked by running:

$ cargo test -r --test opa --features opa-testutil,serde_json/arbitrary_precision

Currently, Regorus passes all the non-builtin specific tests. See passing tests suites.

The following test suites don't pass fully due to missing builtins:

  • globsmatch
  • graphql
  • invalidkeyerror
  • jsonpatch
  • jwtbuiltins
  • jwtdecodeverify
  • jwtencodesign
  • jwtencodesignheadererrors
  • jwtencodesignpayloaderrors
  • jwtencodesignraw
  • jwtverifyhs256
  • jwtverifyhs384
  • jwtverifyhs512
  • jwtverifyrsa
  • netcidrcontainsmatches
  • netcidrintersects
  • netcidrmerge
  • netcidroverlap
  • netlookupipaddr
  • providers-aws
  • regometadatachain
  • regometadatarule
  • regoparsemodule
  • rendertemplate

They are captured in the following github issues.

Cryptographic builtins are not supported by design. Users that need cryptographic builtins are encouraged to use extensions.

Grammar

The grammar used by Regorus to parse Rego policies is described in grammar.md in both W3C EBNF and RailRoad Diagram formats.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

Languages
Rust 63.5%
Open Policy Agent 15%
C 12.3%
C# 5.2%
C++ 2.4%
Other 1.4%