* feat(compiler): support registered host-await builtins Allow hosts to register function names at compile time so that calls to those names emit HostAwait instructions directly, enabling natural syntax like fetch(x) instead of __builtin_host_await(x, "fetch"). - Add host_await_builtins map and register_host_await_builtin() to Compiler - Validate arg_count == 1 and reject reserved __builtin_host_await name - Extend determine_call_target() resolution: explicit > registered > user > builtin - Both explicit and registered paths emit identical HostAwait bytecode - Add compile_from_policy_with_host_await() entry point in rules.rs - Extended test harness with HostAwaitBuiltinSpec and args assertion - 9 YAML test cases: suspend/resume, run-to-completion, multiple names, queue, shadowing, object packing, arg_count rejection, reserved name rejection, standard builtin override - Documentation: instruction-set.md, architecture.md * Update src/languages/rego/compiler/function_calls.rs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Mark Birger <birgerm@yandex.ru> * fix(compiler): address PR #667 review feedback on host-await registration - Compiler::register_host_await_builtin now rejects duplicate, empty, and whitespace-only names. Previously a duplicate registration would silently overwrite the existing entry, which could mask the host's own registration mistakes. - YAML test cases added: empty registration list as no-op, duplicate name rejection, empty/whitespace name rejection, out-param (a, out) calling syntax with a single-arg registered builtin, and mixed __builtin_host_await + registered builtins in the same policy consuming from their respective identifier queues. - Test harness: replace assert_eq! on HostAwait argument mismatch with anyhow::Error so mismatches propagate through the case reporter instead of panicking and skipping the harness's normal error path. - YAML comment fix: "Registration panics" -> "Registration fails with an error" (registration returns Err, never panics). Addresses anakrish + Copilot inline review comments on PR #667. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * compiler: split CallTarget::HostAwait into explicit and registered variants Addresses PR #667 review item #8: at the emit site in `compile_function_call`, the discrimination between explicit `__builtin_host_await(arg, id)` and a registered host-awaitable builtin was being recovered by string-comparing `original_fcn_path` against `"__builtin_host_await"`. The information was already known in `determine_call_target` and was being thrown away. Replace the single `CallTarget::HostAwait` variant with two: * `ExplicitHostAwait` (unit) — the two-argument call form. The identifier register comes from the user's second argument. * `RegisteredHostAwait { identifier: String }` — the one-argument call form for registered builtins. The identifier is the registered name and is captured in the variant at recognition time, so the emit site never re-derives it from the function path. This removes the magic-string comparison at the emit site (the source of truth is now `determine_call_target`) and makes both match sites in `compile_function_call` exhaustive over the two forms — adding a third host-await form in the future would force a compile error at every match site instead of silently falling through. Arities are now hardcoded in the `expected_args` extraction (`Some(2)` for explicit, `Some(1)` for registered) rather than carried in the variant; registered builtins are constrained to `arg_count == 1` at registration time, so there is no per-call variability to carry. Bytecode output is unchanged; the full RVM test suite (97 cases) and the registered_host_await suite (15 cases) pass without modification. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(compiler): clarify registered host-await intercepts unqualified calls only PR #667 review (Medium): the docs implied registered host-await names shadow user functions and builtins unconditionally, but determine_call_target matches only the bare original_fcn_path. A package-qualified call such as data.demo.resolve(x) is therefore not intercepted -- it resolves through the normal path like any other call. Rather than expand registration to qualified paths (which would let a registered name leak into every package exposing a same-named rule), document the unqualified-only behavior and pin it with tests. - register_host_await_builtin: doc now states only the unqualified call form is intercepted; qualified calls resolve normally. - determine_call_target: inline comment explaining the deliberate original_fcn_path-only match. - docs/rvm/instruction-set.md: describe qualified-call resolution, including that builtins have no qualified form. - tests: cross-package and same-package qualified calls resolve to the rule; bare-name shadowing of a standard builtin; Unknown-function outcome when no rule exists at the qualified path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(tests): compare host-await argument without re-running process_value PR #667 review (Low): the suspendable test harness compared the host-await argument via process_value(argument), but argument is already a runtime Value. process_value is a YAML-fixture decoder -- it rewrites "#undefined" to Undefined, {set!: [...]} to a set, and errors on a runtime Value::Set. Re-running it on the runtime argument could coerce a legitimate payload into a fixture sentinel (passing for the wrong reason) or error outright on sets. Compare the runtime argument directly against the expected value, which is already decoded once at YAML load time. Add a regression case (registered_builtin_suspendable_set_argument) that passes a set payload: it fails under the old double-processing ("unexpected set in value read from json/yaml") and passes with the fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(tests): reject `args:` payload expectations in run-to-completion mode PR #667 review (Low): a run-to-completion host-await response could carry an `args:` payload expectation, but RTC execution pre-loads responses and never surfaces the call argument to the harness, so the expectation was parsed and silently dropped. A case with `args: "WRONG"` passed as long as the result matched -- asserting a payload that was never checked. Reject `args:` for run-to-completion fixtures at load time, directing the author to suspendable mode where arguments are validated. Also only build the run-to-completion response vector when the case actually runs in RTC mode, so a suspendable case using the shared host_await_responses field with `args:` is not wrongly rejected. Route the fixture-load error through the same want_error handling used for compilation errors, and add registered_builtin_run_to_completion_rejects_args which now fails loudly instead of passing silently. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compiler): reject host-await builtin names with surrounding whitespace PR #667 review (Low): register_host_await_builtin rejected all-whitespace names via name.trim().is_empty(), but accepted padded names like " lookup" or "lookup ". Those were inserted into host_await_builtins, but Rego function-call paths produce the trimmed identifier, so a padded registration could never match -- a silent dead registration. Reject any name that is not already trimmed (name != name.trim()) in addition to empty names, and update the error message accordingly. Add test cases for leading and trailing whitespace. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: Mark Birger <birgerm@yandex.ru> Co-authored-by: Mark Birger <markbirger@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Regorus
Regorus is
- Rego-Rus(t) - A fast, light-weight Rego interpreter written in Rust.
- Rigorous - A rigorous enforcer of well-defined Rego semantics.
Regorus is also
-
cross-platform - Written in platform-agnostic Rust.
-
no_std compatible - Regorus can be used in
no_stdenvironments too. Most of the builtins are supported. -
current - We strive to keep Regorus up to date with latest OPA release. Regorus defaults to
v1of the Rego language. -
compliant - Regorus is mostly compliant with the latest OPA release v1.2.0. See OPA Conformance for details. Note that while we behaviorally produce the same results, we don't yet support all the builtins.
-
extensible - Extend the Rego language by implementing custom stateful builtins in Rust. See add_extension. Support for extensibility using other languages coming soon.
-
polyglot - In addition to Rust, Regorus can be used from C, C++, C#, Golang, Java, Javascript, Python, and Ruby. This is made possible by the excellent FFI tools available in the Rust ecosystem. See bindings for information on how to use Regorus from different languages.
To try out a Javascript(WASM) compiled version of Regorus from your browser, visit Regorus Playground.
Regorus is available as a library that can be easily integrated into your Rust projects. Here is an example of evaluating a simple Rego policy:
fn main() -> anyhow::Result<()> {
// Create an engine for evaluating Rego policies.
let mut engine = regorus::Engine::new();
let policy = String::from(
r#"
package example
allow if {
## All actions are allowed for admins.
input.principal == "admin"
} else if {
## Check if action is allowed for given user.
input.action in data.allowed_actions[input.principal]
}
"#,
);
// Add policy to the engine.
engine.add_policy(String::from("policy.rego"), policy)?;
// Add data to engine.
engine.add_data(regorus::Value::from_json_str(
r#"{
"allowed_actions": {
"user1" : ["read", "write"],
"user2" : ["read"]
}}"#,
)?)?;
// Set input and evaluate whether user1 can write.
engine.set_input(regorus::Value::from_json_str(
r#"{
"principal": "user1",
"action": "write"
}"#,
)?);
let r = engine.eval_rule(String::from("data.example.allow"))?;
assert_eq!(r, regorus::Value::from(true));
// Set input and evaluate whether user2 can write.
engine.set_input(regorus::Value::from_json_str(
r#"{
"principal": "user2",
"action": "write"
}"#,
)?);
let r = engine.eval_rule(String::from("data.example.allow"))?;
assert_eq!(r, regorus::Value::Undefined);
Ok(())
}
Regorus is designed with Confidential Computing in mind. In Confidential Computing environments, it is important to be able to control exactly what is being run. Regorus allows enabling and disabling various components using cargo features. By default all features are enabled.
The default build of regorus example program is 6.3M:
$ cargo build -r --example regorus; strip target/release/examples/regorus; ls -lh target/release/examples/regorus
-rwxr-xr-x 1 anand staff 6.3M May 11 22:03 target/release/examples/regorus*
When all default features are disabled, the binary size drops down to 1.9M.
$ cargo build -r --example regorus --no-default-features; strip target/release/examples/regorus; ls -lh target/release/examples/regorus
-rwxr-xr-x 1 anand staff 1.9M May 11 22:04 target/release/examples/regorus*
Regorus passes the OPA v1.2.0 test-suite barring a few builtins. See OPA Conformance below.
Bindings
Regorus can be used from a variety of languages:
- C: C binding is generated using cbindgen. corrosion-rs can be used to seamlessly use Regorous in your CMake based projects. See bindings/c.
- C freestanding: bindings/c_no_std shows how to use Regorus from C environments without a libc.
- C++: C++ binding is generated using cbindgen. corrosion-rs can be used to seamlessly use Regorous in your CMake based projects. See bindings/cpp.
- C#: C# binding is generated using csbindgen. See bindings/csharp for an example of how to build and use Regorus in your C# projects.
- Golang: The C bindings are exposed to Golang via CGo. See bindings/go for an example of how to build and use Regorus in your Go projects.
- Python: Python bindings are generated using pyo3. Wheels are created using maturin. See bindings/python.
- Java: Java bindings are developed using jni-rs. See bindings/java.
- Javascript: Regorus is compiled to WASM using wasmpack. See bindings/wasm for an example of using Regorus from nodejs. To try out a Javascript(WASM) compiled version of Regorus from your browser, visit Regorus Playground.
- Ruby: Ruby bindings are developed using magnus. See bindings/ruby.
To avoid operational overhead, we currently don't publish these bindings to various repositories. It is straight-forward to build these bindings yourself.
Getting Started
examples/regorus is an example program that shows how to integrate Regorus into your project and evaluate Rego policies.
To build and install it, do
$ cargo install --example regorus --path .
Check that the regorus example program is working
$ regorus
Usage: regorus <COMMAND>
Commands:
ast Parse a Rego policy and dump AST
eval Evaluate a Rego Query
lex Tokenize a Rego policy
parse Parse a Rego policy
help Print this message or the help of the given subcommand(s)
Options:
-h, --help Print help
-V, --version Print version
First, let's evaluate a simple Rego expression 1*2+3
$ regorus eval "1*2+3"
This produces the following output
{
"result": [
{
"expressions": [
{
"value": 5,
"text": "1*2+3",
"location": {
"row": 1,
"col": 1
}
}
]
}
]
}
Next, evaluate a sample policy and input (borrowed from Rego tutorial):
$ regorus eval -d examples/server/allowed_server.rego -i examples/server/input.json data.example
Finally, evaluate real-world policies used in Azure Container Instances (ACI)
$ regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.policy.mount_overlay=x
Policy coverage
Regorus allows determining which lines of a policy have been executed using the coverage feature (enabled by default).
We can try it out using the regorus example program by passing in the --coverage flag.
$ regorus eval -d examples/server/allowed_server.rego -i examples/server/input.json data.example --coverage
It produces the following coverage report which shows that all lines are executed except the line that sets allow to true.
See Engine::get_coverage_report for details. Policy coverage information is useful for debugging your policy as well as to write tests for your policy so that all lines of the policy are exercised by the tests.
ACI Policies
Regorus successfully passes the ACI policy test-suite. It is fast and can run each of the tests in a few milliseconds.
$ cargo test -r --test aci
Finished release [optimized + debuginfo] target(s) in 0.05s
Running tests/aci/main.rs (target/release/deps/aci-2cd8d21a893a2450)
aci/mount_device passed 3.863292ms
aci/mount_overlay passed 3.6905ms
aci/scratch_mount passed 3.643041ms
aci/create_container passed 5.046333ms
aci/shutdown_container passed 3.632ms
aci/scratch_unmount passed 3.631333ms
aci/unmount_overlay passed 3.609916ms
aci/unmount_device passed 3.626875ms
aci/load_fragment passed 4.045167ms
Run the ACI policies in the tests/aci directory, using data tests/aci/data.json and input tests/aci/input.json:
$ regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.policy.mount_overlay=x
Verify that OPA produces the same output
$ diff <(regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x) \
<(opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x)
Azure Policy (Preview)
Regorus can evaluate Azure Policy
definitions natively. A dedicated compiler translates Azure Policy JSON
directly into RVM (Regorus Virtual Machine) bytecode — the same VM that
powers Rego evaluation — so you don't have to rewrite policies in Rego.
Enable it with the azure_policy cargo feature.
Most of the policy language is supported: conditions with field, count,
and value; logical connectives (allOf, anyOf, not); comparison
operators; template expressions like parameters(), concat(),
dateTimeAdd(), and utcNow(); and effects including Deny, Audit, Modify,
Append, AuditIfNotExists, and DeployIfNotExists. An alias registry handles
the translation from fully-qualified alias names to the flattened ARM resource
shape expected by the engine.
Quick start
cargo install --example regorus --features azure_policy --path .
# Evaluate a policy against a non-compliant storage account (→ Deny)
regorus azure-policy-eval \
--policy-definition examples/regorus/azure_policy_data/require_https_storage.json \
--resource examples/regorus/azure_policy_data/non_compliant_storage.json \
--aliases tests/azure_policy/aliases/test_aliases.json
# Same policy against a compliant resource (→ undefined, no effect)
regorus azure-policy-eval \
--policy-definition examples/regorus/azure_policy_data/require_https_storage.json \
--resource examples/regorus/azure_policy_data/compliant_storage.json \
--aliases tests/azure_policy/aliases/test_aliases.json
# List aliases for a resource type
regorus azure-policy-aliases \
--aliases tests/azure_policy/aliases/test_aliases.json \
--resource-type Microsoft.Storage
The test suite covers conditions, effects, template functions, alias resolution, and end-to-end scenarios across YAML-driven test files:
cargo test --features azure_policy -- azure_policy
Performance
To check how fast Regorus runs on your system, first install a tool like hyperfine.
$ cargo install hyperfine
Then benchmark evaluation of the ACI policies,
$ hyperfine "regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x"
Benchmark 1: regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x
Time (mean ± σ): 4.6 ms ± 0.2 ms [User: 4.1 ms, System: 0.4 ms]
Range (min … max): 4.4 ms … 6.0 ms 422 runs
Compare it with OPA
$ hyperfine "opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x"
Benchmark 1: opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x
Time (mean ± σ): 45.2 ms ± 0.6 ms [User: 68.8 ms, System: 5.1 ms]
Range (min … max): 43.8 ms … 46.7 ms 62 runs
Contributor Workflow
Regorus uses a small companion CLI under the xtask package to keep CI and local development in sync.
The commands mirror our GitHub Actions jobs, making it easy to dry-run CI steps before sending a pull request.
- Run the full release pipeline with
cargo xtask ci-releaseand the debug checks withcargo xtask ci-debug. - Exercise language bindings through focused helpers such as
cargo xtask test-java --release --frozenorcargo xtask test-go. - Use
cargo xtask test-musl --release --frozenfor the cross-compilation matrix andcargo xtask test-no-stdfor embedded targets. - Formatting (
cargo xtask fmt) and linting (cargo xtask clippy --sarif) wrap the usual Cargo tooling while matching CI defaults.
The workflows in .github/workflows invoke the same commands, so keeping local runs green is usually enough to satisfy the checks enforced on main.
OPA Conformance
Regorus has been verified to be compliant with OPA v1.2.0 using a test driver that loads and runs the OPA testsuite using Regorus, and verifies that expected outputs are produced.
The test driver can be invoked by running:
$ cargo test -r --test opa --features opa-testutil,serde_json/arbitrary_precision
Currently, Regorus passes all the non-builtin specific tests. See passing tests suites.
The following test suites don't pass fully due to missing builtins:
globsmatchgraphqlinvalidkeyerrorjsonpatchjwtbuiltinsjwtdecodeverifyjwtencodesignjwtencodesignheadererrorsjwtencodesignpayloaderrorsjwtencodesignrawjwtverifyhs256jwtverifyhs384jwtverifyhs512jwtverifyrsanetcidrcontainsmatchesnetcidrintersectsnetcidrmergenetcidroverlapnetlookupipaddrproviders-awsregometadatachainregometadataruleregoparsemodulerendertemplate
They are captured in the following github issues.
Cryptographic builtins are not supported by design. Users that need cryptographic builtins are encouraged to use extensions.
Grammar
The grammar used by Regorus to parse Rego policies is described in grammar.md in both W3C EBNF and RailRoad Diagram formats.
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.
