Compare commits

...

23 Commits

Author SHA1 Message Date
Anand Krishnamoorthi
0a39e434db chore: release (#226)
* chore: release
2024-05-07 18:26:50 -07:00
Anand Krishnamoorthi
9832a297ed Improve example in readme (#224)
- Use eval_rule
- Show functions add_policy, add_data, set_input
- Show Engine, Value types

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-02 12:46:01 -07:00
Anand Krishnamoorthi
c6fb8cf044 Add tests for kata containers policies (#221)
closes #220

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-27 06:31:09 -07:00
Anand Krishnamoorthi
55abbb2b42 Update to OPA v0.64.0 (#219)
Implement json.marshal_with_options builtin

closes #215, closes #218

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-26 04:52:11 -07:00
Anand Krishnamoorthi
3743f32edc Enable policy files greater than 64KB in size (#217)
fixes #214

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-26 01:04:21 -07:00
Anand Krishnamoorthi
744dad6126 OPA Conformance: Do not interpret # within regular string (#216)
fixes #213

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-25 23:51:52 -07:00
Anand Krishnamoorthi
7fde3382f6 chore: release (#210)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-22 15:09:01 -07:00
Anand Krishnamoorthi
316f3a7692 early return (#189)
If a rule is written to produce a constant value, then not all iterations of loops
within it need to be executed. Execution can stop via early return once the first iteration
that produces a value has been executed.

This brings forth the question : What if one of the subsequent iterations would have resulted
in an error?
e.g:
x {
  [1, "hello"][_] + 1
}

Such errors are not raised; consistent with OPA.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-22 07:41:35 -07:00
Anand Krishnamoorthi
de56cce7cb Fix anyhow dependency issues (#208)
- Do not require backtrace feature
- Starting version 1.0.77, anyhow gathers backtrace is std feature (enabled by default)
  is specified even if backtrace feature is not enabled.
  Therefore specify default features as false.
- Specify version 1.0.45 since that is the minimul version required to successfully
  compile regorus

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-20 09:57:52 -07:00
Anand Krishnamoorthi
72ced23366 build: remove unused compact-rc dependency (#207)
Remove unused compact-rc dependency, to avoid a build error:

error[E0658]: use of unstable library feature 'ptr_addr_eq'
<...>/registry/src/index.crates.io-6f17d22bba15001f/compact-rc-0.5.4/src/base.rs:319:9
    |
319 |         std::ptr::addr_eq(Self::as_ptr(this), Self::as_ptr(other))

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Co-authored-by: Dan Mihai <dmihai@microsoft.com>
2024-04-20 06:47:18 -07:00
Anand Krishnamoorthi
3fa2847e6f chore: release (#205)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-11 07:03:10 -07:00
Anand Krishnamoorthi
82c86437cb Add a note in example to prefer eval_rule over eval_query (#204)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-11 06:38:45 -07:00
Anand Krishnamoorthi
05e91da06e Do not enable serde_json/arbitrary_precision by default (#203)
The feature does not interoperate well with other serde_json features like untagged enums.

Fixes #199

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-11 06:23:14 -07:00
Anand Krishnamoorthi
8c69dd491b Rewrite so that code compiles with chrono_tz 0.8.5 and 0.9.0 (#201)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-11 04:37:38 +05:30
thedavemarshall
6a167143cb update ruby bindings (#200) 2024-04-10 07:28:25 +05:30
Anand Krishnamoorthi
d2049d07f3 Store Value instances in AST for strings, numbers and idents (#197)
This avoids having to create value instances during evaluation

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-07 18:21:32 +05:30
Anand Krishnamoorthi
e326f3c629 From<serde_json::Value> and From<serde_yaml::Value> (#196)
Provide wrappers around serde_json::from_value and serde_yaml::from_value since
they may not be apparent and the user may end up serializing to json/yaml and
rereading as a regorus::Value

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-06 16:40:57 +05:30
Anand Krishnamoorthi
3c7674e7c2 Build dependency on git only if opa.runtime feature is enabled. (#194)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-03 10:17:51 +05:30
Anand Krishnamoorthi
947c9490fa Update to opa v0.63.0 (#192)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-03 09:28:55 +05:30
dependabot[bot]
0ebcb568cc Update pyo3 requirement from 0.20.2 to 0.21.0 (#190)
Updates the requirements on [pyo3](https://github.com/pyo3/pyo3) to permit the latest version.
- [Release notes](https://github.com/pyo3/pyo3/releases)
- [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pyo3/pyo3/compare/v0.20.2...v0.21.0)

---
updated-dependencies:
- dependency-name: pyo3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-02 08:14:50 +05:30
thedavemarshall
e86801bdd9 Ruby bindings for existing FFI methods, plus eval_rule() (#188)
* Ruby binding

* use relative regorus crate in ruby instead of a published version, also cargo fmt

* remove unnecessary Cargo.toml, include the top level Cargo.lock in ruby gem

* ruby bindings continued- add eval_rule, fix _json methods, update README.md

also added rubocop-minitest and rubocop-rake, and added more test coverage

* update README.md to include Ruby bindings

Closes #191
2024-04-02 08:14:29 +05:30
Anand Krishnamoorthi
3d98c3b12e eval_rule: Evaluate rules directly instead of queries (#186)
closes #185

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-03-26 05:24:21 +05:30
Anand Krishnamoorthi
330a6dff72 Remove cruft. (#184)
Logging wasn't implemented fully nor getting used much.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-03-24 18:37:24 +05:30
64 changed files with 9474 additions and 377 deletions

View File

@@ -30,15 +30,19 @@ jobs:
run: cargo test -r --verbose
- name: Run tests (ACI)
run: cargo test -r --test aci
- name: Run tests (KATA)
run: cargo test -r --test kata
- name: Run tests (OPA Conformance)
run: >-
cargo test -r --test opa --features opa-testutil -- $(tr '\n' ' ' < tests/opa.passing)
cargo test -r --test opa --features opa-testutil,serde_json/arbitrary_precision -- $(tr '\n' ' ' < tests/opa.passing)
- name: Build (MUSL)
run: cargo build --verbose --all-targets --target x86_64-unknown-linux-musl
- name: Run tests (MUSL)
run: cargo test -r --verbose --target x86_64-unknown-linux-musl
- name: Run tests (MUSL ACI)
run: cargo test -r --test aci --target x86_64-unknown-linux-musl
- name: Run tests (KATA ACI)
run: cargo test -r --test kata --target x86_64-unknown-linux-musl
- name: Run tests (MUSL OPA Conformance)
run: >-
cargo test -r --test opa --features opa-testutil --target x86_64-unknown-linux-musl -- $(tr '\n' ' ' < tests/opa.passing)
cargo test -r --test opa --features opa-testutil,serde_json/arbitrary_precision --target x86_64-unknown-linux-musl -- $(tr '\n' ' ' < tests/opa.passing)

View File

@@ -6,6 +6,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.1.5](https://github.com/microsoft/regorus/compare/regorus-v0.1.4...regorus-v0.1.5) - 2024-05-07
### Added
- Support policy files greater than 64KB in size ([#217](https://github.com/microsoft/regorus/pull/217))
- Add tests for kata containers policies ([#221](https://github.com/microsoft/regorus/pull/221))
- Support for OPA v0.64.0 ([#219](https://github.com/microsoft/regorus/pull/219))
- New builtin `json.marshal_with_options`
### Changed
- Improve example in readme ([#224](https://github.com/microsoft/regorus/pull/224))
### Fixed
- OPA Conformance: Do not interpret # within regular string ([#216](https://github.com/microsoft/regorus/pull/216))
## [0.1.4](https://github.com/microsoft/regorus/compare/regorus-v0.1.3...regorus-v0.1.4) - 2024-04-22
### Other
- early return ([#189](https://github.com/microsoft/regorus/pull/189))
- Fix anyhow dependency issues ([#208](https://github.com/microsoft/regorus/pull/208))
- remove unused compact-rc dependency ([#207](https://github.com/microsoft/regorus/pull/207))
## [0.1.3](https://github.com/microsoft/regorus/compare/regorus-v0.1.2...regorus-v0.1.3) - 2024-04-11
### Other
- Add a note in example to prefer eval_rule over eval_query ([#204](https://github.com/microsoft/regorus/pull/204))
- Do not enable serde_json/arbitrary_precision by default ([#203](https://github.com/microsoft/regorus/pull/203))
- Rewrite so that code compiles with chrono_tz 0.8.5 and 0.9.0 ([#201](https://github.com/microsoft/regorus/pull/201))
- update ruby bindings ([#200](https://github.com/microsoft/regorus/pull/200))
- Store Value instances in AST for strings, numbers and idents ([#197](https://github.com/microsoft/regorus/pull/197))
- :Value> and From<serde_yaml::Value> ([#196](https://github.com/microsoft/regorus/pull/196))
- Build dependency on git only if opa.runtime feature is enabled. ([#194](https://github.com/microsoft/regorus/pull/194))
- Update to opa v0.63.0 ([#192](https://github.com/microsoft/regorus/pull/192))
- Update pyo3 requirement from 0.20.2 to 0.21.0 ([#190](https://github.com/microsoft/regorus/pull/190))
- Ruby bindings for existing FFI methods, plus eval_rule() ([#188](https://github.com/microsoft/regorus/pull/188))
- Evaluate rules directly instead of queries ([#186](https://github.com/microsoft/regorus/pull/186))
- Remove cruft. ([#184](https://github.com/microsoft/regorus/pull/184))
## [0.1.2](https://github.com/microsoft/regorus/compare/regorus-v0.1.1...regorus-v0.1.2) - 2024-03-22
### Other

View File

@@ -5,12 +5,13 @@ members = [
"bindings/python",
"bindings/wasm",
"bindings/java",
"bindings/ruby/ext/regorusrb",
]
[package]
name = "regorus"
description = "A fast, lightweight Rego (OPA policy language) interpreter"
version = "0.1.2"
version = "0.1.5"
edition = "2021"
license-file = "LICENSE"
repository = "https://github.com/microsoft/regorus"
@@ -68,12 +69,10 @@ full-opa = [
opa-testutil = []
[dependencies]
anyhow = {version = "1.0.66", features = ["backtrace"] }
anyhow = { version = "1.0.45", default-features=false }
serde = {version = "1.0.150", features = ["derive", "rc"] }
serde_json = {version = "1.0.89", features = ["arbitrary_precision"] }
serde_json = "1.0.89"
serde_yaml = {version = "0.9.16", optional = true }
log = "0.4.17"
env_logger="0.11.1"
lazy_static = "1.4.0"
rand = "0.8.5"
num = "0.4.1"
@@ -97,20 +96,19 @@ uuid = { version = "1.6.1", features = ["v4", "fast-rng"], optional = true }
jsonschema = { version = "0.17.1", default-features = false, optional = true }
chrono = { version = "0.4.31", optional = true }
chrono-tz = { version = "0.8.5", optional = true }
compact-rc = "0.5.2"
jsonwebtoken = { version = "9.2.0", optional = true }
itertools = "0.12.1"
[dev-dependencies]
cfg-if = "1.0.0"
clap = { version = "4.4.7", features = ["derive"] }
colored-diff = "0.2.3"
prettydiff = { version = "0.6.4", default-features = false }
serde_yaml = "0.9.16"
test-generator = "0.3.1"
walkdir = "2.3.2"
[build-dependencies]
anyhow = "1.0.66"
anyhow = "1.0"
[profile.release]
debug = true
@@ -126,6 +124,11 @@ name="aci"
harness=false
test=false
[[test]]
name="kata"
harness=false
test=false
[package.metadata.docs.rs]
# To build locally:
# RUSTDOCFLAGS="--cfg docsrs" cargo +nightly doc --all-features --no-deps

View File

@@ -9,11 +9,11 @@
Regorus is also
- *cross-platform* - Written in platform-agnostic Rust.
- *current* - We strive to keep Regorus up to date with latest OPA release. Regorus supports `import rego.v1`.
- *compliant* - Regorus is mostly compliant with the latest [OPA release v0.62.0](https://github.com/open-policy-agent/opa/releases/tag/v0.62.0). See [OPA Conformance](#opa-conformance) for details. Note that while we behaviorally produce the same results, we don't yet support all the builtins.
- *compliant* - Regorus is mostly compliant with the latest [OPA release v0.64.0](https://github.com/open-policy-agent/opa/releases/tag/v0.64.0). See [OPA Conformance](#opa-conformance) for details. Note that while we behaviorally produce the same results, we don't yet support all the builtins.
- *extensible* - Extend the Rego language by implementing custom stateful builtins in Rust.
See [add_extension](https://github.com/microsoft/regorus/blob/fc68bf9c8bea36427dae9401a7d1f6ada771f7ab/src/engine.rs#L352).
Support for extensibility using other languages coming soon.
- *polyglot* - In addition to Rust, Regorus can be used from *C*, *C++*, *C#*, *Golang*, *Java*, *Javascript* and *Python*.
- *polyglot* - In addition to Rust, Regorus can be used from *C*, *C++*, *C#*, *Golang*, *Java*, *Javascript*, *Python*, and *Ruby*.
This is made possible by the excellent FFI tools available in the Rust ecosystem. See [bindings](#bindings) for information on how to use Regorus from different languages.
To try out a *Javascript(WASM)* compiled version of Regorus from your browser, visit [Regorus Playground](https://anakrish.github.io/regorus-playground/).
@@ -24,31 +24,60 @@ Regorus is available as a library that can be easily integrated into your Rust p
Here is an example of evaluating a simple Rego policy:
```rust
use anyhow::Result;
use regorus::*;
use serde_json;
fn main() -> anyhow::Result<()> {
// Create an engine for evaluating Rego policies.
let mut engine = regorus::Engine::new();
fn main() -> Result<()> {
// Create an engine for evaluating Rego policies.
let mut engine = Engine::new();
let policy = String::from(
r#"
package example
import rego.v1
// Add policy to the engine.
engine.add_policy(
// Filename to be associated with the policy.
"hello.rego".to_string(),
allow if {
## All actions are allowed for admins.
input.principal == "admin"
} else if {
## Check if action is allowed for given user.
input.action in data.allowed_actions[input.principal]
}
"#,
);
// Rego policy that just sets a message.
r#"
package test
message = "Hello, World!"
"#.to_string()
)?;
// Add policy to the engine.
engine.add_policy(String::from("policy.rego"), policy)?;
// Evaluate the policy, fetch the message and print it.
let results = engine.eval_query("data.test.message".to_string(), false)?;
println!("{}", serde_json::to_string_pretty(&results)?);
// Add data to engine.
engine.add_data(regorus::Value::from_json_str(
r#"{
"allowed_actions": {
"user1" : ["read", "write"],
"user2" : ["read"]
}}"#,
)?)?;
Ok(())
// Set input and evaluate whether user1 can write.
engine.set_input(regorus::Value::from_json_str(
r#"{
"principal": "user1",
"action": "write"
}"#,
)?);
let r = engine.eval_rule(String::from("data.example.allow"))?;
assert_eq!(r, regorus::Value::from(true));
// Set input and evaluate whether user2 can write.
engine.set_input(regorus::Value::from_json_str(
r#"{
"principal": "user2",
"action": "write"
}"#,
)?);
let r = engine.eval_rule(String::from("data.example.allow"))?;
assert_eq!(r, regorus::Value::Undefined);
Ok(())
}
```
@@ -69,7 +98,7 @@ $ cargo build -r --example regorus --features "yaml" --no-default-features; stri
-rwxr-xr-x 1 anand staff 2.9M Jan 19 11:26 target/release/examples/regorus*
```
Regorus passes the [OPA v0.61.0 test-suite](https://www.openpolicyagent.org/docs/latest/ir/#test-suite) barring a few
Regorus passes the [OPA v0.64.0 test-suite](https://www.openpolicyagent.org/docs/latest/ir/#test-suite) barring a few
builtins. See [OPA Conformance](#opa-conformance) below.
## Bindings
@@ -90,6 +119,8 @@ Regorus can be used from a variety of languages:
- *Javascript*: Regorus is compiled to WASM using [wasmpack](https://github.com/rustwasm/wasm-pack).
See [bindings/wasm](https://github.com/microsoft/regorus/tree/main/bindings/wasm) for an example of using Regorus from nodejs.
To try out a *Javascript(WASM)* compiled version of Regorus from your browser, visit [Regorus Playground](https://anakrish.github.io/regorus-playground/).
- *Ruby*: Ruby bindings are developed using [magnus](https://github.com/matsadler/magnus).
See [bindings/ruby](https://github.com/microsoft/regorus/tree/main/bindings/ruby).
To avoid operational overhead, we currently don't publish these bindings to various repositories.
It is straight-forward to build these bindings yourself.
@@ -179,7 +210,7 @@ It produces the following coverage report which shows that all lines are execute
![coverage.png](https://github.com/microsoft/regorus/blob/main/docs/coverage.png?raw=true)
See [Engine::get_coverage_report](https://docs.rs/regorus/latest/regorus/struct.Engine.html#method.get_coverage_report) for details.
Policy coverage information is useful for debugging your policy as well as to write tests for your policy so that all
Policy coverage information is useful for debugging your policy as well as to write tests for your policy so that all
lines of the policy are exercised by the tests.
## ACI Policies
@@ -243,14 +274,13 @@ Benchmark 1: opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.jso
```
## OPA Conformance
Regorus has been verified to be compliant with [OPA v0.61.0](https://github.com/open-policy-agent/opa/releases/tag/v0.61.0)
using a [test driver](https://github.com/microsoft/regorus/blob/main/tests/opa.rs) that loads and runs the OPA testsuite using Regorus, and verifies that expected outputs
are produced.
Regorus has been verified to be compliant with [OPA v0.64.0](https://github.com/open-policy-agent/opa/releases/tag/v0.64.0)
using a [test driver](https://github.com/microsoft/regorus/blob/main/tests/opa.rs) that loads and runs the OPA testsuite using Regorus, and verifies that expected outputs are produced.
The test driver can be invoked by running:
```bash
$ cargo test -r --test opa
$ cargo test -r --test opa --features opa-testutil,serde_json/arbitrary_precision
```
Currently, Regorus passes all the non-builtin specific tests.

View File

@@ -8,7 +8,7 @@ edition = "2021"
crate-type = ["cdylib"]
[dependencies]
anyhow = "1.0.79"
anyhow = "1.0"
regorus = { path = "../.." }
serde_json = "1.0.113"

View File

@@ -11,7 +11,7 @@ keywords = ["interpreter", "opa", "policy-as-code", "rego"]
crate-type = ["cdylib"]
[dependencies]
anyhow = "1.0.79"
anyhow = "1.0"
serde_json = "1.0.112"
jni = "0.21.1"
regorus = { path = "../.." }

View File

@@ -12,9 +12,9 @@ keywords = ["interpreter", "opa", "policy-as-code", "rego"]
crate-type = ["cdylib"]
[dependencies]
anyhow = "1.0.79"
anyhow = "1.0"
ordered-float = "4.2.0"
pyo3 = {version = "0.20.2", features = ["anyhow", "extension-module"] }
pyo3 = {version = "0.21.0", features = ["anyhow", "extension-module"] }
regorus = { path = "../.." }
serde_json = "1.0.112"

14
bindings/ruby/.gitignore vendored Normal file
View File

@@ -0,0 +1,14 @@
/.bundle/
/.yardoc
/_yardoc/
/coverage/
/doc/
/pkg/
/spec/reports/
/tmp/
*.bundle
*.so
*.o
*.a
mkmf.log
target/

View File

@@ -0,0 +1,29 @@
require:
- rubocop-minitest
- rubocop-rake
AllCops:
TargetRubyVersion: 3.0
NewCops: enable
Layout/LineLength:
Max: 180
Lint/EmptyClass:
Enabled: false
Metrics/ClassLength:
Exclude:
- 'test/**/*.rb'
Metrics/MethodLength:
Enabled: false
Style/StringLiterals:
EnforcedStyle: double_quotes
Style/StringLiteralsInInterpolation:
EnforcedStyle: double_quotes
Style/WordArray:
Enabled: false

View File

@@ -0,0 +1 @@
ruby 3.3.0

View File

@@ -0,0 +1,5 @@
## [Unreleased]
## [0.1.0] - 2024-03-29
- Initial release

3
bindings/ruby/Cargo.toml Normal file
View File

@@ -0,0 +1,3 @@
[workspace]
members = ["ext/regorusrb"]
resolver = "2"

16
bindings/ruby/Gemfile Normal file
View File

@@ -0,0 +1,16 @@
# frozen_string_literal: true
source "https://rubygems.org"
# Specify your gem's dependencies in regorusrb.gemspec
gemspec
# These gems are required for local development and testing,
# but won't be included in the published gem
gem "minitest", "~> 5.16"
gem "rake", "~> 13.0"
gem "rake-compiler"
gem "rake-compiler-dock"
gem "rubocop", "~> 1.62", require: false
gem "rubocop-minitest", require: false
gem "rubocop-rake", require: false

View File

@@ -0,0 +1,63 @@
PATH
remote: .
specs:
regorusrb (0.1.0)
rb_sys (~> 0.9.91)
GEM
remote: https://rubygems.org/
specs:
ast (2.4.2)
json (2.7.2)
language_server-protocol (3.17.0.3)
minitest (5.22.3)
parallel (1.24.0)
parser (3.3.0.5)
ast (~> 2.4.1)
racc
racc (1.7.3)
rainbow (3.1.1)
rake (13.2.1)
rake-compiler (1.2.7)
rake
rake-compiler-dock (1.4.0)
rb_sys (0.9.91)
regexp_parser (2.9.0)
rexml (3.2.6)
rubocop (1.63.0)
json (~> 2.3)
language_server-protocol (>= 3.17.0)
parallel (~> 1.10)
parser (>= 3.3.0.2)
rainbow (>= 2.2.2, < 4.0)
regexp_parser (>= 1.8, < 3.0)
rexml (>= 3.2.5, < 4.0)
rubocop-ast (>= 1.31.1, < 2.0)
ruby-progressbar (~> 1.7)
unicode-display_width (>= 2.4.0, < 3.0)
rubocop-ast (1.31.2)
parser (>= 3.3.0.4)
rubocop-minitest (0.35.0)
rubocop (>= 1.61, < 2.0)
rubocop-ast (>= 1.31.1, < 2.0)
rubocop-rake (0.6.0)
rubocop (~> 1.0)
ruby-progressbar (1.13.0)
unicode-display_width (2.5.0)
PLATFORMS
ruby
x86_64-linux
DEPENDENCIES
minitest (~> 5.16)
rake (~> 13.0)
rake-compiler
rake-compiler-dock
regorusrb!
rubocop (~> 1.62)
rubocop-minitest
rubocop-rake
BUNDLED WITH
2.5.7

21
bindings/ruby/LICENSE.txt Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) Microsoft Corporation.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE

99
bindings/ruby/README.md Normal file
View File

@@ -0,0 +1,99 @@
# Regorusrb
**Regorus** is
- *Rego*-*Rus(t)* - A fast, light-weight [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/)
interpreter written in Rust.
- *Rigorous* - A rigorous enforcer of well-defined Rego semantics.
## Installation
Regorus can be used in Ruby by configuring bundler to build from the remote git source.
If using [Bundler](https://bundler.io/) to manage gems (recommended), edit your gemfile to include the following
`
gem "regorusrb", git: "https://github.com/microsoft/regorus/", glob: "bindings/ruby/*.gemspec"
`
or manually install checkout the source and build the gem
`
git clone https://github.com/microsoft/regorus/
cd regorus/bindings/ruby
rake && rake build # should eventually output 'regorusrb 0.1.0 built to pkg/regorusrb-0.1.0.gem.'
gem install --local ./pkg/regorusrb-0.1.0.gem
`
It is not yet available in rubygems.
See [Repository](https://github.com/microsoft/regorus).
To build this gem locally without bundler,
`rake build`
then to install the gem and build the native extensions
`gem install --local ./pkg/regorusrb-0.1.0.gem`
## Usage
```ruby
require "regorus"
engine = Regorus::Engine.new
engine.add_policy_from_file('../../tests/aci/framework.rego')
engine.add_policy_from_file('../../tests/aci/api.rego')
engine.add_policy_from_file('../../tests/aci/policy.rego')
# can be strings or symbols
data = {
metadata: {
devices: {
"/run/layers/p0-layer0": "1b80f120dbd88e4355d6241b519c3e25290215c469516b49dece9cf07175a766",
"/run/layers/p0-layer1": "e769d7487cc314d3ee748a4440805317c19262c7acd2fdbdb0d47d2e4613a15c",
"/run/layers/p0-layer2": "eb36921e1f82af46dfe248ef8f1b3afb6a5230a64181d960d10237a08cd73c79",
"/run/layers/p0-layer3": "41d64cdeb347bf236b4c13b7403b633ff11f1cf94dbc7cf881a44d6da88c5156",
"/run/layers/p0-layer4": "4dedae42847c704da891a28c25d32201a1ae440bce2aecccfa8e6f03b97a6a6c",
"/run/layers/p0-layer5": "fe84c9d5bfddd07a2624d00333cf13c1a9c941f3a261f13ead44fc6a93bc0e7a"
}
}
}
engine.add_data(data)
input = {
"containerID": "container0",
"layerPaths": [
"/run/layers/p0-layer0",
"/run/layers/p0-layer1",
"/run/layers/p0-layer2",
"/run/layers/p0-layer3",
"/run/layers/p0-layer4",
"/run/layers/p0-layer5"
],
"target": "/run/gcs/c/container0/rootfs"
}
engine.set_input(input)
# Evaluate a specife rule
rule_results = engine.eval_rule('data.framework.mount_overlay')
puts rule_results # { "allowed" => true, "metadata" => [...]}
# Or evalute a full policy document
query_results = engine.eval_query('data.framework')
puts query_results[:result][0]
# Query results can can also be returned as JSON strings instead of Ruby Hash structure
results_json = engine.eval_query_as_json('data.framework.mount_overlay=x')
puts results_json
```
## Development
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).

29
bindings/ruby/Rakefile Normal file
View File

@@ -0,0 +1,29 @@
# frozen_string_literal: true
require "bundler/gem_tasks"
require "minitest/test_task"
require "rake/extensiontask"
require "rubocop/rake_task"
require "rb_sys/extensiontask"
Minitest::TestTask.create
RuboCop::RakeTask.new
desc "build the .gem file, including native extensions, according to the .gemspec"
task build: :compile
GEMSPEC = Gem::Specification.load("regorusrb.gemspec")
RbSys::ExtensionTask.new("regorusrb", GEMSPEC) do |ext|
ext.lib_dir = "lib/regorus"
ext.cross_compile = true
ext.cross_platform = %w[x86-mingw32 x64-mingw-ucrt x64-mingw32 x86-linux x86_64-linux x86_64-darwin arm64-darwin]
end
task default: %i[compile test rubocop]
desc "Build native extension for a given platform (i.e. rake 'native[x86_64-linux]')"
task :native, [:platform] do |_t, platform:|
sh "bundle", "exec", "rb-sys-dock", "--platform", platform, "--build"
end

11
bindings/ruby/bin/console Executable file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env ruby
# frozen_string_literal: true
require "bundler/setup"
require "regorus"
# You can add fixtures and/or initialization code here to make experimenting
# with your gem easier. You can also use a different console, if you like.
require "irb"
IRB.start(__FILE__)

8
bindings/ruby/bin/setup Executable file
View File

@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
set -vx
bundle install
# Do any other automated setup that you need to do here

View File

@@ -0,0 +1,16 @@
[package]
name = "regorusrb"
version = "0.1.0"
edition = "2021"
description = "Ruby bindings for Regorus - a fast, lightweight Rego interpreter written in Rust"
publish = false
[lib]
crate-type = ["cdylib"]
path = "src/lib.rs"
[dependencies]
magnus = { version = "0.6.3" }
regorus = { git = "https://github.com/microsoft/regorus" }
serde_json = "1.0.115"
serde_magnus = "0.8.1"

View File

@@ -0,0 +1,8 @@
# frozen_string_literal: true
require "mkmf"
require "rb_sys/mkmf"
create_rust_makefile("regorus/regorusrb") do |r|
r.auto_install_rust_toolchain = true
end

View File

@@ -0,0 +1,241 @@
use magnus::{exception::runtime_error, method, module, prelude::*, Error, Ruby};
use regorus::Engine as RegorusEngine;
use std::cell::RefCell;
use std::cmp::Ordering;
// `Value` exists under magnus, regorus, and serde_json, so be explicit
#[derive(Default)]
#[magnus::wrap(class = "Regorus::Engine")]
pub struct Engine {
engine: RefCell<RegorusEngine>,
}
impl Clone for Engine {
fn clone(&self) -> Self {
Self {
engine: self.engine.clone(),
}
}
}
impl Engine {
fn initialize(&self) {
let engine = RegorusEngine::new();
*self.engine.borrow_mut() = engine;
}
fn compare(&self, other: &Self) -> Result<i32, Error> {
let self_ptr: *const _ = &*self.engine.borrow();
let other_ptr: *const _ = &*other.engine.borrow();
match self_ptr.partial_cmp(&other_ptr) {
Some(Ordering::Less) => Ok(-1),
Some(Ordering::Equal) => Ok(0),
Some(Ordering::Greater) => Ok(1),
None => Err(Error::new(runtime_error(), "Comparison failed")),
}
}
fn add_policy(&self, path: String, rego: String) -> Result<(), Error> {
self.engine
.borrow_mut()
.add_policy(path, rego)
.map_err(|e| Error::new(runtime_error(), format!("Failed to add policy: {}", e)))
}
fn add_policy_from_file(&self, path: String) -> Result<(), Error> {
self.engine
.borrow_mut()
.add_policy_from_file(path)
.map_err(|e| Error::new(runtime_error(), format!("Failed to add policy: {}", e)))
}
fn add_data(&self, ruby_hash: magnus::RHash) -> Result<(), Error> {
let data_value: regorus::Value = serde_magnus::deserialize(ruby_hash).map_err(|e| {
Error::new(
runtime_error(),
format!("Failed to deserialize Ruby value: {}", e),
)
})?;
self.engine
.borrow_mut()
.add_data(data_value)
.map_err(|e| Error::new(runtime_error(), format!("Failed to add data: {}", e)))
}
fn add_data_json(&self, json_string: String) -> Result<(), Error> {
self.engine
.borrow_mut()
.add_data_json(&json_string)
.map_err(|e| Error::new(runtime_error(), format!("Failed to add data json: {}", e)))
}
fn add_data_from_json_file(&self, path: String) -> Result<(), Error> {
let json_data = regorus::Value::from_json_file(&path).map_err(|e| {
Error::new(
runtime_error(),
format!("Failed to parse JSON data file: {}", e),
)
})?;
self.engine.borrow_mut().add_data(json_data).map_err(|e| {
Error::new(
runtime_error(),
format!("Failed to add data from file: {}", e),
)
})
}
fn clear_data(&self) -> Result<(), Error> {
self.engine.borrow_mut().clear_data();
Ok(())
}
fn set_input(&self, ruby_hash: magnus::RHash) -> Result<(), Error> {
let input_value: regorus::Value = serde_magnus::deserialize(ruby_hash).map_err(|e| {
Error::new(
runtime_error(),
format!("Failed to deserialize Ruby value: {}", e),
)
})?;
self.engine.borrow_mut().set_input(input_value);
Ok(())
}
fn set_input_json(&self, json_string: String) -> Result<(), Error> {
self.engine
.borrow_mut()
.set_input_json(&json_string)
.map_err(|e| Error::new(runtime_error(), format!("Failed to set input JSON: {}", e)))
}
fn add_input_from_json_file(&self, path: String) -> Result<(), Error> {
let json_data = regorus::Value::from_json_file(&path).map_err(|e| {
Error::new(
runtime_error(),
format!("Failed to parse JSON input file: {}", e),
)
})?;
self.engine.borrow_mut().set_input(json_data);
Ok(())
}
fn eval_query(&self, query: String) -> Result<magnus::Value, Error> {
let results = self
.engine
.borrow_mut()
.eval_query(query, false)
.map_err(|e| Error::new(runtime_error(), format!("Failed to evaluate query: {}", e)))?;
serde_magnus::serialize(&results).map_err(|e| {
Error::new(
runtime_error(),
format!("Failed to serailzie query results: {}", e),
)
})
}
fn eval_query_as_json(&self, query: String) -> Result<String, Error> {
let results = self
.engine
.borrow_mut()
.eval_query(query, false)
.map_err(|e| {
Error::new(
runtime_error(),
format!("Failed to evaluate query as json: {}", e),
)
})?;
serde_json::to_string(&results).map_err(|e| {
Error::new(
runtime_error(),
format!("Failed to serialize query results: {}", e),
)
})
}
fn eval_rule(&self, query: String) -> Result<Option<magnus::Value>, Error> {
let result =
self.engine.borrow_mut().eval_rule(query).map_err(|e| {
Error::new(runtime_error(), format!("Failed to evaluate rule: {}", e))
})?;
match result {
regorus::Value::Undefined => Ok(None), // Convert undefined to Ruby's nil
_ => serde_magnus::serialize(&result) // Serialize other results normally
.map(Some)
.map_err(|e| {
magnus::Error::new(
runtime_error(),
format!("Failed to serialize the rule evaluation result: {}", e),
)
}),
}
}
fn eval_bool_query(&self, query: String) -> Result<bool, Error> {
self.engine
.borrow_mut()
.eval_bool_query(query, false)
.map_err(|e| Error::new(runtime_error(), format!("Failed to evaluate query: {}", e)))
}
fn eval_allow_query(&self, query: String) -> Result<bool, Error> {
Ok(self.engine.borrow_mut().eval_allow_query(query, false))
}
fn eval_deny_query(&self, query: String) -> Result<bool, Error> {
Ok(self.engine.borrow_mut().eval_deny_query(query, false))
}
}
#[magnus::init]
fn init(ruby: &Ruby) -> Result<(), Error> {
let regorus_module = ruby.define_module("Regorus")?;
let engine_class = regorus_module.define_class("Engine", ruby.class_object())?;
// ruby object methods
engine_class.define_alloc_func::<Engine>();
engine_class.define_method("initialize", method!(Engine::initialize, 0))?;
engine_class.define_method("clone", method!(Engine::clone, 0))?;
engine_class.define_method("<=>", method!(Engine::compare, 1))?;
// defines <, <=, >, >=, and == based on <=>
engine_class.include_module(module::comparable())?;
// policy operations
engine_class.define_method("add_policy", method!(Engine::add_policy, 2))?;
engine_class.define_method(
"add_policy_from_file",
method!(Engine::add_policy_from_file, 1),
)?;
// data operations
engine_class.define_method("add_data", method!(Engine::add_data, 1))?;
engine_class.define_method("add_data_json", method!(Engine::add_data_json, 1))?;
engine_class.define_method(
"add_data_from_json_file",
method!(Engine::add_data_from_json_file, 1),
)?;
engine_class.define_method("clear_data", method!(Engine::clear_data, 0))?;
// input operations
engine_class.define_method("set_input", method!(Engine::set_input, 1))?;
engine_class.define_method("set_input_json", method!(Engine::set_input_json, 1))?;
engine_class.define_method(
"add_input_from_json_file",
method!(Engine::add_input_from_json_file, 1),
)?;
// query operations
engine_class.define_method("eval_query", method!(Engine::eval_query, 1))?;
engine_class.define_method("eval_query_as_json", method!(Engine::eval_query_as_json, 1))?;
engine_class.define_method("eval_rule", method!(Engine::eval_rule, 1))?;
engine_class.define_method("eval_bool_query", method!(Engine::eval_bool_query, 1))?;
engine_class.define_method("eval_allow_query", method!(Engine::eval_allow_query, 1))?;
engine_class.define_method("eval_deny_query", method!(Engine::eval_deny_query, 1))?;
Ok(())
}

View File

@@ -0,0 +1,8 @@
# frozen_string_literal: true
require_relative "regorus/version"
require_relative "regorus/regorusrb"
module Regorus
class Engine; end
end

View File

@@ -0,0 +1,5 @@
# frozen_string_literal: true
module Regorus
VERSION = "0.1.0"
end

View File

@@ -0,0 +1,30 @@
# frozen_string_literal: true
require_relative "lib/regorus/version"
Gem::Specification.new do |spec|
spec.name = "regorusrb"
spec.version = Regorus::VERSION
spec.authors = ["David Marshall"]
spec.summary = "Ruby bindings for Regorus - a fast, lightweight Rego interpreter written in Rust"
spec.homepage = "https://github.com/microsoft/regorus/blob/main/bindings/ruby"
spec.license = "MIT"
spec.required_ruby_version = ">= 3.0.0"
spec.required_rubygems_version = ">= 3.3.11"
spec.metadata["allowed_push_host"] = "TODO: Set to your gem server 'https://example.com'"
spec.metadata["homepage_uri"] = spec.homepage
spec.metadata["source_code_uri"] = spec.homepage
spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/main/bindings/ruby/CHANGELOG.md"
spec.metadata["rubygems_mfa_required"] = "true"
spec.files = Dir["lib/*.rb", "lib/regorus/*.rb", "ext/**/*.{rs,rb,lock,toml}", "Cargo.{lock,toml}", "LICENSE.txt", "README.md"]
spec.bindir = "exe"
spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
spec.require_paths = ["lib"]
spec.extensions = ["ext/regorusrb/extconf.rb"]
spec.add_dependency "rb_sys", "~> 0.9.91"
end

View File

@@ -0,0 +1,4 @@
module Regorus
VERSION: String
# See the writing guide of rbs: https://github.com/ruby/rbs#guides
end

View File

@@ -0,0 +1,6 @@
# frozen_string_literal: true
$LOAD_PATH.unshift File.expand_path("../lib", __dir__)
require "regorus"
require "minitest/autorun"

View File

@@ -0,0 +1,204 @@
# frozen_string_literal: true
require "test_helper"
require "json"
class TestRegorus < Minitest::Test
ALICE = "Alice"
BOB = "Bob"
CARLOS = "Carlos"
def setup
@engine = ::Regorus::Engine.new
@engine.add_policy("regorus_test.rego", example_policy)
@engine.add_data(example_data)
end
def example_policy
<<~REGO
package regorus_test
is_manager {
input.name == data.managers[_]
}
is_employee {
input.name == data.employees[_]
}
# Set a default value for to return false instead of nil
default is_manager_bool = false
default is_employee_bool = false
is_manager_bool {
is_manager
}
is_employee_bool {
is_employee
}
REGO
end
def example_data
{
"managers" => [ALICE],
"employees" => [ALICE, BOB]
}
end
def input_for(name)
{ "name" => name }
end
def test_version_number_presence
refute_nil ::Regorus::VERSION
end
def test_engine_creation
assert_instance_of ::Regorus::Engine, ::Regorus::Engine.new
end
def test_policy_addition
assert_silent { @engine.add_policy("example.rego", example_policy) }
end
def test_object_creation_with_new
refute_same ::Regorus::Engine.new, ::Regorus::Engine.new
end
def test_data_addition
assert_silent { @engine.add_data(example_data) }
end
def test_data_addition_as_json
assert_silent { @engine.add_data_json(example_data.to_json) }
end
def test_query_evaluation_for_alice
@engine.set_input(input_for(ALICE))
assert_equal alice_results, @engine.eval_query("data.regorus_test")
end
def test_query_evaluation_for_bob
@engine.set_input(input_for(BOB))
assert_equal bob_results, @engine.eval_query("data.regorus_test")
end
def test_query_evaluation_as_json
@engine.set_input(input_for(ALICE))
assert_equal alice_results.to_json, @engine.eval_query_as_json("data.regorus_test")
end
def test_rule_evaluation_for_alice
@engine.set_input(input_for(ALICE))
assert @engine.eval_rule("data.regorus_test.is_employee")
assert @engine.eval_rule("data.regorus_test.is_employee_bool")
assert @engine.eval_rule("data.regorus_test.is_manager")
assert @engine.eval_rule("data.regorus_test.is_manager_bool")
end
def test_rule_evaluation_for_bob
@engine.set_input(input_for(BOB))
assert @engine.eval_rule("data.regorus_test.is_employee")
assert @engine.eval_rule("data.regorus_test.is_employee_bool")
assert_nil @engine.eval_rule("data.regorus_test.is_manager")
refute @engine.eval_rule("data.regorus_test.is_manager_bool")
end
def test_rule_evaluation_for_carlos
@engine.set_input(input_for(CARLOS))
assert_nil @engine.eval_rule("data.regorus_test.is_employee")
refute @engine.eval_rule("data.regorus_test.is_employee_bool")
assert_nil @engine.eval_rule("data.regorus_test.is_manager")
refute @engine.eval_rule("data.regorus_test.is_manager_bool")
end
def test_eval_bool_query
assert @engine.eval_bool_query("1 < 2")
refute @engine.eval_bool_query("1 > 2")
assert_raises(RuntimeError) { @engine.eval_bool_query("1 + 1") }
assert_raises(RuntimeError) { @engine.eval_bool_query("true; true") }
assert_raises(RuntimeError) { @engine.eval_bool_query("true; false; true") }
end
def test_eval_allow_query
assert @engine.eval_allow_query("1 < 2")
refute @engine.eval_allow_query("1 > 2")
refute @engine.eval_allow_query("1 + 1")
refute @engine.eval_allow_query("true; true")
refute @engine.eval_allow_query("true; false; true")
end
def test_eval_deny_query
assert @engine.eval_deny_query("1 < 2")
refute @engine.eval_deny_query("1 > 2")
assert @engine.eval_deny_query("1 + 1")
assert @engine.eval_deny_query("true; true")
assert @engine.eval_deny_query("true; false; true")
end
def test_missing_rules_handling
@engine.set_input(input_for(ALICE))
assert_raises(RuntimeError) { @engine.eval_rule("data.regorus_test.not_a_rule") }
end
def test_engine_cloning
cloned_engine = @engine.clone
assert_instance_of ::Regorus::Engine, cloned_engine
refute_same @engine, cloned_engine
end
def alice_results
{
result: [
{
expressions: [
{
value: {
"is_employee" => true,
"is_employee_bool" => true,
"is_manager" => true,
"is_manager_bool" => true
},
text: "data.regorus_test",
location: {
row: 1,
col: 1
}
}
]
}
]
}
end
def bob_results
{
result: [
{
expressions: [
{
value: {
"is_employee" => true,
"is_employee_bool" => true,
"is_manager_bool" => false
},
text: "data.regorus_test",
location: {
row: 1,
col: 1
}
}
]
}
]
}
end
end

View File

@@ -3,7 +3,6 @@
use anyhow::Result;
use std::path::Path;
use std::process::Command;
fn main() -> Result<()> {
// Copy hooks to appropriate location so that git will run them.
@@ -14,12 +13,15 @@ fn main() -> Result<()> {
}
// Supply information as compile-time environment variables.
let output = Command::new("git")
.args(["rev-parse", "HEAD"])
.output()
.unwrap();
let git_hash = String::from_utf8(output.stdout).unwrap();
println!("cargo:rustc-env=GIT_HASH={}", git_hash);
#[cfg(feature = "opa-runtime")]
{
let output = std::process::Command::new("git")
.args(["rev-parse", "HEAD"])
.output()
.expect("`git rev-parse HEAD` failed.");
let git_hash = String::from_utf8(output.stdout).unwrap();
println!("cargo:rustc-env=GIT_HASH={}", git_hash);
}
Ok(())
}

View File

@@ -159,24 +159,25 @@ In future, each builtin will be associated with a feature (many builtins could b
| [type_name](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-types-type_name) | _ |
- [Encoding](https://www.openpolicyagent.org/docs/latest/policy-reference/#encoding)
| Builtin | Feature |
|----------------------------------------------------------------------------------------------------------------------------------|-------------|
| [base64.is_valid](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-base64is_valid) | `base64` |
| [base64url.decode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-base64urldecode) | `base64` |
| [base64url.encode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-base64urlencode) | `base64url` |
| [base64url.encode_no_pad](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-base64urlencode_no_pad) | `base64url` |
| [hex.decode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-hexdecode) | `hex` |
| [hex.encode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-hexencode) | `hex` |
| [json.is_valid](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-jsonis_valid) | _ |
| [json.marshal](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-jsonmarshal) | _ |
| [json.unmarshal](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-jsonunmarshal) | _ |
| [urlquery.decode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-urlquerydecode) | `urlquery` |
| [urlquery.decode_object](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-urlquerydecode_object) | `urlquery` |
| [urlquery.encode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-urlqueryencode) | `urlquery` |
| [urlquery.encode_object](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-urlqueryencode_object) | `urlquery` |
| [yaml.is_valid](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-yamlis_valid) | `yaml` |
| [yaml.marshal](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-yamlmarshal) | `yaml` |
| [yaml.unmarshal](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-yamlunmarshal) | `yaml` |
| Builtin | Feature |
|--------------------------------------------------------------------------------------------------------------------------------------|-------------|
| [base64.is_valid](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-base64is_valid) | `base64` |
| [base64url.decode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-base64urldecode) | `base64` |
| [base64url.encode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-base64urlencode) | `base64url` |
| [base64url.encode_no_pad](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-base64urlencode_no_pad) | `base64url` |
| [hex.decode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-hexdecode) | `hex` |
| [hex.encode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-hexencode) | `hex` |
| [json.is_valid](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-jsonis_valid) | _ |
| [json.marshal](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-jsonmarshal) | _ |
| [json.marshal_with_options](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-jsonmarshal_with_options) | _ |
| [json.unmarshal](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-jsonunmarshal) | _ |
| [urlquery.decode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-urlquerydecode) | `urlquery` |
| [urlquery.decode_object](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-urlquerydecode_object) | `urlquery` |
| [urlquery.encode](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-urlqueryencode) | `urlquery` |
| [urlquery.encode_object](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-urlqueryencode_object) | `urlquery` |
| [yaml.is_valid](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-yamlis_valid) | `yaml` |
| [yaml.marshal](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-yamlmarshal) | `yaml` |
| [yaml.unmarshal](https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-encoding-yamlunmarshal) | `yaml` |
- [Time](https://www.openpolicyagent.org/docs/latest/policy-reference/#time)
| Builtin | Feature |

View File

@@ -70,7 +70,14 @@ fn rego_eval(
engine.set_input(input);
}
// Evaluate query.
// Note: The `eval_query` function is used below since it produces output
// in the same format as OPA. It also allows evaluating arbitrary statements
// as queries.
//
// Most applications will want to use `eval_rule` instead.
// It is faster since it does not have to parse the query string.
// It also returns the value of the rule directly and thus is easier
// to use.
let results = engine.eval_query(query, enable_tracing)?;
println!("{}", serde_json::to_string_pretty(&results)?);
@@ -184,10 +191,6 @@ struct Cli {
fn main() -> Result<()> {
use clap::Parser;
env_logger::builder()
.format_level(false)
.format_timestamp(None)
.init();
// Parse and dispatch command.
let cli = Cli::parse();

View File

@@ -15,7 +15,8 @@ if [ -f Cargo.toml ]; then
# Ensure that all tests pass
cargo test -r
cargo test -r --test aci
cargo test -r --test kata
# Ensure that OPA conformance tests don't regress.
cargo test -r --features opa-testutil --test opa -- $(tr '\n' ' ' < tests/opa.passing)
cargo test -r --features opa-testutil,serde_json/arbitrary_precision --test opa -- $(tr '\n' ' ' < tests/opa.passing)
fi

View File

@@ -2,6 +2,7 @@
// Licensed under the MIT License.
use crate::lexer::*;
use crate::value::Value;
use crate::Rc;
use std::ops::Deref;
@@ -98,13 +99,13 @@ pub type Ref<T> = NodeRef<T>;
#[derive(Debug)]
pub enum Expr {
// Simple items that only have a span as content.
String(Span),
RawString(Span),
Number(Span),
String((Span, Value)),
RawString((Span, Value)),
Number((Span, Value)),
True(Span),
False(Span),
Null(Span),
Var(Span),
Var((Span, Value)),
// array
Array {
@@ -158,7 +159,7 @@ pub enum Expr {
RefDot {
span: Span,
refr: Ref<Expr>,
field: Span,
field: (Span, Value),
},
RefBrack {
@@ -207,7 +208,8 @@ impl Expr {
pub fn span(&self) -> &Span {
use Expr::*;
match self {
String(s) | RawString(s) | Number(s) | True(s) | False(s) | Null(s) | Var(s) => s,
String(s) | RawString(s) | Number(s) | Var(s) => &s.0,
True(s) | False(s) | Null(s) => s,
Array { span, .. }
| Set { span, .. }
| Object { span, .. }

View File

@@ -42,6 +42,7 @@ pub fn register(m: &mut HashMap<&'static str, builtins::BuiltinFcn>) {
}
m.insert("json.is_valid", (json_is_valid, 1));
m.insert("json.marshal", (json_marshal, 1));
m.insert("json.marshal_with_options", (json_marshal_with_options, 2));
m.insert("json.unmarshal", (json_unmarshal, 1));
#[cfg(feature = "yaml")]
@@ -368,6 +369,74 @@ fn json_marshal(span: &Span, params: &[Ref<Expr>], args: &[Value], _strict: bool
))
}
fn json_marshal_with_options(
span: &Span,
params: &[Ref<Expr>],
args: &[Value],
_strict: bool,
) -> Result<Value> {
let name = "json.marshal_with_options";
ensure_args_count(span, name, params, args, 2)?;
let options = ensure_object(name, &params[1], args[1].clone())?;
let (mut pretty, mut indent, mut prefix) = (true, Some("\t".to_owned()), None);
for (option, option_value) in options.iter() {
match option {
Value::String(s) if s.as_ref() == "pretty" && option_value.as_bool().is_ok() => {
pretty = option_value == &Value::Bool(true);
}
Value::String(s) if s.as_ref() == "pretty" => bail!(params[1]
.span()
.error("marshaling option `pretty` must be true or false")),
Value::String(s) if s.as_ref() == "prefix" && option_value.as_string().is_ok() => {
prefix = Some(option_value.as_string()?.as_ref().to_string());
}
Value::String(s) if s.as_ref() == "prefix" => bail!(params[1]
.span()
.error("marshaling option `pretty` must be string")),
Value::String(s) if s.as_ref() == "indent" && option_value.as_string().is_ok() => {
indent = Some(option_value.as_string()?.as_ref().to_string());
}
Value::String(s) if s.as_ref() == "indent" => bail!(params[1]
.span()
.error("marshaling option `pretty` must be string")),
_ => bail!(params[1]
.span()
.error("marshaling option must be one of `indent`, `prefix` or `pretty`")),
}
}
if !pretty || options.is_empty() {
return Ok(Value::String(
serde_json::to_string(&args[0])
.with_context(|| span.error("could not serialize to json"))?
.into(),
));
}
let lines: Vec<String> = serde_json::to_string_pretty(&args[0])
.with_context(|| span.error("could not serialize to json"))?
.split('\n')
.map(|line| {
let mut line = line.to_string();
if let Some(indent) = &indent {
let start_trimmed = line.trim_start();
let leading_spaces = line.len() - start_trimmed.len();
let indentation_level = leading_spaces / 2;
line = indent.repeat(indentation_level) + start_trimmed;
}
if let Some(prefix) = &prefix {
line = prefix.to_owned() + &line;
}
line
})
.collect();
Ok(Value::from(lines.join("\n")))
}
fn json_unmarshal(
span: &Span,
params: &[Ref<Expr>],

View File

@@ -12,7 +12,7 @@ use semver::Version;
use std::cmp::Ordering;
use std::collections::HashMap;
use anyhow::{Ok, Result};
use anyhow::Result;
pub fn register(m: &mut HashMap<&'static str, builtins::BuiltinFcn>) {
m.insert("semver.compare", (compare, 2));

View File

@@ -9,7 +9,7 @@ use crate::value::Value;
use std::collections::HashMap;
use anyhow::{anyhow, bail, Result};
use anyhow::{bail, Result};
use chrono::{
DateTime, Datelike, Days, FixedOffset, Local, Months, SecondsFormat, TimeZone, Timelike, Utc,
@@ -248,7 +248,10 @@ fn parse_epoch(
"UTC" | "" => Utc.timestamp_nanos(ns).fixed_offset(),
"Local" => Local.timestamp_nanos(ns).fixed_offset(),
_ => {
let tz: Tz = tz.parse().map_err(|err: String| anyhow!(err))?;
let tz: Tz = match tz.parse() {
Ok(tz) => tz,
Err(e) => bail!(e),
};
tz.timestamp_nanos(ns).fixed_offset()
}
};

View File

@@ -9,7 +9,7 @@ use crate::value::Value;
use std::collections::{BTreeMap, HashMap};
use anyhow::{Ok, Result};
use anyhow::Result;
use uuid::{Timestamp, Uuid};
pub fn register(m: &mut HashMap<&'static str, builtins::BuiltinFcn>) {

View File

@@ -65,7 +65,7 @@ impl Engine {
/// ```
///
pub fn add_policy(&mut self, path: String, rego: String) -> Result<()> {
let source = Source::new(path, rego);
let source = Source::from_contents(path, rego)?;
let mut parser = Parser::new(&source)?;
self.modules.push(Ref::new(parser.parse()?));
// if policies change, interpreter needs to be prepared again
@@ -208,6 +208,52 @@ impl Engine {
&self.modules
}
/// Evaluate rule(s) at given path.
///
/// [`Engine::eval_rule`] is often faster than [`Engine::eval_query`] and should be preferred if
/// OPA style [`QueryResults`] are not needed.
///
/// ```
/// # use regorus::*;
/// # fn main() -> anyhow::Result<()> {
/// let mut engine = Engine::new();
///
/// // Add policy
/// engine.add_policy(
/// "policy.rego".to_string(),
/// r#"
/// package example
/// import rego.v1
///
/// x = [1, 2]
///
/// y := 5 if input.a > 2
/// "#.to_string())?;
///
/// // Evaluate rule.
/// let v = engine.eval_rule("data.example.x".to_string())?;
/// assert_eq!(v, Value::from(vec![Value::from(1), Value::from(2)]));
///
/// // y evaluates to undefined.
/// let v = engine.eval_rule("data.example.y".to_string())?;
/// assert_eq!(v, Value::Undefined);
///
/// // Evaluating a non-existent rule is an error.
/// let r = engine.eval_rule("data.exaample.x".to_string());
/// assert!(r.is_err());
///
/// // Path must be valid rule paths.
/// assert!( engine.eval_rule("data".to_string()).is_err());
/// assert!( engine.eval_rule("data.example".to_string()).is_err());
/// # Ok(())
/// # }
/// ```
pub fn eval_rule(&mut self, path: String) -> Result<Value> {
self.prepare_for_eval(false)?;
self.interpreter.clean_internal_evaluation_state();
self.interpreter.eval_rule_in_path(path)
}
/// Evaluate a Rego query.
///
/// ```
@@ -248,15 +294,15 @@ impl Engine {
self.interpreter.create_rule_prefixes()?;
let query_module = {
let source = Source::new(
let source = Source::from_contents(
"<query_module.rego>".to_owned(),
"package __internal_query_module".to_owned(),
);
)?;
Ref::new(Parser::new(&source)?.parse()?)
};
// Parse the query.
let query_source = Source::new("<query.rego>".to_string(), query);
let query_source = Source::from_contents("<query.rego>".to_string(), query)?;
let mut parser = Parser::new(&query_source)?;
let query_node = parser.parse_user_query()?;
if query_node.span.text() == "data" {
@@ -365,15 +411,15 @@ impl Engine {
self.eval_modules(enable_tracing)?;
let query_module = {
let source = Source::new(
let source = Source::from_contents(
"<query_module.rego>".to_owned(),
"package __internal_query_module".to_owned(),
);
)?;
Ref::new(Parser::new(&source)?.parse()?)
};
// Parse the query.
let query_source = Source::new("<query.rego>".to_string(), query);
let query_source = Source::from_contents("<query.rego>".to_string(), query)?;
let mut parser = Parser::new(&query_source)?;
let query_node = parser.parse_user_query()?;
let query_schedule = Analyzer::new().analyze_query_snippet(&self.modules, &query_node)?;
@@ -419,7 +465,7 @@ impl Engine {
}
#[doc(hidden)]
pub fn eval_rule(
pub fn eval_rule_in_module(
&mut self,
module: &Ref<Module>,
rule: &Ref<Rule>,

View File

@@ -4,7 +4,6 @@
use crate::ast::*;
use crate::builtins::{self, BuiltinFcn};
use crate::lexer::*;
use crate::number::*;
use crate::parser::Parser;
use crate::scheduler::*;
use crate::utils::*;
@@ -13,11 +12,9 @@ use crate::Rc;
use crate::{Expression, Extension, Location, QueryResult, QueryResults};
use anyhow::{anyhow, bail, Result};
use log::info;
use std::collections::btree_map::Entry as BTreeMapEntry;
use std::collections::{hash_map::Entry, BTreeMap, BTreeSet, HashMap};
use std::collections::{hash_map::Entry, BTreeMap, BTreeSet, HashMap, HashSet};
use std::ops::Bound::*;
use std::str::FromStr;
type Scope = BTreeMap<SourceStr, Value>;
@@ -76,6 +73,7 @@ pub struct Interpreter {
gather_prints: bool,
prints: Vec<String>,
rule_paths: HashSet<String>,
}
impl Default for Interpreter {
@@ -96,6 +94,8 @@ struct Context {
rule_value: Value,
is_set: bool,
is_old_style_set: bool,
output_constness_determined: bool,
early_return: bool,
}
impl Default for Context {
@@ -111,6 +111,8 @@ impl Default for Context {
rule_value: Value::new_object(),
is_set: false,
is_old_style_set: false,
output_constness_determined: false,
early_return: false,
}
}
}
@@ -196,6 +198,7 @@ impl Interpreter {
gather_prints: false,
prints: Vec::default(),
rule_paths: HashSet::new(),
}
}
@@ -236,7 +239,6 @@ impl Interpreter {
pub fn set_input(&mut self, input: Value) {
self.input = input;
info!("input: {:#?}", self.input);
}
pub fn init_with_document(&mut self) -> Result<()> {
@@ -323,18 +325,18 @@ impl Interpreter {
// Stop path collection upon encountering the leading variable.
Expr::Var(v) => {
path.reverse();
return self.lookup_var(v, &path[..], false);
return self.lookup_var(&v.0, &path[..], false);
}
// Accumulate chained . field accesses.
Expr::RefDot { refr, field, .. } => {
expr = refr;
path.push(field.text());
path.push(field.0.text());
}
Expr::RefBrack { refr, index, .. } => match index.as_ref() {
// refr["field"] is the same as refr.field
Expr::String(s) => {
expr = refr;
path.push(s.text());
path.push(s.0.text());
}
// Handle other forms of refr.
// Note, we have the choice to evaluate a non-string index
@@ -416,8 +418,8 @@ impl Interpreter {
// Then hoist the current bracket operation.
let mut indices = Vec::with_capacity(1);
let _ = traverse(index, &mut |e| match e.as_ref() {
Var(ident) if self.is_loop_index_var(&ident.source_str()) => {
indices.push(ident.source_str());
Var(ident) if self.is_loop_index_var(&ident.0.source_str()) => {
indices.push(ident.0.source_str());
Ok(false)
}
Array { .. } | Object { .. } => Ok(true),
@@ -586,16 +588,16 @@ impl Interpreter {
AssignOp::Eq => {
match (lhs.as_ref(), rhs.as_ref()) {
(_, Expr::Var(var))
if var.source_str().text() != "input"
&& self.lookup_var(var, &[], true)? == Value::Undefined =>
if var.0.source_str().text() != "input"
&& self.lookup_var(&var.0, &[], true)? == Value::Undefined =>
{
(var.source_str(), self.eval_expr(lhs)?)
(var.0.source_str(), self.eval_expr(lhs)?)
}
(Expr::Var(var), _)
if var.source_str().text() != "input"
&& self.lookup_var(var, &[], true)? == Value::Undefined =>
if var.0.source_str().text() != "input"
&& self.lookup_var(&var.0, &[], true)? == Value::Undefined =>
{
(var.source_str(), self.eval_expr(rhs)?)
(var.0.source_str(), self.eval_expr(rhs)?)
}
(
Expr::Array {
@@ -696,8 +698,8 @@ impl Interpreter {
return Ok(rhs_value);
}
let name = if let Expr::Var(span) = lhs.as_ref() {
span.source_str()
let name = if let Expr::Var(s) = lhs.as_ref() {
s.0.source_str()
} else {
let mut cache = BTreeMap::new();
let mut type_match = BTreeSet::new();
@@ -731,11 +733,6 @@ impl Interpreter {
// TODO: optimize this
self.variables_assignment(&name, &value)?;
info!(
"eval_assign_expr before, op: {:?}, lhs: {:?}, rhs: {:?}",
op, lhs, rhs
);
Ok(Value::Bool(true))
}
@@ -834,16 +831,16 @@ impl Interpreter {
let raise_error = is_last && type_match.get(expr).is_none();
match (expr.as_ref(), value) {
(Expr::Var(ident), _) if ident.text() == "_" => Ok(true),
(Expr::Var(ident), _) if ident.0.text() == "_" => Ok(true),
(Expr::Var(ident), _)
if check_existing_value
&& self.lookup_local_var(&ident.source_str()) == Some(value.clone()) =>
&& self.lookup_local_var(&ident.0.source_str()) == Some(value.clone()) =>
{
Ok(false)
}
(Expr::Var(ident), _) => {
self.add_variable(&ident.source_str(), value.clone())?;
self.add_variable(&ident.0.source_str(), value.clone())?;
Ok(true)
}
@@ -1323,13 +1320,6 @@ impl Interpreter {
}
fn eval_stmt(&mut self, stmt: &LiteralStmt, stmts: &[&LiteralStmt]) -> Result<bool> {
debug_new_group!(
"eval_stmt {}:{} {}",
stmt.span.line,
stmt.span.col,
stmt.span.text()
);
let (saved_state, skip_exec) = self.apply_with_modifiers(stmt)?;
let r = if !skip_exec {
self.eval_stmt_impl(stmt, stmts)
@@ -1401,7 +1391,7 @@ impl Interpreter {
// then evaluate statements only if the index applies to this collection.
let loop_expr_index = loop_expr.index();
if let Some(Expr::Var(index_var)) = loop_expr_index.as_ref().map(|r| r.as_ref()) {
if let Some(idx) = self.lookup_local_var(&index_var.source_str()) {
if let Some(idx) = self.lookup_local_var(&index_var.0.source_str()) {
if loop_expr_value[&idx] != Value::Undefined {
result = self.eval_stmts_in_loop(stmts, &loops[1..])? || result;
return Ok(result);
@@ -1442,6 +1432,9 @@ impl Interpreter {
Self::clear_scope(self.current_scope_mut()?);
if let Some(ctx) = self.contexts.last_mut() {
ctx.result = query_result.clone();
if ctx.early_return {
break;
}
}
}
@@ -1466,6 +1459,9 @@ impl Interpreter {
Self::clear_scope(self.current_scope_mut()?);
if let Some(ctx) = self.contexts.last_mut() {
ctx.result = query_result.clone();
if ctx.early_return {
break;
}
}
}
self.loop_var_values.remove(&loop_expr.expr());
@@ -1488,6 +1484,9 @@ impl Interpreter {
Self::clear_scope(self.current_scope_mut()?);
if let Some(ctx) = self.contexts.last_mut() {
ctx.result = query_result.clone();
if ctx.early_return {
break;
}
}
}
self.loop_var_values.remove(&loop_expr.expr());
@@ -1514,7 +1513,7 @@ impl Interpreter {
loop {
match expr.as_ref() {
Expr::Var(v) => {
comps.push(Value::String(v.text().into()));
comps.push(Value::String(v.0.text().into()));
break;
}
Expr::RefBrack { refr, index, .. } => {
@@ -1522,7 +1521,7 @@ impl Interpreter {
expr = refr;
}
Expr::RefDot { refr, field, .. } => {
comps.push(Value::String(field.text().into()));
comps.push(Value::String(field.0.text().into()));
expr = refr;
}
_ => {
@@ -1600,30 +1599,95 @@ impl Interpreter {
Ok(())
}
// A ref is a constant ref, if it does not contain any local variables.
// For now, we restrict constant refs to those that contain only simple literals.
fn is_constant_ref(&self, mut expr: &Ref<Expr>) -> Result<bool> {
loop {
match expr.as_ref() {
Expr::Var(_) => break,
Expr::RefDot { refr, .. } => expr = refr,
Expr::RefBrack { refr, index, .. } if self.is_simple_literal(index)? => expr = refr,
_ => return Ok(false),
}
}
Ok(true)
}
fn is_simple_literal(&self, expr: &Ref<Expr>) -> Result<bool> {
Ok(matches!(
expr.as_ref(),
Expr::String(_)
| Expr::RawString(_)
| Expr::True(_)
| Expr::False(_)
| Expr::Null(_)
| Expr::Number(_)
))
}
// A rule's output expression is constant if it does not contain local variables.
// For now, we restrict output expressions to those that contain only simple literals.
fn is_constant_output(
&self,
key_expr: &Option<Ref<Expr>>,
output_expr: &Ref<Expr>,
) -> Result<bool> {
let mut is_const = true;
if let Some(key_expr) = key_expr {
is_const = self.is_simple_literal(key_expr)?;
}
Ok(is_const && self.is_simple_literal(output_expr)?)
}
fn eval_output_expr_in_loop(&mut self, loops: &[LoopExpr]) -> Result<bool> {
if loops.is_empty() {
let (key_expr, output_expr) = self.get_exprs_from_context()?;
let ctx = self.get_current_context()?;
let (is_set, is_old_style_set) = (ctx.is_set, ctx.is_old_style_set);
let (is_set, is_old_style_set, is_rule, constness_determined) = (
ctx.is_set,
ctx.is_old_style_set,
!ctx.is_compr,
ctx.output_constness_determined,
);
if let Some(rule_ref) = ctx.rule_ref.clone() {
let mut is_const_rule = if is_rule && !constness_determined {
self.is_constant_ref(&rule_ref)?
} else {
// Constness has already been determined or is not a rule.
// Treat the expression as not constant.
false
};
let mut comps = self.eval_rule_ref(&rule_ref)?;
if let Some(ke) = &key_expr {
comps.push(self.eval_expr(ke)?);
}
let output = if let Some(oe) = &output_expr {
// Rule is constant only if its ref, key and output are constant.
is_const_rule = is_const_rule && self.is_constant_output(&key_expr, oe)?;
self.eval_expr(oe)?
} else if is_old_style_set && !comps.is_empty() {
// Rule's constness is determined only by its ref.
let output = comps[comps.len() - 1].clone();
comps.pop();
output
} else {
// Rule's constness is determined only by its ref.
Value::Bool(true)
};
let comps_defined = comps.iter().all(|v| v != &Value::Undefined);
let ctx = self.contexts.last_mut().expect("no current context");
if is_const_rule {
ctx.early_return = true;
}
if is_rule {
ctx.output_constness_determined = true;
}
if output == Value::Undefined || !comps_defined {
return Ok(false);
}
@@ -2404,12 +2468,12 @@ impl Interpreter {
if let Some(ea) = extra_arg {
match ea.as_ref() {
Expr::Var(var)
if allow_return_arg && self.lookup_local_var(&var.source_str()).is_none() =>
if allow_return_arg && self.lookup_local_var(&var.0.source_str()).is_none() =>
{
let value =
self.eval_call_impl(span, expr, fcn, &params[..params.len() - 1])?;
if var.text() != "_" {
self.add_variable(&var.source_str(), value)?;
if var.0.text() != "_" {
self.add_variable(&var.0.source_str(), value)?;
}
Ok(Value::Bool(true))
}
@@ -2543,7 +2607,6 @@ impl Interpreter {
fn lookup_var(&mut self, span: &Span, fields: &[&str], no_error: bool) -> Result<Value> {
let name = span.source_str();
debug_new_group!("lookup_var: name={name}, fields={fields:?}, no_error={no_error}");
// Return local variable/argument.
if let Some(v) = self.lookup_local_var(&name) {
@@ -2648,13 +2711,6 @@ impl Interpreter {
}
fn eval_expr(&mut self, expr: &ExprRef) -> Result<Value> {
debug_new_group!(
"eval_expr: {}:{} {}",
expr.span().line,
expr.span().col,
expr.span().text()
);
#[cfg(feature = "coverage")]
if self.enable_coverage {
let span = expr.span();
@@ -2684,24 +2740,10 @@ impl Interpreter {
Expr::Null(_) => Ok(Value::Null),
Expr::True(_) => Ok(Value::Bool(true)),
Expr::False(_) => Ok(Value::Bool(false)),
Expr::Number(span) => {
let v = match Number::from_str(span.text()) {
Ok(v) => Ok(Value::Number(v)),
Err(_) => Err(span
.source
.error(span.line, span.col, "could not parse number")),
};
v
}
Expr::Number((_, v)) => Ok(v.clone()),
// TODO: Handle string vs rawstring
Expr::String(span) => {
match serde_json::from_str::<Value>(format!("\"{}\"", span.text()).as_str()) {
Ok(s) => Ok(s),
Err(e) => bail!(span.error(format!("invalid string literal. {e}").as_str())),
}
}
Expr::RawString(span) => Ok(Value::String(span.text().to_string().into())),
Expr::String((_, v)) => Ok(v.clone()),
Expr::RawString((_, v)) => Ok(v.clone()),
// TODO: Handle undefined variables
Expr::Var(_) => self.eval_chained_ref_dot_or_brack(expr),
Expr::RefDot { .. } => self.eval_chained_ref_dot_or_brack(expr),
@@ -2929,17 +2971,17 @@ impl Interpreter {
while let Some(e) = expr {
match e {
Expr::RefDot { refr, field, .. } => {
comps.push(field.text());
comps.push(field.0.text());
expr = Some(refr);
}
Expr::RefBrack { refr, index, .. } if matches!(index.as_ref(), Expr::String(_)) => {
if let Expr::String(s) = index.as_ref() {
comps.push(s.text());
comps.push(s.0.text());
expr = Some(refr);
}
}
Expr::Var(v) => {
comps.push(v.text());
comps.push(v.0.text());
expr = None;
}
_ => bail!(e.span().error("invalid ref expression")),
@@ -3005,7 +3047,7 @@ impl Interpreter {
}
// The following may evaluate to undefined.
Var(span) => ("var", span),
Var((span, _)) => ("var", span),
Call { span, .. } => ("call", span),
UnaryExpr { span, .. } => ("unaryexpr", span),
RefDot { span, .. } => ("ref", span),
@@ -3379,19 +3421,19 @@ impl Interpreter {
loop {
refr = match refr.as_ref() {
Expr::Var(v) => {
components.push(v.text().into());
components.push(v.0.text().into());
break;
}
Expr::RefBrack { refr, index, .. } => {
if let Expr::String(s) = index.as_ref() {
components.push(s.text().into());
components.push(s.0.text().into());
} else {
components.clear();
}
refr
}
Expr::RefDot { refr, field, .. } => {
components.push(field.text().into());
components.push(field.0.text().into());
refr
}
_ => break,
@@ -3402,12 +3444,8 @@ impl Interpreter {
}
pub fn create_rule_prefixes(&mut self) -> Result<()> {
debug_new_group!("create_rule_prefixes");
debug!("data before: {}", self.data);
for module in self.modules.clone() {
let module_path = Self::get_rule_path_components(&module.package.refr)?;
debug!("processing module {module_path:?}");
for rule in &module.policy {
let rule_refr = Self::get_rule_refr(rule);
@@ -3443,7 +3481,7 @@ impl Interpreter {
}
}
}
debug!("data after: {}", self.data);
Ok(())
}
@@ -3452,6 +3490,9 @@ impl Interpreter {
let comps: Vec<&str> = comps.iter().map(|s| s.text()).collect();
for c in 0..comps.len() {
let path = self.current_module_path.clone() + "." + &comps[0..c + 1].join(".");
if c + 1 == comps.len() {
self.rule_paths.insert(path.clone());
}
match self.rules.entry(path) {
Entry::Occupied(o) => {
@@ -3476,6 +3517,10 @@ impl Interpreter {
let comps: Vec<&str> = comps.iter().map(|s| s.text()).collect();
for (idx, c) in (0..comps.len()).enumerate() {
let path = self.current_module_path.clone() + "." + &comps[0..c + 1].join(".");
if c + 1 == comps.len() {
self.rule_paths.insert(path.clone());
}
match self.default_rules.entry(path) {
Entry::Occupied(o) => {
if idx + 1 == comps.len() {
@@ -3509,12 +3554,12 @@ impl Interpreter {
let target = match &import.r#as {
Some(s) => s.text(),
_ => match import.refr.as_ref() {
Expr::RefDot { field, .. } => field.text(),
Expr::RefDot { field, .. } => field.0.text(),
Expr::RefBrack { index, .. } => match index.as_ref() {
Expr::String(s) => s.text(),
Expr::String(s) => s.0.text(),
_ => "",
},
Expr::Var(v) if v.text() == "input" => {
Expr::Var(v) if v.0.text() == "input" => {
// Warn redundant import of input. Ignore it.
eprintln!(
"{}",
@@ -3733,4 +3778,14 @@ impl Interpreter {
pub fn take_prints(&mut self) -> Result<Vec<String>> {
Ok(std::mem::take(&mut self.prints))
}
pub fn eval_rule_in_path(&mut self, path: String) -> Result<Value> {
if !self.rule_paths.contains(&path) {
bail!("not a valid rule path");
}
self.ensure_rule_evaluated(path.clone())?;
let parts: Vec<&str> = path.split('.').collect();
Ok(Self::get_value_chained(self.data.clone(), &parts[1..]))
}
}

View File

@@ -18,7 +18,7 @@ use anyhow::{anyhow, bail, Result};
struct SourceInternal {
pub file: String,
pub contents: String,
pub lines: Vec<(u16, u16)>,
pub lines: Vec<(u32, u32)>,
}
#[derive(Clone)]
@@ -61,8 +61,8 @@ impl Debug for Source {
#[derive(Clone)]
pub struct SourceStr {
source: Source,
start: u16,
end: u16,
start: u32,
end: u32,
}
impl Debug for SourceStr {
@@ -78,7 +78,7 @@ impl std::fmt::Display for SourceStr {
}
impl SourceStr {
pub fn new(source: Source, start: u16, end: u16) -> Self {
pub fn new(source: Source, start: u32, end: u32) -> Self {
Self { source, start, end }
}
@@ -116,39 +116,43 @@ impl std::cmp::Ord for SourceStr {
}
impl Source {
pub fn new(file: String, contents: String) -> Source {
pub fn from_contents(file: String, contents: String) -> Result<Source> {
let max_size = u32::MAX as usize - 2; // Account for rows, cols possibly starting at 1, EOF etc.
if contents.len() > max_size {
bail!("{file} exceeds maximum allowed policy file size {max_size}");
}
let mut lines = vec![];
let mut prev_ch = ' ';
let mut prev_pos = 0u16;
let mut start = 0u16;
let mut prev_pos = 0u32;
let mut start = 0u32;
for (i, ch) in contents.char_indices() {
if ch == '\n' {
let end = match prev_ch {
'\r' => prev_pos,
_ => i as u16,
_ => i as u32,
};
lines.push((start, end));
start = i as u16 + 1;
start = i as u32 + 1;
}
prev_ch = ch;
prev_pos = i as u16;
prev_pos = i as u32;
}
if (start as usize) < contents.len() {
lines.push((start, contents.len() as u16));
lines.push((start, contents.len() as u32));
} else if contents.is_empty() {
lines.push((0, 0));
} else {
let s = (contents.len() - 1) as u16;
let s = (contents.len() - 1) as u32;
lines.push((s, s));
}
Self {
Ok(Self {
src: Rc::new(SourceInternal {
file,
contents,
lines,
}),
}
})
}
pub fn from_file<P: AsRef<Path>>(path: P) -> Result<Source> {
@@ -157,10 +161,7 @@ impl Source {
Err(e) => bail!("Failed to read {}. {e}", path.as_ref().display()),
};
// TODO: retain path instead of converting to string
Ok(Self::new(
path.as_ref().to_string_lossy().to_string(),
contents,
))
Self::from_contents(path.as_ref().to_string_lossy().to_string(), contents)
}
pub fn file(&self) -> &String {
@@ -169,7 +170,7 @@ impl Source {
pub fn contents(&self) -> &String {
&self.src.contents
}
pub fn line(&self, idx: u16) -> &str {
pub fn line(&self, idx: u32) -> &str {
let idx = idx as usize;
if idx < self.src.lines.len() {
let (start, end) = self.src.lines[idx];
@@ -179,7 +180,7 @@ impl Source {
}
}
pub fn message(&self, line: u16, col: u16, kind: &str, msg: &str) -> String {
pub fn message(&self, line: u32, col: u32, kind: &str, msg: &str) -> String {
if line as usize > self.src.lines.len() {
return format!("{}: invalid line {} specified", self.src.file, line);
}
@@ -206,7 +207,7 @@ impl Source {
)
}
pub fn error(&self, line: u16, col: u16, msg: &str) -> anyhow::Error {
pub fn error(&self, line: u32, col: u32, msg: &str) -> anyhow::Error {
anyhow!(self.message(line, col, "error", msg))
}
}
@@ -214,10 +215,10 @@ impl Source {
#[derive(Clone)]
pub struct Span {
pub source: Source,
pub line: u16,
pub col: u16,
pub start: u16,
pub end: u16,
pub line: u32,
pub col: u32,
pub start: u32,
pub end: u32,
}
impl Span {
@@ -272,8 +273,8 @@ pub struct Token(pub TokenKind, pub Span);
pub struct Lexer<'source> {
source: Source,
iter: Peekable<CharIndices<'source>>,
line: u16,
col: u16,
line: u32,
col: u32,
}
impl<'source> Lexer<'source> {
@@ -312,15 +313,15 @@ impl<'source> Lexer<'source> {
}
}
let end = self.peek().0;
self.col += (end - start) as u16;
self.col += (end - start) as u32;
Ok(Token(
TokenKind::Ident,
Span {
source: self.source.clone(),
line: self.line,
col,
start: start as u16,
end: end as u16,
start: start as u32,
end: end as u32,
},
))
}
@@ -363,7 +364,7 @@ impl<'source> Lexer<'source> {
}
let end = self.peek().0;
self.col += (end - start) as u16;
self.col += (end - start) as u32;
// Check for invalid number.Valid number cannot be followed by
// these characters:
@@ -403,8 +404,8 @@ impl<'source> Lexer<'source> {
source: self.source.clone(),
line: self.line,
col,
start: start as u16,
end: end as u16,
start: start as u32,
end: end as u32,
},
))
}
@@ -440,8 +441,8 @@ impl<'source> Lexer<'source> {
source: self.source.clone(),
line,
col,
start: start as u16,
end: end as u16 - 1,
start: start as u32,
end: end as u32 - 1,
},
))
}
@@ -453,9 +454,9 @@ impl<'source> Lexer<'source> {
let (start, _) = self.peek();
loop {
let (offset, ch) = self.peek();
let col = self.col + (offset - start) as u16;
let col = self.col + (offset - start) as u32;
match ch {
'"' | '#' | '\x00' => {
'"' | '\x00' => {
break;
}
'\\' => {
@@ -468,7 +469,7 @@ impl<'source> Lexer<'source> {
'u' => {
for _i in 0..4 {
let (offset, ch) = self.peek();
let col = self.col + (offset - start) as u16;
let col = self.col + (offset - start) as u32;
if !ch.is_ascii_hexdigit() {
return Err(self.source.error(
line,
@@ -484,7 +485,7 @@ impl<'source> Lexer<'source> {
}
_ => {
// check for valid json chars
let col = self.col + (offset - start) as u16;
let col = self.col + (offset - start) as u32;
if !('\u{0020}'..='\u{10FFFF}').contains(&ch) {
return Err(self.source.error(line, col, "invalid character in string"));
}
@@ -499,7 +500,7 @@ impl<'source> Lexer<'source> {
self.iter.next();
let end = self.peek().0;
self.col += (end - start) as u16;
self.col += (end - start) as u32;
// Ensure that the string is parsable in Rust.
match serde_json::from_str::<String>(&self.source.contents()[start - 1..end]) {
@@ -522,8 +523,8 @@ impl<'source> Lexer<'source> {
source: self.source.clone(),
line,
col: col + 1,
start: start as u16,
end: end as u16 - 1,
start: start as u32,
end: end as u32 - 1,
},
))
}
@@ -593,14 +594,14 @@ impl<'source> Lexer<'source> {
source: self.source.clone(),
line: self.line,
col,
start: start as u16,
end: start as u16 + 1,
start: start as u32,
end: start as u32 + 1,
}))
}
':' => {
self.col += 1;
self.iter.next();
let mut end = start as u16 + 1;
let mut end = start as u32 + 1;
if self.peek().1 == '=' {
self.col += 1;
self.iter.next();
@@ -610,7 +611,7 @@ impl<'source> Lexer<'source> {
source: self.source.clone(),
line: self.line,
col,
start: start as u16,
start: start as u32,
end
}))
}
@@ -626,8 +627,8 @@ impl<'source> Lexer<'source> {
source: self.source.clone(),
line: self.line,
col,
start: start as u16,
end: self.peek().0 as u16,
start: start as u32,
end: self.peek().0 as u32,
}))
}
'!' if self.peekahead(1).1 == '=' => {
@@ -638,8 +639,8 @@ impl<'source> Lexer<'source> {
source: self.source.clone(),
line: self.line,
col,
start: start as u16,
end: self.peek().0 as u16,
start: start as u32,
end: self.peek().0 as u32,
}))
}
'"' => self.read_string(),
@@ -648,8 +649,8 @@ impl<'source> Lexer<'source> {
source: self.source.clone(),
line:self.line,
col,
start: start as u16,
end: start as u16
start: start as u32,
end: start as u32
})),
_ if chr.is_ascii_digit() => self.read_number(),
_ if chr.is_ascii_alphabetic() || chr == '_' => {

View File

@@ -47,9 +47,9 @@ use std::rc::Rc;
#[derive(Debug, Clone, Serialize, Eq, PartialEq)]
pub struct Location {
/// Line number. Starts at 1.
pub row: u16,
pub row: u32,
/// Column number. Starts at 1.
pub col: u16,
pub col: u32,
}
/// An expression in a Rego query.

View File

@@ -3,7 +3,11 @@
use crate::ast::*;
use crate::lexer::*;
use crate::number::*;
use crate::value::*;
use std::collections::BTreeMap;
use std::str::FromStr;
use anyhow::{anyhow, bail, Result};
@@ -12,8 +16,8 @@ pub struct Parser<'source> {
source: Source,
lexer: Lexer<'source>,
tok: Token,
line: u16,
end: u16,
line: u32,
end: u32,
future_keywords: BTreeMap<String, Span>,
rego_v1: bool,
}
@@ -102,15 +106,25 @@ impl<'source> Parser<'source> {
match refr.as_ref() {
Expr::RefDot { refr, field, .. } => {
Self::get_path_ref_components_into(refr, comps)?;
comps.push(field.clone());
comps.push(field.0.clone());
}
Expr::RefBrack { refr, index, .. } => {
Self::get_path_ref_components_into(refr, comps)?;
Self::get_path_ref_components_into(index, comps)?;
}
Expr::Var(v) => comps.push(v.clone()),
Expr::String(s) => comps.push(s.clone()),
_ => bail!("internal error: not a simple ref"),
Expr::Var(v) => comps.push(v.0.clone()),
Expr::String(s) => comps.push(s.0.clone()),
Expr::True(s) | Expr::False(s) | Expr::Null(s) => comps.push(s.clone()),
Expr::Number(s) => {
// Ensure that the span will be the serialized representation.
if *s.0.text() == s.1.to_json_str()? {
comps.push(s.0.clone());
} else {
bail!(refr.span().error("not a valid ref"));
}
}
_ => bail!(refr.span().error("not a valid ref")),
}
Ok(())
}
@@ -231,17 +245,38 @@ impl<'source> Parser<'source> {
}
}
fn read_number(span: Span) -> Result<Expr> {
match Number::from_str(span.text()) {
Ok(v) => Ok(Expr::Number((span, Value::Number(v)))),
Err(_) => bail!(span.error("could not parse number")),
}
}
fn parse_scalar_or_var(&mut self) -> Result<Expr> {
let span = self.tok.1.clone();
let node = match &self.tok.0 {
TokenKind::Number => Expr::Number(span),
TokenKind::String => Expr::String(span),
TokenKind::RawString => Expr::RawString(span),
TokenKind::Number => Self::read_number(span)?,
TokenKind::String => {
let v = match serde_json::from_str::<Value>(format!("\"{}\"", span.text()).as_str())
{
Ok(v) => v,
Err(e) => bail!(span.error(format!("invalid string literal. {e}").as_str())),
};
Expr::String((span, v))
}
TokenKind::RawString => {
let v = Value::from(span.text().to_string());
Expr::RawString((span, v))
}
TokenKind::Ident => match self.token_text() {
"null" => Expr::Null(span),
"true" => Expr::True(span),
"false" => Expr::False(span),
_ => return Ok(Expr::Var(self.parse_var()?)),
_ => {
let ident = self.parse_var()?;
let v = Value::from(ident.text());
return Ok(Expr::Var((ident, v)));
}
},
_ => {
return Err(self.source.error(
@@ -529,10 +564,11 @@ impl<'source> Parser<'source> {
)
);
}
let fieldv = Value::from(field.text());
term = Expr::RefDot {
span,
refr: Ref::new(term),
field,
field: (field, fieldv),
};
}
"[" => {
@@ -632,7 +668,7 @@ impl<'source> Parser<'source> {
rhs_span.col += 1;
self.next_token()?;
Expr::Number(rhs_span)
Self::read_number(rhs_span)?
} else {
self.next_token()?;
self.parse_mul_div_mod_expr()?
@@ -717,7 +753,7 @@ impl<'source> Parser<'source> {
fn parse_membership_tail(
&mut self,
start: u16,
start: u32,
mut expr1: Expr,
mut expr2: Option<Expr>,
) -> Result<Expr> {
@@ -786,10 +822,10 @@ impl<'source> Parser<'source> {
"=" => AssignOp::Eq,
":=" if self.rego_v1 => {
if let Expr::Var(v) = &expr {
if v.text() == "input" {
if v.0.text() == "input" {
bail!(span.error("input cannot be shadowed"));
}
if v.text() == "data" {
if v.0.text() == "data" {
bail!(span.error("data cannot be shadowed"));
}
}
@@ -1055,11 +1091,16 @@ impl<'source> Parser<'source> {
}))
}
fn span_and_value(s: Span) -> (Span, Value) {
let v = Value::from(s.text());
(s, v)
}
fn parse_path_ref(&mut self) -> Result<Expr> {
let start = self.tok.1.start;
let var = self.parse_var()?;
let mut refr = Expr::Var(var);
let mut refr = Expr::Var(Self::span_and_value(var));
loop {
let mut span = self.tok.1.clone();
let sep_pos = span.start;
@@ -1095,13 +1136,13 @@ impl<'source> Parser<'source> {
refr = Expr::RefDot {
span,
refr: Ref::new(refr),
field,
field: Self::span_and_value(field),
};
}
"[" => {
self.next_token()?;
let index = match &self.tok.0 {
TokenKind::String => Expr::String(self.tok.1.clone()),
TokenKind::String => Expr::String(Self::span_and_value(self.tok.1.clone())),
_ => {
return Err(self.source.error(
self.tok.1.line,
@@ -1140,7 +1181,7 @@ impl<'source> Parser<'source> {
bail!(span.error("data cannot be shadowed"));
}
}
Expr::Var(v)
Expr::Var(Self::span_and_value(v))
} else {
return Err(self.source.error(
span.line,
@@ -1184,7 +1225,7 @@ impl<'source> Parser<'source> {
term = Expr::RefDot {
span,
refr: Ref::new(term),
field,
field: Self::span_and_value(field),
};
}
"[" => {
@@ -1488,7 +1529,10 @@ impl<'source> Parser<'source> {
Ok(Rule::Default {
span,
refr: rule_ref,
args: args.into_iter().map(|a| Ref::new(Expr::Var(a))).collect(),
args: args
.into_iter()
.map(|a| Ref::new(Expr::Var(Self::span_and_value(a))))
.collect(),
op,
value,
})

View File

@@ -299,23 +299,23 @@ fn gather_assigned_vars(
) -> Result<()> {
traverse(expr, &mut |e| match e.as_ref() {
// Ignore _, input, data.
Var(v) if matches!(v.text(), "_" | "input" | "data") => Ok(false),
Var(v) if matches!(v.0.text(), "_" | "input" | "data") => Ok(false),
// Record local var that can shadow input var.
Var(v) if can_shadow => {
scope.locals.insert(v.source_str(), v.clone());
scope.locals.insert(v.0.source_str(), v.0.clone());
Ok(false)
}
// Record input vars.
Var(v) if var_exists(v, parent_scopes) => {
scope.inputs.insert(v.source_str());
Var(v) if var_exists(&v.0, parent_scopes) => {
scope.inputs.insert(v.0.source_str());
Ok(false)
}
// Record local var.
Var(v) => {
scope.unscoped.insert(v.source_str());
scope.unscoped.insert(v.0.source_str());
Ok(false)
}
@@ -327,8 +327,10 @@ fn gather_assigned_vars(
fn gather_input_vars(expr: &Ref<Expr>, parent_scopes: &[Scope], scope: &mut Scope) -> Result<()> {
traverse(expr, &mut |e| match e.as_ref() {
Var(v) if !scope.unscoped.contains(&v.source_str()) && var_exists(v, parent_scopes) => {
scope.inputs.insert(v.source_str());
Var(v)
if !scope.unscoped.contains(&v.0.source_str()) && var_exists(&v.0, parent_scopes) =>
{
scope.inputs.insert(v.0.source_str());
Ok(false)
}
_ => Ok(true),
@@ -537,7 +539,7 @@ impl Analyzer {
for a in args.iter() {
traverse(a, &mut |e| {
if let Var(v) = e.as_ref() {
scope.unscoped.insert(v.source_str());
scope.unscoped.insert(v.0.source_str());
}
Ok(true)
})?;
@@ -630,10 +632,10 @@ impl Analyzer {
let full_expr = expr;
std::convert::identity(&full_expr);
traverse(expr, &mut |e| match e.as_ref() {
Var(v) if !matches!(v.text(), "_" | "input" | "data") => {
let name = v.source_str();
Var(v) if !matches!(v.0.text(), "_" | "input" | "data") => {
let name = v.0.source_str();
let is_extra_arg = match assigned_vars {
Some(vars) => vars.contains(&v.source_str()),
Some(vars) => vars.contains(&v.0.source_str()),
_ => false,
};
@@ -642,7 +644,7 @@ impl Analyzer {
{
if !is_extra_arg {
used_vars.push(name.clone());
first_use.entry(name).or_insert(v.clone());
first_use.entry(name).or_insert(v.0.clone());
}
} else if !scope.inputs.contains(&name) {
#[cfg(feature = "deprecated")]
@@ -656,7 +658,9 @@ impl Analyzer {
}
}
}
bail!(v.error(format!("use of undefined variable `{name}` is unsafe").as_str()));
bail!(v
.0
.error(format!("use of undefined variable `{name}` is unsafe").as_str()));
}
Ok(false)
}
@@ -664,7 +668,7 @@ impl Analyzer {
RefBrack { refr, index, .. } => {
traverse(index, &mut |e| match e.as_ref() {
Var(v) => {
let var = v.source_str();
let var = v.0.source_str();
if scope.locals.contains_key(&var) || scope.unscoped.contains(&var) {
let (rb_used_vars, rb_comprs) =
Self::gather_used_vars_comprs_index_vars(
@@ -758,10 +762,10 @@ impl Analyzer {
let mut vars = vec![];
traverse(expr, &mut |e| match e.as_ref() {
Var(v) => {
let var = v.source_str();
let var = v.0.source_str();
if scope.locals.contains_key(&var) {
if check_first_use {
Self::check_first_use(v, first_use)?;
Self::check_first_use(&v.0, first_use)?;
}
vars.push(var);
} else if scope.unscoped.contains(&var) {
@@ -947,8 +951,8 @@ impl Analyzer {
non_vars: &mut Vec<Ref<Expr>>,
) -> Result<()> {
traverse(expr, &mut |e| match e.as_ref() {
Var(v) if scope.locals.contains_key(&v.source_str()) => {
vars.push(v.source_str());
Var(v) if scope.locals.contains_key(&v.0.source_str()) => {
vars.push(v.0.source_str());
Ok(false)
}
// TODO: Object key/value

View File

@@ -73,10 +73,10 @@ fn match_values(computed: &Value, expected: &Value) -> Result<()> {
if computed != expected {
panic!(
"{}",
colored_diff::PrettyDifference {
expected: &serde_yaml::to_string(&expected)?,
actual: &serde_yaml::to_string(&computed)?
}
prettydiff::diff_chars(
&serde_yaml::to_string(&expected)?,
&serde_yaml::to_string(&computed)?
)
);
}
Ok(())
@@ -347,7 +347,7 @@ fn yaml_test(file: &str) -> Result<()> {
Err(e) => {
// If Err is returned, it doesn't always get printed by cargo test.
// Therefore, panic with the error.
panic!("{}", e);
panic!("{e}");
}
}
}

View File

@@ -38,7 +38,10 @@ fn analyze_file(regos: &[String], expected_scopes: &[Scope]) -> Result<()> {
let mut sources = vec![];
let mut modules = vec![];
for (idx, _) in regos.iter().enumerate() {
sources.push(Source::new(format!("rego_{idx}"), regos[idx].clone()));
sources.push(Source::from_contents(
format!("rego_{idx}"),
regos[idx].clone(),
)?);
}
for source in &sources {

View File

@@ -8,95 +8,23 @@ use crate::lexer::*;
use std::collections::BTreeMap;
use anyhow::{bail, Result};
#[cfg(debug_assertions)]
macro_rules! debug {
($($arg:tt)+) => {
{
if log::log_enabled!(log::Level::Debug) {
print!("{}:{}:", file!(), line!());
crate::utils::NESTING.with(|f| {
print!("{}", " ".repeat(*f.borrow() as usize));
});
println!($($arg)+);
}
}
}
}
#[cfg(not(debug_assertions))]
macro_rules! debug {
($($arg:tt)+) => {};
}
#[allow(unused)]
pub(crate) use debug;
#[cfg(debug_assertions)]
#[allow(unused)]
macro_rules! debug_new_group {
($($arg:tt)+) => {
debug!($($arg)+);
let _group = DebugNesting::new();
};
}
#[cfg(not(debug_assertions))]
macro_rules! debug_new_group {
($($arg:tt)+) => {};
}
#[allow(unused)]
pub(crate) use debug_new_group;
#[allow(unused)]
pub struct DebugNesting {}
#[cfg(debug_assertions)]
thread_local!(pub static NESTING: std::cell::RefCell<u32> = std::cell::RefCell::new(1));
impl DebugNesting {
#[cfg(debug_assertions)]
#[allow(unused)]
pub fn new() -> DebugNesting {
NESTING.with(|f| {
*f.borrow_mut() += 1;
});
DebugNesting {}
}
}
#[allow(unused)]
impl Drop for DebugNesting {
#[cfg(debug_assertions)]
fn drop(&mut self) {
NESTING.with(|f| {
*f.borrow_mut() -= 1;
});
}
#[cfg(not(debug_assertions))]
fn drop(&mut self) {}
}
pub fn get_path_string(refr: &Expr, document: Option<&str>) -> Result<String> {
let mut comps: Vec<&str> = vec![];
let mut expr = Some(refr);
while expr.is_some() {
match expr {
Some(Expr::RefDot { refr, field, .. }) => {
comps.push(field.text());
comps.push(field.0.text());
expr = Some(refr);
}
Some(Expr::RefBrack { refr, index, .. }) => {
if let Expr::String(s) = index.as_ref() {
comps.push(s.text());
comps.push(s.0.text());
}
expr = Some(refr);
}
Some(Expr::Var(v)) => {
comps.push(v.text());
comps.push(v.0.text());
expr = None;
}
_ => bail!("internal error: not a simple ref {expr:?}"),
@@ -193,7 +121,7 @@ pub fn get_root_var(mut expr: &Expr) -> Result<SourceStr> {
let empty = expr.span().source_str().clone_empty();
loop {
match expr {
Expr::Var(v) => return Ok(v.source_str()),
Expr::Var(v) => return Ok(v.0.source_str()),
Expr::RefDot { refr, .. } | Expr::RefBrack { refr, .. } => expr = refr,
_ => return Ok(empty),
}

View File

@@ -460,8 +460,8 @@ impl From<u128> for Value {
/// # use regorus::*;
/// # fn main() -> anyhow::Result<()> {
/// assert_eq!(
/// Value::from(340_282_366_920_938_463_463_374_607_431_768_211_455u128),
/// Value::from_json_str("340282366920938463463374607431768211455")?);
/// Value::from(340_282_366_920_938_463_463_374_607_431_768_211_455u128).as_u128()?,
/// 340_282_366_920_938_463_463_374_607_431_768_211_455u128);
/// # Ok(())
/// # }
fn from(n: u128) -> Self {
@@ -475,8 +475,8 @@ impl From<i128> for Value {
/// # use regorus::*;
/// # fn main() -> anyhow::Result<()> {
/// assert_eq!(
/// Value::from(-170141183460469231731687303715884105728i128),
/// Value::from_json_str("-170141183460469231731687303715884105728")?);
/// Value::from(-170141183460469231731687303715884105728i128).as_i128()?,
/// -170141183460469231731687303715884105728i128);
/// # Ok(())
/// # }
fn from(n: i128) -> Self {
@@ -551,7 +551,7 @@ impl From<f64> for Value {
/// # fn main() -> anyhow::Result<()> {
/// assert_eq!(
/// Value::from(3.141592653589793),
/// Value::from_json_str("3.141592653589793")?);
/// Value::from_numeric_string("3.141592653589793")?);
/// # Ok(())
/// # }
/// ```
@@ -559,19 +559,19 @@ impl From<f64> for Value {
/// Note, f64 can store only around 15 digits of precision whereas [`Value::Number`]
/// can store arbitrary precision. Adding an extra digit to the f64 literal in the above
/// example causes loss of precision and the Value created from f64 does not match the
/// Value parsed from json string (which is more precise).
/// Value parsed from numeric string (which is more precise).
/// ```
/// # use regorus::*;
/// # fn main() -> anyhow::Result<()> {
/// // The last digit is lost in f64.
/// assert_ne!(
/// Value::from(3.1415926535897932),
/// Value::from_json_str("3.141592653589793232")?);
/// Value::from_numeric_string("3.141592653589793232")?);
///
/// // The value, in this case is equal to parsing the json number with last digit omitted.
/// assert_ne!(
/// Value::from(3.1415926535897932),
/// Value::from_json_str("3.14159265358979323")?);
/// Value::from_numeric_string("3.14159265358979323")?);
/// # Ok(())
/// # }
/// ```
@@ -584,6 +584,55 @@ impl From<f64> for Value {
}
}
impl From<serde_json::Value> for Value {
/// Create a [`Value`] from [`serde_json::Value`].
///
/// Returns [`Value::Undefined`] in case of error.
/// ```
/// # use regorus::*;
/// # fn main() -> anyhow::Result<()> {
/// let json_v = serde_json::json!({ "x":10, "y": 20 });
/// let v = Value::from(json_v);
///
/// assert_eq!(v["x"].as_u64()?, 10);
/// assert_eq!(v["y"].as_u64()?, 20);
/// # Ok(())
/// # }
fn from(v: serde_json::Value) -> Self {
match serde_json::from_value(v) {
Ok(v) => v,
_ => Value::Undefined,
}
}
}
#[cfg(feature = "yaml")]
impl From<serde_yaml::Value> for Value {
/// Create a [`Value`] from [`serde_yaml::Value`].
///
/// Returns [`Value::Undefined`] in case of error.
/// ```
/// # use regorus::*;
/// # fn main() -> anyhow::Result<()> {
/// let yaml = "
/// x: 10
/// y: 20
/// ";
/// let yaml_v : serde_yaml::Value = serde_yaml::from_str(&yaml).unwrap();
/// let v = Value::from(yaml_v);
///
/// assert_eq!(v["x"].as_u64()?, 10);
/// assert_eq!(v["y"].as_u64()?, 20);
/// # Ok(())
/// # }
fn from(v: serde_yaml::Value) -> Self {
match serde_yaml::from_value(v) {
Ok(v) => v,
_ => Value::Undefined,
}
}
}
impl Value {
/// Create a [`Value::Number`] from a string containing numeric representation of a number.
///
@@ -594,9 +643,9 @@ impl Value {
/// # fn main() -> anyhow::Result<()> {
/// let v = Value::from_numeric_string("3.14159265358979323846264338327950288419716939937510")?;
///
/// assert_eq!(
/// v.to_json_str()?,
/// "3.1415926535897932384626433832795028841971693993751");
/// println!("{}", v.to_json_str()?);
/// // Prints 3.1415926535897932384626433832795028841971693993751 if serde_json/arbitrary_precision feature is enabled.
/// // Prints 3.141592653589793 if serde_json/arbitrary_precision is not enabled.
/// # Ok(())
/// # }
/// ```

View File

@@ -92,10 +92,10 @@ fn run_aci_tests(dir: &Path) -> Result<()> {
Ok(actual) => {
println!(
"DIFF {}",
colored_diff::PrettyDifference {
expected: &serde_yaml::to_string(&case.want_result)?,
actual: &serde_yaml::to_string(&actual)?
}
prettydiff::diff_chars(
&serde_yaml::to_string(&case.want_result)?,
&serde_yaml::to_string(&actual)?
)
);
nfailures += 1;

View File

@@ -21,3 +21,87 @@ cases:
x1: [[1, 0], [2, 1], [3, 2], [4, 3]]
x2: [[1, 1], [2, 2], [3, 3], [4, 4]]
x3: [["q", "p"], ["s", "r"]]
- note: early return
data: {}
modules:
- |
package test
import future.keywords
a = [1, "hello"]
# Implicit value
b1 {
a[_] + 1
}
# Literals
b2 := true { a[_] + 1 }
b3 := false { a[_] + 1 }
b4 := 1 { a[_] + 1 }
b5 := null { a[_] + 1 }
b6 := "hello" { a[_] + 1 }
b7 := `world` { a[_] + 1 }
# constant refs
c[null] := true { a[_] + 1 }
c["hello"] := false { a[_] + 1 }
c[`world`] := 1 { a[_] + 1 }
c[true] := null { a[_] + 1 }
c[false] := "hello" { a[_] + 1 }
c[7] := `world` { a[_] + 1 }
# Old style set must should also be considered for early return.
old.style { a[_] + 1 }
# Multi part constant refactor
multi[1]["hello"] := 5 { a[_] +1 }
# Two elements must be produced
d = [1 | [1,2][_] ]
# Non simple ref must not result in early return.
f[p] = 5 {
p := a[_]
}
f1[p] = 5 {
a[p]
}
# Contains syntax
g contains p if {
p := a[_]
}
query: data.test
want_result:
a: [1, "hello"]
b1: true
b2: true
b3: false
b4: 1
b5: null
b6: "hello"
b7: "world"
c:
null: true
"hello": false
"world": 1
true: null
false: "hello"
7: "world"
d: [1, 1]
f:
1: 5
"hello": 5
f1:
0: 5
1: 5
g:
set!: [1, "hello"]
multi:
1:
"hello": 5
old:
set!: ["style"]

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,19 @@
[
false,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true
]

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,19 @@
[
false,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true
]

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,19 @@
[
false,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true
]

File diff suppressed because it is too large Load Diff

2620
tests/kata/data/large.rego Normal file

File diff suppressed because it is too large Load Diff

115
tests/kata/main.rs Normal file
View File

@@ -0,0 +1,115 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
use regorus::*;
use std::path::Path;
use anyhow::Result;
use clap::Parser;
use walkdir::WalkDir;
fn run_kata_tests(tests_dir: &Path, generate: bool) -> Result<()> {
for entry in WalkDir::new(tests_dir)
.max_depth(1) // Do not recurse
.sort_by_file_name()
.into_iter()
.filter_map(|e| e.ok())
{
let path = entry.path();
if path == tests_dir || !path.is_dir() {
continue;
}
let policy_file = path.join("policy.rego");
let inputs_file = path.join("inputs.txt");
let outputs_file = path.join("output.json");
let mut engine = Engine::new();
engine.add_policy_from_file(&policy_file)?;
engine.set_gather_prints(true);
engine.set_strict_builtin_errors(false);
#[cfg(feature = "coverage")]
engine.set_enable_coverage(true);
// Keep a copy of the engine.
let engine_base = engine.clone();
let mut results = if generate {
vec![]
} else {
Value::from_json_str(&std::fs::read_to_string(&outputs_file)?)?
.as_array()?
.iter()
.cloned()
.rev()
.collect()
};
let inputs = std::fs::read_to_string(&inputs_file)?;
for (lineno, line) in inputs.split('\n').enumerate() {
let line = line.trim();
if line.is_empty() {
continue;
}
// Remove "ep":
let line = line.replace("\"ep\":", "");
// Remove trailing ,
let line = &line[0..line.len() - 1];
let request = Value::from_json_str(line)?;
let rule = format!("data.agent_policy.{}", request[0].as_string()?.as_ref());
let input = request[1].clone();
// Evaluate using engine.
engine.set_input(input.clone());
let r = engine.eval_rule(rule.clone())?;
// Evaluate using fresh engine.
let mut new_engine = engine_base.clone();
new_engine.set_input(input);
let r_new = new_engine.eval_rule(rule)?;
// Ensure that both evaluations produced the same result.
assert_eq!(r, r_new);
if generate {
results.push(r);
} else {
let expected = results.pop().unwrap();
assert_eq!(r, expected, "{lineno} failed in {}", inputs_file.display());
}
}
if generate {
std::fs::write(outputs_file, Value::from(results).to_json_str()?)?;
}
#[cfg(feature = "coverage")]
{
let report = engine.get_coverage_report()?;
println!("{}", report.to_colored_string()?);
}
}
println!("kata tests passed");
Ok(())
}
#[derive(clap::Parser)]
#[command(author, version, about, long_about = None)]
struct Cli {
/// Path to Kata test suite.
#[arg(long, short)]
#[clap(default_value = "tests/kata/data")]
test_dir: String,
/// Generate outputs instead of testing.
#[arg(long, short)]
#[clap(default_value = "false")]
generate: bool,
}
fn main() -> Result<()> {
let cli = Cli::parse();
run_kata_tests(&Path::new(&cli.test_dir), cli.generate)
}

View File

@@ -56,11 +56,9 @@ cases:
- note: comments-within-rstring
rego: |
#Comments aren't allowed within strings,
"#This is not a comment
"#c
tokens:
error: unmatched "
#Comments aren't lexed within raw strings.
"#This is not a comment"#c
tokens: ["#This is not a comment", ""]
- note: comment-integer-break
rego: |

View File

@@ -77,7 +77,7 @@ fn yaml_test_impl(file: &str) -> Result<()> {
let test: Test = serde_yaml::from_str(&yaml)?;
for case in &test.cases {
let source = Source::new("case.rego".to_string(), case.rego.clone());
let source = Source::from_contents("case.rego".to_string(), case.rego.clone())?;
print!("case {} ", &case.note);
match get_tokens(&source) {
@@ -162,7 +162,7 @@ fn run(path: &str) {
#[test]
fn debug() -> Result<()> {
let rego = "\"This string is 35 characters long.\"\"short string\"";
let source = Source::new("case.rego".to_string(), rego.to_string());
let source = Source::from_contents("case.rego".to_string(), rego.to_string())?;
let mut lexer = Lexer::new(&source);
let tok = lexer.next_token()?;
@@ -184,7 +184,7 @@ fn debug() -> Result<()> {
#[test]
fn tab() -> Result<()> {
let rego = r#" "This string is 35 characters long."`raw string`p"#;
let source = Source::new("case.rego".to_string(), rego.to_string());
let source = Source::from_contents("case.rego".to_string(), rego.to_string())?;
let mut lexer = Lexer::new(&source);
@@ -214,7 +214,7 @@ fn tab() -> Result<()> {
#[test]
fn invalid_line() -> Result<()> {
let rego = "";
let source = Source::new("case.rego".to_string(), rego.to_string());
let source = Source::from_contents("case.rego".to_string(), rego.to_string())?;
assert_eq!(
source.message(2, 0, "", ""),
@@ -223,3 +223,21 @@ fn invalid_line() -> Result<()> {
Ok(())
}
#[test]
fn file_more_than_64_kb_size() -> Result<()> {
let source = Source::from_file("tests/kata/data/large.rego")?;
let mut lexer = Lexer::new(&source);
let mut count = 0;
// Read tokens until EOF.
loop {
let token = lexer.next_token()?;
count += 1;
if token.0 == TokenKind::Eof {
break;
}
}
assert_eq!(count, 8789);
Ok(())
}

View File

@@ -13,7 +13,7 @@ use serde::{Deserialize, Serialize};
use walkdir::WalkDir;
const OPA_REPO: &str = "https://github.com/open-policy-agent/opa";
const OPA_BRANCH: &str = "v0.62.0";
const OPA_BRANCH: &str = "v0.64.0";
#[derive(Serialize, Deserialize, PartialEq, Debug)]
#[serde(deny_unknown_fields)]

View File

@@ -102,13 +102,13 @@ fn match_expr_impl(e: &Expr, v: &Value) -> Result<()> {
return Ok(());
}
match e {
Expr::String(s) => match_span(s, &v["string"]),
Expr::RawString(s) => match_span(s, &v["rawstring"]),
Expr::Number(s) => match_span(s, &v["number"]),
Expr::String(s) => match_span(&s.0, &v["string"]),
Expr::RawString(s) => match_span(&s.0, &v["rawstring"]),
Expr::Number(s) => match_span(&s.0, &v["number"]),
Expr::True(s) => match_span(s, v),
Expr::False(s) => match_span(s, v),
Expr::Null(s) => match_span(s, v),
Expr::Var(s) => match_span(s, &v["var"]),
Expr::Var(s) => match_span(&s.0, &v["var"]),
Expr::Array { span, items } => match_vec(span, items, &v["array"]),
Expr::Set { span, items } => match_vec(span, items, &v["set"]),
Expr::Object { span, fields } => match_object(span, fields, &v["object"]),
@@ -141,7 +141,7 @@ fn match_expr_impl(e: &Expr, v: &Value) -> Result<()> {
Expr::RefDot { span, refr, field } => {
match_span_opt(span, &v["refdot"]["span"])?;
match_expr(refr, &v["refdot"]["refr"])?;
match_span(field, &v["refdot"]["field"])
match_span(&field.0, &v["refdot"]["field"])
}
Expr::RefBrack { span, refr, index } => {
match_span_opt(span, &v["refbrack"]["span"])?;
@@ -630,7 +630,7 @@ fn yaml_test_impl(file: &str) -> Result<()> {
for case in &test.cases {
print!("\ncase {} ", case.note);
let source = Source::new("case.rego".to_string(), case.rego.clone());
let source = Source::from_contents("case.rego".to_string(), case.rego.clone())?;
let mut parser = Parser::new(&source)?;
match parser.parse() {
Ok(module) => {