Commit Graph

125 Commits

Author SHA1 Message Date
Anand Krishnamoorthi
3ac9b1121e Lockdown kata test prints as well as prints of various values (#249)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-23 08:30:41 -07:00
Anand Krishnamoorthi
56bd8841d0 Provide a way to obtain package names of loaded policies (#239)
Engine::
- add_policy and add_policy_from_file return the package name.
- get_packages returns the names of loaded packages.

closes #229

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-18 08:02:54 -07:00
Anand Krishnamoorthi
e86b590f91 no_std support (#232)
- Disable default features in dependencies
- Use anyhow::Error::msg to map errors. Note: anyhow will itself be removed later.
- lazy_static/spin_no_std used in no_std environments
- ensure_no_std binary is built to target  thumbv7m-none-eabi to ensure that
  there are no std dependencies.  thumbv7m-none-eabi target has no std support.
- The opa-no-std feature enables only those Regorus features that work with no_std.
- Enable tests with no_std
- Update sizes of regorus binary in  README.md
- Ensure that regorus example can be built with only std
- Ensure that regorus example can be built with no_std

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-13 06:42:35 -07:00
Anand Krishnamoorthi
01fc234a33 add std feature (#231)
- `std` feature is enabled by default
- By default enable #![no_std] compilation
- Import std create if `std` feature is enabled or if testing
- Use core, alloc types
- Make it clear where std types are being used
- In no std, use BTreeMap in place of HashMap.
   HashMap is not available in no std due to lack of a
   secure random number generator

Note: The project does not yet compile without std feature being specified.
But it's really close to being able to do so.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-09 11:28:42 -07:00
Anand Krishnamoorthi
69d6426663 Use alloc, core instead of std (#225)
- Replace std with alloc, core in most places in src
  Tests, bindings aren't changed.
- Introduce BuiltinsMap type alias inplace of HashMap.
  In no_std case, this could be aliases to BTreeMap
- Fix clippy warnings

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-07 18:41:09 -07:00
Anand Krishnamoorthi
55abbb2b42 Update to OPA v0.64.0 (#219)
Implement json.marshal_with_options builtin

closes #215, closes #218

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-26 04:52:11 -07:00
Anand Krishnamoorthi
3743f32edc Enable policy files greater than 64KB in size (#217)
fixes #214

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-26 01:04:21 -07:00
Anand Krishnamoorthi
744dad6126 OPA Conformance: Do not interpret # within regular string (#216)
fixes #213

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-25 23:51:52 -07:00
Anand Krishnamoorthi
316f3a7692 early return (#189)
If a rule is written to produce a constant value, then not all iterations of loops
within it need to be executed. Execution can stop via early return once the first iteration
that produces a value has been executed.

This brings forth the question : What if one of the subsequent iterations would have resulted
in an error?
e.g:
x {
  [1, "hello"][_] + 1
}

Such errors are not raised; consistent with OPA.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-22 07:41:35 -07:00
Anand Krishnamoorthi
de56cce7cb Fix anyhow dependency issues (#208)
- Do not require backtrace feature
- Starting version 1.0.77, anyhow gathers backtrace is std feature (enabled by default)
  is specified even if backtrace feature is not enabled.
  Therefore specify default features as false.
- Specify version 1.0.45 since that is the minimul version required to successfully
  compile regorus

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-20 09:57:52 -07:00
Anand Krishnamoorthi
05e91da06e Do not enable serde_json/arbitrary_precision by default (#203)
The feature does not interoperate well with other serde_json features like untagged enums.

Fixes #199

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-11 06:23:14 -07:00
Anand Krishnamoorthi
8c69dd491b Rewrite so that code compiles with chrono_tz 0.8.5 and 0.9.0 (#201)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-11 04:37:38 +05:30
Anand Krishnamoorthi
d2049d07f3 Store Value instances in AST for strings, numbers and idents (#197)
This avoids having to create value instances during evaluation

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-07 18:21:32 +05:30
Anand Krishnamoorthi
e326f3c629 From<serde_json::Value> and From<serde_yaml::Value> (#196)
Provide wrappers around serde_json::from_value and serde_yaml::from_value since
they may not be apparent and the user may end up serializing to json/yaml and
rereading as a regorus::Value

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-06 16:40:57 +05:30
Anand Krishnamoorthi
3d98c3b12e eval_rule: Evaluate rules directly instead of queries (#186)
closes #185

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-03-26 05:24:21 +05:30
Anand Krishnamoorthi
330a6dff72 Remove cruft. (#184)
Logging wasn't implemented fully nor getting used much.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-03-24 18:37:24 +05:30
Anand Krishnamoorthi
7e3fc08a14 Handle non simple refs in chained expressions (#182)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-03-15 07:13:54 -07:00
Anand Krishnamoorthi
48982222c5 Ability to gather print statements (#179)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-03-11 16:44:05 -07:00
Anand Krishnamoorthi
90757210bc Top-down evaluation (#177)
When executing a query, only those rules that are used
by the query will be evaluated.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-03-10 19:46:11 -07:00
Anand Krishnamoorthi
7bc9a50a52 Make unary - operator OPA compatible. (#175)
OPA supports unary - operator only in the following cases:
-\s+numeric literal

We match OPAs behavior for now. This can be revisited later.
2024-03-10 09:03:11 -07:00
Burak
08f3007b5c Don't use deprecated chrono Duration methods (#173)
Some panicking methods on `Duration` are deprecated
as part of chrono `0.4.35`. We switched to use `Duration::try_*`
of those APIs.
2024-03-09 14:35:08 -08:00
Anand Krishnamoorthi
863601c2d5 Propagate Undefined in object expressions (#171)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-03-05 09:56:15 -08:00
Anand Krishnamoorthi
fbfed6b49c Fix regression (#164)
Second lookup of an object rule without fully qualified path, resulted
in returning the object instead of the requested field.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-26 17:31:48 -08:00
Anand Krishnamoorthi
595f9d34d5 Separately keep track of whether rules have been evaluated or not (#163)
Previously we used to rely on whether there was a value in the
data document for a given rule path. This approach cannot handle
the case of evaluating a.b when a.b.c has been evaluated but
a.b.d has not been evaluated. Upon evaluating a.b.c, the data document
will already have a value of a.b even though a.b.d has not yet
been evaluated.

Hence we need to keep track of evaluated rules separately.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-25 23:36:48 -08:00
Anand Krishnamoorthi
f51731e584 Handle else block without body (#155)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-22 21:04:20 -08:00
Anand Krishnamoorthi
10f2caf0c0 Ignore errors from builtin functions in non strict mode (#154)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-22 15:46:51 -08:00
Anand Krishnamoorthi
3a86c83827 Document coverage feature; Convenience query functions (#152)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-22 14:59:06 -08:00
Anand Krishnamoorthi
d3d5367fd4 Policy Coverage (#149)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-19 19:10:13 -08:00
Anand Krishnamoorthi
f3d9652a73 Initial implementation of policy coverage (#146)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-18 22:16:53 -08:00
Anand Krishnamoorthi
8d282f1ffd Preserve false in single-expression queries (#145)
Note: 1 = 2 is different from 1 == 2
See issue for details

fixes #144

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-16 06:11:07 -08:00
Anand Krishnamoorthi
53b990f97d arc feature to enable using Engine and other data structures from multiple threads (#142)
* `arc` feature to make engine usable from multiple threads.

`arc` is turned on by default. When enabled, std::sync::Arc
will be used instead of std::rc::Rc. The former makes regorus
types like Engine, Value, ast nodes etc Send, allowing for
usability from multiple threads.
Arc would add a performance overhead though since the reference
counting will now become atomic.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* Make engine and related types Debug

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* Input, Data as json. Evaluate bool queries.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-13 10:23:19 -08:00
Anand Krishnamoorthi
13eb06e4be genpolicy tweaks (#141)
Allow `import input` instead of erroring out.
This import is redundant and has no effect.

Emit `print` messages to stderr onstead of stdout.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-11 16:52:12 -08:00
Anand Krishnamoorthi
3b2e639918 io.jwt.decode (#140)
* io.jwt.decode

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* Update README

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* Install musl-tools to compile ring crate

ring crate is a dependency of jsonwebtoken

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-09 23:36:16 -08:00
Anand Krishnamoorthi
a381c38a90 Use compact_rc (#139)
There is not much use for weak_counts in Rc for us.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-09 23:22:09 -08:00
Anand Krishnamoorthi
a95a9d21b3 Ability to add custom builtin functions (#132)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-07 15:15:55 -08:00
Burak
5717f9c249 Partially implement Go's time format (#130)
* Partially implement Go's time format

* Parse date only values

* Fix leap year handling in `time.diff`

* Disable failing test case
2024-02-07 15:12:34 -08:00
Anand Krishnamoorthi
beea2274d3 Conform to OPA 0.61.0. (#118)
Implement `import rego.v1`
https://www.openpolicyagent.org/docs/latest/policy-language/#the-regov1-import

- `if` required before rule body
- import rego.v1 automatically imports future.keywords
- handle import shadowing
- data, input cannot be shadowed
- deprecated functions as disallowed
- rules must have assignment or body
- `contains` required for parital set

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-31 21:39:02 -08:00
Anand Krishnamoorthi
8ca863c661 Python bindings (#115)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-28 14:39:59 -08:00
Anand Krishnamoorthi
6eca85b497 Improve crate documentation (#111)
- Document QueryResults
- Delete snippets folder
- Document Value

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-19 14:51:14 -08:00
Anand Krishnamoorthi
d39200a52c - Document Location, Expression, QueryResult (#109)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-15 17:21:49 -08:00
Anand Krishnamoorthi
723c9379a0 Add crate documentation (#106)
- Separate out public, unstable and internal APIs.
- Cleanup README.md and include it as the crate documentation.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-14 21:15:48 -08:00
Burak
49bd657a41 Implement builtin time.parse_duration_ns method (#100)
* Implement builtin `time.parse_duration_ns` method

* Parse durations properly in `test.sleep` method
2024-01-08 02:21:35 -08:00
Anand Krishnamoorthi
c0fa1c1a42 Implement import keyword (#101)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-08 01:58:37 -08:00
Anand Krishnamoorthi
3e1ddac2a9 OPA conformance: Pass refheads test suite (#90)
- parser: allow non-string index at any position; not just last
- impl Default for Context
- Fix width of OPA test results table
- Allow non string compoenent anywhere in rule ref; not just as last item.
- Normalize want_result before comparison.
- Ensure that object rules are created even if no definition succeed,
- Sort want_result values for "refheads/general, multiple result-set entries"
  The entries are in reverse order of how OPA and regorus produce.
- Emit PASS status for each OPA testpoint
- Detect rule conflicts

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-05 11:13:25 -08:00
Anand Krishnamoorthi
25dec7e59b OPA conformance: Ensure that withkeyword OPA tests pass (#88)
- ignore worktrees
- feature guard time module
- Apply with modifiers before evaluating loop expressions
- Support value modifier for functions
- stubs for http.send and io.jwt.decode_verify
- Initialize with-document after initializing init data
- In case of conflict, with modifier override init-data values.
- In case of conflict, subsequent with modifier overrides earlier ones.
- Ensure that zero parameter functions are evaluated and added to document
- opa.runtime builtin
  returns:
   - git commit hash
   - environment vars
   - regorus features enabled
   - builtins available
   - deprecated builtins available
- If `sort_bindings` is specified, sort the bindings in OPA tests
- gather inputs, used vars and comprehensions in with modifiers
- For refs starting with `data`, ensure that modules are evaluated before looking up
  value of the expression. Thie ensures that modules that have only been partly populated
  (E.g via with mods) are completely evaluated before the value is looked up
- Mark rules overridden using with modifiers are evaluated.
- Exclude env vars in opa.runtime.
- Include regorus version in OPA runtime
- update to opa v0.60.0
- scheduler: Handle function refs in with modifers. Error out only if
  a truly undefined ref.
- Handle undefined params, parameter expression evaluation errors before
  applying with modifiers.
- When applying with modifiers, first determine whether the target is a
  function. If so, handle cleanly.
- concat: raise error only in strict mode
- In strict mode, propagate errors raised by function rule execution
  in case of multiple function definitions for same rule
- skip "withkeyword/builtin-builtin: arity 0" test which can never pass.
- When a mock has is being applied, clear with_function so that
  other mocks won't be applied during the evaluation of the mock.
- Ability to specify strictness in tests

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-04 01:13:04 -08:00
Anand Krishnamoorthi
b470c3fb7f Handle walk builtin as a loop expression (#86)
The walk builtin generates values and implicitly creates a loop over the values.
Hoist walk calls as loops and handle them. Also handle cases where return value
is bound to an extra parameter.

Closes #83
2024-01-01 15:16:57 -08:00
Burak
d2eb3ecd1f Implement most of the builtin time module (#82)
* Implement builtin `time.add_date` method

* Implement builtin `time.clock` method

* Implement builtin `time.date` method

* Implement builtin `time.diff` method

* Implement builtin `time.format` method

* Migrate `time.now_ns` to `chrono`

* Implement builtin `time.parse_ns` method

* Implement builtin `time.parse_rfc3339_ns` method

* Implement builtin `time.weekday` method

* Add conditional `test` module for OPA tests

* Cache result of `time.now_ns`

* Fail in strict mode if timestamp is outside of range

* Add `ensure_i32` util

* Move `diff_between_datetimes` into its own file and include appropriate license
2023-12-31 17:07:39 -08:00
Anand Krishnamoorthi
2292774446 OPA Conformance
- Disable default features of jsonschema to reduce binary size (#85)
- graph.reachable, graph.reachable_paths builtins.
- In progress walk builtin
Note: graph.reachable_paths is buggy in upstream OPA and its semantics
  are not well defined.
  https://github.com/open-policy-agent/opa/issues/5871
  https://github.com/open-policy-agent/opa/issues/6128
- Use correct feature for regex
- Ensure that regorus complies with all features disabled.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2023-12-31 13:51:53 -08:00
Anand Krishnamoorthi
84bedda1c6 OPA conformance (#81)
- Switch to scientific crate. Large values are printed in scientific notations.
  Regular values are printed as u64, i64 or f64.
- Skip copying commit hooks in git worktrees
- urlquery.decode, urlquery.encode, urlquery.encode_object
- substring, indexof_n string builtins
- Make sprintf more OPA conformant

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2023-12-30 08:36:15 -08:00
Anand Krishnamoorthi
e549882b07 More OPA conformance (#77)
- object.union
- object.union_n
- treat negative integers as two separate tokens (Sub and Number)
  when seen in arithmetic expressions
- Ensure that fully query string is parsed
- Evaluate queries in a separate module instead of the last read module.
  This correctly handles queries of the form `x = data.test.y` where x is
  already a ref in `data.test`
- Handle queries producing multiple outputs in test infrastructure
- Add tests for engine
- Add tests locking down valid queries
- Update opa.passing

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2023-12-27 17:00:40 -08:00