Commit Graph

24 Commits

Author SHA1 Message Date
Anand Krishnamoorthi
ee3dff9a3d fix(ci): skip mimalloc FFI and disable isolation for Miri (#621)
- Add cfg(not(miri)) guards to mimalloc module, global allocator, and
  allocator-memory-limits code paths so Miri falls back to the default
  system allocator instead of calling unsupported FFI functions.
- Set MIRIFLAGS="-Zmiri-disable-isolation" in the workflow so tests
  that perform file I/O can run under Miri.
- Skip units/parse tests under Miri due to Float-vs-BigInt Number
  representation mismatch with Miri's soft-float emulation.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2026-03-11 15:13:51 -05:00
Anand Krishnamoorthi
394625d4bc feat!: add cooperative execution-time limits across engine, VM, and binding (#539)
- Introduce ExecutionTimer/ExecutionTimerConfig to allow limiting evaluating time.
- To amortize time checking costs, checking interval can be configured via the notion of work units
- A global fallback time limit can be set to universally limit all evaluation in addition to engine level limit setting.
- Implement limnits in interpreter and RVM. In RVM, also handle suspend/resume so that time during pause is not counted.
- Add engine-level APIs to set/clear per-engine timer configuration and apply global fallback defaults.
- Surface execution-time limits through FFI and C# bindings
- Add C# tests and example usage to validate engine overrides, global fallback behavior, and compiled policy enforcement.
- Expand docs for execution-time limit
- Add interpreter YAML cases and VM unit tests for time-limit behavior and deterministic time sources.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2026-01-28 05:58:03 +05:30
Anand Krishnamoorthi
249dcd0b43 chore: Add clippy lints (#529)
Lints are added (deny) at crate level.

In each offending file, the failing lints are explicitly allowed.
Each file will be fixed in subsequent PRs.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-12-23 15:59:34 -06:00
Anand Krishnamoorthi
6bc1249dc8 feat: Safeguard lookup use
Detect invalid indexes and raise internal errors.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-12-19 11:33:24 -06:00
Anand Krishnamoorthi
a3a20a1235 feat!: Rego -> RVM Compiler and extensive testsuite (#506)
# RVM compiler test cases

Coverage:
- arithmetic
- arrays
- chained lookups
- comparisons
- comprehensions
- default rules
- destructuring
- function rules
- loops/quantifiers
- multiple entrypoints
- objects/sets
- variables
- negative/edge scenarios such as data/rule conflicts
- virtual data lookups
- etc

 # Modify interpreter and compiled policy for RVM Compilation

- Interpreter::eval_default_rule_for_compiler:
   evaluates a named default rule in isolation - allows compiler to emit a constant value instead of instructions
   for the default value

#  feat: Rego Compiler Scaffolding

- Introduce the rego::compiler module surface and entry point wiring
- Add the core compiler concepts:
  - register allocator
  - scope tracking
  - literal/builtin tables
  - rule worklists
  - instruction emit helpers
  - compiler-specific error types
  - context structs for rules, comprehensions, and loops to support later lowering passes.

# feat: Compile Rules/Queries

- add compiler::compile_from_policy workflow plus rule worklist, entry-point wiring, and recursion checks
- implement query lowering:
  - scheduling-aware statement ordering
  - loop hoisting
  - “every/some” semantics
  - context yields
  -  literal assertions
- finalize Program construction

# feat: Expression Lowering

- add compile_rego_expr and helpers to translate every AST expression into RVM instructions,
- interop with binding plans, comprehensions, and membership checks.
- implement collection literal builders (ArrayCreate, SetCreate, ObjectCreate)
  - dedupe literal keys and handle mixed literal/dynamic fields via instruction data blocks.
- operations:
  - arithmetic/boolean/bin operators
  - membership
  - unary minus
  - set unions/intersections
  - etc
- user-defined and builtin function calls
- reference handling
  - analyse chained refs
  - distinguishe data/input/local roots
  - perform rule dispatch or virtual document lookups
  - emits optimized Index/ChainedIndex instructions.

# feat: Comprehensions & Loops

- shared comprehension emitter
 - wraps array/set/object comprehensions with ComprehensionBegin/End
 - context management
- loop lowering utilities
 - read hoisting metadata
 - emit LoopStart/LoopNext
 - some in lowering
 - every quantifiers
 - index iteration
 - propagate binding plans into stored registers so downstream statements see bound variables.

# feat: Destructuring Lowering

- destructuring planner integration
 - assignment/parameter/loop bindings use hoisted plans instead of re-walking ASTs.
- handle :=, =, wildcard matches, and equality
 - evaluate RHS
 - applying destructuring plans
 - emit assert condition as needed
- support nested array/object destructuring, dynamic keys, and some ... in forms

# test: Shared Testing + RVM Suites

- move YAML test helpers into test_utils.rs and re-export via common.rs for use by interpreter and vm test suites
- comprehensive compiler test suite
  - compiles policies with the new Rego→RVM compiler
  - runs them through RegoVM
  - compares against interpreter behavior
  - supports multiple entry points
  - provides assembly listings
  - filterable YAML suites.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-11-24 12:08:37 -06:00
Anand Krishnamoorthi
49bd3c22f3 feat!: add Rego Virtual Machine (RVM) implementation (#495)
* feat!: add Rego Virtual Machine (RVM) implementation

This commit introduces a register-based virtual machine for executing Rego
policies with bytecode-style instructions. Unlike the existing tree-walking
interpreter, the RVM compiles policies into instruction sequences that operate
on virtual registers, offering better performance and optimization potential.

Core Components:

Instruction Set Architecture:
- Define instruction types for data operations, control flow, and builtins
- Implement instruction parameter encoding and display formatting
- Add instruction parser with comprehensive test coverage

Virtual Machine Engine:
- Register-based execution model with program counter management
- Loop execution supporting iterators, comprehensions, and quantifiers
- Function call handling with argument evaluation and context management
- Rule evaluation with default value resolution and virtual data support
- Arithmetic and comparison operation implementations

Program Representation:
- Program listing builder with instruction sequencing
- Rule tree construction for organizing policy rules
- Binary and JSON serialization for compiled programs
- Recompilation support for program modification

Testing Infrastructure:
- Extensive YAML test suites covering all VM features
- Rust unit tests for VM execution and instruction parsing
- Test suites for loops, comprehensions, builtins, and control flow

BREAKING CHANGE: Introduces new VM execution path alongside interpreter

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* docs: add detailed RVM architecture references

Introduce architecture.md explaining program artifacts, serialization, and runtime subsystems.
Document the full opcode catalog in instruction-set.md, including operands, parameter tables, and outcomes.
Walk through execution flow, stacks, and operational guidance in vm-runtime.md, tying the runtime to the new architecture docs.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-11-14 11:43:19 -06:00
Anand Krishnamoorthi
1e4ff952e6 feat!: Introduce structured destructuring plans for bindings (#485)
- add a dedicated `compiler/destructuring_planner` feature that precomputes binding plans for assignments, parameters, and `some in` expressions
- enrich `ScopeContext` with same-scope tracking, local scheduling hints, and module globals so the planner enforces := shadowing rules without blocking parent scopes
- wire the planner through compiler, hoist, interpreter, and engine paths while updating binding plan variants and adding query traversal helpers for dependency analysis
- document the new planner architecture and ship interpreter regressions that exercise nested destructuring, shadowing, and error reporting

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-10-21 15:57:49 -05:00
Anand Krishnamoorthi
5d8387f4d9 feat(hoist): pre-compute loop hoisting metadata at compilation time (#483)
Introduce a compiler pass that analyzes and pre-computes loop hoisting information
during policy compilation. This hoisted metadata is stored in lookup tables and made
available to downstream consumers:

- interpreter: use HoistedLoop entries during evaluation (replaces runtime scanning)
- type inference: can leverage pre-computed loop structure for type propagation
- RVM compiler: will consume hoisting metadata for optimized bytecode generation

Changes:
- populate loop hoisting tables during engine preparation and query snippet execution
- refactor eval_stmts_in_loop and eval_output_expr_in_loop to consume HoistedLoop directly
- add helper methods for accessing loop expressions, collections, and indices from HoistedLoop
- extend Lookup with get_checked and into_slots for safe query context access and merging

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-10-08 11:11:30 -05:00
Anand Krishnamoorthi
85753aaf37 feat: Implement efficient node lookup table using node indices (#463)
Major Changes:
- Add generic Lookup<T> structure for efficient O(1) module-level data access
- Combine separate scope and order lookups into unified QuerySchedule structure
- Add query_schedule field to Interpreter for dedicated user query scheduling
- Refactor loop hoising to separate module
- Use efficient lookup for loop vars
- Also added more tests for loops

Key Concept:
- Ensure module context and indexing stay synchronized during function calls

Testing:
- All scheduler and interpreter tests passing

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-26 15:01:45 -05:00
Anand Krishnamoorthi
cc917ea75d feat: Complete target system with C# bindings and resource inference (#458)
* feat: Add Schema Registry and Validation Framework

This commit introduces a comprehensive schema registry and validation framework, providing schema-based validation of resources and policy effects.

- Thread-safe, in-memory registry for schema storage and management
- Global registry patterns for effects and resources
- Concurrent access with proper error handling
- Unicode schema names support

- JSON Schema-compliant validation for all primitive types
- Advanced constraint validation (patterns, ranges, length limits)
- Discriminated union support with anyOf schemas
- Detailed error reporting with nested validation paths
- Discriminated subobject validation for polymorphic schemas

- **Registry Tests**: All registry operations
- **Effect Tests**: Policy effect validation
- **Resource Tests**: Resource validation
- **Validation Tests**: Core validation engine
- Thread-safety, error handling, integration scenarios, edge cases

- **Dependencies**: dashmap, once_cell, regex
- **Thread Safety**: Minimal locking with Rc<Schema> sharing
- **Error Types**: TypeMismatch, OutOfRange, PatternMismatch, etc.

- Complete schema registry and validation subsystem
- Comprehensive test coverage
- Foundation for policy validation in Regorus

Benchmarks:

- Criterion benchmarks for basic types, effects and Azure resources
- Performance range: 3.22ns (string) to 34.74µs (Azure VM resource schema validation)
- String withs patterns validation: 30.2µs. Need to explore whether regex caching helps
  bring this down.
- Azure policy effects: 188ns-1.4µs

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* feat: Complete target system with C# bindings and resource inference

- Add comprehensive target system with TargetRegistry and target-aware compilation
- Implement resource type inference from policy equality expressions
- Create modular C# bindings with separate wrapper classes for each concept
- Add thread-safe CompiledPolicy with reference counting for safe disposal
- Enhance FFI with detailed error propagation and target functionality
- Create TargetExampleApp demonstrating Azure Policy integration
- Add CI/CD pipeline testing for all C# applications
- Support target definitions with schema validation and resource selectors
- Implement PolicyModule struct and target-aware compilation methods
- Add comprehensive test coverage for target functionality

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-19 20:23:43 -05:00
Anand Krishnamoorthi
f46ab5b697 fix!: Fix glob.match behavior in presence of : (#390)
glob.match("api://*/appId", null, "api://foo.com/appId") wasn't
being handled correctly. Switch to globset crate.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-04-09 11:11:57 -07:00
Anand Krishnamoorthi
c963e477a3 feat: Update to OPA v1.2.0 (#373)
Regorus now defaults to rego v1. `import rego.v1` is no longer needed.
Additionally, `future` keywords are automatically imported.

See
https://www.openpolicyagent.org/docs/latest/v0-upgrade/#changes-to-rego-in-opa-v10
to understand the differences between rego v1 and v0.

BREAKING CHANGE:

v0 style policies will error out by default. To enable v0 behavior, call engine.set_rego_v0(true) before
loading policies.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-03-10 11:56:01 -07:00
Anand Krishnamoorthi
7565ec3ecf feat: or keyword (#315)
Add `or` operator to Rego languages. Available via `rego-extensions`
Cargo feature.

If the evaluated lhs value is not false, null or undefined it is returned.
Otherwise rhs is evaluated and returned.

or operator has least precedence, and is left-associative.

closes #314
2024-09-13 16:39:19 -07:00
Anand Krishnamoorthi
ef549a6528 fix: Merge data to init document (#293)
Init document is the aggregated data documen that the user has
specified using multiple `add_data` calls. Each query evaluation
starts of by initializing the current data to the init document.

Previously `add_data` was incorrectly added to the current document,
causing the added data to be lost if the addition happened after query
evaluation.

With this fix, scenarios where data addition may be interspersed with
query evaluation calls are supported.

Also provide a get_data method to obtain the (init) data document.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-08-07 11:39:23 -07:00
Anand Krishnamoorthi
3ac9b1121e Lockdown kata test prints as well as prints of various values (#249)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-23 08:30:41 -07:00
Anand Krishnamoorthi
e86b590f91 no_std support (#232)
- Disable default features in dependencies
- Use anyhow::Error::msg to map errors. Note: anyhow will itself be removed later.
- lazy_static/spin_no_std used in no_std environments
- ensure_no_std binary is built to target  thumbv7m-none-eabi to ensure that
  there are no std dependencies.  thumbv7m-none-eabi target has no std support.
- The opa-no-std feature enables only those Regorus features that work with no_std.
- Enable tests with no_std
- Update sizes of regorus binary in  README.md
- Ensure that regorus example can be built with only std
- Ensure that regorus example can be built with no_std

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-13 06:42:35 -07:00
Anand Krishnamoorthi
01fc234a33 add std feature (#231)
- `std` feature is enabled by default
- By default enable #![no_std] compilation
- Import std create if `std` feature is enabled or if testing
- Use core, alloc types
- Make it clear where std types are being used
- In no std, use BTreeMap in place of HashMap.
   HashMap is not available in no std due to lack of a
   secure random number generator

Note: The project does not yet compile without std feature being specified.
But it's really close to being able to do so.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-05-09 11:28:42 -07:00
Anand Krishnamoorthi
3743f32edc Enable policy files greater than 64KB in size (#217)
fixes #214

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-26 01:04:21 -07:00
Anand Krishnamoorthi
316f3a7692 early return (#189)
If a rule is written to produce a constant value, then not all iterations of loops
within it need to be executed. Execution can stop via early return once the first iteration
that produces a value has been executed.

This brings forth the question : What if one of the subsequent iterations would have resulted
in an error?
e.g:
x {
  [1, "hello"][_] + 1
}

Such errors are not raised; consistent with OPA.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-04-22 07:41:35 -07:00
Anand Krishnamoorthi
90757210bc Top-down evaluation (#177)
When executing a query, only those rules that are used
by the query will be evaluated.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-03-10 19:46:11 -07:00
Anand Krishnamoorthi
d3d5367fd4 Policy Coverage (#149)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-19 19:10:13 -08:00
Anand Krishnamoorthi
8d282f1ffd Preserve false in single-expression queries (#145)
Note: 1 = 2 is different from 1 == 2
See issue for details

fixes #144

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-02-16 06:11:07 -08:00
Anand Krishnamoorthi
beea2274d3 Conform to OPA 0.61.0. (#118)
Implement `import rego.v1`
https://www.openpolicyagent.org/docs/latest/policy-language/#the-regov1-import

- `if` required before rule body
- import rego.v1 automatically imports future.keywords
- handle import shadowing
- data, input cannot be shadowed
- deprecated functions as disallowed
- rules must have assignment or body
- `contains` required for parital set

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-31 21:39:02 -08:00
Anand Krishnamoorthi
723c9379a0 Add crate documentation (#106)
- Separate out public, unstable and internal APIs.
- Cleanup README.md and include it as the crate documentation.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-01-14 21:15:48 -08:00