Commit Graph

22 Commits

Author SHA1 Message Date
Anand Krishnamoorthi
d561531613 feat: add multi-threaded evaluation benchmark suite with comprehensive C# implementation (#457)
This commit introduces a complete multi-threaded evaluation benchmark suite for both Rust and C# implementations of Regorus.

- Implemented engine evaluation benchmark with input and engine cloning strategies
- Implemented compiled policy evaluation benchmark with input cloning and shared compiled policy strategies.

- Created EngineEvaluationBenchmark.cs and CompiledPolicyEvaluationBenchmark.cs with time-based execution (3s warmup + 3s evaluation)
- Implemented configuration options matching Rust implementation (useClonedEngines, useSharedPolicies parameters)

- Created markdown analysis documentation with cross-platform performance analysis
- C# seems to achieve 58-89% of Rust performance on test machine.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-22 11:40:39 -05:00
Anand Krishnamoorthi
cc917ea75d feat: Complete target system with C# bindings and resource inference (#458)
* feat: Add Schema Registry and Validation Framework

This commit introduces a comprehensive schema registry and validation framework, providing schema-based validation of resources and policy effects.

- Thread-safe, in-memory registry for schema storage and management
- Global registry patterns for effects and resources
- Concurrent access with proper error handling
- Unicode schema names support

- JSON Schema-compliant validation for all primitive types
- Advanced constraint validation (patterns, ranges, length limits)
- Discriminated union support with anyOf schemas
- Detailed error reporting with nested validation paths
- Discriminated subobject validation for polymorphic schemas

- **Registry Tests**: All registry operations
- **Effect Tests**: Policy effect validation
- **Resource Tests**: Resource validation
- **Validation Tests**: Core validation engine
- Thread-safety, error handling, integration scenarios, edge cases

- **Dependencies**: dashmap, once_cell, regex
- **Thread Safety**: Minimal locking with Rc<Schema> sharing
- **Error Types**: TypeMismatch, OutOfRange, PatternMismatch, etc.

- Complete schema registry and validation subsystem
- Comprehensive test coverage
- Foundation for policy validation in Regorus

Benchmarks:

- Criterion benchmarks for basic types, effects and Azure resources
- Performance range: 3.22ns (string) to 34.74µs (Azure VM resource schema validation)
- String withs patterns validation: 30.2µs. Need to explore whether regex caching helps
  bring this down.
- Azure policy effects: 188ns-1.4µs

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* feat: Complete target system with C# bindings and resource inference

- Add comprehensive target system with TargetRegistry and target-aware compilation
- Implement resource type inference from policy equality expressions
- Create modular C# bindings with separate wrapper classes for each concept
- Add thread-safe CompiledPolicy with reference counting for safe disposal
- Enhance FFI with detailed error propagation and target functionality
- Create TargetExampleApp demonstrating Azure Policy integration
- Add CI/CD pipeline testing for all C# applications
- Support target definitions with schema validation and resource selectors
- Implement PolicyModule struct and target-aware compilation methods
- Add comprehensive test coverage for target functionality

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-19 20:23:43 -05:00
dependabot[bot]
3c33d31d08 build(deps): bump clap from 4.5.43 to 4.5.45 (#459)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.43 to 4.5.45.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.43...clap_complete-v4.5.45)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.5.45
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-18 06:40:36 -05:00
dependabot[bot]
965daa0a46 build(deps): bump dashmap from 5.5.3 to 6.1.0 (#460)
Bumps [dashmap](https://github.com/xacrimon/dashmap) from 5.5.3 to 6.1.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases)
- [Commits](https://github.com/xacrimon/dashmap/compare/v.5.5.3...v6.1.0)

---
updated-dependencies:
- dependency-name: dashmap
  dependency-version: 6.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-18 05:03:37 -05:00
Anand Krishnamoorthi
db718654b5 feat: Add Schema Registry and Validation Framework (#456)
* feat: Add Schema Registry and Validation Framework

This commit introduces a comprehensive schema registry and validation framework, providing schema-based validation of resources and policy effects.

- Thread-safe, in-memory registry for schema storage and management
- Global registry patterns for effects and resources
- Concurrent access with proper error handling
- Unicode schema names support

- JSON Schema-compliant validation for all primitive types
- Advanced constraint validation (patterns, ranges, length limits)
- Discriminated union support with anyOf schemas
- Detailed error reporting with nested validation paths
- Discriminated subobject validation for polymorphic schemas

- **Registry Tests**: All registry operations
- **Effect Tests**: Policy effect validation
- **Resource Tests**: Resource validation
- **Validation Tests**: Core validation engine
- Thread-safety, error handling, integration scenarios, edge cases

- **Dependencies**: dashmap, once_cell, regex
- **Thread Safety**: Minimal locking with Rc<Schema> sharing
- **Error Types**: TypeMismatch, OutOfRange, PatternMismatch, etc.

- Complete schema registry and validation subsystem
- Comprehensive test coverage
- Foundation for policy validation in Regorus

Benchmarks:

- Criterion benchmarks for basic types, effects and Azure resources
- Performance range: 3.22ns (string) to 34.74µs (Azure VM resource schema validation)
- String withs patterns validation: 30.2µs. Need to explore whether regex caching helps
  bring this down.
- Azure policy effects: 188ns-1.4µs

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* Address PR feedback

- move error to a separate file
- use meaningful var names

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* Refactor

- Reusable Registry struct
- Split and simplify tests

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-14 15:59:30 -05:00
Anand Krishnamoorthi
77f8544868 feat: Type System (#452)
Details:

- Implement complete Type enum with 12 variants: Any, Integer, Number, Boolean,
  Null, String, Array, Set, Object, Enum, Const, AnyOf
- Add Schema wrapper struct with reference counting for efficient sharing
- Support JSON Schema-compatible deserialization with serde
- Implement discriminated subobjects for polymorphic type definitions
- Add comprehensive test suite covering all type variants
- Include Azure resource schema examples (Storage, VM, Key Vault, App Service)
- Create meta-schema validation system with lazy static validator
- Add extensive edge case and corner case test coverage
- Implement custom deserializers for complex schema patterns

This establishes the foundation for type checking and validation of Rego
policies, particularly useful for cloud resource schemas and policy validation.

Regorus's type system is a first of many features intended to
enable type checking and various other constraints on Rego policies.

The type system is inspired from:
   - JSON schema
   - Bicep

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-11 15:40:45 -05:00
dependabot[bot]
2fcd5e3eb9 build(deps): bump clap from 4.5.42 to 4.5.43 (#454)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.42 to 4.5.43.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.42...clap_complete-v4.5.43)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.5.43
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-11 07:07:53 -05:00
dependabot[bot]
30f0d4e781 build(deps): bump criterion from 0.6.0 to 0.7.0 (#455)
Bumps [criterion](https://github.com/bheisler/criterion.rs) from 0.6.0 to 0.7.0.
- [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bheisler/criterion.rs/compare/0.6.0...0.7.0)

---
updated-dependencies:
- dependency-name: criterion
  dependency-version: 0.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-11 06:34:50 -05:00
Anand Krishnamoorthi
fc09802bfb deps: Update dependencies across all Cargo.toml files and ignore .sln files (#450)
Dependencies updated:
- cc: 1.2.29 -> 1.2.31
- chrono-tz: 0.10.3 -> 0.10.4
- clap: 4.5.40 -> 4.5.42
- clap_builder: 4.5.40 -> 4.5.42
- clap_derive: 4.5.40 -> 4.5.41
- phf: 0.11.3 -> 0.12.1
- phf_shared: 0.11.3 -> 0.12.1
- rand: 0.9.1 -> 0.9.2
- rb-sys: 0.9.116 -> 0.9.117
- rb-sys-build: 0.9.116 -> 0.9.117
- redox_syscall: 0.5.13 -> 0.5.17
- rustix: 1.0.7 -> 1.0.8
- serde_json: 1.0.140 -> 1.0.142
- windows-targets: 0.53.2 -> 0.53.3
- winnow: 0.7.11 -> 0.7.12

Removed obsolete build dependencies:
- chrono-tz-build, parse-zoneinfo, phf_codegen, phf_generator, rand_core

Updated in: main, ffi, java, python, ruby, and wasm bindings
Added *.sln to .gitignore to exclude Visual Studio solution files

Also fix python publishing pipeline by removing non-existent dependency.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-04 19:56:27 -05:00
Anand Krishnamoorthi
9487defa20 chore: Regorus v0.5.0 release (#432)
Also update binding versions and lock files.
Note:
- Ruby binding is not updated
- C# binding is v0.7.0. We will make it match Regorus version later.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-07-08 16:40:06 -05:00
Anand Krishnamoorthi
444b2970a1 feat!: Indexes for nodes in the AST (#414)
Indexes allow associating extra data with nodes in the AST
using an array and then quickly looking up the array to fetch
the extra data.

- Index eidx for expressions
- Index sidx for statements
- Index qidx for queries.

AST nodes are not cloneable. Therefore once a module is created,
it is not possible to accidentally create two nodes with the same
index inadvertently via clone.

Also added IndexChecker in debug builds. When a module is parsed,
it will assert that indexes have been constructed correctly.

AST Cleanup
- Make literal expressions (null, val, number, string etc) also structs
  to match all other expressions
- Merge True and False nodes into a single Bool node.

Also update dependencies.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-06-20 15:09:07 -05:00
dependabot[bot]
c631d44154 build(deps): bump clap from 4.5.38 to 4.5.39 (#415)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.38 to 4.5.39.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.38...clap_complete-v4.5.39)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.5.39
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-02 08:59:41 -05:00
Anand Krishnamoorthi
39f10326cc build(deps): Update criterion and other deps (#412)
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-05-23 14:00:14 -05:00
Burak Varlı
60ac4a7a7c Basic benchmarking setup with Criterion (#408)
* Basic benchmarking setup with Criterion
* Fix Clippy warnings

Signed-off-by: Burak Varlı <burakvar@amazon.co.uk>
2025-05-14 10:41:06 -05:00
dependabot[bot]
2f6c39753c build(deps): bump clap from 4.5.37 to 4.5.38 (#406)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.37 to 4.5.38.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.37...clap_complete-v4.5.38)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.5.38
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-12 10:33:03 -05:00
Anand Krishnamoorthi
130f9685fd feat: Updates for Policy Framework (#405)
- Documentation
  - Regorus Engine is intended to be used from a single thread
  - Clone the engine after adding policies and data to use from another thread

- Builtin errors strictness:
  - default to less strict for OPA compatibility
  - Provide API to change strictness

- Expose GetAstAsJson to C#,
  This can allow writing policy validations in C#.

- Use spectre mitigated msvc crt libs (binskim compliance)

- Update dependencies

fixes #404

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-04-30 15:33:40 -05:00
Anand Krishnamoorthi
6719456468 build: Update dependencies (#401)
Also bump up C# package version

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-04-29 11:06:05 -05:00
Anand Krishnamoorthi
9e43bd9878 fix!: Remove cryptographic builtins (#396)
Cryptographic builtins are removed due to various reasons like FIPS
compliance. Users needing crypto builtins are encouraged to use
extensions.

Deprecated functions are also removed.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-04-16 12:24:19 -07:00
dependabot[bot]
667cb0d90f build(deps): bump clap from 4.5.35 to 4.5.36 (#395)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.35 to 4.5.36.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.35...clap_complete-v4.5.36)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.5.36
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-15 08:33:41 -07:00
Anand Krishnamoorthi
f46ab5b697 fix!: Fix glob.match behavior in presence of : (#390)
glob.match("api://*/appId", null, "api://foo.com/appId") wasn't
being handled correctly. Switch to globset crate.

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-04-09 11:11:57 -07:00
dependabot[bot]
77cdac0fef build(deps): bump clap from 4.5.34 to 4.5.35 (#389)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.34 to 4.5.35.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.34...clap_complete-v4.5.35)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.5.35
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-06 21:14:33 -07:00
Anand Krishnamoorthi
c28bde3f56 build: Check-in Cargo.lock files and lockdown .net (#384)
Use frozen and locked builds

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-03-31 07:55:47 -07:00